Meter reading device, meter reading method, and program

Abstract

To improve work efficiency and improve reading accuracy.SOLUTION: According to an embodiment, a meter reading device includes a user interface, a camera that images a meter having a numeric display frame to obtain multiple pieces of image frame data, a memory that has a buffer area, and a processor that processes the image frame data. The processor includes an image processing unit, an input control unit, and an output control unit. The image processing unit recognizes characters within the display frame for each of the multiple pieces of image frame data and generates a reading result. The input control unit temporarily stores the reading result for each piece of image frame data in the buffer area. The output control unit compares the reading result stored in the buffer area and outputs to the user interface a numerical value corresponding to the reading result whose number of generation times exceeds a predetermined threshold.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] Embodiments of the present invention relate to a meter reading device, a meter reading method, and a program.

Background Art

[0002] There is an operation of periodically reading meter values for maintaining and operating various devices installed in factories, machine rooms of buildings and facilities, and calculating usage fees. In this type of operation, conventionally, inspection workers have filled in and totaled the results of visual reading in paper forms.

[0003] In recent years, services that support facility inspections using tablets have been provided. For example, a service that supports the needle inspection of meters indicating the state of facilities is known. In this type of service, software that processes image data obtained by photographing a meter and converts it into a numerical value is used to read the meter value. By not only photographing the meter value but also reading the value from the photographed image by a computer, human reading errors can be prevented. In addition, by saving the read value as a numerical value, entry errors can be prevented, the accuracy of the record can be ensured, and convenience such as graphing can be improved.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The display mode of the meter value varies widely for each meter. Also, there is a wide variety of meters themselves, such as analog meters, digital meters, rectangular meters, or round meters. In order to accommodate such diversity, there is a need to improve the workability of reading the information displayed on the meter and the reading accuracy.

[0006] Therefore, an object is to provide a meter reading device, a meter reading method, and a program that enhance workability and improve reading accuracy.

Means for Solving the Problem

[0007] According to an embodiment, a meter reading device includes a user interface, a camera that captures an image of a meter having a numerical display frame to obtain a plurality of image frame data, a memory having a buffer area, and a processor that processes the image frame data. The processor includes an image processing unit, an input control unit, and an output control unit. The image processing unit recognizes the characters within the display frame for each of the plurality of image frame data and generates a reading result. The input control unit temporarily stores the reading result for each image frame data in the buffer area. The output control unit compares the reading results stored in the buffer area and outputs, to the user interface, a numerical value corresponding to the reading result whose number of generations exceeds a predetermined threshold value.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

MODE FOR CARRYING OUT THE INVENTION

[0009] Hereinafter, an embodiment will be described with reference to the drawings. (Configuration) FIG. 1 is a diagram showing an example of a cloud system that provides an equipment inspection support service according to an embodiment. Hereinafter, application examples to the needle inspection work of meters used in factories, office equipment, or various measuring instruments will be described.

[0010] The cloud system shown in FIG. 1 is a system centered around a cloud server 13 connected to the Internet 14, for example. Through the Internet 14, the mobile terminal 100 and the site PC 11 can access the cloud server 13.

[0011] The site PC 11 is a desktop or laptop personal computer. The site PC 11 is installed, for example, in the management department of a meter reading service vendor and is operated by the operator of that department. The site PC 11 has functions such as acquiring data files uploaded from the mobile terminal 100 to the cloud server 13, referring to, viewing, editing, aggregating, data conversion / file creation, and issuing forms. The acquired data files can also be saved to the external medium 12.

[0012] The cloud server 13 provides services such as storing, managing, data processing of data uploaded from the mobile terminal 100 or the site PC 11, or data sharing among users. The base station 200 is connected to the Internet 14 via the gateway (GW) 400 and communicably connects the mobile terminal 100 and the Internet 14.

[0013] The mobile terminal 100 is a smartphone, tablet, or notebook computer, etc., and is carried by the inspector who performs the meter reading work. The mobile terminal 100 is used with the form creation support application according to the embodiment installed. The form creation support application includes, for example, a meter reading application. The mobile terminal 100 is equipped with a camera and optically reads the measured values of the meters MT1 to MTn installed at the site. The form creation support application creates form data based on the read measured values and transmits it to the cloud server 13.

[0014] The meters MT1 to MTn are related to various facilities and measuring instruments, and display, for example, measured values such as current, voltage, pressure and flow rate of gas or liquid, and values indicating the operating state of the facilities and equipment in the display frame. There are two types of meters MT1 to MTn: numerical meters and analog meters, and both types can be mixed as the objects of meter reading by the inspector.

[0015] The numerical meter is either rotary or digital, and visually displays measurement values and the like. The digital numerical meter uses, for example, a plurality of 7-segment displays on which numbers from 0 to 9 are displayed to indicate numerical values. The rotary numerical meter includes, for example, a mechanism in which a plurality of number wheels with arithmetic numbers from 0 to 9 are arranged in series, and the number wheels are rotated to indicate numerical values.

[0016] The analog meter indicates numerical values by the position of a pointer that moves on a display panel on which arithmetic numbers and scales are marked. That is, numerical values such as measurement values are analogically indicated by the relative positional relationship between the numbers, scales, and the pointer.

[0017] FIG. 2 is a functional block diagram showing an example of the mobile terminal 100. The mobile terminal 100 as an example of a meter reading device includes a communication unit 101, an input unit 102, a display unit 103, a camera 104, a GNSS (Global Navigation Satellite System) reception unit 105, a storage 106, a memory 107, and a processor 110. That is, the mobile terminal is a computer including a processor and a memory.

[0018] The processor 110 controls the inside of the mobile terminal 100 and also controls communication with the outside of the meter reading device. The communication unit 101 is a wireless communication interface, establishes a wireless communication link with the base station 200, and communicates with the cloud server 13 via the Internet 14. In addition, the communication unit 101 performs interface processing necessary for data transfer, such as adding / removing the header part of the packet.

[0019] The input unit 102 includes input devices such as a touch panel of the display unit 103 and key switches provided on the exterior of the mobile terminal 100, and receives input operations and instructions from inspection workers. The display unit 103 is a user interface that forms a GUI (Graphical User Interface) environment on a display device such as a touch panel to provide visual information to the inspection operator. The touch panel accepts information input by a stylus or a finger. There are various types of touch panels, such as capacitive, resistive film, and projection infrared types. Examples of display devices include liquid crystal panels, organic EL (Electro Luminescence) panels, and electronic paper.

[0020] The display unit 103, for example, displays input fields for information, soft keys, and various images (photos, CG (Computer Graphics) images) on the display device. The display related to the input fields for information and soft keys in the display unit 103 is controlled by the processor 110 so as to correspond to operations on the input unit 102.

[0021] The camera 104 is a digital camera that generates digital image data. The camera 104, for example, photographs meters MT1 to MTn (Fig. 1) and acquires image data including data such as images of the indicated numbers. The image data includes a plurality of image frame data. When the user holds (brings close) the mobile terminal 100 over the meters MT1 to MTn to be photographed and a predetermined time elapses with the camera 104 facing the meters, the processor 110 activates the camera 104. Thereby, the acquisition of the image frame data is started.

[0022] The camera 104 includes an optical system including a lens, an imaging unit equipped with an image sensor such as a CMOS (Complementary MOS), and a signal processing unit that generates image data in a predetermined format (e.g., JPEG (Joint Photographic Experts Group)) from the imaging signal from the imaging unit. The image frame data is processed by the processor 110. The processor 110 may add metadata (e.g., Exif (Exchangeable image file format) data) indicating the shooting date and time, location, and shooting conditions (such as shutter speed, aperture value, angle of view, latitude, and longitude) to the image data.

[0023] When the mobile terminal 100 is brought close to each meter MT1 to MTn, the camera 104 is turned on and imaging is started. The image data acquired by imaging and the reading result of the image data are temporarily stored in the buffer area 5 of the memory 107. The capacity of the buffer area may be, for example, a size that can hold the reading results for up to X times at most, and after X times, the old information is overwritten and updated.

[0024] The processor 110 processes the image data captured by the camera 104 and extracts characters such as numbers. For numerical reading, for example, AI (Artificial Intelligence) can be utilized.

[0025] The reading result data processed by the processor 110 is stored in the buffer 5. The captured image and the reading result data are stored in the storage 106. Alternatively, it may be an external medium 12 or a cloud server 13. When using these storage destinations, data may be compressed to save network resources, or the storage process itself may be periodically performed as a so-called batch process.

[0026] The GNSS receiver unit 105 receives positioning signals transmitted from a plurality of GNSS satellites typified by GPS (Global Positioning System) satellites and ground reference stations, etc., and measures the position of the mobile terminal 100. The positioning signal also includes time information, and the current time can also be known using this.

[0027] The storage 106 is a recording device such as a flash memory like RAM (Random Access Memory), ROM (Read Only Memory), SSD (Solid State Drive), or HDD (Hard Disk Drive). Different types of recording devices may be provided in combination according to the characteristics of the data to be handled.

[0028] The storage 106 stores, for example, data generated along with the operation of the processor 110, various parameters, data input from inspection workers, image data acquired by the camera 104 (for example, image data obtained by photographing the measured values of the meters MT1 to MTn), data acquired (downloaded) from the cloud server 13, or temporary data for information processing, etc. In the embodiment, the storage 106 stores invoice data 106a, setting information data 106b, image data 106c, learned model 106d, and program 106e.

[0029] The invoice data 106a is created by aggregating the pointer values of each meter for each predetermined meter reading period. As soon as new invoice data 106a is created, it is immediately uploaded to the cloud server 13. The invoice data 106a is data created for each meter reading site, for example, and includes information such as "meter information", "meter image", "current meter reading value", "difference from the previous time", "previous meter reading value", "difference obtained previously". When a plurality of meters are installed at the meter reading site, the invoice data 106a includes the above information for each meter.

[0030] Here, the "meter information" includes a master code for identifying the meter, identification information indicating the object to be measured by the meter (such as a tenant, a parking lot, a shared space, etc.), and a type (such as gas, electricity, water, etc.). The "meter image" is an image that serves as the basis for the meter reading value, that is, an image taken during the meter reading.

[0031] The "current meter reading value" is the value read in the current meter reading. The "difference from the previous time" is the difference between the current meter reading value and the previous meter reading value. The "previous meter reading value" is the value read in the previous meter reading. The "difference obtained in the previous time" is the difference between the previous meter reading value and the meter reading value of the time before the previous time.

[0032] The setting information data 106b is setting information necessary for aggregating the meter reading values of each meter. The setting information data 106b is created for each meter at each meter reading site, similar to the form data 106a.

[0033] The image data 106c is image data generated by photographing the meter indicating the meter reading value with the camera 104, and is recorded in the form data 106a as an evidentiary meter image. At that time, only the necessary part including the numerical value may be selected and cut out from the image immediately after shooting.

[0034] The learned model 106d is generated by machine learning a neural network such as a DNN (Deep Neural Network) or a CNN (Convolutional Neural Network). For example, learning data including digital image data including an image of a numerical value and the slope of the numerical value shown in the digital image data is repeatedly input into the neural network, and the learned model 106d can be generated by machine learning that minimizes the error between the output slope and the correct value by a method such as gradient descent.

[0035] The 106e of the storage 106 includes an OS (Operating System), firmware, application software, etc., and causes the mobile terminal 100 to execute various functions according to the embodiment.

[0036] The processor 110 includes processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), a chipset, RAM, ROM, etc., and controls each part of the mobile terminal 100. The ROM stores firmware, set values (various parameters), etc. The processor 110 reads the program 106e in the storage 106 into the RAM, and uses the RAM as a work area to realize various functions.

[0037] Here, the term "processor" used in the embodiment means, for example, a CPU, a GPU (Graphical Processing Unit), or a circuit of an application specific integrated circuit (ASIC), a programmable logic device (for example, a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)).

[0038] The processor 110 realizes each of the above parts by reading and executing the program 106e stored in the storage 106. Instead of storing the program in the storage 106, it may be configured to directly incorporate the program into the circuit of the processor 110. In this case, the processor 110 realizes each of the above parts by reading and executing the program incorporated in the circuit.

[0039] Processor 110 executes the instructions included in program 106e to implement at least the following functions. That is, processor 110 functions as communication control unit 111, input control unit 112, display control unit 113, image processing unit 114, position information processing unit 115, and needle inspection data aggregation processing unit 116. Here, program 106e includes instructions for causing mobile terminal 100 as a computer to function as storage 106, communication control unit 111, input control unit 112, display control unit 113, image processing unit 114, position information processing unit 115, and needle inspection data aggregation processing unit 116.

[0040] Communication control unit 111 controls the communication between mobile terminal 100 and base station 200, Internet 14, and cloud server 13 according to a predetermined communication protocol. Communication control unit 111, for example, downloads files and data from cloud server 13 and uploads data to cloud server 13.

[0041] Image processing unit 114 performs image processing on the image data captured by camera 104. That is, image processing unit 114 recognizes the characters within the display frame for each of a plurality of image frame data and generates a recognition result. Image processing unit 114, for example, has functions as a recognition engine and an analysis engine.

[0042] The recognition engine recognizes various information from an image based on the image data acquired by camera 104, for example, by pattern recognition applying AI. The recognition engine, for example, has functions of detecting a QR code (registered trademark), distinguishing (determining) whether a meter shown in an image is numerical data or analog data, detecting the portion of the display panel of the meter where the measured value is displayed, and detecting an object within the display panel (a numerical sequence of the measured value of numerical data, a pointer or notation number of analog data, etc.) and its arrangement (coordinates).

[0043] The analysis engine performs a process of analyzing the information detected by the recognition engine. For example, for the image of the meter part, the analysis engine decodes a QR code (registered trademark), performs character recognition processing such as optical character recognition (OCR), or converts the measured value of numerical data into text by AI, or detects the value indicated by the pointer from the positional relationship of the objects described in the meter part of the analog meter.

[0044] In addition, if the character spacing of the readable character string within the display frame is equal to or less than a predetermined interval, the image processing unit 114 recognizes the character string as the same character group. Further, the image processing unit 114 recognizes the characters within the display frame digit by digit and generates a recognition result for each digit.

[0045] Under the condition that the character spacing is equal to or less than the threshold value and is recognized as one character group, for example, for the captured image when all 5 digits of 12345 match, increment the count of the number of occurrences. By this method, it can be confirmed that the imaging state is stable without camera shake, and the correct value can be read.

[0046] Also, under the condition that the character spacing is equal to or less than the threshold value and is recognized as one character group, for example, combine the characters with the maximum number of occurrences or occurrence frequencies for each digit of 12345 to newly generate one character group. By doing so, it can be confirmed that the imaging state is stable, and the correct value can be read.

[0047] Regarding the character recognition processing, a plurality of types of analysis engines are prepared in advance. Each analysis engine corresponds to, for example, each type of meter, and uses an analysis algorithm suitable for the notation numbers of the corresponding meter to convert the displayed measured value into text. That is, since the font of the arithmetic numbers varies depending on the meter, and the combination of the notation numbers and the background color also varies, analysis engines with analysis algorithms suitable for the display of each meter are prepared.

[0048] The input control unit 112 interprets the requests of the inspection worker from the operations on the input unit 102, accepts the input of information (such as character strings), the import of data, and the change of the association between data, etc. Further, the input control unit 112 temporarily stores the reading result for each image frame data in the buffer area 5 of the memory 107, or saves it once in the storage 106. Alternatively, not only the reading result, but the input control unit 112 may temporarily store a plurality of image frame data in the buffer area 5. The input control unit 112 acquires the image data generated by the camera 104, transfers it to the storage 106, and stores it as the image data 106c for storage.

[0049] The display control unit 113 as the output control unit displays various information including characters and images, various icons, input fields for information, soft keys, etc. on the display unit 103. Further, the display control unit 113 performs a guide display indicating the capture position of the image. Furthermore, the display control unit 113 compares the reading results stored in the buffer area 5, and outputs the numerical value corresponding to the reading result whose generation count exceeds the preset threshold to the display unit 103. Here, the generation count of the reading result may be read as the generation frequency. That is, the threshold determination may be performed based on the generation count per unit time.

[0050] In addition, the display control unit 113 outputs the image frame data corresponding to the reading result whose generation count exceeds the threshold to the display unit 103. Also, the display control unit 113 outputs the image frame data captured after the generation count exceeds the threshold to the display unit 103.

[0051] In addition, the display control unit 113 notifies the user such as the inspection worker from the display unit 103 that the generation count has exceeded the threshold. Also, the display control unit 113 notifies the user from the display unit 103 that the generation count of the numerical value recognized from the reading result stored in the buffer area 5 has exceeded the preset threshold.

[0052] In addition, the display control unit 113 counts the reading results of the character strings recognized as the same character group by the image processing unit 114, and sets the count value as the number of times the reading result is generated. Further, the display control unit 113 compares the reading results by the image processing unit 114 digit by digit, and outputs to the display unit 103 a numerical value obtained by combining digit by digit the reading results whose generation times exceed a predetermined threshold value.

[0053] The display control unit 113 reads out from the buffer area 5 or the storage 106 the image frame data corresponding to the reading result whose generation times exceed the threshold value, and displays it on the display unit 103.

[0054] Furthermore, the display control unit 113 displays on the display unit 103, in the user interface, a column for displaying the image data of the display frame, a column for displaying the numerical value corresponding to the reading result, and an operation unit for correcting the numerical value displayed in the column for displaying the numerical value corresponding to the reading result.

[0055] The position information processing unit 115 includes an acceleration sensor, a gyro sensor, etc., and measures the position of the mobile terminal 100 based on the detection results of these, the positioning result by the GNSS reception unit 105, and / or the positioning information obtained from the base station 200 by the communication control unit 111. The needle inspection data aggregation processing unit 116 comprehensively controls various processes related to the aggregation of needle inspection data.

[0056] FIG. 3 is a diagram showing a display example of the display unit 103 of the mobile terminal 100. The display unit 103 displays a needle inspection value input screen as shown in FIG. 3, for example. The needle inspection value input screen includes a “photo of the current month's needle inspection value” area 61, a “input field for the current month's needle inspection value” area 62, a “voice input result of the current month's needle inspection value” area 63, and a voice input start icon 64.

[0057] The "Photo of Current Month's Needle Inspection Value" area 61 corresponds to the column for displaying the image data of the display frame. The "Input Field for Current Month's Needle Inspection Value" area 62 corresponds to the column for displaying the numerical value corresponding to the reading result. Further, the "Input Field for Current Month's Needle Inspection Value" area 62 also corresponds to the operation part for correcting the numerical value displayed in the column for displaying the numerical value corresponding to the reading result.

[0058] Area 61 displays the image of the needle inspection value obtained by photographing the meter to be needle-inspected with the camera 104. Area 62 is a field for the inspection operator to input the needle inspection value. The inspection operator can input the needle inspection value in two ways: by looking at the actual meters MT1 to MTn and inputting, or by looking at the image in area 61 and inputting.

[0059] Area 62 displays the digit wheels corresponding to each digit of the numerical value. By operating the digit wheels by the inspection operator through the GUI, the digit wheels can be rotated in the vertical direction. That is, the inspection operator inputs the needle inspection value by operating the digit wheels of each digit displayed in area 62.

[0060] Area 63 is an area where the result of voice input by the inspection operator by reading out the needle inspection values of the meters MT1 to MTn is displayed. The display unit 103 displays the needle inspection value of the voice input result in area 63. Also, the display unit 103 displays a warning message "Please confirm the needle inspection value" in the warning display area 65 which is an area for displaying a warning against incorrect needle inspection. Thereby, the mobile terminal 100 can prompt the inspection operator to confirm again whether the current needle inspection value that may have been incorrectly input is the correct value.

[0061] The voice input start icon 64 is an icon that the inspection operator presses when starting voice input. In response to the inspection operator pressing the voice input start icon 64, the input control unit 112 (Figure 2) enables (activates) the voice input unit 232 of the mobile terminal 100.

[0062] As shown in FIG. 3, the camera 104 is provided, for example, on the back surface of the mobile terminal 100. Also, the operation button 53 is provided at the lower center of the display unit 103 of the mobile terminal 100. This operation button 53 may be included in the input unit 102 (FIG. 2).

[0063] (Operation) Next, the operation in the above configuration will be described.

[0064] [Method of counting the number of times of complete character group matching] FIG. 4 is a diagram for explaining the [method of counting the number of times of complete character group matching]. A numerical display frame is cut out from the image frame data obtained by photographing the meter, and further, for example, by character recognition using AI, the numbers for each digit are taken out as characters. The image processing unit 114 performs reading for each digit. Here, if the distance between characters (character interval) is equal to or less than a predetermined threshold interval, the image processing unit 114 recognizes the characters in the display frame as the same one character group.

[0065] In FIG. 4, the number of occurrences of the numerical value "13868" is counted. For example, if the same numerical value is obtained 8 times out of 10 times, as in (1) to (10) (refer to the part of FIG. 4(a)), assuming that the threshold is exceeded, an image for storage is automatically captured. Next, the numerical value "13868" is recorded as the read value (refer to FIG. 4(b)). This is the read result in terms of numerical units.

[0066] The image processing unit 114 extracts the characters that can be read from the image, obtains the distance between the characters, and determines whether the characters are included in the same character string group or are treated as part of another character string group. This determination process will be described later.

[0067] FIG. 5 is a diagram for explaining the application of the [method of counting the number of times of complete character group matching] to a plurality of meters. When a plurality of meters are included in the field of view of the camera 104, the processor 110 sets a plurality of buffer areas 5 in the memory 107. This is why there are two sets of (1) to (10) in the figure (see FIGS. 5(a) and 5(b)). And the recognition result for each meter is held in each buffer area 5.

[0068] FIG. 6 is a diagram for explaining the [method of selecting the most frequent value for each digit and using the result of combining each digit as the reading value]. A numerical display frame is cut out from the image frame data obtained by photographing the meter, and further, for example, by character recognition using AI, the numbers for each digit are taken out as characters. The number of appearances of each digit is counted. For example, in the leftmost digit, the value 1 is read from (1) to (10). Therefore, the most frequent value is 1. In the digit second from the left adjacent to it, 3 appears 8 times and 8 appears 2 times. Thus, the most frequent value of the reading value is 3. In this way, the most frequent values of each digit are combined to obtain "13868".

[0069] The image processing unit 114 reads digit by digit and counts the appearance frequency of the numerical values in each digit. And the numerical value that reaches the predetermined number of appearances is taken as the confirmed result of that digit. For example, in FIG. 6, the most frequent values of each digit are "1", "3", "8", "6", "8", and these are used as the values of each digit. The meter reading value is the numerical value "13868" obtained by combining the most frequent values of each digit. This is called reading by digit unit. This numerical value "13868" is displayed on the display unit 103.

[0070] Here, the timing when the shutter of the camera 104 starts may be the timing when the operation button 53 is operated by the user, or alternatively, at the timing when a certain time has elapsed after the camera 104 is started with a timer setting. In the description, the number of readings is set to 10, but any value can be set for the number of times.

[0071] FIG. 7 is a flowchart showing an example of a processing procedure related to numerical value reading. In FIG. 7, for example, the camera 104 is activated from an application installed on the mobile terminal 100 (step S10), held against the meter (step S11), and the camera 104 is brought closer to the meter until the target meter can be photographed.

[0072] Next, the image processing unit 114 reads the characters (step S12). Next, it is determined whether or not a predetermined condition has been achieved (step S13). If Yes, as a condition achievement, a final meter image is captured by the camera 104 (step S14), and the meter image together with the read numerical value is displayed on the display unit 103 (step S14).

[0073] If No in step S13, the processing procedure returns to step S12. Then, the reading process is looped until the condition is achieved. Here, the condition in step S13 is, for example, whether or not the frequency or the number of times the read character string (numerical string) completely matches exceeds a predetermined threshold value. When this achievement condition is satisfied, the reading result is displayed in units of numerical values or in units of digits with the most frequent value of each digit as the read value. In the processing procedure of FIG. 7, the loop is repeated until the condition is satisfied, and a case is shown where there is no setting such as a timer for determining the upper limit time until shooting or shooting with a manual shutter.

[0074] FIG. 8 is a flowchart showing another example of a processing procedure related to numerical value reading. In FIG. 8, the camera 104 is activated (step S20), held against the meter (step S21), and the camera 104 is brought closer to the meter until the target meter can be photographed. The image processing unit 114 reads the characters (step S22).

[0075] Next, the processor 110 determines whether the timer setting time has elapsed (step S23). If the answer is No, the processor 110 determines whether a manual shutter operation has been performed, that is, whether the operation button 53 has been pressed (step S24). If the answer is also No in step S24, the processing procedure returns to step S22 until the condition is met (Yes in step S25), and the character reading process is looped.

[0076] When the timer setting time has elapsed or the manual shutter is pressed, the loop is exited by an interrupt. When the loop is exited or the condition in step S25 is met, the final meter image is captured by the camera 104 (step S26), and the meter image is displayed on the display unit 103 together with the read numerical value (step S27).

[0077] In the processing procedure of FIG. 8, since there is no branch in one path between "imaging" in step S26 and "numerical value output (numerical unit)" in step S27, the determination is always made by an exact match of the target character group. This processing procedure mainly focuses on real-time performance (pre-reading).

[0078] FIG. 9 is a flowchart showing another example of the processing procedure related to numerical value reading. In FIG. 9, the camera 104 is activated (step S30), held in front of the meter (step S31), and the camera 104 is brought closer to the meter until the target meter can be photographed. The image processing unit 114 performs character reading (step S32).

[0079] Next, the processor 110 determines whether the timer setting time has elapsed since the start of reading (step S33). If the answer is No, the processor 110 determines whether the operation button 53 has been pressed (step S34). If the answer is also No in step S34, the processing procedure returns to step S32 until the condition is met (Yes in step S35), and the character reading process is looped.

[0080] In step S34, it is determined whether the user manually presses the camera shutter. Before the mobile terminal 100 performs automatic imaging when the timer set time has passed or conditions are met, etc., the processing procedure when the user manually presses the shutter is shown.

[0081] In step S34, if the shutter is pressed, it is Yes, and "imaging" in step 38 is performed. If the camera shutter is not pressed, it is No, and the determination in the next step 35 is made. In step S35, if the characters (numbers) read in step S32 exceed the specified number of occurrences or frequency, the threshold is exceeded, and it is considered that the shooting conditions are met, and the process proceeds to step S36. If it is No in step S35, the processing procedure returns to step S32.

[0082] When the timer set time has elapsed or the manual shutter is pressed, the loop is exited by an interrupt. After exiting the loop, the processing procedure proceeds to step S38, and the final meter image is captured by the camera 104 (step S38). This is the processing when the imaging conditions are not reached until the timer set time is exceeded. Next, the numerical value in digit units is output and displayed on the display unit 103 (step S39). In step S39, the reading result obtained by combining the most frequent values of each digit shown in [the method of selecting the most frequent value of each digit and using the result of combining each digit as the reading value] is output.

[0083] On the other hand, even when the condition in step S35 is met, the final meter image is captured by the camera 104 (step S36). The meter image is displayed on the display unit 103 together with the read numerical value (step S37).

[0084] In step S37, similar to the method of counting the number of times of exact character group matching, the image captured by the exact character group matching is displayed on the display unit 103. In step S37, "numerical output (numerical unit)" is an exact match, and it is difficult to exceed the threshold including the shooting conditions. On the other hand, in the "numerical output (digit unit)" of step S39, it is easy to obtain the reading value as a result from the threshold of each digit. For example, when the number of digits of the character group increases, it is also assumed that the exact match of "numerical output (numerical unit)" takes more time to achieve the condition.

[0085] In the flowcharts of FIGS. 7, 8, and 9, the object to be read is described as a numerical value. However, this is not limited thereto, and other characters may be included in the object to be read, and the same processing procedure is established.

[0086] FIG. 10 is a flowchart showing an example of the processing procedure in the reading process. The details of the processing in steps S12, S22, and S32 are shown. When jumping to the reading process of FIG. 10, the processor 110 identifies the display frame of the numerical value in the image and starts reading the characters within the frame (step S40). Next, the processor 110 determines whether the display frame in which the numerical value is displayed has been read (step S41). If it is Yes, the reading is successful and the process proceeds to the next step. If it has not been read, it becomes No in step S41, and the processing procedure returns to step S40.

[0087] If it is Yes in step S41, the processor 110 determines whether it can be recognized as the first character string group or the next character string group (step S42). In this step, after reading one character at a time, based on the interval between the characters, it is determined whether to recognize them as one character string group. If the character interval is narrower than the threshold, it is treated as one character string group and becomes Yes. Then, the processor 110 temporarily stores the reading result of each character string group in the memory 107 (step S44), and the process returns to the calling source.

[0088] On the other hand, if the character spacing is wider than the threshold value, it is recognized as a different character group, so the result in step S42 is No. The processor 110 recognizes it as a different character string group (step S43), and the process returns to the calling source.

[0089] FIG. 11 is a diagram for comparing a conventional reading method with a case using a plurality of image frame data. The conventional reading method is shown on the left side in the figure (see (a) of FIG. 11). After capturing one image, the value is read. The image processing unit 114 may change the contrast of one image to pseudo-create and process a plurality of images. From the time when the camera 104 is activated and character reading becomes possible, acquisition of image frame data and storage of the reading result in the buffer area 5 are started. For example, from 10 comparison images, the numbers that appear most frequently for each digit are tabulated and output to estimate the read value of the numerical value.

[0090] In the conventional method, a numerical value was read from only one image frame. However, in the method of FIG. 11, the information amount of the image can be significantly increased, so the reading rate and reading accuracy are improved accordingly. In FIG. 11, 13868 obtained by combining the most frequent values is obtained as the read value. When the read value is obtained, the processor 110 controls the camera 104 to acquire image data for storage and for display on the display unit 103. Alternatively, the display control unit 113 may read out the image frame data corresponding to the read value from the buffer area 5 and output it to the display unit 103.

[0091] FIG. 12 is a block diagram showing an example of a computer capable of implementing the functions of the meter reading device. In FIG. 12, an information processing device 20 as a general-purpose computer includes a control device such as a CPU 86, a storage device such as a ROM (Read Only Memory) 88, a RAM (Random Access Memory) 90, and an HDD (hard disk drive) 92, an I / F unit 82 that is an interface with various devices, an output unit 80 that outputs various information such as output information, an input unit 94 that receives operations by the user, and a bus 96 that connects each unit.

[0092] In the above configuration, the CPU 86 reads a program from the ROM 88 and executes it on the RAM 90, whereby each part shown in FIG. 2 is realized on a general-purpose computer.

[0093] Note that the program for executing each of the above processes executed by the information processing apparatus 20 of the above embodiment may be stored in the HDD 92. Further, the program for executing each of the above processes executed by the information processing apparatus 20 of the above embodiment may be provided by being pre-installed in the ROM 88.

[0094] Further, the program for executing the above process executed by the information processing apparatus 20 may be stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), flexible disk (FD), etc. in an installable format or an executable format file, and provided as a computer program product. Further, the program for executing the above process executed by the information processing apparatus 20 of the above embodiment may be stored on a computer connected to a network such as the Internet, and provided by being downloaded via the network. Further, the program for executing the above process executed by the information processing apparatus 20 of the above embodiment may be provided or distributed via a network such as the Internet.

[0095] (Effect) As described above, according to the embodiment, (1) the mobile terminal 100 is brought close to the meter to be inspected, and the camera 104 is activated. (2) The camera 104 is held facing the meter. (3) The user presses the operation button 53. (4) The user checks the displayed image and its read value. (5) Move to the next meter. By this procedure, the user (inspection worker) can collect the data of the meter.

[0096] In the existing method, only numerical values are read from a single captured image, so the accuracy varies greatly depending on the state of the image. In contrast, in the embodiment, by performing imaging and reading in real time from the point when the camera 104 is held in front of the meter, a plurality of reading result information can be obtained.

[0097] That is, it is possible to realize the operation of "when it becomes possible to shoot stably without camera shake, notify the user. By the notification, the user presses the shutter for imaging or performs imaging by automatic shooting. Transmit the imaging result and the value reading result to the user by image display." If the user gets used to it, it is also possible to realize the operation of "starting from the time when it becomes possible to read the meter value from the captured image, when the set time or the set number of acquired images is exceeded, automatically press the shutter and shoot. Then, continue to shoot automatically with different captured images in the same way."

[0098] If the matching rate of the read characters of the captured image by the camera 104 exceeds the threshold value, or when a certain period of time has elapsed since the activation of the camera 104, set it to automatic shooting, then simple and high accuracy can be realized.

[0099] That is, according to the embodiment, the reading accuracy of the meter value is improved, and an image that can be surely read without shooting blur, insufficient illumination, etc. can be used as a saved image. In addition, since the shooting and reading of the camera that is activated only by bringing the mobile terminal closer to the meter are started, camera shake caused by the operation of pressing the shutter during shooting can be prevented. In addition, by automatically shooting the same object multiple times, the reading accuracy can be improved. In addition, it is possible to confirm the brightness, the position and angle of the terminal, etc. that can be surely read without shooting blur, reflection of light on the protective glass in the meter display frame, reflection of the photographer himself or herself, or the terminal, etc., and an effect of promoting the proficiency of the operator can be expected. According to the embodiment, a rhythm is generated in the work, enabling efficient shooting and reading, while achieving both improvement in workability and reading accuracy. Further, by repeating shooting at the set timing, a rhythm is generated in the work, enabling efficient shooting and reading.

[0100] From these, according to the embodiment, it is possible to provide a meter reading device, a meter reading method, and a program that enhance workability and improve reading accuracy.

[0101] Note that the present invention is not limited to the above-described embodiment. For example, the application examples are not limited to factories, office facilities, or various measuring instruments. For example, the meters MT1 to MTn may indicate not only measured values such as current, voltage, pressure, and flow rate of gas or liquid, and the operating states of facilities and devices, but also other physical quantities and states.

[0102] Also, the Internet 14 shown in FIG. 1 can be replaced with a LAN (Local Area Network), and it can also be implemented as an on-premises service. In this case, the base station 200 may be an access point that implements a wireless LAN (IEEE802.11 series).

[0103] Alternatively, if the Internet 14 is a public or local mobile communication network, the base station 200 is a wireless base station such as 3G, LTE (registered trademark), 4G, 5G, etc. Also, the air interface between the mobile terminal 100 and the base station 200 in FIG. 1 is not limited to wireless LAN, 4G, 5G, etc. In short, any wireless communication method may be applied as long as the mobile terminal 100 can be connected to the Internet 14 by a wireless link.

[0104] In addition, it is also possible to integrate some or all of the functions included in the processor 110 in FIG. 2 with other functions, or to divide each function into a plurality of functional blocks from another perspective and explain them in another expression. Also, the neural network that forms the basis of the learned model 106d is not limited to CNN or DNN, and neural networks with different forms such as SVM (Support Vector Machine) can be appropriately used.

[0105] In addition, all or some of the functions of the mobile terminal 100 may be implemented on the site PC 11 or the cloud server 13. For example, the image acquired by the mobile terminal 100 may be transferred to the site PC 11, and processing such as image processing, character recognition, and numerical recognition may be performed on the site PC 11. It is possible to flexibly determine how much processing is shared between the edge (mobile terminal 100) and the cloud (site PC 11, cloud server 13) according to the system requirements.

[0106] Furthermore, the service provided by the cloud server 13 may be provided in a subscription manner to the mobile terminal 100 or the site PC 11 owned (or leased) by the client receiving the service.

[0107] Also, the method for inputting the meter reading value in each digit wheel displayed in the "Input Field for Meter Reading Value of the Current Month" area 62 in FIG. 3 is not limited to the method described above. For example, an input method in which a pull-down menu that allows selection of numbers from 0 to 9 is displayed when touching each digit wheel, or an input method in which a frame that allows input of numbers from 0 to 9 is displayed when touching each digit wheel may be used. Any input method related to known number input may be used for the input method of the meter reading value in the input field of the "Input Field for Meter Reading Value of the Current Month" area 62.

[0108] Although embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalent scope thereof.

Explanation of Reference Numerals

[0109] 5... Buffer area, 11... Site PC, 12... External media, 13... Cloud server, 14... Internet, 20... Information processing device, 53... Operation button, 64... Voice input start icon, 65... Alarm display area, 80... Output unit, 82... I / F unit, 86... CPU, 88... ROM, 90... RAM, 94... Input unit, 96... Bus, 100... Mobile terminal, 101... Communication unit, 102... Input unit, 103... Display unit, 104... Camera, 105... GNSS receiving unit, 105... Receiving unit, 106... Storage, 106a... Invoice data, 106b... Setting information data, 106c... Image data, 106d... Learned model, 106e... Program, 107... Memory, 110... Processor, 111... Communication control unit, 112... Input control unit, 113... Display control unit, 114... Image processing unit, 115... Position information processing unit, 116... Needle inspection data aggregation processing unit, 200... Base station, 232... Voice input unit, 400... Gateway, LAN... Wireless, MT1 to MTn... Meters.

Claims

1. A user interface, a camera that captures a meter having a numerical display frame to obtain a plurality of image frame data, a memory having a buffer area, and a processor that processes the image frame data, wherein the processor includes an image processing unit that recognizes characters within the display frame for each of the plurality of image frame data to generate a reading result, an input control unit that temporarily stores the reading result for each of the image frame data in the buffer area, and an output control unit that compares the reading results stored in the buffer area and outputs a numerical value corresponding to a reading result whose generation count exceeds a predetermined threshold to the user interface, wherein the output control unit outputs image frame data captured after the generation count exceeds the threshold to the user interface, a meter reading device.

2. A user interface, a camera that captures a meter having a numerical display frame to obtain a plurality of image frame data, a memory having a buffer area, and a processor that processes the image frame data, wherein the processor includes an image processing unit that recognizes characters within the display frame for each of the plurality of image frame data to generate a reading result, an input control unit that temporarily stores the reading result for each of the image frame data in the buffer area, and an output control unit that compares the reading results stored in the buffer area and outputs a numerical value corresponding to a reading result whose generation count exceeds a predetermined threshold to the user interface, wherein the output control unit notifies the user from the user interface that the generation count has exceeded the threshold, a meter reading device.

3. A user interface, a camera that captures a meter having a numerical display frame to obtain a plurality of image frame data, a memory having a buffer area, and a processor that processes the image frame data, wherein the processor includes an image processing unit that recognizes characters within the display frame for each of the plurality of image frame data to generate a reading result, an input control unit that temporarily stores the reading result for each of the image frame data in the buffer area, and an output control unit that compares the reading results stored in the buffer area and outputs a numerical value corresponding to a reading result whose generation count exceeds a predetermined threshold to the user interface, The output control unit is a meter reading device that notifies the user from the user interface that the number of times of generation of a numerical value recognized from the reading result stored in the buffer area has exceeded a preset threshold value. **Claim 4**: The meter reading device according to any one of claims 1 to 3, wherein the output control unit outputs image frame data corresponding to a reading result in which the number of generations exceeds the threshold value to the user interface. **Claim 5**: The input control unit temporarily stores the plurality of image frame data in the buffer area. **Claim 4**: The meter reading device according to claim 4, wherein the output control unit reads out image frame data corresponding to a reading result in which the number of generations exceeds the threshold value from the buffer area and outputs it to the user interface. **Claim 6**: The meter reading device according to any one of claims 1 to 3, wherein the processor controls the camera to start acquiring the image frame data when a preset time has elapsed while the camera is facing the meter. **Claim 7** The image processing unit The meter reading device according to any one of claims 1 to 3, wherein if the distance between characters of a readable character string within the display frame is equal to or less than a preset threshold interval, the character string is recognized as the same character group. **Claim 8** **Claim 7**: The meter reading device according to claim 7, wherein the output control unit counts the reading results of the character strings recognized as the same character group as the number of generations. **Claim 9**: The image processing unit recognizes the characters in the display frame digit by digit and generates a reading result for each digit. **Claim 11**: The meter reading device according to any one of claims 1 to 3, wherein the output control unit compares the reading results digit by digit and outputs to the user interface a numerical value obtained by combining the reading results in which the number of generations exceeds a preset threshold value digit by digit. **Claim 10**: The output control unit A column for displaying the image data of the display frame on the user interface, A column for displaying the numerical value corresponding to the reading result, The meter reading device according to any one of claims 1 to 3, which displays an operation unit for correcting the numerical value displayed in the column for displaying the numerical value corresponding to the reading result. In the method for reading a meter by the processor of a computer comprising a user interface, a camera, a memory having a buffer area, and a processor, the step of the processor capturing an image of a meter having a numerical display frame to obtain a plurality of image frame data; the step of the processor recognizing the characters within the display frame for each of the plurality of image frame data to generate a reading result; the step of the processor temporarily storing the reading result for each of the image frame data in the buffer area; the step of the processor comparing the reading results stored in the buffer area and outputting, to the user interface, a numerical value corresponding to a reading result whose generation count exceeds a predetermined threshold; the step of the processor outputting, to the user interface, the image frame data captured after the generation count exceeds the threshold. A method for reading a meter, comprising: **Claim 12** In the method for reading a meter by the processor of a computer comprising a user interface, a camera, a memory having a buffer area, and a processor, the step of the processor capturing an image of a meter having a numerical display frame to obtain a plurality of image frame data; the step of the processor recognizing the characters within the display frame for each of the plurality of image frame data to generate a reading result; the step of the processor temporarily storing the reading result for each of the image frame data in the buffer area; the step of the processor comparing the reading results stored in the buffer area and outputting, to the user interface, a numerical value corresponding to a reading result whose generation count exceeds a predetermined threshold; the step of the processor notifying the user, via the user interface, that the generation count has exceeded the threshold. A method for reading a meter, comprising: **Claim 13** In the method for reading a meter by the processor of a computer comprising a user interface, a camera, a memory having a buffer area, and a processor, the step of the processor capturing an image of a meter having a numerical display frame to obtain a plurality of image frame data; the step of the processor recognizing the characters within the display frame for each of the plurality of image frame data to generate a reading result; The step of the processor temporarily storing the reading result for each of the image frame data in the buffer area; The step of the processor comparing the reading results stored in the buffer area and outputting a numerical value corresponding to the reading result whose generation count exceeds a predetermined threshold to the user interface; A meter reading method comprising: the step of the processor notifying the user from the user interface that the generation count of the numerical value recognized from the reading result stored in the buffer area exceeds a predetermined threshold.

14. A program executable by a processor of a computer including a user interface, a camera, a memory having a buffer area, and a processor, An instruction for causing the processor to capture a meter having a display frame for a numerical value with the camera to obtain a plurality of image frame data; An instruction for causing the processor to function as an image processing unit that recognizes characters within the display frame for each of the plurality of image frame data and generates a reading result; An instruction for causing the processor to function as an input control unit that temporarily stores the reading result for each of the image frame data in the buffer area; An instruction for causing the processor to function as an output control unit that compares the reading results stored in the buffer area, outputs a numerical value corresponding to the reading result whose generation count exceeds a predetermined threshold to the user interface, and outputs the image frame data captured after the generation count exceeds the threshold to the user interface; A program including the above.

15. A program executable by a processor of a computer including a user interface, a camera, a memory having a buffer area, and a processor, An instruction for causing the processor to capture a meter having a display frame for a numerical value with the camera to obtain a plurality of image frame data; An instruction for causing the processor to function as an image processing unit that recognizes characters within the display frame for each of the plurality of image frame data and generates a reading result; An instruction for causing the processor to function as an input control unit that temporarily stores the reading result for each of the image frame data in the buffer area; A program including an instruction to cause the processor to compare the read results stored in the buffer area, output a numerical value corresponding to the read results whose generation count exceeds a predetermined threshold to the user interface, and function as an output control unit to notify the user from the user interface that the generation count has exceeded the threshold. [

16. ] A program executable by a processor of a computer comprising a user interface, a camera, a memory having a buffer area, and a processor, an instruction to cause the processor to capture, by the camera, a meter having a display frame for a numerical value and acquire a plurality of image frame data; an instruction to cause the processor to function as an image processing unit that recognizes characters within the display frame for each of the plurality of image frame data and generates a read result; an instruction to cause the processor to function as an input control unit that temporarily stores the read result for each of the image frame data in the buffer area; a program including an instruction to cause the processor to compare the read results stored in the buffer area, output a numerical value corresponding to the read results whose generation count exceeds a predetermined threshold to the user interface, and function as an output control unit to notify the user from the user interface that the generation count of the numerical value recognized from the read results stored in the buffer area has exceeded the predetermined threshold.

Images