Automatic reading device and automatic reading method for mechanical meter

Abstract

To propose an automatic reading device for a mechanical meter that can automatically read a pointer value of the mechanical meter by reducing the risk of misrecognition.SOLUTION: An automatic reading device 1 comprises: a template image storage unit 11 that stores a template image 31 created from character shape data for one full rotation of a numeral wheel of a mechanical meter M; an image acquisition unit 12 that acquires a photographed image 32 of a display panel D captured by an imaging unit 2; a numeral image extraction unit 17 that extracts each numeral display range indicating respective digits of a pointer value as a target numeral image 33 on the basis of the photographed image 32; a numeral recognition unit 18 that performs template matching to extract feature maps of the template image 31 and the target numeral images 33 by using a CNN and recognize numbers of respective digits from a similarity between the feature maps; and a pointer value synthesis unit 19 that synthesizes the recognized numbers of respective digits and reads them as the pointer value.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an automatic reading device and an automatic reading method for mechanical meters, and more particularly to an automatic reading device that captures an image of the display panel of a number rotating wheel type mechanical meter, analyzes the captured image, and automatically reads the pointer value of the number rotating wheel type mechanical meter, and an automatic reading method for automatically reading the pointer value of such a meter. [Background technology]

[0002] In meters that display the cumulative flow rate of fluids, such as kerosene meters, gas meters, and water meters, and meters that display the mileage of automobiles, a number-rotating-wheel-type mechanical meter (hereinafter simply referred to as a "mechanical meter") is used, in which a rotating wheel with numbers 0 to 9 arranged at each digit position rotates mechanically to display the cumulative flow rate, mileage, etc., on a display panel. For example, such mechanical meters are used in meter readings, where a meter reader visually reads and records the indicated value displayed on the display panel. Reading and recording the indicated value manually in this way poses problems such as the risk of human error, such as misreading or transcription errors, and the time required to convert the value into electronic data. In addition, given the recent labor shortage, devices for automatically reading mechanical meters have been proposed (see, for example, Patent Documents 1 and 2).

[0003] Patent Document 1 discloses an automatic reading device for mechanical meters (current character meter recognition processing device) that prepares and stores in advance a template image (matching model image) in which the numbers 0 to 9 are arranged vertically, and compares and searches to determine which position of the number wheel in the template image best matches the displayed value in the captured image obtained by capturing an image of the display panel (indicated value display section) of a mechanical meter (current character meter), and calculates the displayed value in decimal units from the detected position information.

[0004] Patent Document 2 also discloses an automatic reading device (character recognition device) that includes a filter processing unit (102) that generates a binary image by binarizing an image obtained by capturing an image of a display panel (counter unit) of a mechanical meter, a character extraction unit (104) that extracts a plurality of partial images including each of a plurality of characters in the captured image based on a vertical projection that calculates a total brightness value for each ordinate of the binary image and a horizontal projection that calculates a total brightness value for each abscissa of the binary image, and a matching processing unit (107) that matches each of the plurality of partial images with a template image (dictionary data) stored in advance to recognize characters. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-242444 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-207910 Summary of the Invention [Problem to be solved by the invention]

[0006] Many mechanical meters read by automatic reading devices for such mechanical meters have a number wheel that rotates gradually and continuously in the depth direction, and the numbers do not necessarily face forward. Furthermore, the reflected light from the display panel can make the captured image of the display panel difficult to read. Therefore, when automatically reading the pointer value of such a mechanical meter and attempting to recognize the pointer value from the captured image of the display panel, the rotation position of the number wheel at the time of capture can cause image distortion, resulting in deformation of the numbers and misalignment of the numbers' position or size, which can lead to misrecognition. Furthermore, misalignment of the numbers in the image extracted during reading can lead to misrecognition (see FIG. 8(a)). Furthermore, the difference between the captured image, which is difficult to read due to the reflected light, and the template image can become large, leading to misrecognition (see FIG. 8(b)). Therefore, there is a constant demand for new devices and methods that can read the pointer value of a mechanical meter from the captured image of the mechanical meter's display panel without misrecognition.

[0007] In view of the above, an object of the present invention is to propose an automatic reading device for a mechanical meter that can automatically read the pointer value of a mechanical meter while reducing the risk of erroneous recognition, and also to propose an automatic reading method for a mechanical meter that can automatically read the pointer value of such a meter while reducing the risk of erroneous recognition. [Means for solving the problem]

[0008] The automatic reading device for a mechanical meter according to the present invention is an automatic reading device for a mechanical meter that automatically reads the pointer value of a number rotating wheel type mechanical meter, a template image storage unit for storing a template image created from character data for one revolution of a number wheel of a mechanical meter to be read; an imaging unit that captures an image of a display panel of a mechanical meter; an image acquisition unit that acquires a captured image of the display panel captured by the imaging unit; a number image extraction unit that extracts, as target number images, each of the number display ranges that indicate each digit of the pointer value based on the captured image acquired by the image acquisition unit; A number recognition unit uses CNN (Convolutional Neural Network), a machine learning technique, to extract feature maps for the template image and the target number image, and performs template matching to recognize each digit based on the similarity between the feature maps. a pointer value synthesis unit that synthesizes the numbers of each digit recognized by the number recognition unit and reads them as a pointer value; The present invention is characterized by comprising:

[0009] In this automatic reading device for mechanical meters, the template matching performed in the number recognition unit preferably involves setting a comparison window to select a feature map of the same size as the feature map of the target number image from the feature map of the template image, calculating the similarity between the feature map within the comparison window and the feature map of the target number image for one revolution while shifting the position of the comparison window placed on the feature map of the template image, and determining the number to be recognized from the position where the similarity is greatest.

[0010] In addition, in this automatic reading device for mechanical meters, a binarization unit that binarizes the captured image acquired by the image acquisition unit using an adaptive threshold; a maximum area extraction unit that extracts an area with a maximum black area from the captured image binarized by the binarization unit; a contour extraction unit that extracts the contour of the largest area extracted by the largest area extraction unit; a line detection unit that detects a line from the contour extracted by the contour extraction unit; Further provided with It is preferable that the number image extraction unit estimates the position of the number from the straight line detected by the straight line detection unit, cuts out the number image at the estimated position from the captured image acquired by the image acquisition unit, and extracts it as the target number image for each digit.

[0011] In this automatic reading device for mechanical meters, the straight line detection section preferably detects straight lines by a Hough transform.

[0012] In addition, in this automatic reading device for mechanical meters, it is preferable that the number image extraction unit corrects distortion using projective transformation when extracting the number image.

[0013] In addition, in this automatic reading device for mechanical meters, The number recognition section a learning dataset storage process for storing a set of number images cut out from the captured images acquired by the image acquisition unit and correct answer labels indicating visual readings of the numbers in the number images as a learning dataset; a learning image selection process for randomly selecting a predetermined number of number images from the learning dataset stored in the learning dataset storage process; A learning image transformation process that translates, scales, rotates, fills in parts of, or randomly transforms the number images selected in the learning image selection process; A learning digit recognition process that extracts a feature map of the digit image converted by the learning image conversion process using CNN, performs digit recognition on the feature map of the digit image by template matching, and calculates the similarity at each position; A CNN parameter update process that updates the CNN parameters using the backpropagation algorithm so that the similarity calculated in the training digit recognition process becomes high at the position corresponding to the correct label; It is preferable to perform template matching to recognize each digit using CNN parameters that are optimized by repeating the above steps.

[0014] This automatic reading device for mechanical meters is suitable for use as a device for reading the indicator value of a kerosene meter that measures the flow rate of kerosene.

[0015] The automatic reading method for a mechanical meter according to the present invention is a method for automatically reading a pointer value of a number rotating wheel type mechanical meter, comprising: a template image preparation step of preparing a template image created from character shape data for one circumference of the number wheel of a mechanical meter to be read; an imaging step of imaging a display panel of a mechanical meter; an image acquisition step of acquiring a captured image of the display panel captured in the imaging step; a number image extraction step of extracting, as target number images, number display ranges indicating each digit of the pointer value based on the captured image acquired in the image acquisition step; A number recognition process uses a machine learning technique called CNN (Convolutional Neural Network) to extract feature maps for the template image and the target number image, and performs template matching to recognize each digit based on the similarity between the feature maps. a pointer value synthesis step of synthesizing the numbers of each digit recognized in the number recognition step and reading it as a pointer value; The present invention is characterized by carrying out the following.

[0016] In this method for automatically reading mechanical meters, the template matching performed in the number recognition step preferably involves setting a comparison window for selecting a feature map of the same size as the feature map of the target number image from the feature map of the template image, calculating the similarity between the feature map within the comparison window and the feature map of the target number image for one revolution while shifting the position of the comparison window placed on the feature map of the template image, and determining the number to be recognized from the position where the similarity is greatest.

[0017] In addition, in this automatic reading method for a mechanical meter, a binarization step of binarizing the captured image acquired in the image acquisition step using an adaptive threshold; a maximum area extraction step of extracting an area with the maximum black area from the captured image binarized in the binarization step; a contour extraction step of extracting a contour of the largest area extracted in the largest area extraction step; a line detection step of detecting a line from the contour extracted in the contour extraction step; Further provided with In the number image extraction step, it is preferable to estimate the position of the number from the straight lines detected in the straight line detection step, and cut out the number image to extract it as a target number image.

[0018] In addition, in this automatic reading method for a mechanical meter, In the line detection step, it is preferable to detect lines by a Hough transform.

[0019] In addition, in this automatic reading method for a mechanical meter, In the number image extraction step, it is preferable to correct distortion using a projective transformation when cutting out the number image.

[0020] In addition, in this automatic reading method for a mechanical meter, a learning dataset storage step of storing a set of the number images cut out from the captured images acquired in the image acquisition step and correct answer labels indicating visual readings of the numbers in the number images as a learning dataset; a learning image selection step of randomly selecting a predetermined number of number images from the learning dataset stored in the learning dataset storage step; a learning image transformation step of randomly transforming the number images selected in the learning image selection step by translating, scaling, rotating, filling in a part of the number images, or by other methods; A learning digit recognition process that uses CNN to extract a feature map of the digit image converted in the learning image conversion process, performs digit recognition on the feature map of the digit image by template matching, and calculates the similarity at each position; a CNN parameter updating step of updating the CNN parameters by backpropagation so that the similarity calculated in the training digit recognition step becomes high at the position corresponding to the correct label; It is preferable to further perform the following.

[0021] This method for automatically reading a mechanical meter is suitable for use in reading the pointer value of a kerosene meter that measures the flow rate of kerosene. [Effects of the Invention]

[0022] According to the present invention, it is possible to provide an automatic reading device for a mechanical meter that can automatically read the pointer value of a mechanical meter while reducing the risk of erroneous recognition, and also to provide an automatic reading method for a mechanical meter that automatically reads the pointer value of such a meter while reducing the risk of erroneous recognition. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a diagram illustrating the overall configuration of an automatic reading device for a mechanical meter according to an embodiment. [Figure 2] 2 is a block diagram showing each functional unit that performs a reading function in the processing unit of the automatic reading device of FIG. 1. FIG. [Figure 3] 3 is a flowchart showing the processing performed in each functional unit that realizes the reading function of FIG. 2. FIG. [Figure 4] FIG. 2 is an explanatory diagram of template matching using CNN performed in the automatic reading device of FIG. 1. [Figure 5] FIG. 2 is a flow chart showing the learning process of the CNN used in the automatic reading device of FIG. 1. [Figure 6] 2 is a flowchart showing a method for automatically reading a mechanical meter performed by the automatic reading device of FIG. 1. [Figure 7] 7 is a flowchart showing a CNN learning method in the automatic reading method of FIG. 6. [Figure 8] FIG. 1 is an explanatory diagram showing an example of erroneous recognition in conventional template matching. DETAILED DESCRIPTION OF THE INVENTION

[0024] Hereinafter, an embodiment of an automatic reading device for a mechanical meter to which the present invention is applied will be described with reference to the drawings. Note that the following embodiment shows an example of the present invention, and is not intended to limit the present invention to the configuration of the embodiment.

[0025] (Overall configuration of automatic reading device 1) FIG. 1 is a diagram illustrating the overall configuration of an automatic mechanical meter reading device 1 according to an embodiment of the present invention. The automatic mechanical meter reading device 1 (hereinafter simply referred to as "automatic reading device 1") is a device for capturing an image of a display panel D of a number wheel-type mechanical meter M, which is commonly used in meter reading meters that display the cumulative flow rate of fluids, such as kerosene meters, gas meters, and water meters, and automatically reading the pointer value from the captured image. As an example, the automatic reading device 1 is configured as a device that automatically reads the pointer value of a kerosene meter. The automatic reading device 1 has a reading function that automatically reads the pointer value. As shown in FIG. 1, the automatic reading device 1 includes an imaging unit 2 with an imaging function, an input / output unit 3 that receives input instructions from a user and outputs the reading results and device information so that the user can confirm them, and a processing unit 4 that controls the entire device.

[0026] The imaging unit 2 is a built-in or external camera that captures an image of the display panel D of the mechanical meter M and transmits the captured image 32 to the processing unit 4, which is electrically connected to the imaging unit 2 and will be described later.

[0027] The input / output unit 3 has input devices such as operation buttons and an operation touch panel, and output devices such as a liquid crystal panel, a speaker, and LED elements, which are appropriately selected as an interface with the user depending on the specifications. The input / output unit 3 also has connection terminals for wired connection with external devices and wireless communication devices for wireless communication with external devices, which are appropriately selected as connection interfaces with external devices depending on the specifications. The input / output unit 3 transmits input data input from the input devices or components to the electrically connected processing unit 4, which will be described later, and presents output data transmitted from the processing unit 4 to the user or an external terminal.

[0028] The processing unit 4 is electrically connected to the imaging unit 2 and the input / output unit 3, and controls the entire device while exchanging data with these units. The processing unit 4 is a computer having a calculation unit 4a and a storage unit 4b, and has multiple functional units corresponding to the execution processes that cause the calculation unit 4a to perform execution processes in accordance with computer processing programs stored in the storage unit 4b in order to perform various functions of the automatic reading device 1. The storage unit 4b stores not only the computer processing programs that indicate the execution processes of the calculation unit 4a, but also temporary data required when performing the execution processes. Below, each of the multiple functional units in the processing unit 4 that perform the reading function of the automatic reading device 1 will be described in detail.

[0029] (Each functional unit that performs the reading function) Fig. 2 is a block diagram showing each functional unit that performs the reading function in the processing unit 4 of the automatic reading device 1, Fig. 3 is a flow diagram showing the processing performed in each functional unit that performs these reading functions, and Fig. 4 is an explanatory diagram of template matching using CNN performed in the automatic reading device 1. With reference to these figures, each functional unit that performs the reading function will be explained. As shown in Fig. 2, the automatic reading device 1 includes a template image storage unit 11, an image acquisition unit 12, a binarization unit 13, a maximum area extraction unit 14, a contour extraction unit 15, a straight line detection unit 16, a number image extraction unit 17, a number recognition unit 18, and a pointer value synthesis unit 19 as functional units that perform processing to automatically read the pointer value of the mechanical meter M in the processing unit 4.

[0030] The template image storage unit 11 stores a template image 31 (see FIGS. 3 and 4) created from character shape data (data in which numbers 0 to 9 are arranged vertically) for one revolution of the number rotating wheel N of the mechanical meter M to be read, which data is input in advance from the input / output unit 3. The template image storage unit 11 also stores the positions in the template image 31 to which each number corresponds.

[0031] The image acquisition unit 12 acquires a captured image 32 of the display panel D of the mechanical meter M captured by the imaging unit 2 (see FIG. 3(a)).

[0032] The binarization unit 13 performs binarization processing to convert the captured image 32 acquired by the image acquisition unit 12 into two colors, black and white (see FIG. 3(b)). At this time, it is preferable to use an adaptive threshold calculated for each small region in the image as the threshold for determining black and white.

[0033] The maximum area extraction unit 14 extracts the area with the largest black area from the captured image 32 binarized by the binarization unit 13 (see FIG. 3(c)).

[0034] The contour extraction unit 15 extracts the contour of the largest area extracted by the largest area extraction unit 14 (see FIG. 3(d)).

[0035] The line detection unit 16 detects lines from the contours extracted by the contour extraction unit 15 (see FIG. 3(e)). At this time, it is preferable to use a Hough transform.

[0036] The number image extraction unit 17 extracts each number display range indicating each digit of the pointer value as a target number image 33 based on the captured image 32 acquired by the image acquisition unit 12. More specifically, the number image extraction unit 17 processes the captured image 32 acquired by the image acquisition unit 12 using the binarization unit 13, maximum area extraction unit 14, contour extraction unit 15, and line detection unit 16 to estimate the position of the numbers from the line detected by the line detection unit 16, and cuts out the number image at the estimated position from the captured image 32 acquired by the image acquisition unit 12 and extracts it as a target number image 33 for each digit (see the upper parts of Figures 3(f) to 3(g)). When cutting out the numbers, the number image extraction unit 17 corrects distortion using projective transformation.

[0037] The number recognition unit 18 uses a machine learning technique called CNN (Convolutional Neural Network) to perform template matching, which recognizes each digit of a number based on the similarity between the feature map of the template image 31 and the feature map of the target number image 33 (see FIG. 3(g)). More specifically, as shown in FIG. 4, the number recognition unit 18 uses CNN to extract feature maps for the template image 31 and the target number image 33, and recognizes each digit of the number based on the similarity between the feature maps as template matching. More specifically, the number recognition unit 18 first creates a feature map by extracting the features of each image from the template image 31 stored in the template image storage unit 11 and the target number image 33 extracted by the number image extraction unit 17 using a filter for extracting features in a feature extraction unit of the CNN, which is composed of a convolutional layer, a pooling layer, etc. Next, a comparison window W is set to select a feature map of the same size as the feature map of the target number image 33 from the feature map of the template image 31. The comparison window W is then shifted over the feature map of the template image 31 to calculate the similarity between the feature map in the comparison window and the feature map of the target number image 33 for one rotation. The number to be recognized is then determined from the position where the similarity is maximized. For example, Figure 4 shows the similarity between the feature map in the comparison window and the feature map of the target number image 33 for one rotation, with the horizontal axis representing position and the vertical axis representing similarity. When the position is exactly "2," the similarity on the vertical axis reaches its maximum at 0.8. As a result, the number recognition unit 18 recognizes the target number image 33 as "2.0." The filter parameters used in the CNN feature extraction unit (hereinafter referred to as "CNN parameters") are updated through the CNN learning process described below.

[0038] The pointer value synthesis unit 19 synthesizes the numbers of each digit recognized by the number recognition unit 18 and reads it as a pointer value (see FIG. 3(h)). For example, in FIG. 3, the number recognition unit 18 recognizes the numbers from the left digit onwards as "0.0", "0.0", "0.0", "1.0", "6.0", "6.0", and "5.8", respectively, and the pointer value synthesis unit 19 synthesizes these and reads the pointer value as "00016.658".

[0039] (CNN learning process) Figure 5 is a flow diagram showing the learning process of the CNN used in the automatic reading device 1. Referring also to Figure 5, the learning of the CNN used in the digit recognition unit 18 will be described. In the automatic reading device 1, the CNN parameters used in the digit recognition unit 18 are updated to CNN parameters optimized by CNN learning by a computer terminal connected via the input / output unit 3 as appropriate. As shown in Figure 5, the CNN parameters used in the digit recognition unit 18 are updated by importing, from the input / output unit 3, the CNN parameters optimized by the connected computer terminal through a learning dataset storage process 21, a learning image selection process 22, a learning image conversion process 23, a learning digit recognition process 24, and a CNN parameter update process 25.

[0040] In the learning dataset storage process 21, the computer terminal captures number images cut out from the captured image 32 acquired by the image acquisition unit 12, as shown in Fig. 5(a), and stores the captured number images and correct labels indicating visually read values ​​of the numbers in the number images as sets of learning datasets. For example, in Fig. 5(a), the learning dataset storage process 21 stores at least a set of a number image in which the number "3" faces straight ahead and a correct label of "3.0", a set of a number image in which the number "2" is shifted downward and missing its bottom half and a correct label of "1.8", a set of a number image in which the number "7" is shifted slightly downward and a correct label of "7.0", a set of a number image in which the number "8" is shifted slightly downward and slightly to the left from the straight ahead and a correct label of "8.0", and a set of a number image in which the number "4" faces straight ahead and is shifted slightly to the right and a correct label of "4.0".

[0041] In the learning image selection process 22, as shown in Fig. 5(b), the computer terminal randomly selects a predetermined number of number images from the learning dataset stored in the learning dataset storage process 21. For example, in Fig. 5(b), a number image in which the number "4" faces straight ahead and is slightly shifted to the right, and a number image in which the number "2" is shifted downward with the bottom missing, are selected.

[0042] In the learning image conversion process 23, the computer terminal randomly converts the number images selected in the learning image selection process 22 by translating, scaling, rotating, filling in a part of them, or by other methods, as shown in Fig. 5(c). For example, in Fig. 5(c), the number "4" is converted by translating it to the right, and the number "2" is converted by filling in a part of the upper left side of the number "2".

[0043] In the training digit recognition process 24, the computer terminal extracts a feature map of the digit image converted in the training image conversion process 23 using CNN, and performs digit recognition on the feature map of the digit image by template matching, calculating the similarity at each position, as shown in Fig. 5(d). The template matching method performed here is the same as the template matching performed in the digit recognition unit 18 described above.

[0044] In the CNN parameter update process 25, the computer terminal updates the CNN parameters using the backpropagation method so that the similarity calculated in the training digit recognition process 24 becomes high at the position corresponding to the correct label, as shown in Figure 5(e).

[0045] In the automatic reading device 1, the CNN parameters are optimized through learning by repeatedly performing a processing cycle of a learning image selection process 22, a learning image conversion process 23, a learning digit recognition process 24, and a CNN parameter update process 25. As a result, in the automatic reading device 1, the reading accuracy of the mechanical meter M improves as the learning is repeated.

[0046] (Mechanical meter M automatic reading method) Fig. 6 is a flowchart showing the automatic reading method of a mechanical meter M performed by the automatic reading device 1. With reference to Fig. 6, the automatic reading method for automatically reading the pointer value of a number rotating wheel type mechanical meter M performed by the automatic reading device 1 will be described. Note that each step performed in this automatic reading method is the same as the processing performed by each functional unit that performs the reading function in the automatic reading device 1 described above. For this reason, in the following explanation, specific examples will be omitted.

[0047] In this automatic reading method, as shown in FIG. 6, first, in a template image preparation step S1, a template image 31 is prepared from character data for one revolution of the digit wheel of the mechanical meter M to be read. Next, in an imaging step S2, an image of the display panel D of the mechanical meter M is captured. Next, in an image acquisition step S3, a captured image 32 of the display panel D captured in the imaging step S2 is acquired. Next, in a binarization step S4, the captured image 32 captured in the image acquisition step S3 is binarized using an adaptive threshold. Next, in a maximum area extraction step S5, an area with the largest black area is extracted from the captured image 32 binarized in the binarization step S4. Next, in a contour extraction step S6, the contour of the largest area extracted in the maximum area extraction step S5 is extracted. Next, in a line detection step S7, a Hough transform is used to detect lines from the contours extracted in the contour extraction step S6. Next, in the number image extraction step S8, the position of the numbers is estimated from the line detected in the line detection step S7 based on the captured image 32 acquired in the image acquisition step S3, and the number image is extracted to extract each number display area representing each digit of the pointer value as a target number image 33. In the number image extraction step S8, distortion is preferably corrected using projective transformation when extracting the numbers. Next, in the number recognition step S9, a machine learning technique called CNN (convolutional neural network) is used to extract feature maps from the template image 31 and the target number image 33, and template matching is performed to recognize each digit based on the similarity between the feature maps. In the template matching performed in this number recognition step S9, a comparison window W is set to select a feature map of the same size as the feature map of the target number image 33 from the feature map of the template image 31. The position of the comparison window W placed on the feature map of the template image 31 is shifted to calculate the similarity between the feature map within the comparison window W and the feature map of the target number image 33 for one rotation, and the number to be recognized is determined from the position where the similarity is greatest. Then, in a pointer value synthesis step S10, the numbers of each digit recognized in the number recognition step S9 are synthesized and read as a pointer value. In this automatic reading method, the CNN parameters are updated by machine learning.

[0048] (CNN training method) FIG. 7 is a flowchart showing the CNN learning method in this automatic reading method. The CNN learning method performed in this automatic reading method will be described with reference to FIG. 7. Note that each step performed in this learning method is the same as the CNN learning process in the automatic reading device 1 described above. Therefore, in the following explanation, specific examples will be omitted.

[0049] In the CNN training method performed in this automatic reading method, as shown in FIG. 7, first, in a training dataset storage step S11, number images extracted from the captured image 32 acquired in the image acquisition step S3 and correct labels indicating the visual readings of the numbers in the number images are stored as a set as a training dataset. Next, in a training image selection step S12, a predetermined number of number images are randomly selected from the training dataset stored in the training dataset storage step S11. Next, in a training image transformation step S13, the number images selected in the training image selection step S12 are randomly transformed by translation, scaling, rotation, partial filling, or other methods. Next, in a training digit recognition step S14, a feature map of the number images transformed in the training image transformation step S13 is extracted using a CNN, and digit recognition is performed on the number image feature map by template matching to calculate the similarity at each position. Next, in a CNN parameter update step S15, the CNN parameters are updated using backpropagation so that the similarity calculated in the training digit recognition step S14 is high at positions corresponding to the correct labels. In this automatic reading method, the learning data set stored in the learning data set storage step S11 is updated as appropriate, and the CNN parameters are optimized through learning by repeatedly performing the learning image selection step S12, the learning image conversion step S13, the learning digit recognition step S14, and the CNN parameter update step S15 until a factor for stopping learning occurs. As a result, in this automatic reading method, the reading accuracy of the mechanical meter M improves the more learning is repeated.

[0050] (Actions and Effects) The automatic reading device 1 according to the embodiment includes a number image extraction unit 17 that extracts each of the number display ranges representing each digit of the indicator reading as a target number image 33 based on the captured image 32 of the display panel D acquired by the image acquisition unit 12, and a number recognition unit 18 that uses CNN, a machine learning technique, to extract feature maps for each of the template image 31 stored in the template image storage unit 11 and the target number image 33 extracted by the number image extraction unit 17, and performs template matching to recognize each digit based on the similarity between the feature maps. Therefore, the automatic reading device 1 recognizes numbers based on the similarity between feature maps using CNN, which boasts high accuracy in image recognition tasks such as handwritten number recognition, rather than the similarity between the original images, thereby improving number recognition accuracy and enabling the reading of the indicator reading of a mechanical meter M with high accuracy. As a result, the automatic reading device 1 is an automatic reading device for mechanical meters that can automatically read the indicator reading of a mechanical meter M with reduced risk of misrecognition.

[0051] In this automatic reading device 1, the template matching performed by the number recognition unit 18 sets a comparison window W to select a feature map of the same size as the target number image 33 from the feature map of the template image 31, and calculates the similarity of one rotation between the feature map in the comparison window W and the feature map of the target number image 33 while shifting the position of the comparison window W placed on the feature map of the template image 31, and determines the number to be recognized from the position where the similarity is maximum. According to the automatic reading device 1, by setting the comparison window W and adjusting the comparison conditions, template matching between feature maps is possible.

[0052] In addition, in this automatic reading device 1, the binarization unit 13, maximum area extraction unit 14, contour extraction unit 15, and straight line detection unit 16 perform various processes to binarize the captured image 32, extract the area with the largest black area, extract the contour of the largest area, detect straight lines from the contour, and process the image to make it easier to detect and accurately estimate the positions of the numbers.Then, the number image extraction unit 17 cuts out the number image from the captured image 32 and extracts it as the target number image 33.The automatic reading device 1 makes it possible to cut out the number image by performing the processes of each functional unit in a stepwise manner.

[0053] Furthermore, according to this automatic reading device 1, the straight line detection unit 16 detects straight lines using a Hough transform, which has excellent figure detection accuracy, so that the position of the number image can be estimated with high accuracy, and the target number image 33 can be extracted in a way that makes it less likely to be recognized incorrectly.

[0054] Furthermore, according to this automatic reading device 1, when the number image extraction unit 17 cuts out the number image, it corrects distortion using projective transformation, so that the target number image 33 can be extracted in a way that makes it even less likely to be misrecognized.

[0055] The automatic reading device 1 also connects to a computer terminal via the input / output unit 3. The computer terminal stores a set of number images extracted from the captured image 32 and a correct answer label indicating the visual reading of the numbers in the number image as a learning dataset (training dataset storage process 21). The computer also performs a training image selection process 22, a training image conversion process 23, a training digit recognition process 24, and a CNN parameter update process 25. A predetermined number of number images are randomly selected from the training dataset, and the selected number images are randomly transformed by translation, scaling, rotation, partial filling, or other methods. The transformed number image feature map is subjected to digit recognition by template matching, and the similarity at each position is calculated. The digit recognition unit 18 recognizes numbers using CNN parameters updated by backpropagation so that the calculated similarity is high at the position corresponding to the correct answer label. The automatic reading device 1 randomly transforms the number images, allowing it to learn different images each time, reducing misrecognition errors for various number image patterns, including misaligned numbers. Furthermore, according to the automatic reading device 1, by using the CNN parameters optimized using the learning method in this way, it becomes possible to read the pointer value of the mechanical meter M with further reduced recognition errors.

[0056] The automatic reading method according to the embodiment executes steps similar to the processing by each functional unit that performs the reading function in the automatic reading device 1 and the learning processing of the CNN. Therefore, the automatic reading method according to the embodiment achieves the same effects as the effects of the automatic reading device 1. Therefore, the automatic reading method according to the embodiment is an automatic reading method for a mechanical meter that automatically reads the indicator value of a mechanical meter M while reducing the risk of erroneous recognition.

[0057] [Other forms] Although the present invention has been described based on the above embodiment, the present invention is not limited to the above embodiment and can be embodied in various forms without departing from the spirit of the present invention, and for example, the following modifications are also possible.

[0058] (1) The shapes, positions, and arrangements of the numbers and templates described in the above embodiments, the processing order in the processing unit, the conversion method, the extraction conditions, the processing contents, etc. are examples and can be changed within the scope that does not impair the effects of the present invention.

[0059] (2) In the above embodiment, the template matching is described using a diagram in which each digit of a number is recognized from the pinpoint maximum position of the similarity between feature maps, but the present invention is not limited to this. For example, a secondary process such as calculating an average value for each range may be performed, and each digit of a number may be recognized from the maximum value or the like.

[0060] (3) In the above embodiment, the position of the number is estimated by the processes of the binarization unit 13, the maximum area extraction unit 14, the contour extraction unit 15, and the straight line detection unit 16 so that the number image extraction unit 17 extracts the target number image 33. However, the present invention is not limited to this. For example, the position of the number may be estimated by adding a process by another functional unit in addition to the process by these functional units, or the position of the number may be estimated by a process by a different functional unit.

[0061] (4) In the above embodiment, the computer terminal connected to the automatic reading device 1 via the input / output unit 3 performs CNN learning, imports the learned data, updates the CNN parameters as needed, and uses the CNN parameters to cause the number recognition unit 18 of the automatic reading device 1 to recognize numbers. However, the present invention is not limited to this. For example, the CNN learning function performed by the computer terminal in the embodiment may be installed in the automatic reading device.

[0062] (5) In the above embodiment, the automatic reading device 1 was described as automatically reading the pointer value of a mechanical meter M with a rotating number wheel, which is often used in meter reading meters that display the cumulative flow rate of fluids, such as kerosene meters, gas meters, and water meters. However, the present invention is not limited to this. For example, the automatic reading device 1 may automatically read the pointer value of a mechanical meter that displays distance, such as an automobile odometer. [Explanation of symbols]

[0063] 1...automatic reading device, 2...imaging unit, 3...input / output unit, 4...processing unit, 4a...calculation unit, 4b...storage unit, 11...template image storage unit, 12...image acquisition unit, 13...binarization unit, 14...maximum area extraction unit, 15...contour extraction unit, 16...straight line detection unit, 17...digit image extraction unit, 18...digit recognition unit, 19...guideline value synthesis unit, 21...learning data set storage processing, 22...learning image selection processing, 23...learning image conversion processing, 24...learning digit recognition processing, 25...CNN parameter update processing, 31...template image, 32...captured image, 33 ...Target number image, D...display panel, M...mechanical meter, N...digit rotating wheel, S1...template image preparation process, S2...imaging process, S3...image acquisition process, S4...binarization process, S5...maximum area extraction process, S6...contour extraction process, S7...straight line detection process, S8...digit image extraction process, S9...digit recognition process, S10...guide value synthesis process, S11...learning dataset storage process, S12...learning image selection process, S13...learning image conversion process, S14...learning digit recognition process, S15...CNN parameter update process, W...comparison window

Claims

1. An automatic reading device for a mechanical meter that automatically reads the pointer value of a number rotating wheel type mechanical meter, a template image storage unit for storing a template image created from character shape data for one revolution of the number rotating wheel of the mechanical meter to be read; an imaging unit that captures an image of a display panel of the mechanical meter; an image acquisition unit that acquires a captured image of the display panel captured by the imaging unit; a number image extraction unit that extracts, as target number images, number display ranges that indicate the respective digits of the indicator value based on the captured image acquired by the image acquisition unit; a number recognition unit that uses a machine learning technique, CNN (Convolutional Neural Network), to extract feature maps of the template image and the target number image, and performs template matching to recognize numbers of each digit based on the similarity between the feature maps; a pointer value synthesis unit that synthesizes the numbers of each digit recognized by the number recognition unit and reads the number as the pointer value; An automatic reading device for a mechanical meter, comprising:

2. 2. The automatic reading device for a mechanical meter according to claim 1, The template matching performed by the number recognition unit sets a comparison window to select a feature map of the same size as the feature map of the target number image from the feature map of the template image, calculates the similarity between the feature map in the comparison window and the feature map of the target number image for one rotation while shifting the position of the comparison window placed on the feature map of the template image, and determines the number to be recognized from the position where the similarity is maximum.

3. 2. The automatic reading device for a mechanical meter according to claim 1, a binarization unit that binarizes the captured image acquired by the image acquisition unit using an adaptive threshold; a maximum area extraction unit that extracts an area with a maximum black area from the captured image binarized by the binarization unit; a contour extraction unit that extracts a contour of the largest area extracted by the largest area extraction unit; a straight line detection unit that detects straight lines from the contour extracted by the contour extraction unit; Further provided with The number image extraction unit estimates the position of the number from the straight line detected by the straight line detection unit, and cuts out the number image at the estimated position from the captured image acquired by the image acquisition unit and extracts it as the target number image for each digit. This is an automatic reading device for mechanical meters.

4. 4. The automatic reading device for a mechanical meter according to claim 3, The line detection unit detects the line using a Hough transform.

5. 4. The automatic reading device for a mechanical meter according to claim 3, The automatic reading device for mechanical meters, wherein the number image extraction unit corrects distortion using projective transformation when extracting the number image.

6. The automatic reading device for a mechanical meter according to any one of claims 1 to 5, The number recognition unit a learning dataset storage process for storing the number image cut out from the captured image acquired by the image acquisition unit and a correct answer label indicating a visual reading value of the number of the number image as a set as a learning dataset; a learning image selection process for randomly selecting a predetermined number of number images from the learning dataset stored in the learning dataset storage process; a learning image transformation process for randomly transforming the number images selected in the learning image selection process by translation, scaling, rotation, filling in a part of the number images, or other methods; A learning number recognition process that extracts a feature map of the number image converted by the learning image conversion process using the CNN, performs number recognition on the feature map of the number image by template matching, and calculates the similarity at each position; a CNN parameter updating process for updating CNN parameters by backpropagation so that the similarity calculated in the learning digit recognition process becomes high at the position corresponding to the correct label; The automatic reading device for mechanical meters performs template matching to recognize each digit of the number using the CNN parameters that are optimized by repeating the above steps.

7. 7. The automatic reading device for a mechanical meter according to claim 6, An automatic mechanical meter reading device for reading the pointer value of a mechanical meter configured as a kerosene meter that measures the flow rate of kerosene.

8. An automatic reading method for a mechanical meter for automatically reading the pointer value of a number rotating wheel type mechanical meter, comprising: a template image preparation step of preparing a template image created from character shape data for one circumference of the number wheel of the mechanical meter to be read; an imaging step of imaging a display panel of the mechanical meter; an image acquisition step of acquiring a captured image of the display panel captured in the imaging step; a number image extraction step of extracting, as target number images, number display ranges indicating each digit of the indicator value based on the captured image acquired in the image acquisition step; a number recognition process that uses a machine learning technique called CNN (Convolutional Neural Network) to extract feature maps of the template image and the target number image, and performs template matching to recognize each digit based on the similarity between the feature maps; a pointer value synthesis step of synthesizing the numbers of each digit recognized in the number recognition step and reading the synthesized number as the pointer value; A method for automatically reading a mechanical meter, comprising:

9. 9. The method for automatically reading a mechanical meter according to claim 8, The template matching performed in the number recognition process sets a comparison window for selecting a feature map of the same size as the feature map of the target number image from the feature map of the template image, calculates the similarity between the feature map in the comparison window and the feature map of the target number image for one rotation while shifting the position of the comparison window placed on the feature map of the template image, and determines the number to be recognized from the position where the similarity is maximum.

10. 9. The method for automatically reading a mechanical meter according to claim 8, a binarization step of binarizing the captured image acquired in the image acquisition step using an adaptive threshold; a maximum area extraction step of extracting an area with a maximum black area from the captured image binarized in the binarization step; a contour extraction step of extracting a contour of the largest region extracted in the largest region extraction step; a straight line detection step of detecting a straight line from the contour extracted in the contour extraction step; Further provided with In the number image extraction step, the position of the number is estimated from the straight line detected in the straight line detection step, and the number image is cut out and extracted as the target number image.

11. The method for automatically reading a mechanical meter according to claim 10, In the line detection step, the line is detected by a Hough transform.

12. The method for automatically reading a mechanical meter according to claim 10, In the number image extraction step, distortion is corrected using a projective transformation when the number image is extracted.

13. The method for automatically reading a mechanical meter according to any one of claims 8 to 12, a learning dataset storage step of storing the number image cut out from the captured image acquired in the image acquisition step and a correct answer label indicating a visual reading value of the number in the number image as a set as a learning dataset; a learning image selection step of randomly selecting a predetermined number of number images from the learning data set stored in the learning data set storage step; a learning image transformation step of randomly transforming the number images selected in the learning image selection step by translating, scaling, rotating, filling in a part of the number images, or by other methods; a learning number recognition step of extracting a feature map of the number image converted in the learning image conversion step using the CNN, performing number recognition on the feature map of the number image by template matching, and calculating the similarity at each position; a CNN parameter updating step of updating CNN parameters by backpropagation so that the similarity calculated in the learning digit recognition step becomes high at the position corresponding to the correct label; The method for automatically reading a mechanical meter further comprises:

14. The method for automatically reading a mechanical meter according to claim 13, A method for automatically reading a mechanical meter for reading the pointer value of a mechanical meter configured as a kerosene meter for measuring the flow rate of kerosene.

Images