IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND IMAGE PROCESSING PROGRAM

Abstract

To provide an image processing device, image processing method and image processing program which allow a display needle and a meter board surface to be clearly discriminated by employing a new determination method that is not affected by a peripheral light environment and thereby improve the accuracy of position determination of the display needle.SOLUTION: An image processing device comprises: a calculation unit 3 which takes an average value of brightness of the whole pixels in a needle movable region on the basis of image data extracted by an image extraction unit 2 and calculates a relative value for each pixel from a difference between the average value and brightness of each pixel in the needle movable region; and a needle value calculation unit 4 which determines the position of the display needle in the needle movable region by sectioning the relative value of each pixel in the needle movable region calculated by the calculation unit 3 with a predetermined set value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and an image processing program that can improve the recognition accuracy of an object in both a dark environment and a bright environment in image recognition processing in which an image of an indicator needle such as a meter needle is photographed as an object. [Background technology]

[0002] As a technique for recognizing images of meter needles and the like, image processing devices disclosed in Patent Documents 1 and 2 are known. Specifically, the automatic reading device for analog meters shown in Patent Document 1 has a measurement means for photographing the analog meter to obtain a measurement image indicating the indicated value of the display needle, a needle rotation angle calculation means for determining the rotation angle of the display needle based on polar coordinates, and an indication value calculation means for determining the indicated value of the display needle shown by the measurement image from the rotation angle and a reference image that serves as a master image.

[0003] Furthermore, in the data collection system for control panels shown in Patent Document 2, a digital camera for photographing the control panel of an old-fashioned machine tool is retrofitted to the panel, and the digital camera photographs the control panel while shining light from a lamp on it. The captured image is then analyzed and processed to convert it into data, thereby obtaining the current value of the display needle. Furthermore, in the image analysis processing unit of Patent Document 2, the digital image of the control panel taken with a digital camera is compared with a pre-registered reference image, the monitoring area is cut out from the digital image, and appropriate correction processing is performed on the image of each monitoring area, thereby recognizing the position indicated by the control panel. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2006-120133 A [Patent Document 2] JP 2017-069963 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the image analysis processing disclosed in Patent Documents 1 and 2, the value indicated by the indicator needle of an analog meter is recognized by comparing a reference image captured in advance with an image captured during measurement. In this situation, the image analysis process disclosed in Patent Document 2 in particular is adapted to recognize the position indicated by the display hand in the area illuminated by the light of the lamp. However, in the above-mentioned image analysis process, the conditions of the lamp light illuminating the indicator needle are often affected by the surrounding environment. For example, when there is little surrounding light, such as at night, the difference in color between the indicator needle and the meter face is difficult to discern, resulting in the problem that the indicator needle and the meter face cannot be clearly distinguished. Similarly, in Patent Document 1, which does not use lamp light, there is a problem in that the display hands cannot be recognized in a photographed image due to poor surrounding lighting conditions, and a new solution has been anticipated.

[0006] The present invention has been made in consideration of the above-mentioned circumstances, and provides an image processing device, an image processing method, and an image processing program that employ a new determination method that is not affected by the surrounding light environment, thereby making it possible to clearly distinguish between the display needle and the background meter face, thereby improving the accuracy of determining the position of the display needle. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention proposes the following means. The image processing device shown in the first aspect of the present invention is characterized by having an image capturing unit that captures an image of a display dial and a display needle that is provided on the display dial and moves within a predetermined range to obtain image data of a needle value recognition target area, an image extraction unit that cuts out image data of a needle movement area corresponding to the movable range of the display needle from the image data captured by the image capturing unit, a calculation unit that takes an average value of the brightness of all pixels within the needle movement area based on the image data extracted by the image extraction unit and calculates a relative value for each pixel from the difference between the average value and the brightness of each pixel within the needle movement area, and a needle value calculation unit that determines the position of the display needle within the needle movement area by dividing the relative values ​​of each pixel within the needle movement area calculated by the calculation unit by a predetermined set value.

[0008] The image processing method shown in the second aspect of the present invention is characterized by having an image capturing step of capturing an image of a display dial and a display hand that is provided on the display dial and moves within a predetermined range to obtain image data of a needle value recognition target area; an image extraction step of extracting image data of a needle movement area corresponding to the movable range of the display hand from the image data captured in the image capturing step; a calculation step of taking an average value of the brightness of all pixels within the needle movement area based on the image data extracted in the image extraction step, and calculating a relative value for each pixel from the difference between the average value and the brightness of each pixel within the needle movement area; and a needle value calculation step of determining the position of the display hand within the needle movement area by dividing the relative values ​​of each pixel within the needle movement area calculated in the calculation step by a predetermined set value.

[0009] The image processing program shown in the third aspect of the present invention is characterized by having an image capturing step of capturing an image of a display dial and a display hand that is provided on the display dial and moves within a predetermined range to obtain image data of a needle value recognition target area; an image extraction step of extracting image data of a needle movement area corresponding to the movable range of the display hand from the image data captured in the image capturing step; a calculation step of taking an average value of the brightness of all pixels within the needle movement area based on the image data extracted in the image extraction step, and calculating a relative value for each pixel from the difference between the average value and the brightness of each pixel within the needle movement area; and a needle value calculation step of determining the position of the display hand within the needle movement area by dividing the relative values ​​of each pixel within the needle movement area calculated in the calculation step by a predetermined set value. Effect of the Invention

[0010] In the present invention, by adopting a new determination method based on relative values ​​that is not affected by the surrounding light environment, it is possible to clearly distinguish between the display hand and its background, thereby improving the accuracy of determining the position of the display hand. [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic diagram showing a minimum configuration of an image processing device according to the present invention; [Diagram 2] 1 is a schematic configuration diagram of an image processing apparatus according to an embodiment of the present invention; [Diagram 3] 13 is a flowchart showing a master image setting process executed by an initial setting unit of the first processing unit. [Figure 4] 10 is a flowchart showing a needle value recognition process executed by a needle value recognition section of a second processing section. [Diagram 5] 13 is a flowchart for normalizing the needle movement area of ​​the display hand. [Figure 6] 11A to 11C are explanatory diagrams showing a process for defining a needle movable area. [Figure 7] 11 is an explanatory diagram showing a process for specifying a hand movement area of ​​the display hand. FIG. [Figure 8]FIG. 13 is a diagram showing a relative value distribution for each pixel. [Figure 9] (A) A diagram showing how to identify the display needle when the meter image is generally dark, (B) A diagram showing how to identify the display needle when the meter image is generally bright, and (C) A diagram showing the case where the position of the display needle is clarified within the needle movable area. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The minimum configuration of an image processing device 100 according to the present invention will be described with reference to FIG. This image processing device 100 mainly comprises an image capturing unit 1, an image extracting unit 2, a calculation unit 3, and a needle value calculating unit 4.

[0013] The image capturing unit 1 captures an image of the display panel and a display hand that is provided on the display panel and moves within a predetermined range, and obtains image data of the needle value recognition target area.

[0014] The image extraction unit 2 cuts out image data of the needle movement area of ​​the display needle from the image data of the needle value recognition target area captured by the image capture unit 1.

[0015] Based on the image data extracted by the image extraction unit 2, the calculation unit 3 calculates the average brightness value of all pixels within the needle movement area, and calculates the relative value for each pixel from the difference between the average brightness value and the brightness of each pixel within the needle movement area. For example, the calculation unit 3 calculates a deviation value, which is a relative value for each pixel, from the difference between the average brightness of all pixels in the needle movable area and the brightness of each pixel in the needle movable area based on the image data.

[0016] The needle value calculation unit 4 determines the position of the display needle within the needle movement area by dividing the relative values ​​of each pixel within the needle movement area calculated by the calculation unit 3 by a predetermined set value.

[0017] In the image processing device 100 configured as above, first, the image capturing unit 1 captures an image of the display dial and a display hand that is provided on the display dial and moves within a predetermined range, and obtains image data thereof. Then, the image extraction unit 2 extracts image data within the needle movement area of ​​the display hand from the image data captured by the image capture unit 1, and then the calculation unit 3 calculates the average brightness of all pixels within the needle movement area extracted by the image extraction unit 2, and calculates a relative value for each pixel from the difference between the average brightness and the brightness of each pixel within the needle movement area. Thereafter, the needle value calculation unit 4 determines the position of the display needle within the needle movement area by dividing the relative values ​​of each pixel within the needle movement area calculated by the calculation unit 3 by a predetermined set value.

[0018] As a result, in the image processing device 100, by utilizing the relative value of each pixel which represents the difference between the brightness of each pixel and the brightness of the entire needle movement area and indicates the relative brightness within the needle movement area, it becomes possible to reliably separate the display needle from its background, and regardless of whether the image is generally dark or generally bright, it is always possible to separate the needle from the background using a constant threshold value. In other words, the image processing device 100 of the present invention employs a new determination method based on relative values ​​that is not affected by the surrounding light environment, making it possible to clearly distinguish between the display hand and the background, i.e., the meter face, thereby improving the accuracy of determining the position of the display hand.

[0019] (Embodiment) Next, an image processing device 101 according to an embodiment of the present invention will be described with reference to FIGS. As shown in FIG. 2, this image processing device 101 has a first processing unit 20A that predefines a master image for determining a needle value recognition target area (see FIG. 6) provided on the display panel of the meter, and a second processing unit 20B that recognizes the current position of the display needle on the display panel of the meter within the needle value recognition target area defined by the master image.

[0020] The first processing unit 20A has a meter image capturing unit 21 that captures an image of the display panel 30 of the meter M (see Figure 6 (A)), an initial setting unit 22 that determines a needle movable area A2 of the display needle 31 from the needle value recognition target area A1 of the meter image captured by the meter image capturing unit 21, and a master image storage unit 23 that stores the needle movable area A2 determined by the initial setting unit 22 as a master image. The master image setting process executed by the initial setting section 22 of the first processing section 20A will be described later with reference to FIG.

[0021] The second processing unit 20B includes a meter image photographing unit 11, a needle movable range extracting unit 12, a brightness normalizing unit 13, and a needle value calculating unit 14 as main components.

[0022] The meter image photographing unit 11 photographs the display panel 30 and the display needle 31 (see FIG. 6(A)) that is provided on the display panel 30 and moves within a predetermined movable range in response to an input signal, and obtains image data of the needle value recognition target area A1.

[0023] The needle movable range extraction unit 12 extracts image data of a needle movable area A2 corresponding to the movable range of the display needle 31 from the image data taken into the meter image photographing unit 11.

[0024] Based on the image data extracted by the needle movement range extraction unit 12, the brightness normalization unit 13 calculates the average value of the brightness (in this example, the brightness expressed in 255 levels) of all pixels within the needle movement area A2, and calculates the relative value for each pixel from the difference between the average value and the brightness of each pixel within the needle movement area A2. For example, the brightness normalization unit 13 calculates a deviation value, which is a relative value for each pixel, from the difference between the average brightness of all pixels within the needle movable area A2 and the brightness of each pixel within the needle movable area A2 based on the image data.

[0025] The needle value calculation unit 14 determines the position of the display needle 31 within the needle movable area A2 by dividing the relative values ​​of each pixel within the needle movable area A2 calculated by the brightness normalization unit 13 by a predetermined set value. In the second processing unit 20B, the needle value recognition unit 15 is configured by the needle movement range extraction unit 12, the brightness normalization unit 13, and the needle value calculation unit 14. The processing executed by this needle value recognition unit 15 will be described later with reference to Figs. 4 and 5.

[0026] Next, with reference to FIG. 3, the master image setting process executed by the initial setting unit 22 of the first processing unit 20A will be described for each step S. In this flowchart, a needle movable area A2, which is the movable range of the display needle 31, is determined by executing the angle detection process (steps S2 to S4) multiple times for the operating meter.

[0027] [Step S1] It is determined whether or not the process of detecting the angle of the display hand 31 of the meter M has been completed a plurality of times, and if YES, the process proceeds to the next step S5, and if NO, the process proceeds to the next step S2.

[0028] [Step S2] After the angle of the indicator needle 31 is detected from the image captured by the meter image capturing unit 21, the process proceeds to the next step S3.

[0029] [Step S3] If the detected angle of the display hand 31 detected in step S2 is the smallest among the measured angles, the detected angle is determined to be the "minimum angle" and registered (see Figures 6(A) and (B)), and then the process proceeds to the next step S4.

[0030] [Step S4] If the detected angle of the display hand 31 detected in step S2 is the largest among the measured angles, the detected angle is determined to be the "maximum angle" and registered (see Figures 6(A) and (C)), and then the process proceeds to step S4.

[0031] [Step S5] When the image reading process is completed (YES in step S1), a needle movement area A2, which is the range of movement of the display needle 31, is determined based on the "minimum angle" registered in step S3 and the "maximum angle" registered in step S4, and the needle movement area A2 is registered as a master image G (see Figure 7), after which this flowchart ends.

[0032] Next, the needle value recognition process executed by the needle value recognition section 15 of the second processing section 20B will be described for each step S with reference to FIG.

[0033] [Steps S10 to S11] First, the needle movable area A2 of the display hand 31 obtained in step S5 is used as a master image G (step S10), and then the needle movable area A2 is clarified by normalization (step S11). Note that the normalization process of step S11 is executed simultaneously with the processing of steps S2 to S4 in Fig. 3. Also, the normalization process of step S11 will be described later with reference to Fig. 5.

[0034] [Step S12] The needle movement area A2 of the display needle 31 existing within the needle value recognition target area A1 is identified from the image data showing the needle value recognition target area A1 of the display panel 30 photographed by the meter image photographing unit 11 and the master image G showing the needle movement area A2 acquired in step S5 of FIG. 3 (see FIG. 7). That is, in this step S12, a process is performed to extract the needle movable area A2 of the display needle 31 from the needle value recognition target area A1 of the display dial 30 photographed by the meter image photographing unit 11, using the master image G showing the needle movable area A2 acquired in step S5 of Figure 3.

[0035] [Step S13] The hand movement area A2 of the display hand 31 extracted in step S12 is normalized. The normalization process in step S13 will be described later with reference to FIG.

[0036] [Step S14] The relative value for each pixel representing the display hand 31 calculated by normalization in step S13 is compared with a preset value.

[0037] [Step S15] The relative value representing the display hand 31 calculated by normalization in step S13 is divided by a predetermined set value to determine the position of the display hand 31 within the hand movable area A2, and then this flowchart ends.

[0038] For example, in step S15, the position of the display hand 31 within the hand movable area A2 is clarified by dividing the deviation value indicating the relative value for each pixel representing the display hand 31 by a predetermined setting value (in this example, the setting value = 50) as shown in Figure 8. As a result, in step S15, even if the meter M is an overall dark image as shown in FIG. 9(A) or an overall bright image as shown in FIG. 9(B), by adopting the discrimination method shown in FIG. 8, it is possible to clarify the position of the display needle 31 within the needle movable area A2 as shown in FIG. 9(C).

[0039] Next, the normalization process shown in steps S11 and S13 in FIG. 4 will be described with reference to FIG.

[0040] [Step S20] After taking the average brightness value of all pixels within the needle movement area A2 of the display needle 31, the process proceeds to the next step 21.

[0041] [Step S21] For each pixel (section shown in FIG. 8) that represents a predetermined pixel area within the needle movable area A2, the difference from the average value calculated in step S20 is calculated.

[0042] [Step S22] This flowchart is then terminated after calculating a deviation value (see Figure 8) which is a relative value for each pixel from the difference between the average brightness of all pixels within the needle movable area A2 and the brightness of each pixel area within the needle movable area A2.

[0043] In the image processing device 101 configured as described above, the meter image capturing unit 11 captures an image of the display panel 30 and the display needle 31 that is provided on the display panel 30 and moves within a predetermined movable range in response to an input signal, and obtains image data thereof. Thereafter, the needle movable range extracting section 12 extracts image data of a needle movable area A2 corresponding to the movable range of the display needle 31 from the image data taken into the meter image photographing section 11. Thereafter, the brightness normalization unit 13 calculates the average brightness of all pixels within the needle movable area A2 based on the image data extracted by the needle movable range extraction unit 12, and calculates a relative value for each pixel from the difference between the average brightness and the brightness of each pixel within the needle movable area A2 (step S13 in Figure 4) (see Figure 8). Thereafter, the needle value calculation unit 14 determines the position of the display needle 31 within the needle movable area A2 by dividing the relative values ​​of each pixel within the needle movable area A2 calculated by the brightness normalization unit 13 by a predetermined set value (steps S14 to S15 in Figure 4) (see Figure 8).

[0044] As a result, in the image processing device 101, by utilizing the relative value of each pixel which represents the difference between the brightness of each pixel and the brightness of the entire needle movement area A2 and indicates the relative brightness within the needle movement area A2, it becomes possible to reliably separate the display needle 31 from its background (see Figure 8), and regardless of whether the image is generally dark or generally bright, it becomes possible to always separate the needle from the background using a constant threshold value (see Figure 9). In other words, in the image processing device 101 of this embodiment, by adopting a new determination method based on relative values ​​that is not affected by the surrounding light environment, it is possible to clearly distinguish between the display needle 31 and the background such as the meter display panel 30, thereby improving the accuracy of determining the position of the display needle 31.

[0045] In addition to using the deviation value as the above relative value, a relative value for each pixel may be calculated from the difference between the average brightness of all pixels within the needle movable area A2 and the brightness of each pixel within the needle movable area A2 by using an appropriately determined calculation formula.

[0046] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design modifications and the like are also included within the scope of the present invention that does not deviate from the gist of the present invention. [Industrial Applicability]

[0047] The present invention relates to an image processing device, an image processing method, and an image processing program capable of improving the recognition accuracy of an object in a dark environment or a bright environment in image recognition processing in which an image of an indicator needle such as a meter needle is taken as an object. [Explanation of symbols]

[0048] 1. Image capture unit 2 Extraction part 3 Arithmetic section 4 Needle value calculation section 11 Meter image capture unit 12 Needle movement range cutout 13 Brightness normalization section 14 Needle value calculation section 15 Needle value recognition unit 20A First Processing Section 20B Second processing section 21 Meter image capture unit 22 Initial setting section 23 Master image storage unit 30 Display panel 31 Display hand 100 Image processing device 101 Image processing device A1 needle value recognition area A2 Needle movable area G Master Image M Meter

Claims

1. an image capturing unit that captures an image of the display board and a display needle that is provided on the display board and moves within a predetermined range to obtain image data of a needle value recognition target area; an image extraction unit that extracts image data of a needle movable area corresponding to the movable range of the display hand from the image data captured by the image capture unit; a calculation unit which calculates an average value of brightness of all pixels within the needle movable area based on the image data extracted by the image extraction unit, and calculates a relative value for each pixel from the difference between the average value and the brightness of each pixel within the needle movable area; and a needle value calculation unit that determines the position of the display needle within the needle movable area by dividing the relative values ​​of each pixel within the needle movable area calculated by the calculation unit by a predetermined set value.

2. 2. The image processing apparatus according to claim 1, wherein the relative value calculated by the calculation unit is a deviation value.

3. The image processing device according to claim 1 or claim 2, characterized in that the image extraction unit pre-stores a master image showing the needle movable area, and extracts an image corresponding to the needle movable area from the image data captured by the image capture unit based on this master image.

4. 4. The image processing device according to claim 3, wherein the image extraction unit detects the positions of the minimum and maximum angles of the display hand based on image data captured by the image capture unit as the master image, and recognizes in advance that a hand movable area exists between the minimum and maximum angles of the display hand.

5. 5. The image processing device according to claim 1, wherein the calculation section detects brightness of pixels based on luminance of the image data extracted by the image extraction section.

6. an image capturing step of capturing an image of a display dial and a display needle disposed on the display dial and moving within a predetermined range to obtain image data of a needle value recognition target area; an image extraction step of extracting image data of a needle movement area corresponding to the movable range of the display needle from the image data captured in the image capturing step; a calculation step of calculating an average value of brightness of all pixels in the needle movable area based on the image data extracted in the image extraction step, and calculating a relative value for each pixel from the difference between the average value and the brightness of each pixel in the needle movable area; and a needle value calculation step of determining the position of the display needle within the needle movable area by dividing the relative values ​​of each pixel within the needle movable area calculated in this calculation step by a predetermined set value.

7. an image capturing step of capturing an image of a display dial and a display needle disposed on the display dial and moving within a predetermined range to obtain image data of a needle value recognition target area; an image extraction step of extracting image data of a needle movement area corresponding to the movable range of the display needle from the image data captured in the image capturing step; a calculation step of calculating an average value of brightness of all pixels in the needle movable area based on the image data extracted in the image extraction step, and calculating a relative value for each pixel from the difference between the average value and the brightness of each pixel in the needle movable area; and a needle value calculation step of determining the position of the display needle within the needle movable area by dividing the relative values ​​of each pixel within the needle movable area calculated in this calculation step by a predetermined set value.

Images