Processing methods, computing devices, and storage media

Abstract

The processing method, the computing device, and the storage medium of the embodiment perform polar coordinate transformation on at least a part of an image of an instrument with a center of the instrument as a reference, the instrument having a pointer rotating around a rotation axis, a display disc, and a plurality of scales arranged circumferentially on the display disc around the rotation axis, and at each point in a first direction corresponding to the circumferential direction, calculate an integrated value of the luminance of at least a part of a second direction corresponding to a radial direction from the center toward the scale and a deviation of the luminance of at least a part of the second direction.

Description

[0001] This application is a divisional application of the invention patent application with the same title of "Reading system, reading method, program, and storage medium" filed in the People's Republic of China on March 20, 2019, with the application number of 201980028538.4 by the same applicant. TECHNICAL FIELD

[0002] Embodiments of the present application relate to a processing method, a computing device, and a storage medium. BACKGROUND

[0003] There is a system that reads a value indicated by a circular instrument. In this system, it is desirable to have high accuracy in reading the value.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2017-126187 SUMMARY

[0007] Technical Problem to be Solved by the Invention

[0008] The technical problem to be solved by the present application is to provide a processing method, a computing device, and a storage medium that can improve the accuracy of reading a value indicated by a circular instrument.

[0009] Means for Solving the Technical Problem

[0010] The processing method of an embodiment performs polar coordinate transformation on at least a portion of an image of an instrument taken with the center of the instrument as a reference, the instrument having a pointer that rotates with a rotation axis as a center, a display dial, and a plurality of scales arranged in a circumferential direction on the display dial around the rotation axis, and calculates, at each point in a first direction corresponding to the circumferential direction, an integrated value of luminance in at least a portion of a second direction corresponding to a radial direction from the center toward the scales and a deviation in luminance in at least a portion of the second direction.

[0011] The computing device of an embodiment executes the processing method described above.

[0012] The storage medium of an embodiment stores a program that causes a computer to execute the processing method described above.

[0013] A reading system of an embodiment has a calculation section and a determination section. The calculation section performs polar coordinate transformation on at least a part of an image of an instrument having a pointer that rotates around a rotation axis, a display dial, and a plurality of scales arranged in a circumferential direction on the display dial around the rotation axis, with the center of the instrument as a reference. The calculation section calculates, at each point in a first direction corresponding to the circumferential direction, an integrated value of luminance in at least a part of a second direction corresponding to a radial direction from the center toward the scales and a deviation of luminance in at least a part of the second direction. The determination section determines a position where the pointer is present using the integrated value and the deviation. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a block diagram showing the structure of a reading system of an embodiment.

[0015] Figure 2 is a diagram illustrating the processing of a reading system of an embodiment.

[0016] Figure 3 is a flowchart showing the operation of a reading system of an embodiment.

[0017] Figure 4 is a graph showing an example of a calculation result related to luminance.

[0018] Figure 5 is a flowchart of another operation of a reading system of an embodiment.

[0019] Figure 6 is a block diagram showing the structure of a reading system of a first modified example of an embodiment.

[0020] Figure 7 is a block diagram showing the structure of a reading system of a second modified example of an embodiment.

[0021] Figure 8 is a diagram illustrating the processing of a reading system of the second modified example of an embodiment.

[0022] Figure 9 is a block diagram showing the structure of a reading system of a third modified example of an embodiment.

[0023] Figure 10 is a diagram illustrating the processing of a reading system of the third modified example of an embodiment.

[0024] Figure 11 is a block diagram showing the structure of a reading system of a fourth modified example of an embodiment.

[0025] Figure 12 is a diagram illustrating the processing of a reading system of the fourth modified example of an embodiment.

[0026] Figure 13 is a block diagram showing a hardware configuration of a reading system for implementing the embodiments. DETAILED DESCRIPTION

[0027] Hereinafter, the embodiments of the present application will be described with reference to the drawings.

[0028] In the present application specification and the drawings, the same reference numerals are applied to the same elements that have been described, and detailed description is appropriately omitted.

[0029] Figure 1 is a block diagram showing a configuration of a reading system of the embodiments.

[0030] The reading system 1 of the embodiments is used for reading a value indicated by a pointer in a meter from an image of the meter taken. The reading system 1 of the embodiments is suitable for use in a meter having a pointer that rotates around a rotation axis and a plurality of scales arranged in a circumferential direction around the rotation axis. In the meter of a reading target, the plurality of scales can be arranged in a circular shape or in a circular arc shape. The meter includes a display disc on which the plurality of scales are marked and a value corresponding to at least a part of the plurality of scales. In such a meter, an outer edge, an outer frame, or the like of the display disc is typically circular. In the following description of the embodiments, such a meter is referred to as a circular meter.

[0031] As shown in Figure 1 , the reading system 1 of the embodiments is provided with an imaging section 11, a meter extraction section 12, a correction section 13, a calculation section 14, a determination section 15, a relative angle calculation section 16, a conversion section 17, and a storage section 19.

[0032] Figure 2 is a diagram illustrating a process of the reading system of the embodiments. With reference to Figure 2 , the process of the reading system 1 of the embodiments will be specifically described.

[0033] The imaging section 11 takes an image of a circular meter, and acquires a still image. The imaging section 11 outputs the acquired image to the meter extraction section 12. The imaging section 11 can also store the image in the storage section 19. When a moving image is taken by the imaging section 11, a still image is cut out from the moving image and output to the meter extraction section 12. In the taken image, an object other than the circular meter can also be captured.

[0034] The meter extraction section 12 extracts a portion in which a circular meter is taken from the image acquired by the imaging section 11. The meter extraction section 12 outputs the extracted portion to the correction section 13. Figure 2 (a) of

[0035] As a specific example, the instrument extraction section 12 has a contour extraction section 12a and a selection section 12b. For explanation, an image captured by the imaging section 11 and input to the instrument extraction section 12 is referred to as an input image.

[0036] The contour extraction section 12a extracts a contour of a region included in the input image, for example, based on a luminance difference in the input image. In extracting the contour, the contour extraction section 12a can also appropriately process the image. For example, the contour extraction section 12a performs binarization after converting the input image to a gray scale, and extracts a contour of a region represented by white from the binary image.

[0037] The selection section 12b calculates an area of a region surrounded by the contour. In extracting a plurality of contours, the selection section 12b calculates an area of each region. The selection section 12b compares each of the calculated areas with a prescribed threshold value, and selects only a region whose area is equal to or larger than the threshold value. Thus, a region having an excessively small area is excluded, and only a region in which a circular instrument is highly likely to be captured is selected. The selection section 12b extracts a portion (cut-out) corresponding to the selected region from the input image. The selection section 12b outputs the extracted portion to the correction section 13.

[0038] The correction section 13 corrects an image input thereto to an image viewed from the front when the input image is distorted. The correction section 13 corrects the image, for example, by performing a projective transformation. In the example of the image shown in (a) of FIG. 10, the outer edge of the display dial is elliptical. By the correction performed by the correction section 13, the image shown in (a) of FIG. 10 is corrected to the image shown in (b) of FIG. 10. Thus, the outer edge of the display dial approaches a perfect circle. The correction section 13 outputs the corrected image to the calculation section 14. Figure 2 Figure 2 Figure 2 The calculation section 14 has, for example, a polar coordinate conversion section 14a, a first calculation section 14b, and a second calculation section 14c.

[0039] The polar coordinate conversion section 14a detects a center of a circular instrument included in the converted image.

[0040] The polar coordinate conversion section 14a detects a center of a circular instrument included in the converted image.

[0041] The calculation section 14 detects the outer edge of the display dial of the circular instrument, for example, based on a luminance difference in the image. When the outer edge of the display dial is circular, the calculation section 14 approximates the outer edge to a perfect circle. The calculation section 14 takes the center of the perfect circle as the center of the circular instrument.

[0042] When the outer edge of the display dial is not circular, the calculation section 14 can also detect the center of the circular instrument using a plurality of scales marked on the display dial. Specifically, when the scales are straight lines, the calculation section 14 generates virtual straight lines along the respective scales. The calculation section 14 detects intersection points of the straight lines, and takes a position of a barycenter of the plurality of intersection points as the center of the circular instrument. ​​

[0043] The polar coordinate transformation unit 14a performs a polar coordinate transformation on the image based on the center of the detected circular instrument. For example, the following equations (1) and (2) are used in the polar coordinate transformation. The image may also be appropriately grayscaled or binarized before the polar coordinate transformation. For example, for... Figure 2 The image shown in (b) is transformed into polar coordinates to generate Figure 2 The image shown in (d) is as follows. Figure 2 The circumferential (angle) θ of the circular instrument shown in (c) is related to... Figure 2 The X direction (first direction) shown in (d) corresponds to the direction of the circular instrument. The radial direction R corresponds to the Y direction (second direction) which is orthogonal to the X direction.

[0044] X=Rcosθ (1)

[0045] Y = Rsinθ (2)

[0046] Angles in a circular instrument are calculated using a defined line as a reference. For example, the polar coordinate transformation unit 14a generates a virtual half-straight line extending horizontally to the right from the center of the circular instrument. When calculating the angle of a point in the image, the polar coordinate transformation unit 14a generates another half-straight line connecting that point to the vertex (the center of the circular instrument). The polar coordinate transformation unit 14a calculates the angle between the reference half-straight line and the other half-straight line as the angle of that point.

[0047] The first calculation unit 14b uses the result of polar coordinate transformation to calculate the total brightness in the Y direction of each point in the X direction. The second calculation unit 14c calculates the deviation of the brightness in the Y direction of each point in the X direction. The first calculation unit 14b and the second calculation unit 14c output the calculated results to the determination unit 15.

[0048] When pixel brightness is represented by 256 gray levels from 0 to 255, the total brightness of all pixels arranged in the Y direction is summed, and this value is used as the total brightness. The same applies when the image is represented in binary. For all pixels arranged in the Y direction, the values ​​representing brightness, either "0" or "1", are summed, and this value is used as the total brightness. A larger total value indicates that more bright pixels are arranged in the Y direction.

[0049] Brightness deviation can be calculated, for example, as follows: First, calculate the average brightness in the Y direction. Then, using this average value, calculate the standard deviation of the brightness in the Y direction. Set this standard deviation as the brightness deviation. Alternatively, the variance, root mean square, contrast ratio, or interquartile range (IQR) of the brightness in the Y direction can also be used as the brightness deviation.

[0050] The contrast ratio CR is expressed, for example, by the following equation (3). max It is the maximum brightness in the Y direction. min It is the minimum brightness in the Y direction.

[0051] CR=(I max -I min ) / (I max +I min (3)

[0052] For example, targeting Figure 2 For the image shown in (d), calculate the total brightness and brightness deviation at each point in the X direction. Figure 2 (e) represents the result of its calculation. Figure 2 In the curve graph of (e), the circles represent the total brightness value, and the quadrilaterals represent the standard deviation (brightness deviation). The horizontal axis represents the position (i.e., angle) in the X direction. The left vertical axis represents the total brightness value. The right vertical axis represents the brightness deviation. Figure 2 For convenience, the angles corresponding to each position are recorded on the horizontal axis of (e).

[0053] The determination unit 15 uses the total brightness value and the brightness deviation to determine the position of the pointer of the circular instrument in the X direction. In the image after polar coordinate transformation, the position in the X direction represents the angle with the center of the circular instrument as the vertex in the image before polar coordinate transformation. That is, by determining the position of the pointer in the X direction, the angle of the pointer is obtained.

[0054] As a specific example, the determination unit 15 includes an extraction unit 15a and a discrimination unit 15b. In a circular instrument that is the object of reading, the case where the color of the display panel is bright and the color of the pointer is darker will be explained.

[0055] Extraction unit 15a uses the calculation results from calculation unit 14 to extract the position with minimal brightness in the X direction from the image after polar coordinate transformation. Multiple positions can be extracted. The extracted positions are candidates for the position where the pointer is located. This is because when the pointer's color is darker than the display disk's color, the total brightness value in the Y direction is lower at the pointer's location than at other locations.

[0056] The discrimination unit 15b refers to the brightness deviation at each extracted location. For example, the discrimination unit 15b compares the brightness deviation of each location with a first threshold. The discrimination unit 15b determines the location where the pointer exists from the extracted locations where the deviation is below the first threshold.

[0057] The candidates extracted by the extraction unit 15a sometimes include multiple positions. These positions include, for example, the position of one end of the pointer and the position of the other end of the pointer. One end and the other end are the ends in the direction in which the pointer extends. One end of the pointer is the part that indicates the area where the scale is arranged, and the other end of the pointer is located at the opposite position across the rotation axis. One end is located closer to the area where the scale is arranged compared to the other end. The distance between one end and the rotation axis is longer than the distance between the other end and the rotation axis. Hereinafter, the aforementioned one end of the pointer will be referred to as the "front end", and the aforementioned other end will be referred to as the "rear end".

[0058] In a circular instrument, the rear end of the pointer protrudes to the opposite side of the front end compared to the center of the display dial. In this case, the positions of the pointer's front end and rear end can be extracted as candidates for the pointer's location.

[0059] However, in this case, the distance between the tip of the pointer and the center is also greater than the distance between the back end and the center. Therefore, at the position where the tip of the pointer is located, there are fewer pixels of color on the display in the Y direction compared to the position where the back end of the pointer is located. That is, at the position where the tip of the pointer is located, there are more low-brightness pixels and fewer high-brightness pixels compared to the position where the back end of the pointer is located. As a result, the brightness deviation in the Y direction is smaller at the position where the tip of the pointer is located compared to the position where the back end of the pointer is located. Therefore, by using the brightness deviation, the position of the pointer can be determined.

[0060] exist Figure 2 In the example shown in (d), the extraction unit 15a extracts multiple locations P0 to P9 with extremely low brightness. At locations P4 and P9, the total brightness value is significantly smaller than at other locations. This indicates that the front end of the pointer exists at one of locations P4 and P9, and the back end of the pointer exists at the other. Based on the extracted multiple locations, the discrimination unit 15b determines that location P4, where the brightness deviation is below a first threshold th1, is a location where a pointer exists.

[0061] In the image after polar coordinate transformation, the position in the X direction is as follows: Figure 3 As shown on the horizontal axis of (d), it represents the angle with the center of the circular instrument as its vertex. Therefore, it can be known that the pointer exists at the angle corresponding to the determined position.

[0062] The storage unit 19 stores information related to the scale of the circular instrument. Specifically, it stores a table showing the correspondence between the pointer angle and the numerical value indicated by the pointer, as well as the maximum and minimum values ​​of the values ​​appended to the scale. The conversion unit 17 converts the pointer angle into a numerical value based on the information in the storage unit 19. Thus, the numerical value indicated by the pointer is obtained. The conversion unit 17 outputs the converted numerical value to an external device, for example.

[0063] The table of the storage section 19 indicates a correspondence relation between an angle of a pointer in a state where the circular meter is arranged at a certain position as a reference and a value. For example, there is a case where the circular meter photographed by the imaging section 11 is rotated by a certain angle with respect to the reference position. In this case, if the angle at which the pointer is present is converted into a value with reference to the table, an error in the amount of the angle occurs.

[0064] Therefore, the angle acquired by the determination section 15 is preferably corrected by the relative angle calculation section 16 with respect to the above angle. The relative angle calculation section 16 calculates a relative rotation angle (relative angle) between the reference position of the circular meter and the photographed circular meter.

[0065] For example, in the storage section 19, the position relation of the scale corresponding to the minimum value and the scale corresponding to the maximum value is stored with respect to the circular meter of the read target. In a general circular meter, the scales are continuously marked in a certain direction from the scale of the minimum value toward the scale of the maximum value. Also, in the opposite direction thereof, the scale of the minimum value is separated from the scale of the maximum value. In the storage section 19, the direction in which the scale of the minimum value is separated from the scale of the maximum value and the distance between these scales is stored as information of the position relation. The relative angle calculation section 16 detects the position relation between the scale of the minimum value and the scale of the maximum value from the corrected image. The relative angle calculation section 16 compares the detected position relation with the position relation stored in the storage section 19 and calculates the relative angle.

[0066] Alternatively, the relative angle calculation section 16 can calculate the relative angle from the corrected image. For example, the relative angle calculation section 16 recognizes a plurality of values marked by the display disc using a technique such as OCR (Optical Character Recognition), pattern matching, or the like. Next, the relative angle calculation section 16 detects the slopes of the recognized plurality of values and takes the average of the slopes as the slope of the circular meter (relative angle).

[0067] The relative angle calculation section 16 adds or subtracts the calculated rotation angle to or from the angle acquired by the determination section 15. The conversion section 17 converts the angle calculated by the relative angle calculation section 16 into a value. Thereby, the reading accuracy of the value is improved.

[0068] Figure 4 is a flowchart indicating the action of the reading system of the embodiment.

[0069] First, the imaging section 11 images the circular meter, and acquires an image (step Sll). Next, the outline extraction section 12a extracts an outline in the image (step S12a). The selection section 12b selects an outline of a prescribed area or more (step S12b). The correction section 13 corrects distortion of the image corresponding to the selected outline (step S13). The polar coordinate conversion section 14a performs polar coordinate conversion on the corrected image (step S14a). The first calculation section 14b calculates a total value of luminance in the Y direction for each point in the X direction (step S14b). The second calculation section 14c calculates a deviation of luminance in the Y direction for each point in the X direction (step S14c).

[0070] The extraction section 15a extracts a position in the X direction in which the total value of luminance is a minimum (step S15a). The discrimination section 15b compares the deviation of luminance with a first threshold value for each of the extracted positions. Thus, positions in which the deviation is the first threshold value or less are discriminated (step S15b). The relative angle calculation section 16 calculates a relative angle between the imaged circular meter and a state of the circular meter stored in the storage section 19 (step S16). The conversion section 17 converts an angle corresponding to the discriminated position to a numerical value using the calculated relative angle (step S17).

[0071] The order of the above-described steps can be changed as appropriate. For example, the calculation of the deviation of luminance (step S14c) can be performed after the extraction of the position (step S15a). The calculation of the relative angle (step S16) can be performed before the polar coordinate conversion of the image (step S14a) or before the extraction of the position (step S15a). When the relative angle is calculated based on slopes of a plurality of numerical values, the calculation of the slopes can also be performed before the polar coordinate conversion of the image (step S14a).

[0072] Next, the effects of the embodiment will be described.

[0073] First, the reading system of the reference example will be described. In the reading system of the reference example, an image of the imaged circular meter is subjected to polar coordinate conversion. A total value of luminance in the Y direction is calculated for each point in the X direction. Based on the total value, a pointer is detected. According to this method, even in a case where the image is not clear, the pointer can be detected with high precision. Even in a case where a part of the pointer cannot be seen because a seal or the like is attached to the circular meter, the pointer can be detected with high precision.

[0074] However, as described above, in the detection based on the total value of luminance, it is sometimes difficult to discriminate between the leading end and the trailing end of the pointer. When the rotation range of the pointer is 180 degrees or more, if the leading end and the trailing end of the pointer cannot be discriminated, the numerical value indicated by the pointer cannot be read.

[0075] Therefore, in the reading system 1 of the embodiment, the deviation of the luminance is further used to determine the position where the pointer exists. As described above, the deviation of the luminance is different between the position where the front end of the pointer exists and the position where the rear end of the pointer exists. By using the deviation of the luminance, the front end of the pointer can be determined with high accuracy.

[0076] Therefore, according to the reading system 1 of the embodiment, the reading accuracy of the indicated value can be improved, particularly for a circular meter in which the rotation range of the pointer is 180 degrees or more.

[0077] Depending on the shape of the pointer, the resolution of the captured image, and the like, dark pixels can sometimes be included more in the rotation axis portion or the scale portion of the pointer. In this case, the calculation section 14 can also exclude these portions and calculate the total value of the luminance or the deviation of the luminance in the Y direction.

[0078] For example, the first calculation section 14b excludes at least either the portion including the rotation axis of the pointer or the portion including the scale from the image subjected to the polar coordinate transformation and calculates the total value of the luminance in the Y direction. The second calculation section 14c excludes at least either the portion including the rotation axis of the pointer or the portion including the scale from the image subjected to the polar coordinate transformation and calculates the deviation of the luminance in the Y direction.

[0079] Alternatively, the polar coordinate transformation section 14a can exclude at least either the portion including the rotation axis of the pointer or the portion including the scale from the image and perform the polar coordinate transformation on the image. However, the excluded portion is rectangular in the image subjected to the polar coordinate transformation, and thus the processing can be speeded up. In order to speed up the processing, it is preferable to exclude the rotation axis portion and the scale portion in the calculation in the above-described first calculation section 14b and second calculation section 14c.

[0080] A seal or the like is sometimes attached to the front surface of the circular meter. When calculating the total value of the luminance and the deviation of the luminance, if the luminance of the portion where the seal is attached is included in the calculation, the position of the pointer can be erroneously detected. In particular, when the seal is black or red or the like, the reflectance of which is low, the possibility of erroneous detection is high. Therefore, when the seal is attached to the circular meter, the calculation section 14 can exclude the portion where the seal is attached from the image and calculate the total value of the luminance or the deviation of the luminance in the Y direction.

[0081] For example, the polar coordinate transformation section 14a excludes the portion where the seal is attached and performs the polar coordinate transformation on the image. The portion where the seal is attached is stored in the storage section 19 in advance. The first calculation section 14b and the second calculation section 14c calculate the total value of the luminance and the deviation of the luminance from the image subjected to the polar coordinate transformation in which the seal is excluded.

[0082] Alternatively, the first calculation section 14b and the second calculation section 14c can also exclude the portion including the seal from the polar coordinate-transformed image, and calculate the total value of the brightness and the deviation of the brightness. However, in order to speed up the processing, it is preferable to exclude the seal and perform the polar coordinate transformation on the image. Typically, the outer shape of such a seal is rectangular. This is because, if the portion including the seal is polar coordinate-transformed, the portion becomes non-rectangular, and the processing becomes complicated.

[0083] In the above-described example, the case where the color tone of the display disc is darker than the color tone of the bright, the pointer, and the scale was described, but the circular meter that the reading system 1 of the embodiment can read is not limited to this example. It is also possible that, in the circular meter of the reading target, the color tone of the display disc is brighter than the color tone of the pointer and the scale. In this case, in the polar coordinate-transformed image, the number of bright pixels increases at the position where the pointer exists, compared to other positions. Therefore, the extraction section 15a extracts the position where the total value of the brightness in the Y direction is maximum. The discrimination section 15b extracts the position where the pointer exists from the extracted position, using the deviation of the brightness in the Y direction.

[0084] The discrimination section 15b can also discriminate the position where the pointer exists by the following processing when a plurality of positions are extracted by the extraction section 15a. The discrimination section 15b investigates whether or not the plurality of positions include two positions corresponding to the diagonal of the meter. The two positions corresponding to the diagonal of the meter refer to positions where the difference between the angle corresponding to one position and the angle corresponding to the other position is substantially 180 degrees. As for the difference between the angles, an allowable difference can also be set based on the performance of the imaging section 11, the resolution of the image, and the like. When the two positions corresponding to the diagonal exist, it is indicated that the front end and the rear end of the pointer exist at these positions. The discrimination section 15b compares the deviation of the brightness at these positions, and discriminates that the pointer exists at the side where the deviation is smaller.

[0085] In the above-described description, the case where the determination section 15 has the extraction section 15a and the discrimination section 15b was described. The processing in the determination section 15 is not limited to this example. For example, the determination section 15 can also perform the extraction of the position where the total value of the brightness is the extremum and the extraction of the position where the deviation of the brightness is equal to or smaller than the first threshold value in parallel. The determination section 15 determines the position that satisfies both conditions as the position where the pointer exists. Alternatively, the determination section 15 can also extract the range of the position where the deviation of the brightness is equal to or smaller than the first threshold value, and discriminate the position where the total value of the brightness is the extremum within the range. That is, if the position where the total value of the brightness is the minimum value or the maximum value and the deviation of the brightness is equal to or smaller than the first threshold value can be found, the specific processing in the determination section 15 can be appropriately changed.

[0086] In the above description, an example of a reading system 1 including an image capture unit 11, an instrument extraction unit 12, and a correction unit 13 has been described. However, the reading system 1 may also lack these components. For example, if an image whose distortion has been corrected is input into the reading system 1, these components are not required.

[0087] Preferred embodiments and variations of the reading system 1 will be described. The various embodiments, including variations, described below can be appropriately combined to implement the system.

[0088] Figure 4 (a) and Figure 4 (b) is a graph representing an example of the calculation results related to brightness.

[0089] Figure 4 (a) and Figure 2 The chart in (b) and Figure 4 The chart corresponding to (d). Figure 2 (a) targeting Figure 4 The image shown in (c) illustrates the relationship between the position in the X direction and the sum of the black intensity in the Y direction.

[0090] The total intensity of black corresponds to the total luminance. The total intensity of black is calculated as follows: First, calculate the total of the maximum luminance values ​​in the Y direction. For example, when luminance is represented as 255 grayscale, 255 is the maximum luminance value. Multiplying 255 by the number of pixels in the Y direction yields the total of the maximum luminance values. Next, calculate the total luminance values ​​in the Y direction. Subtracting the total luminance value from the total of the maximum values ​​gives the total intensity of black.

[0091] like Figure 4 As shown in (a), a second threshold th2 is preferably set for the total value of black intensity. First, the extraction unit 15a extracts the ranges Ra1 and Ra2 in the X direction where the total value of black intensity exceeds the second threshold th2. Next, the extraction unit 15a extracts the positions in the X direction where the total value of black intensity is maximized within the ranges Ra1 and Ra2. According to this method, positions with maximized total values ​​can be filtered out, reducing the computational load.

[0092] If the extraction unit 15a extracts multiple locations, it may be impossible for the discrimination unit 15b to distinguish only one location. For example, if... Figure 4 As shown in (b), when the first threshold th1 is set, the brightness deviation at positions P0, P2 to P8 is below the first threshold th1. Therefore, it is impossible for the discrimination unit 15b to determine only one position.

[0093] However, by screening the extracted positions, only one position is easily discriminated by the discrimination section 15b. For example, as shown in (a) of Fig. 10, only the ranges Ra1 and Ra2 are searched, and thus only the positions P4 and P9 are extracted. As a result, even when the first threshold value thl shown in (b) of Fig. 10 is used, only the position P4 can be discriminated. Figure 4 Figure 2

[0094] Therefore, according to the above method, the reading accuracy of the numerical value can be further improved.

[0095] The first threshold value thl and the second threshold value th2 can be set by the user in advance, or can be set based on the calculation result of the sum of the intensities of black and the deviation of the luminance. For example, the extraction section 15a sets 70% of the maximum value of the sum of the luminance as the second threshold value th2. The discrimination section 15b sets 70% of the maximum value of the deviation of the luminance as the first threshold value thl. By setting the threshold values based on the calculation result, the position where the pointer exists can be discriminated with higher accuracy than when the threshold values are fixed in advance. As a result, the reading accuracy of the numerical value can be further improved.

[0096] Here, a case where the sum of the intensities of black is set as the first threshold value thl by converting the calculated sum of the luminance into the sum of the intensities of black is described. The first threshold value can be set as the calculation result of the sum of the luminance as shown in (e) of Fig. 9. In this case, the first threshold value is set to, for example, a value of 30% of the maximum sum of the luminance. Figure 5

[0097] In the above example, the position where the sum of the intensities of black is extremely large is extracted using the sum of the intensities of black corresponding to the sum of the luminance. In this way, the candidates of the position of the pointer can also be made using other numerical values corresponding to the sum of the luminance. Such a method can also be considered as substantially extracting the position where the sum of the luminance is extremely small.

[0098] Figure 5 is a flowchart showing other actions of the reading system 1 according to the embodiment.

[0099] Figure 5 shows the actions of the reading system 1 when the sum of the luminance is compared with the second threshold value. In the flowchart of Figure 3 , the processes other than steps S15al and S15a2 are the same as those of the flowchart of Figure 6 .

[0100] ​​​The extraction section 15a extracts a range in the X direction in which the integrated value of the luminance is equal to or lower than the second threshold value by comparing the integrated value of the luminance at each point in the X direction with the second threshold value (step S15al). The extraction section 15a extracts a position in which the integrated value of the luminance is the smallest in the extracted range (step S15a2). The discrimination section 15b discriminates a position in which the deviation of the luminance is equal to or lower than the first threshold value from the extracted position (step S15b).

[0101] (First Modification)

[0102] Figure 7 is a block diagram showing the structure of a reading system according to a first modification of the embodiment. In the reading system la, the processing in the calculation section 14 is different from that in the reading system 1.

[0103] The polar coordinate conversion section 14a extracts only pixels having low luminance when the corrected image is input, and performs polar coordinate conversion on the extracted pixels. For example, when the luminance is expressed in 255 gradations, only pixels having luminance equal to or lower than 127 gradations are subjected to polar coordinate conversion. When the luminance is expressed in binary, only black pixels are subjected to polar coordinate conversion.

[0104] The first calculation section 14b calculates the integrated value of the luminance in the Y direction at each point in the X direction on the basis of the result of the polar coordinate conversion. In the case of the present modification, only the luminance of the pixels subjected to the polar coordinate conversion is used for the calculation.

[0105] The extraction section 15a extracts a position in the X direction in which the integrated value of the luminance is the smallest. The extraction section 15a outputs the extraction result to the second calculation section 14c. The second calculation section 14c calculates the deviation of the luminance at the extracted position with respect to the result of the polar coordinate conversion.

[0106] In the present modification, only pixels having low luminance are subjected to polar coordinate conversion, and therefore information on the luminance exists only in a part of the Y direction. Therefore, the second calculation section 14c calculates the deviation of the luminance with respect to a point in which information on the luminance is absent, for example, as if the luminance were the largest (white). The second calculation section 14c outputs the calculation result to the discrimination section 15b.

[0107] The discrimination section 15b discriminates a position in which the deviation of the luminance is the smallest among the positions extracted by the extraction section 15a as a position in which the pointer exists.

[0108] According to the reading system la of the present modification, only pixels having low luminance are subjected to polar coordinate conversion. The deviation of the luminance is calculated with respect to only a specific position on the basis of the extraction result of the extraction section 15a. Therefore, the amount of calculation can be reduced, and the reading speed of the numerical value can be improved as compared with the reading system 1.

[0109] (Second Modification)

[0110] Figure 8 is a block diagram showing the structure of a reading system of a second modification example of the embodiment.

[0111] Figure 8 is a graph illustrating the processing of the reading system of the second modification example of the embodiment.

[0112] Figure 8 (a) of is a front view of a circular meter. As shown in (a) of Figure 8 , one circular meter 90 sometimes has two pointers 91 and 92. In the example of (a) of Figure 8 , the left side of the display disc indicates pressure and the right side indicates temperature. That is, the range indicated by the pointer 91 and the range indicated by the pointer 92 are separated from each other and are independent respectively. The reading system lb of the second modification example reads the values indicated by the respective pointers with respect to such a circular meter.

[0113] The processing of the imaging section 11, the meter extraction section 12, the correction section 13, and the calculation section 14 in the reading system lb is the same as that of the reading system 1. For example, by performing polar coordinate conversion on the image of the circular meter having two pointers, an image shown in (b) of Figure 8 is generated.

[0114] The first calculation section 14b and the second calculation section 14c calculate the total value of luminance and the deviation of luminance with respect to the image after the polar coordinate conversion respectively. Figure 8 (c) of shows the calculation results of the total value of luminance and the deviation of luminance in the image shown in (b) of Figure 8 . In the graph of (c) of Figure 8 , the plotting of the circle represents the total value of luminance and the plotting of the quadrangle represents the standard deviation (the deviation of luminance). The horizontal axis represents the position in the X direction (i.e., the angle). The vertical axis on the left side represents the total value of luminance. The vertical axis on the right side represents the deviation of luminance. In the horizontal axis of (c) of Figure 8 , the angle corresponding to each position is written for convenience.

[0115] The extraction section 15a extracts the position in the X direction in which the total value of luminance in the Y direction is the minimum, based on the calculation result of the first calculation section 14b. For example, as shown in (c) of Figure 8 , the positions P0 to P9 are extracted.

[0116] For example, in the storage section 19, with respect to the circular meter 90, the angle range in which the pointers 91 and 92 exist respectively is stored. In the example shown in (a) to Figure 8 (c) of Figure 8 , the pointer 91 exists in the range of 0 degrees or more and less than 90 degrees and 270 degrees or more and less than 360 degrees with the half straight line L as a reference. The pointer 92 exists in the range of 90 degrees or more and less than 270 degrees.

[0117] The discrimination unit 15b, referring to the information stored in the storage unit 19, determines, within the range of each pointer, a position where the brightness deviation is lower than the first threshold th1. Figure 9 In the example shown in (c), position P0 is determined within the range where pointer 91 exists, and position P2 is determined within the range where pointer 92 exists. That is, pointer 91 is determined to exist at the angle corresponding to position P0, and pointer 92 is determined to exist at the angle corresponding to position P2.

[0118] The subsequent processing for each identified position (angle) is the same as that of reading system 1. The relative angle calculation unit 16 detects the rotation angle of the circular instrument in the image and appropriately adds or subtracts the relative angle from each angle. The transformation unit 17 transforms the corrected angles into numerical values.

[0119] exist Figure 10 The example described uses a circular instrument with two pointers. The reading system 1b of this embodiment can also be applied to circular instruments with three or more pointers. In this case, the storage unit 19 stores the ranges where each of the three or more pointers exists. The determination unit 15 determines the position of the pointer within each range.

[0120] According to this variation, for a circular instrument with multiple pointers, the values ​​indicated by each pointer can be read with higher precision.

[0121] (Third variation)

[0122] Figure 10 This is a block diagram illustrating the structure of the reading system in the third variation of the implementation method.

[0123] Figure 10 This is a diagram illustrating the processing of a reading system according to a third variation of the implementation method.

[0124] Figure 10 (a) is the front view of the circular instrument. For example... Figure 10 As shown in (a), sometimes a single circular instrument 90 has multiple pointers, all of which have the same indicating range and are different colors. Figure 10 In example (a), the circular instrument 90 has a red pointer RD, a green pointer GR, and a blue pointer BL.

[0125] For example, the circular instrument 90 is a thermometer. Pointer BL indicates the current temperature. Pointer GR indicates the highest temperature recorded during the measurement. Pointer RD indicates the temperature at which an alarm will be triggered. An alarm is triggered when the value indicated by pointer BL exceeds the value indicated by pointer RD.

[0126] In the reading system 1c of the third modification example, the calculation section 14 further has a pointer division section 14d. The pointer division section 14d extracts pixels having a color in a certain color range when the image is input from the correction section 13. The color range is set to include the color of the pointer. The color of the pixel not extracted is set to white, for example. By performing this processing on the color of each pointer, a color space or a color difference plane can be generated for each pointer.

[0127] The color space is a space constituted by components of RGB (Red Green Blue) or components of HSV (Hue Saturation Value). The color difference plane is a plane obtained by performing subtraction operation among R plane, G plane, and B plane with respect to the RGB color space, for example. In this case, there are six kinds of color difference planes generated. By assigning the color difference plane having the highest sensitivity to each pointer of the reading object, the angle of each pointer is determined.

[0128] For example, by the processing of the pointer division section 14d, as shown in (b) to (d) of FIG. 9, an image of the circular meter 90 having only the pointer BL, an image of the circular meter 90 having only the pointer GR, and an image of the circular meter 90 having only the pointer RD are generated. Figure 11 Figure 12

[0129] When the color space or the color difference plane of each pointer is generated, the pointer division section 14d can also perform HSV conversion or the like on the input image. By performing the HSV conversion, the color is determined from the hue (Hue), the brightness (Value) of the region having the hue of red is extracted, and thus an image data having only red can be made. By performing the HSV processing on the image according to the color of each pointer, the detection accuracy of the pointer can be improved.

[0130] When the image of each pointer is generated by the pointer division section 14d, the same processing as the reading system 1, 1a, or 1b is performed on each image. According to the present modification example, even in the case where the circular meter has a plurality of pointers and the indication ranges of the plurality of pointers are the same, the indicated value can be read for each pointer.

[0131] (Fourth Modification Example)

[0132] Figure 12 is a block diagram showing the structure of the reading system of the fourth modification example of the embodiment.

[0133] Figure 12 is a diagram illustrating the processing of the reading system of the fourth modification example of the embodiment.

[0134] ​​The reading system 1d in the fourth variation includes an identification unit 18 instead of the relative angle calculation unit 16 and the storage unit 19. The identification unit 18 identifies the relationship between the scale and the numerical value of the circular instrument based on the image corrected by the correction unit 13. Specifically, the identification unit 18 has a dividing part 18a, a scale identification part 18b, a number identification part 18c, and a scale engagement part 18d.

[0135] For example, input to the dividing section 18a Figure 12 Image A of the circular instrument shown in (a). The segmentation unit 18a extracts line components from the image and extracts the most circle-like components from it using a Hough transform. As... Figure 12 As shown in (b), the dividing unit 18a designates the area around the outer periphery of the extracted circle as the scale area B1. Typically, the background of the display dial is a relatively bright color (e.g., white), while the pointer and numerals are a relatively dark color (e.g., black). Furthermore, the pointer is positioned closer to the center of the display dial than the scale and numerals. Using these points, the dividing unit 18a extracts the numeral area B2 from the area outside the scale area B1. Thus, the scale and numerals are divided from the display dial.

[0136] The scale recognition unit 18b, for example, determines the scale based on the brightness difference in the scale area B1, such as... Figure 12 As shown in (b), the scale C of the display panel is identified. The scale recognition unit 18b can also determine whether the image A has been properly corrected based on the identified scale C. For example, the scale recognition unit 18b performs a polar coordinate transformation on the scales C arranged in the circumferential direction to align them in one direction. Then, the scale recognition unit 18b calculates at least one of the deviation in the spacing between the scales C, the deviation in the width of the scales C, and the deviation in the shape of the scales C. The scale recognition unit 18b compares the calculated deviation with a preset threshold.

[0137] When the deviation is less than the threshold, the process for reading the circular instrument continues. By performing the above determination, only the display area B with smaller distortion can be read, thus improving the reading accuracy of the value indicated by the circular instrument.

[0138] When the deviation exceeds a threshold, the scale recognition unit 18b outputs the determination result to the correction unit 13. When the determination result is input, the correction unit 13 changes the conditions used to correct the image and generates another corrected image. For this other image, the processing based on the segmentation unit 18a is performed again.

[0139] For example, the generation of other images by the correction unit 13 is repeated until the aforementioned deviation amount is less than the threshold. Alternatively, even if multiple other images are generated, sometimes any one of the deviation amounts is not less than the threshold. In this case, the recognition unit 18 processes the image A that has obtained a very small deviation amount.

[0140] In the scale recognition section 18b, when the amount of deviation is determined to be smaller than the threshold value, the numeral recognition section 18c cuts out numerals from the numeral region B2. For example, as shown in (c) of FIG. 12, the numeral recognition section 18c cuts out a rectangle D containing numerals from the numeral region B2. The numeral recognition section 18c recognizes the numerals contained in the rectangle D. Figure 13

[0141] As shown in (c) of FIG. 12, the scale joint section 18d extracts an angle between a straight line E obtained by extending each scale C and a half straight line L as position information of each scale. The scale joint section 18d determines numerals corresponding to the recognized scales C. Through the above processing, for the photographed circular meter, the position of the scale and the correspondence between the scale and the numerical value are recognized. Figure 13

[0142] The conversion section 17 uses the information obtained by the recognition section 18 to convert the angle into a numerical value when converting the position where the pointer exists, which is output from the determination section 15, into an angle.

[0143] According to the present modified example, even when the position of the scale and the relationship between the position of the scale and the numerical value are not stored in the storage section 19, it is possible to read the numerical value indicated by the pointer. In the present modified example, the image used when recognizing the scale and the numerical value is the same as the image used when calculating the angle of the pointer. Therefore, the relative angle calculation section 16 is not needed.

[0144] ​ is a block diagram showing a hardware structure of a reading system for realizing the embodiment.

[0145] For example, the reading system of the embodiment is constituted by the reading device 5 and the photographing device 6 shown in FIG. 1. The reading device 5 is, for example, a computer, and has a ROM (Read Only Memory) 51, a RAM (Random Access Memory) 52, a CPU (Central Processing Unit) 53, and an HDD (Hard Disk Drive) 54. ​ The ROM 51 stores a program for controlling the operation of the computer. In the ROM 51, a program necessary for the computer to function as the meter extraction section 12, the correction section 13, the calculation section 14, the determination section 15, the relative angle calculation section 16, the conversion section 17, the recognition section 18, and the like in the above-described embodiment is stored.

[0146]

[0147] ​​​The RAM 52 serves as a storage area for expansion of a program stored in the ROM 51. The CPU 53 reads a control program stored in the ROM 51 and controls the operation of the computer according to the control program. The CPU 53 expands various data obtained through the operation of the computer in the RAM 52. The HDD 54 stores information required for reading and information obtained during reading.

[0148] The reading device 5 can also have an eMMC (embedded Multi Media Card), an SSD (Solid State Drive), an SSHD (Solid State Drive), or the like instead of the HDD 54.

[0149] The imaging device 6 captures an object (a circular gauge) and transmits the obtained image to the reading device 5. The imaging device 6 is, for example, a camera.

[0150] The output device 7 outputs data (an indicated value of the circular gauge read) output from the reading device 5 in a manner recognizable by a user. The output device 7 is, for example, a monitor, a printer, a speaker, or the like.

[0151] The reading device 5, the imaging device 6, and the output device 7 are connected to each other by wire or wirelessly, for example. Alternatively, they can be connected to each other via a network. Alternatively, at least two of the reading device 5, the imaging device 6, and the output device 7 can be assembled in one device. For example, the reading device 5 can be assembled integrally with an image processing section or the like of the imaging device 6.

[0152] By using the reading system and the reading method of the above-described embodiments, it is possible to read a value indicated in a circular gauge with higher accuracy. Similarly, by using a program for causing a computer to operate as a reading system, it is possible to cause the computer to read a value indicated by a circular gauge with higher accuracy.

[0153] The above illustrates several embodiments of the present application, but these embodiments are suggested as examples and are not intended to limit the scope of the application. These new embodiments can be implemented in various other ways, and various omissions, substitutions, changes, and the like can be made within the scope of the gist of the application. These embodiments and their modified examples are included in the scope and gist of the application, and are included in the scope of the application and its equivalents recited in the claims. In addition, the above-described embodiments can be implemented in combination with each other.

Claims

1. A processing method, At least a portion of the captured image of the instrument is transformed into polar coordinates with respect to the center of the instrument, which has a pointer that rotates about a rotation axis, a display dial, and a plurality of scales arranged circumferentially around the rotation axis on the display dial. At each point in a first direction corresponding to the circumference, calculate the sum of the brightness values ​​of at least a portion of the brightness values ​​in a second direction corresponding to the radial direction from the center toward the scale, and the deviation of the brightness values ​​in at least a portion of the second direction. Extract the position in the first direction where the total value is an extreme value, and use the deviation to determine the position where the pointer exists based on the extracted position.

2. The processing method according to claim 1, wherein, The color of the display dial is brighter than the color of the pointer. At the extracted locations, the total value is extremely small.

3. The processing method according to claim 1, wherein, The color of the display dial is darker than the color of the pointer. At the extracted location, the total value is extremely large.

4. The processing method according to any one of claims 1 to 3, wherein, The deviation at the extracted position is compared with a first threshold, and it is determined that the pointer exists at a position where the deviation is below the first threshold.

5. The processing method according to claim 4, wherein, The first threshold is set based on the calculation result of the deviation.

6. The processing method according to any one of claims 1 to 3, wherein, The deviations at each of the extracted positions are compared with each other, and it is determined that the pointer exists at the position with the smallest deviation.

7. The processing method according to any one of claims 1 to 3, wherein, If two of the extracted positions exist at positions diagonally opposite to the instrument, the deviations at the two positions are compared, and it is determined that the pointer is located at the position with the smaller deviation.

8. The processing method according to any one of claims 1 to 3, wherein, The location is extracted from the range where the total value at each point in the first direction exceeds the second threshold by comparing the total value with the second threshold.

9. The processing method according to claim 8, wherein, The second threshold is set based on the calculation result of the total value.

10. The processing method according to any one of claims 1 to 3, wherein, The instrument has multiple pointers, and when the indication ranges of the multiple pointers are separated from each other, the position of each pointer is determined based on the extracted position within the multiple indication ranges.

11. The processing method according to any one of claims 1 to 3, wherein, When the instrument has multiple pointers and the colors of the multiple pointers are different from each other, a color space or color difference plane is generated for each pointer. In each color space or color difference plane, the position of the pointer is determined according to the extracted position.

12. The processing method according to any one of claims 1 to 3, wherein, Calculate the angle corresponding to the identified position, and convert the calculated angle into a numerical value.

13. The processing method according to any one of claims 1 to 3, wherein, Identify the position of the scale in the instrument and the relationship between the scale and the numerical value. Using the identification result, convert the angle corresponding to the identified position into a numerical value.

14. The processing method according to any one of claims 1 to 3, wherein, The pointer can rotate more than 180 degrees.

15. The processing method according to claim 1, wherein, The pointer is determined to be located in the first direction where the total value is an extreme value and the deviation is below a first threshold.

16. A computing device that performs the processing method according to any one of claims 1 to 15.

17. A storage medium storing a program that causes a computer to perform the processing method according to any one of claims 1 to 15.

Images