Float meter reading program, float meter reading device, and float meter reading method

Abstract

To provide a float meter reading program, float meter reading device and float meter reading method capable of automatically reading and inputting values of a float meter.SOLUTION: A float meter reading program includes: extracting first partial image data including a float meter from verification image data including the float meter having a float vertically moving in the inside; extracting second partial image data including the float from the extracted first partial image data; specifying a first position and a second position in the vertical direction of a first part and a second part of the float respectively from the extracted second partial image data; specifying an upper end position in the vertical direction of the float from the specified first position and second position; and using a relational expression between the respective positions in the vertical direction of the float meter and respective scale values in the float meter to calculate a scale value in the float meter corresponding to the specified upper end position.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present invention relates to a float meter reading program, a float meter reading device, and a float meter reading method. [Background technology]

[0002] For example, factories that manufacture semiconductors or the like generally have many analog meters installed to measure the temperature, pressure, etc. of each device. Specifically, in such factories, for example, float meters are installed to measure the flow rate of liquids or gases flowing through each device.

[0003] Then, for example, a worker patrols the factory at regular intervals and checks the values ​​indicated by each analog meter. After that, the worker performs necessary calculations using the values ​​of each analog meter, for example, and determines whether or not an abnormality (e.g., equipment failure, etc.) has occurred in the factory. As a result, if it is determined that an abnormality has occurred in the factory, the worker makes arrangements, for example, to replace the equipment in which the abnormality has occurred (see Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 7-06552 [Patent Document 2] China Patent Application Publication No. 106289426 Summary of the Invention [Problem to be solved by the invention]

[0005] Here, the reading of the analog meter as described above is generally performed by an operator's visual inspection, and therefore there is a possibility that the reading of the analog meter as described above may result in errors in reading the value or in inputting the value into a management system (not shown).

[0006] In particular, reading a float meter, which has non-uniform intervals between each graduation, is usually more difficult than reading other analog meters, and so reading a float meter is more prone to errors in reading or inputting values ​​than reading other analog meters.

[0007] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide a float meter reading program, a float meter reading device, and a float meter reading method that enable the reading and input of float meter values ​​to be performed automatically. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the float meter reading program of the present invention is characterized in that it causes a computer to execute the following processes: extracting first partial image data including the float meter from verification image data which includes a float meter having a float which moves vertically inside the float; extracting second partial image data including the float from the extracted first partial image data; identifying first and second positions in the vertical direction for the first and second parts included in the float, respectively, from the extracted second partial image data; identifying the top end position of the float in the vertical direction from the identified first and second positions; and calculating the scale value of the float meter corresponding to the identified top end position by using a relational equation between each position in the vertical direction of the float meter and each scale value of the float meter. Effect of the Invention

[0009] According to the float meter reading program, the float meter reading device, and the float meter reading method of the present invention, it is possible to automatically read and input the value of the float meter. [Brief description of the drawings]

[0010] [Figure 1]FIG. 1 is a diagram showing an example of a configuration of an information processing device 1 according to the first embodiment. [Diagram 2] FIG. 2 is a diagram for explaining a specific example of the floatometer 10. As shown in FIG. [Diagram 3] FIG. 3 is a diagram for explaining an outline of the float meter reading process in the first embodiment. [Figure 4] FIG. 4 is a diagram for explaining an outline of the float meter reading process in the first embodiment. [Diagram 5] FIG. 5 is a flow chart illustrating details of the float meter reading process in the first embodiment. [Figure 6] FIG. 6 is a flow chart illustrating details of the float meter reading process in the first embodiment. [Figure 7] FIG. 7 is a flowchart illustrating details of the float meter reading process in the first embodiment. [Figure 8] FIG. 8 is a diagram for explaining details of the float meter reading process in the first embodiment. [Figure 9] FIG. 9 is a diagram for explaining details of the float meter reading process in the first embodiment. [Figure 10] FIG. 10 is a diagram for explaining details of the float meter reading process in the first embodiment. [Figure 11] FIG. 11 is a diagram for explaining details of the float meter reading process in the first embodiment. [Figure 12] FIG. 12 is a diagram for explaining details of the float meter reading process in the first embodiment. [Figure 13] FIG. 13 is a diagram for explaining details of the float meter reading process in the first embodiment. [Figure 14] FIG. 14 is a diagram for explaining details of the float meter reading process in the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the technical scope of the present invention is not limited to these embodiments.

[0012] First, a configuration example of an information processing device 1 (hereinafter also referred to as a float meter reading device 1) in the first embodiment will be described. Fig. 1 is a diagram showing a configuration example of the information processing device 1 in the first embodiment.

[0013] The information processing device 1 is a computer device, for example, a general-purpose PC (Personal Computer). The information processing device 1 performs a process of automatically reading a value indicated by a float meter (not shown) (hereinafter, also referred to as a float meter reading process).

[0014] 1, the information processing device 1 has a hardware configuration of a general-purpose computer device, and includes, for example, a CPU 101 which is a processor, a memory 102, a communication interface 103, and a storage medium 104. Each unit is connected to each other via a bus 105.

[0015] The storage medium 104 has, for example, a program storage area (not shown) for storing a program (not shown) for performing a float meter reading process.

[0016] The storage medium 104 also includes a storage unit 110 (hereinafter also referred to as a storage area 110) that stores information used when performing a float meter reading process. The storage medium 104 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0017] The CPU 101 executes a program loaded from the storage medium 104 into the memory 102 to perform a float meter reading process.

[0018] The communication interface 103 communicates with an operation terminal 2 such as a smartphone via a network NW such as the Internet.

[0019] [Example of a float meter] Next, a specific example of the floatmeter 10 will be described. FIG.

[0020] 2, the float meter 10 has, for example, a cylindrical glass tube 11 extending vertically, and a float 12 that moves vertically inside the glass tube 11. Note that the intervals between the scale marks 11a added to the side surface of the glass tube 11 increase in width from the top to the bottom in the vertical direction, as shown in FIG.

[0021] Then, the operator measures the flow rate of fluid such as gas or liquid that has flowed into the glass tube 11, for example, by reading the value (hereinafter also referred to as the scale value) of the scale 11a corresponding to the position 12a (hereinafter also referred to as the upper end position 12a) corresponding to the upper edge of the float 12.

[0022] 2, the floatmeter 10 has four circular first reference areas 13a, 13b, 13c, and 13d (hereinafter, these are also collectively referred to as first reference areas 13), and two rectangular second reference areas 14a and 14b (hereinafter, these are also collectively referred to as second reference areas 14). Each of the first reference areas 13 is, for example, a circular sticker (e.g., a blue sticker) attached to the floatmeter 10, and each of the second reference areas 14 is, for example, a rectangular sticker (e.g., a yellow sticker) attached to the floatmeter 10. The uses of the first reference areas 13 and the second reference areas 14 will be described later.

[0023] In the following description, the reading of the scale 11a is performed based on the top end position 12a of the float 12, but the reading of the scale 11a may also be performed based on a position other than the top end position 12a of the float 12.

[0024] [Outline of the first embodiment] Fig. 3 and Fig. 4 are diagrams for explaining an outline of the float meter reading process in the first embodiment. Specifically, Fig. 3 is a diagram for explaining the learning stage process in the float meter reading process in the first embodiment. Also, Fig. 4 is a diagram for explaining the inference stage process in the float meter reading process in the first embodiment.

[0025] First, the learning stage process of the float meter reading process in the first embodiment will be described.

[0026] As shown in FIG. 3, the first data generation unit 111 of the information processing device 1 generates learning data (hereinafter also referred to as first learning data) by adding information about the position of the floatmeter 10 in the first learning image data (hereinafter also referred to as position information) to the learning image data (hereinafter also referred to as first learning image data) showing the floatmeter 10.

[0027] Specifically, the first data generation unit 111 generates the first learning data by adding position information of the float meter 10 in the first learning image data to the first learning image data that includes at least the scale 11a and the four first reference areas 13, for example.

[0028] Note that the first data generating unit 111 may generate further first learning data by changing the size, color, or the like of the already-generated first learning data.

[0029] Then, the first model generation unit 112 of the information processing device 1 generates a learning model (hereinafter also referred to as a first learning model) by, for example, performing machine learning using the multiple first learning data generated by the first data generation unit 111. After that, the first model generation unit 112 stores the generated first learning model in the storage area 110.

[0030] That is, the first model generation unit 112 uses multiple first learning data to generate a first learning model to be used when extracting partial image data (hereinafter also referred to as first partial image data) showing the float meter 10 from the detection image in the inference stage.

[0031] In addition, the second data generation unit 113 of the information processing device 1 generates learning data (hereinafter also referred to as second learning data) by adding position information of the float 12 in the second learning image data to learning image data (hereinafter also referred to as second learning image data) that shows the float 12 moving inside the float meter 10.

[0032] Specifically, the second data generation unit 113 generates second learning data by adding position information of the float 12 in the second learning image data that shows, for example, the float 12 and an area vertically above the upper end position 12a of the float 12 (for example, an area whose vertical length is the same as the length of the float 12).

[0033] Note that the second data generation unit 113 may generate further second learning data by changing the size, color, or the like of the second learning data that has already been generated.

[0034] Then, the second model generation unit 114 of the information processing device 1 generates a learning model (hereinafter also referred to as a second learning model) by, for example, performing machine learning using the multiple pieces of second learning data generated by the second data generation unit 113. After that, the second model generation unit 114 stores the generated second learning model in the memory area 110.

[0035] That is, the second model generation unit 114 uses multiple second learning data to generate a second learning model used when extracting partial image data (hereinafter also referred to as second partial image data) about the float 12 from the first partial image data in the inference stage.

[0036] Next, the process of the inference stage of the float meter reading process in the first embodiment will be described.

[0037] The first image extraction unit 121 of the information processing device 1 extracts, for example, first partial image data about the float meter from the verification image data. The verification image data is, for example, image data showing the same float meter 10 as the first learning data.

[0038] Specifically, the first image extraction unit 121 identifies position information indicated by a value output in response to input of the verification image data to the first learning model stored in the memory area 110. Then, the first image extraction unit 121 extracts the first partial image data corresponding to the identified position information from the verification image data.

[0039] Then, the image correction unit 122 of the information processing device 1 performs homography transformation on the first partial image data extracted by the first image extraction unit 121, for example.

[0040] Next, the second image extraction unit 123 of the information processing device 1 extracts second partial image data for the float 12 from the first partial image data extracted by the first image extraction unit 121.

[0041] Specifically, the second image extraction unit 123 identifies position information indicated by a value output in response to input of the first partial image data to the second learning model stored in the memory area 110. Then, the second image extraction unit 123 extracts the second partial image data corresponding to the identified position information from the first partial image data.

[0042] Then, the position determination unit 124 of the information processing device 1 determines, from the second partial image data extracted by the second image extraction unit 123, a first position and a second position in the vertical direction for the first part and the second part constituting the float 12 reflected in the second image data.

[0043] Specifically, the position identifying unit 124 calculates, for example, a sum of pixel values ​​in a direction perpendicular to the vertical direction (hereinafter also referred to as the horizontal direction) at each of a plurality of positions in the vertical direction of the second partial image data extracted by the second image extracting unit 123. Then, the position identifying unit 124 identifies, for example, any two of the positions in the vertical direction of the second partial image data extracted by the second image extracting unit 123 whose calculated sum is equal to or less than a threshold value as a first position and a second position.

[0044] Then, the upper end identifying section 125 of the information processing device 1 identifies the upper end position 12a of the float 12 reflected in the second image data by using the first position and the second position identified by the position identifying section 124.

[0045] Specifically, the upper end determination unit 125 determines the upper end position 12a of the float 12 reflected in the second image data, for example, from the ratio of the length from the upper end position 12a to the second position to the length from the upper end position 12a to the first position, and the first position and the second position in the second partial image data.

[0046] Furthermore, the value calculation unit 126 of the information processing device 1 calculates the scale value of the floatmeter 10 corresponding to the upper end position 12a identified by the upper end identification unit 125, for example, by using a relational equation between each vertical position on the floatmeter 10 and each scale value on the scale 11a of the floatmeter 10.

[0047] Specifically, the operator generates an approximation equation by using multiple combinations of the vertical position of the floatometer 10 shown in the first learning image and the scale value of the scale 11a of the floatometer 10 shown in the first learning image. Then, the value calculation unit 126 calculates the scale value corresponding to the upper end position 12a identified by the upper end identification unit 125, for example, by using the generated approximation equation. In the following description, the approximation equation used to calculate the scale value is a quadratic approximation equation, but this approximation equation may be, for example, a cubic approximation equation, a quartic approximation equation, or a logarithmic approximation equation.

[0048] That is, in a typical float meter, the shape of the portion of the float used to read the scale (e.g., the top end position of the float) may be similar to the shape of other portions of the float. Therefore, for example, when using a single learning model that outputs the top end position 12a of the float 12 in response to input of verification image data showing the float meter 10, the information processing device 1 may identify a position other than the top end position 12a as the top end position 12a, and may output image data that does not include a part of the float 12 as the second partial image data.

[0049] Therefore, in the learning stage, the information processing device 1 in this embodiment generates a second learning model by using second learning data to which position information is added not only for the float 12 but also for an area including the area vertically above the top end position 12a of the float 12. Then, in the inference stage, the information processing device 1 identifies the top end position 12a of the float 12 from the second partial image data output from the second learning model.

[0050] As a result, even if the information processing device 1 in this embodiment specifies a position other than the top end position 12a as the top end position 12a, it is possible to generate a second learning model capable of outputting the second partial image data including the entire float 12. Therefore, the information processing device 1 in this embodiment is able to accurately read the scale 11a even for a float 12 in which the shape of the part used to read the scale 11a is similar to the shape of the other parts.

[0051] In addition, the first learning model and the second learning model generated in this embodiment may be, for example, a learning model based on SSD (Single Shot Multibox Detector) or a learning model based on YOLO (You Only Look Once).

[0052] [Details of the first embodiment] Next, the float meter reading process in the first embodiment will be described in detail. Figures 5 to 7 are flow charts for explaining the details of the float meter reading process in the first embodiment. Figures 8 to 14 are diagrams for explaining the details of the float meter reading process in the first embodiment.

[0053] [Processing during the learning phase] First, a process in the learning stage will be described in detail in the float meter reading process in the first embodiment. Fig. 5 is a diagram for explaining the process in the learning stage.

[0054] 5, the first data generation unit 111 waits until the learning timing comes (NO in S11). The learning timing may be, for example, the timing when the worker inputs the first learning image data via the operation terminal 2. The learning timing may also be, for example, the timing when the worker inputs information to the effect that each learning model will be generated.

[0055] Then, when the learning timing arrives (YES in S11), the first data generation unit 111 generates first learning data by adding position information of the float meter 10 in each image data to each of the first learning image data (S12).

[0056] Specifically, the first data generation unit 111 adds position information of the floatometer 10 to the first learning image data DT11, for example, by using a bounding box B1 including the floatometer 10 (a bounding box B1 specified by the operator), as shown in FIG. 8.

[0057] More specifically, the worker prepares image data corresponding to each frame included in video data captured by an imaging device (not shown), for example, as the first learning image data DT11. Then, the worker adds position information of the floatometer 10 to the first learning image data DT11 by using a bounding box B1. After that, the first data generator 111 generates the first learning data by, for example, associating the first learning image data DT11 with the coordinates (X and Y coordinates) of the upper left corner of the bounding box B1 and the vertical and horizontal lengths of the bounding box B1.

[0058] The imaging device may be, for example, a non-fixed camera. Also, the imaging device may be, for example, one built into the operation terminal 2.

[0059] Thereafter, the first model generation unit 112 generates a first learning model by performing machine learning using the first learning data generated in the process of S12 (S13).

[0060] Next, the second data generation unit 113 generates second learning data by adding position information of the float 12 in each piece of image data to each piece of second learning image data (S14).

[0061] Specifically, as shown in FIG. 9, the second data generation unit 113 adds positional information of the area including the float 12 to the second learning image data DT12, for example, by using a bounding box B2 including the float 12 (a bounding box B2 specified by the worker).

[0062] More specifically, the second data generation unit 113 acquires, for example, image data output in response to input of the first training image data DT11 to the first learning model generated in the process of S13 as the second training image data DT12. Then, the worker adds position information of the area that reflects the float 12 and the vertically upper side of the upper end position 12a of the float 12 to the second training image data DT12 by using the bounding box B2. After that, the second data generation unit 113 generates the second training data by, for example, associating the second training image data DT12 with the coordinates (X coordinate and Y coordinate) of the top left of the bounding box B2 and the vertical and horizontal lengths of the bounding box B2.

[0063] Thereafter, the second model generation unit 114 generates a second learning model by performing machine learning using the second learning data generated in the process of S14 (S15).

[0064] This allows the operator to generate each learning model to be used in the inference stage described below.

[0065] [Processing in the inference stage] Next, a description will be given of the process in the inference stage among the details of the float meter reading process in the first embodiment. Figures 6 and 7 are diagrams for explaining the process in the inference stage.

[0066] 6, the first image extraction unit 121 waits until the determination timing arrives (NO in S21). The determination timing may be, for example, the timing when the worker inputs the verification image data via the operation terminal 2. The determination timing may also be, for example, the timing when the worker inputs information to the effect that a determination will be made on the verification image data.

[0067] Then, when the determination timing arrives (YES in S21), the first image extraction unit 121 acquires verification image data for the floatometer 10 (S22).

[0068] Specifically, as shown in FIG. 10(A), the first image extraction unit 121 receives, for example, image data corresponding to each frame included in video data captured by an imaging device as verification image data DT21.

[0069] Next, the first image extraction unit 121 acquires a value output from the first learning model in response to input of the verification image data acquired in the process of S22 (S23).

[0070] Then, the first image extraction unit 121 extracts the first partial image data corresponding to the value acquired in the process of S23 from the verification image data acquired in the process of S22 (S24).

[0071] Specifically, as shown in FIG. 10(B), the first image extraction unit 121 acquires, for example, image data corresponding to the portion of the verification image data acquired in the processing of S21, which portion includes the float meter 10, as the first partial image data DT22.

[0072] The image correction unit 122 performs homography transformation on the first partial image data extracted in the process of S24 (S25).

[0073] Specifically, the image correction unit 122 performs homography transformation so that, for example, each of the second reference regions 14 in the floatometer 10 reflected in the first partial image data DT22 extracted in the process of S24 is moved to a pre-specified position.

[0074] That is, the image correction unit 122 normalizes the first partial image data DT22 by performing a homography transformation on the first partial image data DT22, and corrects the positions of each of the second reference areas 14 in the floatometer 10 reflected in the first partial image data DT22.

[0075] This allows the information processing device 1 to align the positions of the second reference areas 14 reflected in all of the first partial image data DT22. Therefore, the information processing device 1 can specify the upper end position 12a of the float 12 based on the position of the second reference area 14, as will be described later.

[0076] In addition, the image correction unit 122 may perform, for example, a homography transformation on the first partial image data DT22 after emphasizing the first reference region 13 by performing an enhancement process such as binarization on the first partial image data DT22.

[0077] This enables the information processing device 1 to more easily specify, for example, a first position and a second position, which will be described later.

[0078] Next, the second image extraction unit 123 acquires a value output from the second learning model in response to input of the first partial image data that has been subjected to homography transformation in the process of S25 (S26).

[0079] Then, as shown in FIG. 7, the second image extraction unit 123 extracts the second partial image data corresponding to the value acquired in the process of S26 from the first partial image data subjected to the homography transformation in the process of S25 (S31).

[0080] Specifically, as shown in FIG. 11(B), the second image extraction unit 123 acquires, for example, image data corresponding to the portion of the first partial image data DT22 (FIG. 11(A)) on which homography transformation has been performed in processing S25, in which the float 12 is reflected, as the second partial image data DT23.

[0081] More specifically, the second image extraction unit 123 acquires image data that shows the area vertically above the upper end position 12a of the float 12 in addition to the float 12 as second partial image data DT23, as shown in FIG. 11(B).

[0082] Next, the position identification unit 124 calculates the sum of pixel values ​​in the horizontal direction (hereinafter also simply referred to as the sum) at each of the multiple positions in the vertical direction of the second partial image data extracted in the process of S31 (S32).

[0083] Then, the position identifying unit 124 identifies, for example, any two of the positions whose total value calculated in the process of S32 satisfies a predetermined condition as the first position and the second position (S33). Specifically, the position identifying unit 124 identifies, for example, any two of the positions whose total value calculated in the process of S32 is equal to or less than a threshold value (hereinafter also referred to as a first threshold value) as the first position and the second position. Specific examples of the processes of S32 and S33 will be described below.

[0084] [Specific examples of processing in S32 and S33] 12 and 13 are diagrams for explaining a specific example of the processes of S32 and S33. Specifically, FIG. 12(A) is a diagram for explaining a specific example of float 12, and FIG. 12(B) is a diagram for explaining a specific example of second partial image data DT23. Also, FIG. 13 is a graph showing the sum of pixel values ​​at each position in the vertical direction of the second partial image data DT23. The vertical axis in FIG. 13 indicates the position (coordinate) in the vertical direction of the second partial image data DT23, and the horizontal axis in FIG. 13 indicates the sum of pixel values ​​in the horizontal direction at each position. Note that the following description will be given assuming that the vertical coordinate of each image data increases from the top to the bottom.

[0085] First, a specific example of the float 12 will be described.

[0086] As shown in Fig. 12(A), the float 12 has an upper end position 12a that is a reference for reading the scale 11a. The float 12 also has a recess 12b on the side surface between positions 12b0 and 12b1 in the vertical direction, and a tapered portion 12c extending downward from a position 12c0 at the lower end in the vertical direction. Hereinafter, as shown in Fig. 12(B), the vertical coordinate corresponding to the upper end position in the second partial image data DT23 is also called coordinate b0, the vertical coordinate corresponding to the lower end position in the second partial image data DT23 is also called coordinate b1, the vertical coordinate corresponding to the position 12b0 in the recess 12b is also called coordinate y1, the vertical coordinate corresponding to the position 12b1 in the recess 12b is also called coordinate y2, and the vertical coordinate corresponding to the position c1 in the tapered portion 12c is also called coordinate y3.

[0087] Next, a specific example of the processes in S32 and S33 will be described.

[0088] In the process of S32, the position identification unit 124 calculates, for example, the sum of pixel values ​​in the horizontal direction at each of a plurality of positions in the vertical direction of the second partial image data DT23. The plurality of positions here are, for example, positions corresponding to each pixel in the vertical direction of the second partial image data DT23.

[0089] Next, the position specifying unit 124 generates a graph showing the sum of pixel values ​​at each position in the vertical direction of the second partial image data DT23, as shown in FIG.

[0090] Specifically, for example, if the vertical length of the second partial image data DT23 (the length from coordinate b0 to coordinate b1) is "80", the position identification unit 124 generates a graph in which, for example, coordinate b0 is set to "0" and coordinate b1 is set to "80", as shown in FIG. 13.

[0091] Then, in the process of S33, the position specifying unit 124 refers to the generated graph and specifies, among a plurality of positions in the vertical direction of the second partial image data DT23, a position where the total value of pixel values ​​is equal to or less than a threshold value.

[0092] Specifically, the graph shown in FIG. 13 indicates that the sum of pixel values ​​corresponding to each point whose vertical coordinate is between "10" and "20" and the sum of pixel values ​​corresponding to each point whose vertical coordinate is between "60" and "70" are particularly small. Therefore, the position identification unit 124, for example, identifies the start point (e.g., "14") and end point (e.g., "16") of the range in which the sum of pixel values ​​is the smallest among each point whose vertical coordinate is between "10" and "20" as coordinate y1 and coordinate y2, respectively. In addition, the position identification unit 124, for example, identifies the start point (e.g., "68") of the range in which the sum of pixel values ​​is the smallest among each point whose vertical coordinate is between "60" and "70" as coordinate y3.

[0093] That is, due to the way the light hits the second partial image data DT21 when the verification image data DT21 is captured, the sum of the pixel values ​​in the horizontal direction in the recess 12b and the tapered portion 12c reflected in the second partial image data DT23 is smaller than the sum of the pixel values ​​in other portions of the float 12. Therefore, the position specifying unit 124 can specify the coordinates y1, y2, and y3 by specifying the positions where the sum of the pixel values ​​is equal to or less than the threshold value.

[0094] After that, the position specifying unit 124 specifies, for example, two of the coordinates y1, y2, and y3 as the first position and the second position.

[0095] In the process of S25, for example, when the first reference region 13 is enhanced by performing an enhancement process on the first partial image data DT22 and then homography transformation is performed, the sum of the horizontal pixel values ​​in the recessed portion 12b and the tapered portion 12c reflected in the second partial image data DT23 may be greater than the sum of the pixel values ​​in other portions of the float 12. Therefore, in this case, the position identification unit 124 may identify, for example, any two of the positions where the sum calculated in the process of S32 is equal to or greater than a threshold value (hereinafter, also referred to as a second threshold value) as the first position and the second position (S33). Specifically, the position identification unit 124 may identify, for example, the positions where the sum of the pixel values ​​is equal to or greater than a threshold value, thereby identifying the coordinates y1, y2, and y3.

[0096] Returning to FIG. 7, the upper end determination unit 125 determines the upper end position 12a of the float 12 in the vertical direction from the first position and the second position determined in S33 (S35).

[0097] Specifically, the upper end identification unit 125 calculates the coordinate y0 corresponding to the upper end position 12a by using, for example, any one of the following formulas (1), (2), and (3). In the following formulas (1), (2), and (3), c1, c2, and c3 respectively represent the length from the upper end position 12a to position 12b0, the length from the upper end position 12a to position 12b1, and the length from the upper end position 12a to position 12c0, which are measured using other image data (not shown).

[0098] More specifically, for example, when the coordinates y3 and y2 are identified as the first position and the second position in the process of S34, the upper end identification unit 125 calculates the coordinate y0 according to, for example, the following formula (1).

[0099]

number

[0100] That is, for example, the value obtained by dividing the length from the top end position 12a to the position 12c0 by the length from the top end position 12a to the position 12b1 is the same for all image data showing the float 12. Therefore, the position identification unit 124 calculates the coordinate y0 by using the value obtained by dividing the length from the top end position 12a to the position 12c0 measured using other image data by the length from the top end position 12a to the position 12b1 measured using other image data.

[0101] Furthermore, for example, when the coordinates y3 and y1 are identified as the first position and the second position in the process of S34, the upper end identification unit 125 calculates the coordinate y0 according to, for example, the following formula (2).

[0102]

number

[0103] Furthermore, for example, when the coordinates y2 and y1 are identified as the first position and the second position in the process of S34, the upper end identification unit 125 calculates the coordinate y0 according to, for example, the following formula (3).

[0104]

number

[0105] The upper end determination unit 125 may calculate the coordinate y0 of the upper end position 12a by using, for example, all of the above formulas (1), (2), and (3). In this case, the upper end determination unit 125 may calculate the median or average of the coordinate y0 calculated by using, for example, each of the above formulas (1), (2), and (3).

[0106] Returning to FIG. 7, the value calculation unit 126 calculates the scale value of the float meter 10 corresponding to the upper end position 12a identified in the process of S34 (S35).

[0107] Specifically, the value calculation unit 126 calculates the coordinate in the first partial image data DT22 of the coordinate y0 calculated in the process of S34 (hereinafter also referred to as coordinate Y0) as shown in Fig. 14. Then, the value calculation unit 126 obtains a value indicating the position of the coordinate Y0 in the first partial image data DT22 by dividing the length from the top end position in the vertical direction of the first partial image data DT22 to the top end position 12a of the float 12 (the length from coordinate m0 to coordinate Y0 shown in Fig. 14) by the length from the top end position to the bottom end position in the vertical direction of the first partial image data DT22 (the length from coordinate m0 to coordinate m1 shown in Fig. 14).

[0108] More specifically, the value calculation unit 126 calculates the coordinate Y0 by adding the coordinate b0 in the first partial image data DT22 described in FIG. 12(B) (hereinafter also referred to as the coordinate B0) to the coordinate y0 calculated in the process of S34. Then, the value calculation unit 126 obtains a value indicating the position of the coordinate Y0 in the first partial image data DT22 by following the following equation (4). Note that in the following equation (4), P corresponds to the value indicating the position of the coordinate Y0, and y min corresponds to the coordinate m0 corresponding to the upper end position of the second reference area 14a, and y max corresponds to the coordinate m1 which corresponds to the lower end position of the second reference region 14b.

[0109]

number

[0110] Then, the value calculation unit 126 calculates the scale value corresponding to the coordinate Y0 (the scale value on the scale 11a) by substituting the value indicating the position of the coordinate Y0 into a quadratic approximation equation generated in advance and corresponding to the float meter 10. After that, the value calculation unit 126 outputs the calculated scale value to the operation terminal 2, for example.

[0111] For example, when the pixel size of image data corresponding to each frame included in video data captured by an imaging device (hereinafter also referred to as reference image data) differs from that of the verification image data DT21, the value calculation unit 126 may multiply the verification image data DT21 by a scaling factor of the reference image data relative to the verification image data DT21, and then substitute the result into the quadratic approximation formula. That is, the value calculation unit 126 may substitute a value indicating the position of the coordinate Y0 in the reference image data into the quadratic approximation formula.

[0112] The operator may also generate a learning model (not shown) to be used in calculating the scale value on the scale 11a of the floatometer 10 reflected in the first partial image data DT22, for example, by using learning data (not shown) including the first learning image data DT11 and information indicating the scale value on the scale 11a of the floatometer 10 reflected in the first learning image data DT11. The value calculation unit 126 may specify, for example, a value output in response to the input of the first partial image data DT22 to the generated learning model, as the scale value on the scale 11a of the floatometer 10 reflected in the first partial image data DT22.

[0113] Furthermore, for example, if the correct value of the scale value on the scale 11a corresponding to the upper end position 12a of the float 12 or a correct equation that can calculate the scale value on the scale 11a by using the upper end position 12a of the float 12 is known in advance, the value calculation unit 126 may use this correct value or correct equation to identify the scale value on the scale 11a of the float 10 reflected in the first partial image data DT22.

[0114] In this manner, the information processing device 1 in this embodiment sequentially identifies the floatometer 10 reflected in the verification image data and the float 12 reflected in the first partial image data by using a first learning model that outputs first partial image data showing the float 12 and an area vertically above the upper end position 12a of the float 12 in response to the input of the verification image data, and a second learning model that outputs second partial image data showing the float 12 and an area vertically above the upper end position 12a of the float 12 in response to the input of the first partial image data. Then, the information processing device 1 identifies the upper end position 12a of the float 12 reflected in the second partial image data by using, for example, the first and second positions (positions other than the upper end position 12a of the float) of the float 12 reflected in the second partial image data.

[0115] As a result, even if the information processing device 1 in this embodiment specifies a position other than the top end position 12a as the top end position 12a, it is possible to generate a second learning model capable of outputting the second partial image data including the entire float 12. Therefore, the information processing device 1 in this embodiment is able to accurately read the scale 11a even for a float 12 in which the shape of the part used to read the scale 11a is similar to the shape of the other parts.

[0116] In addition, the first image extraction unit 121 may be configured to continuously acquire verification image data DT21 corresponding to each frame included in the video data captured by the imaging device in the process of S22. Then, the information processing device 1 may be configured to perform the processes of S23 to S35 for each of the acquired verification image data DT21. Thereafter, the value calculation unit 126 may be configured to output the median and average of each of the scale values ​​calculated in the process of S35 to the operation terminal 2.

[0117] Furthermore, the first data generation unit 111 may automatically identify the position of the bounding box B1 from the positions of the reference markers 13 in the process of S12, for example.

[0118] Specifically, the first data generation unit 111 may automatically identify, for example, a position that includes all of the reference markers 13 as the position of the bounding box B1. Then, the first data generation unit 111 may automatically generate the first training data by adding position information of the identified bounding box B1 to the first training image data DT11.

[0119] This enables the information processing device 1 to reduce the workload of an operator involved in generating the first learning data.

[0120] In addition, in the processing of S14, the second data generation unit 113 may automatically determine the position of the bounding box B2, for example, from a graph showing each position of the reference marker 14 and the sum of pixel values ​​corresponding to each position in the vertical direction of the second learning image data DT12 (the graph described in Figure 13).

[0121] Specifically, the second data generation unit 113 may, for example, identify a range in the vertical direction where the sum of pixel values ​​is equal to or less than a threshold value among the vertical positions of the second learning image data DT12. Then, the second data generation unit 113 may, for example, automatically identify a position corresponding to an area obtained by expanding the identified range vertically upward and downward by a predetermined length as the position of the bounding box B2. Thereafter, the second data generation unit 113 may automatically generate the second learning data by adding position information of the identified bounding box B2 to the second learning image data DT12.

[0122] This enables the information processing device 1 to reduce the workload of an operator involved in generating the second learning data. [Explanation of symbols]

[0123] 1: Information processing device 2: Operation terminal 101:CPU 102: Memory 103: Communication interface 104:Storage medium 105: Bus

Claims

1. Extracting a first partial image data including a float meter having a float that moves vertically inside the float meter from the verification image data including the float meter; extracting second partial image data including the float from the extracted first partial image data; Identifying a first position and a second position in the vertical direction for the first portion and the second portion of the float from the extracted second portion image data, respectively; Identifying an upper end position of the float in the vertical direction from the identified first position and second position; calculating a scale value of the float meter corresponding to the identified upper end position by using a relational expression between each position of the float meter in the vertical direction and each scale value of the float meter; A float meter reading program that causes a computer to execute a process.

2. In claim 1, In the process of identifying the first position and the second position, calculating a sum of pixel values ​​in a horizontal direction at each position of the second partial image data in the vertical direction; identifying, as the first position and the second position, any two of the positions of the second partial image data in the vertical direction where the calculated sum satisfies a predetermined condition; A float meter reading program comprising:

3. In claim 2, In the process of identifying the first position and the second position, any two of positions of the second partial image data in the vertical direction where the sum is equal to or less than a first threshold value are identified as the first position and the second position. A float meter reading program comprising:

4. In claim 2, In the process of identifying the first position and the second position, any two of positions of the second partial image data in the vertical direction where the sum is equal to or greater than a second threshold value are identified as the first position and the second position. A float meter reading program comprising:

5. In claim 1, In the process of extracting the second partial image data, partial image data including the float and an area above the float in the vertical direction, the area having a length in the vertical direction equal to or greater than a predetermined length, is extracted as the second partial image data. A float meter reading program comprising:

6. In claim 1, further comprising: performing a homography transformation on the first partial image data; The process is executed by a computer, In the process of extracting the second partial image data, the second partial image data is extracted from the first partial image data that has been subjected to homography transformation. A float meter reading program comprising:

7. In claim 1, In the process of identifying the upper end position, determining an upper end position of the float in the vertical direction from a ratio of a length from the upper end position to the second position to a length from the upper end position to the first position, and from the first position and the second position in the second partial image data; A float meter reading program comprising:

8. In claim 1, further comprising: generating a plurality of first learning data by adding position information of the float meter included in each of the plurality of first learning image data, the position information being included in each of the plurality of first learning image data; generating a first learning model by performing machine learning using the generated plurality of first learning data; The process is executed by a computer, In the process of extracting the first partial image data, acquiring a value output from the first learning model in response to inputting the verification image data; extracting the first partial image data corresponding to the acquired value from the verification image data; A float meter reading program comprising:

9. In claim 1, further comprising: generating a plurality of second learning data by adding position information of the float included in each of the plurality of second learning image data including the float; generating a second learning model by performing machine learning using the generated plurality of second learning data; The process is executed by a computer, In the process of extracting the second partial image data, Obtaining a value output from the second learning model in response to input of the first partial image data; extracting the second partial image data corresponding to the obtained value from the first partial image data; A float meter reading program comprising:

10. Extracting a first partial image data including a float meter having a float that moves vertically inside the float meter from the verification image data including the float meter; extracting second partial image data including the float from the extracted first partial image data; Identifying a first position and a second position in the vertical direction for the first portion and the second portion of the float from the extracted second portion image data, respectively; Identifying an upper end position of the float in the vertical direction from the identified first position and second position; calculating a scale value of the float meter corresponding to the identified upper end position by using a relational expression between each position of the float meter in the vertical direction and each scale value of the float meter; A float meter reading device comprising:

11. Extracting a first partial image data including a float meter having a float that moves vertically inside the float meter from the verification image data including the float meter; extracting second partial image data including the float from the extracted first partial image data; Identifying a first position and a second position in the vertical direction for the first portion and the second portion of the float from the extracted second portion image data, respectively; Identifying an upper end position of the float in the vertical direction from the identified first position and second position; calculating a scale value of the float meter corresponding to the identified upper end position by using a relational expression between each position of the float meter in the vertical direction and each scale value of the float meter; A method for reading a float meter, the method comprising the steps of: executing processing by a computer;

Images