Information processing system, information processing device, information processing method, and program

Abstract

To detect a value indicated by an analog meter from an image including the analog meter installed, the image being captured by an imaging apparatus.SOLUTION: An information processing system is configured to: specify a center point corresponding to a rotation center of a needle of an analog meter in an image that includes the analog meter captured by an imaging apparatus; searches, using an image obtained by binarizing the image including the analog meter, for intersections between a needle part of the analog meter and multiple circles centered at the specified center point; specify the needle part of the analog meter in the image including the analog meter; and obtain a value indicated by the analog meter on the basis of an angle of the specified needle part of the analog meter.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing apparatus, an information processing method, and a program for detecting a value indicated by an analog meter from an image.

Background Art

[0002] The analog meter that indicates a value by a rotating needle is installed and used in a control panel, an operation panel, an instrumentation rack, etc. As a technique for reading the scale value indicated by the needle of the analog meter from an image captured by a camera or the like, the techniques described in Patent Documents 1 and 2 are disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to enable detection of a value indicated by an analog meter from an image including the installed analog meter captured by an imaging device.

Means for Solving the Problems

[0005] The information processing system according to the present invention includes imaging means for imaging an analog meter that indicates a value by a rotating needle, and the analog meter including the analog meter imaged by the imaging means Extraction means for extracting a partial image; first identification means for searching for circular shapes and the center point of the circles within the partial image, and identifying the circle with the largest area among the circular shapes obtained by the search as the scale portion of the analog meter; and the center point of the circle with the largest area corresponding to the scale portion of the analog meter. a rotation center of a needle included in the analog meter as a point first 2 identifying means for identifying, and using an image obtained by binarizing the portion image, the The rotation of the needleThe intersection points of multiple circles centered on the central point and the needle portion of the analog meter are searched, portion The needle portion of the analog meter in the image is identified by 3 The means of identifying and A fourth identification means that, when the position of the intersection is converted into an angle, identifies the angle with the most detected intersections as the angle of the needle portion of the analog meter in the partial image, The system is characterized by having an acquisition means for acquiring the value indicated by the analog meter based on the angle of the needle portion of the specified analog meter. [Effects of the Invention]

[0006] According to the present invention, it is possible to detect the value indicated by an analog meter from an image captured by an imaging device, which includes an installed analog meter. [Brief explanation of the drawing]

[0007] [Figure 1] This figure shows an example of the configuration of the information processing system in this embodiment. [Figure 2] This figure shows an example of the hardware configuration of the server device in this embodiment. [Figure 3] This figure shows an example of the functional configuration of the server device in this embodiment. [Figure 4] This flowchart shows an example of the meter reading process in this embodiment. [Figure 5] This diagram illustrates the meter reading process in this embodiment. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 shows an example of the configuration of an information processing system according to one embodiment of the present invention. Figure 1 shows an example in which the information processing system of this embodiment is applied to a remote monitoring system that enables remote equipment monitoring, etc. Note that the example shown in Figure 1 is just one example, and the information processing system of this embodiment is not limited to remote monitoring systems but can be applied to other systems as well.

[0009] The server device 101 is an information processing device that acquires data related to the status of a remotely monitored object based on images captured of the object. The server device 101 performs image analysis using the image of the control panel 106 captured by the imaging device 102 and acquires the status of the display components installed on the control panel 106. In this embodiment, the server device 101 acquires the value indicated by an analog meter installed on the control panel 106, which indicates a value by a rotating needle. The server device 101 identifies the angle of the needle of the analog meter and acquires the value (scale value) indicated by the needle based on the identified angle. In this example, it is a control panel 106, but it may be other operation panels or instrumentation racks, and any analog meter installed on any component that can be imaged by the imaging device 102 can be targeted.

[0010] The imaging device 102 is an imaging device such as a camera, and is installed in a position where it can image the control panel 106 (at least a position where the target analog meter is included in the field of view). The imaging device 102 is connected to the server device 101 via a communication line 103 such as a carrier line. The imaging device 102 images the control panel 106 at predetermined time intervals, such as every minute or every five minutes, and transmits the image of the control panel 106 to the server device 101 via the communication line 103.

[0011] Furthermore, the imaging device 102 may be configured to perform imaging operations at any time in response to imaging instructions or other user commands. Also, the imaging device 102 and the server device 101 are not limited to being connected by a communication line 103; they may be connected via a network or the like, as long as the connection allows the imaging device 102 to transmit images it has captured to the server device 101. In addition, although Figure 1 illustrates one imaging device 102, multiple imaging devices may be connected to the server device 101.

[0012] In the information processing system according to this embodiment, the server device 101 can be connected to a terminal device 104 used by a user or the like via a network 105 such as the Internet or a LAN (Local Area Network). The terminal device 104 is composed of, for example, a personal computer (PC), a tablet device, a smartphone, or the like. The server device 101 provides data related to the status of the monitored object obtained in response to a request from the terminal device 104. Further, when the obtained status of the monitored object satisfies a predetermined condition, the server device 101 may notify the terminal device 104 or the like.

[0013] FIG. 2 is a diagram showing an example of the hardware configuration of the server device 101 in this embodiment. The server device 101 includes a CPU (Central Processing Unit) 201, a main storage device 202, an auxiliary storage device 203, an input I / F 204, an output I / F 205, and a communication I / F 206. The CPU 201, the main storage device 202, the auxiliary storage device 203, the input I / F 204, the output I / F 205, and the communication I / F 206 are communicably connected to each other via a system bus 207.

[0014] The CPU 201 is a central processing unit that controls the server device 101. The main storage device 202 is a storage device that functions as a work area for the CPU 201 and a temporary storage location for data. The main storage device 202 is implemented using, for example, a RAM (Random Access Memory) or the like. The auxiliary storage device 203 is a storage device that stores programs, image data, information related to various processes, and the like. The auxiliary storage device 203 is implemented using, for example, a ROM (Read Only Memory), a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like.

[0015] The input I / F 204 is an interface used for inputting information from input devices such as a mouse and a keyboard. The output I / F 205 is an interface used for outputting information to output devices such as a display and a speaker. The communication I / F 206 is an interface used for communication with external devices via a network such as the Internet or a LAN, or for communication with imaging devices via a communication line such as a carrier line.

[0016] When the CPU 201 executes processing based on the program stored in the auxiliary storage device 203, each function of the server device 101 shown in FIG. 3 and each process shown in the flowchart of FIG. 4 are realized.

[0017] FIG. 3 is a diagram showing an example of the functional configuration of the server device 101 in the present embodiment. The server device 101 includes an image acquisition unit 301, an image storage unit 302, an image analysis unit 303, a determination unit 304, an information storage unit 305, and an output unit 306.

[0018] The image acquisition unit 301 acquires an image of the control panel 106 captured by the imaging device 102, specifically, an image of the control panel 106 including an analog meter that indicates a value by a rotating needle, which is the target. The image acquired by the image acquisition unit 301 is stored in the image storage unit 302.

[0019] The image analysis unit 303 performs image analysis on the image stored in the image storage unit 302 and outputs an analysis result. In the present embodiment, the image analysis unit 303 performs the data reading process shown in FIG. 4 described later using the image stored in the image storage unit 302, and acquires the value indicated by the needle of the analog meter of the target in the image and outputs it as an analysis result. The image analysis unit 303 is an example of a first specifying means, a second specifying means, an acquiring means, an extracting means, and a correcting means.

[0020] The determination unit 304 determines whether the analysis results obtained by the image analysis unit 303 satisfy predetermined conditions. For example, the determination unit 304 determines whether the value indicated by the needle of the analog meter obtained as an analysis result exceeds a predetermined threshold or is within a predetermined range. Also, for example, the determination unit 304 determines whether the change over time of the value indicated by the needle of the analog meter obtained as an analysis result satisfies predetermined conditions. The information storage unit 305 stores the analysis results obtained by the image analysis unit 303 and the determination results by the determination unit 304, etc.

[0021] The output unit 306 outputs information obtained by the server device 101 to the terminal device 104, etc. For example, in response to a request from the terminal device 104, the output unit 306 provides the terminal device 104 with the analysis results obtained by image analysis (such as the value indicated by the needle of an analog meter). This allows the terminal device 104 to display and present the analysis results to the user. Alternatively, for example, the output unit 306 may notify the terminal device 104 by email or push notification if the determination by the determination unit 304 meets predetermined conditions. This allows the user of the terminal device 104 to be notified that the predetermined conditions have been met. The output unit 306 may also provide the terminal device 104 with an image stored in the image storage unit 302 that corresponds to the analysis results obtained by image analysis.

[0022] Figure 4 is a flowchart showing an example of the meter reading process performed by the server device 101 in this embodiment. The meter reading process shown in Figure 4 is performed, for example, each time the server device 101 receives an image including the control panel 106 captured by the imaging device 102.

[0023] In step S401, the image acquisition unit 301 acquires an image of the control panel 106, including the target analog meter, which has been captured by the imaging device 102. Here, the target analog meter is an analog meter that has a rotating needle and indicates a value by that needle. The image of the control panel 106 acquired by the image acquisition unit 301 is stored in the image storage unit 302. Figure 5(A) shows an example of the acquired image of the control panel 106. In Figure 5(A), 501 is the entire image captured by the imaging device 102. Image 501 includes the control panel 502 on which the target analog meter 511 is installed.

[0024] In step S402, the image analysis unit 303 extracts a partial image containing the target analog meter from the image of the control panel 106 acquired in step S401 by template matching using a pre-registered template image. For example, in the example shown in Figure 5(A), the image analysis unit 303 extracts a partial image 512 from the entire image 501 by template matching using a template image for extracting the target analog meter 511. This results in a partial image containing the analog meter 511 having a rotating needle 521, as shown in Figure 5(B).

[0025] In step S403, the image analysis unit 303 performs distortion correction processing on the partial image extracted in step S402. In this distortion correction processing, the image analysis unit 303 corrects the distortion in the target analog meter portion by applying a projection transformation to the partial image. For example, if a shape that should be circular appears as an ellipse in the partial image, the distortion is corrected by transforming the shape to become circular. This reduces image distortion that occurs when the imaging device 102 is not positioned so that it is directly in front of the target analog meter in both the horizontal and vertical directions. The image analysis unit 303 may also apply processing such as color tone conversion or Gaussian transformation to the partial image as needed to reduce the influence of the image shooting environment.

[0026] In step S404, the image analysis unit 303 performs a process to identify the scale portion of the analog meter in the partial image. The image analysis unit 303 searches for a portion of the partial image that shows a circular shape (including an arc which is part of it) and the center point of that circle. The image analysis unit 303 identifies the scale portion of the analog meter by determining that the circle with the largest area among the circular shapes obtained through the search corresponds to the scale portion of the analog meter. The image analysis unit 303 also identifies that the center point of the circle with the largest area among the circular shapes that correspond to the scale portion of the analog meter is the rotation center of the needle of the target analog meter.

[0027] For example, the image analysis unit 303 performs circle detection on a partial image using the Hough transform. The image analysis unit 303 then performs binarization on the partial image, converting it into an image where the background is white and areas different from the background are black. Based on the binarized partial image, the image analysis unit 303 converts the original points (x,y) on the circumference of the circle into a three-dimensional space with the center point (Xc,Yc) of the circle and radius r.<Xc,Yc,r> The number is added each time. Then, the image analysis unit 303 considers the combination with the maximum number of points on the virtual circle as a circle, converts it back to the original Cartesian coordinate system, and obtains the set of points that most closely resembles a circle, thereby searching for the shape of the circle and its center point. Here, the image is a collection of discontinuous points with integer coordinate systems, whereas the calculation formula in the Hough transform is (x-Xc) 2 +(y-Yc) 2 =r 2 Errors occur because of the continuity. Therefore, in the image, thin circles with an outline of about 1 pixel width may not be converted as perfect circles, and circles need to be drawn with a width of several pixels, but multiple interpretations are possible for circles with a width of several pixels. Therefore, the image analysis unit 303 considers the circle with the largest area to be the one with the most points detected on its circumference.

[0028] As described above, the image analysis unit 303 acquires a circle 531 corresponding to the scale portion of the analog meter 511, and the center point 532 of the circle 531 corresponding to the rotation center of the needle 521 of the analog meter 511, as illustrated in Figure 5(C) as an example.

[0029] In step S405, the image analysis unit 303 performs binarization on the partial image and uses the binarized partial image to identify the needle portion of the analog meter in the partial image. The image analysis unit 303 performs binarization in response to changes in the shooting environment, such as brightness during image capture, so that the needle portion of the analog meter can be recognized regardless of the color of the needle. Here, the image analysis unit 303 identifies the needle portion of the analog meter in the partial image by performing the process described below, for example. In the following description, it is assumed that the binarized partial image has a white background and the portion of the analog meter needle portion that is different from the background is black.

[0030] The image analysis unit 303 searches for the intersection of the circumference centered on the center point obtained by the search in step S404 and the needle portion of the analog meter in the partial image. The image analysis unit 303 searches for the intersection of the circumference centered on the center point and the needle portion of the analog meter in the partial image by searching for the intersection of the black portion in the binarized partial image, which is a portion of the partial image that is different from the background, and the circumference centered on the center point obtained by the search in step S404. Here, there is no particular limit to the number of circumferences used in the search, but considering that a portion of the partial image that is different from the background and different from the needle portion of the analog meter (for example, characters written inside the meter) may also be detected as an intersection, it is preferable to use multiple circumferences with different radii to prevent false detection of the needle portion. On the other hand, the amount of processing required for the search increases as the number of circumferences used in the search increases. Therefore, for example, the search for the intersection is performed using three circumferences that are smaller in radius than the circle corresponding to the scale portion of the analog meter obtained by the search in step S404 and have different radii from each other. Figure 5(D) shows an example. Figure 5(D) shows an example in which the needle portion 545 of the analog meter is identified using three circles 542, 543, and 544 with different radii, centered at the central point 532, in a binarized partial image 541. In the example shown in Figure 5(D), the radius of the circle corresponding to the scale portion of the analog meter obtained by the search in step S404 is R, and 0.9R, which is smaller than radius R, is divided into three equal parts, 0.9R, 0.6R, and 0.3R, which are used as the radii of the circles 542, 543, and 544, respectively.

[0031] The image analysis unit 303 identifies the needle portion of the analog meter in the partial image based on the intersections detected by the search described above. The image analysis unit 303 converts the positions of the intersections detected by the search described above into angles, using the center point obtained by the search in step S404 as the center, and identifies the angle with the most detected intersections as the angle of the needle portion of the analog meter in the partial image.

[0032] In step S406, the image analysis unit 303 obtains the value indicated by the analog meter needle based on the angle of the analog meter needle in the partial image identified in step S405. The image analysis unit 303 performs a conversion process using pre-registered data that associates the angle of the needle with the value indicated by the analog meter needle, and obtains the value indicated by the analog meter needle according to the angle of the needle. By pre-registering data that associates the angle of the needle with the value indicated by the analog meter needle, even if the scale of the scale on the analog meter differs depending on the range of values, the value indicated by the analog meter needle can be appropriately obtained by taking into account the difference in scale.

[0033] In step S407, the image analysis unit 303 stores the value indicated by the needle of the target analog meter, which was acquired in step S406, in the information storage unit 305, and the meter reading process ends.

[0034] As described above, according to this embodiment, a partial image including the target analog meter is extracted from the image of the control panel 106 captured by the imaging device 102 by template matching, and the scale portion of the analog meter is identified in the extracted partial image, thereby identifying the center point that is the rotation center of the needle of the analog meter. Then, the needle portion of the analog meter in the partial image is identified by searching for the intersection point of the circumference centered on that center point and the image, and the value indicated by the analog meter is obtained based on the angle of the identified needle portion of the analog meter. In this way, the value indicated by the installed analog meter can be detected from the image captured by the imaging device 102.

[0035] Furthermore, since the values ​​indicated by the analog meters can be detected from the image of the control panel 106 captured by the imaging device 102, the values ​​of the analog meters installed on the control panel 106 can be checked even from a location far from where the control panel 106 is located, making it possible to remotely check the status of the monitored object. Modification work on the control panel itself to enable remote checks may be subject to constraints such as cost and permit conditions. However, according to this embodiment, since it is only necessary to install an imaging device so that the control panel can be imaged without performing any modification work on the control panel, it is possible to address situations where there are constraints on modification work on the control panel, and remote checks can be easily enabled while suppressing the costs and work involved in the introduction. In addition, even in environments where the brightness during image capture differs, such as in the morning and evening, the values ​​of the meter can be accurately detected by appropriately correcting and adjusting the image to detect the values ​​indicated by the analog meters.

[0036] In the above explanation, a server device 101, located away from the control panel 106 and the imaging device 102 that captures images of it, performs the process of detecting the values ​​of the analog meters from the captured images. However, this is not limited to this configuration. For example, a separate server device installed inside or near the imaging device 102 may be equipped with a function corresponding to the image analysis unit 303, and the process of detecting the values ​​of the analog meters on the side where the control panel 106 is installed may be performed, and the detected values ​​of the analog meters may be transmitted to the server device 101.

[0037] It should be noted that the embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features. [Explanation of symbols]

[0038] 101 Server device (information processing device) 102 Imaging device 103 Communication lines 104 Terminal device 105 Network 106 Control Panel 201 CPU 202 Main storage 203 Auxiliary storage device 204 Input Interface 205 Output I / F 206 Communication I / F 207 System Bus 301 Image acquisition unit 302 Image storage unit 303 Image Analysis Department 304 Judgment section 305 Information storage unit 306 Output section

Claims

1. An imaging means for imaging an analog meter that indicates a value by a rotating needle, An extraction means for extracting a partial image including the analog meter captured by the imaging means, A first identification means that searches for the circular shape and the center point of the circular shape within the aforementioned partial image, and identifies the circle with the largest area among the circular shapes obtained by the search as the scale portion of the analog meter, A second identification means for identifying the center point of the circular shape with the largest area corresponding to the scale portion of the analog meter as the rotation center point of the needle of the analog meter, A third identification means for identifying the analog meter's needle portion in the partial image by using an image obtained by binarizing the partial image to search for the intersection points of multiple circles centered on the rotation center point of the needle and the needle portion of the analog meter, A fourth identification means that, when the position of the intersection is converted into an angle, identifies the angle with the most detected intersections as the angle of the needle portion of the analog meter in the partial image, An information processing system characterized by having an acquisition means for acquiring a value indicated by an analog meter based on the angle of the needle portion of the specified analog meter.

2. The information processing system according to claim 1, characterized in that the third identifying means identifies the needle portion of the analog meter based on the intersection points of the multiple circumferences, each having a smaller area than the radius of the largest circular shape and different radii from one another, with the needle portion of the analog meter.

3. An imaging means for imaging an analog meter that indicates a value by a rotating needle, An extraction means for extracting a partial image including the analog meter captured by the imaging means, A first identification means that searches for the circular shape and the center point of the circle within the aforementioned partial image, and identifies the center point of the circle with the largest area as the first center point corresponding to the rotation center of the needle of the analog meter, A second identification means for identifying the needle portion of the analog meter in the partial image by using an image obtained by binarizing the partial image to search for the intersection points between a plurality of circles centered on the first center point and the needle portion of the analog meter, The system includes an acquisition means for acquiring the value indicated by the analog meter based on the angle of the needle portion of the specified analog meter, The second identification means is an information processing system characterized by identifying the needle portion of the analog meter based on the intersection points of the multiple circumferences, each having a smaller area than the radius of the largest circular shape obtained by the search performed by the first identification means and having different radii from each other, with the needle portion of the analog meter.

4. The information processing system according to any one of claims 1 to 3, characterized in that the acquisition means acquires the value indicated by the analog meter based on data that associates the angle of the needle portion of the analog meter with the value indicated by the analog meter, which has been registered in advance.

5. The information processing system according to any one of claims 1 to 4, characterized in that the extraction means extracts the partial image including the analog meter from the image captured by the imaging means by template matching.

6. The information processing system according to any one of claims 1 to 5, further comprising a correction means for correcting distortion by applying a projection transformation to the partial image.

7. Image acquisition means that acquires an image captured by an imaging means that images an analog meter that indicates a value by a rotating needle, Extraction means for extracting a partial image including the analog meter from the image acquired by the image acquisition means, A first identification means that searches for the circular shape and the center point of the circular shape within the aforementioned partial image, and identifies the circle with the largest area among the circular shapes obtained by the search as the scale portion of the analog meter, A second identification means for identifying the center point of the circular shape with the largest area corresponding to the scale portion of the analog meter as the rotation center point of the needle of the analog meter, A third identification means for identifying the analog meter's needle portion in the partial image by using an image obtained by binarizing the partial image to search for the intersection points of multiple circles centered on the rotation center point of the needle and the needle portion of the analog meter, A fourth identification means that, when the position of the intersection is converted into an angle, identifies the angle with the most detected intersections as the angle of the needle portion of the analog meter in the partial image, An information processing apparatus characterized by having an acquisition means for acquiring a value indicated by an analog meter based on the angle of the needle portion of the specified analog meter.

8. An image acquisition step involves acquiring an image captured by an imaging means that images an analog meter that indicates a value by a rotating needle, and an image acquisition step involves acquiring an image captured by the imaging means. An extraction step for extracting a partial image including the analog meter from the image acquired in the image acquisition step, A first identification step involves searching for the circular shape and the center point of the circle within the aforementioned partial image, and identifying the circle with the largest area among the circular shapes obtained by the search as the scale portion of the analog meter. A second identification step involves identifying the center point of the circular shape with the largest area, which corresponds to the scale portion of the analog meter, as the rotation center point of the needle of the analog meter. A third identification step involves using the image obtained by binarizing the aforementioned partial image to search for the intersection points between multiple circles centered on the rotation center point of the needle and the needle portion of the analog meter, thereby identifying the needle portion of the analog meter in the aforementioned partial image. A fourth identification step involves converting the position of the intersection point into an angle, and identifying the angle with the highest number of detected intersection points as the angle of the needle portion of the analog meter in the partial image. An information processing method characterized by comprising an acquisition step of acquiring a value indicated by the analog meter based on the angle of the needle portion of the specified analog meter.

9. An image acquisition step involves acquiring an image captured by an imaging means that images an analog meter that indicates a value by a rotating needle, and An extraction step is performed to extract a partial image including the analog meter from the image acquired in the image acquisition step, A first identification step involves searching for the circular shape and the center point of the circle within the aforementioned partial image, and identifying the circle with the largest area among the circular shapes obtained by the search as the scale portion of the analog meter. A second identification step involves identifying the center point of the circular shape with the largest area, which corresponds to the scale portion of the analog meter, as the rotation center point of the needle of the analog meter. A third identification step involves using the binarized image of the partial image to search for the intersection points of multiple circles centered on the rotation center of the needle and the needle portion of the analog meter, thereby identifying the needle portion of the analog meter in the partial image. A fourth identification step involves converting the position of the intersection point into an angle, and identifying the angle with the highest number of detected intersection points as the angle of the needle portion of the analog meter in the partial image. A program for causing a computer to perform an acquisition step of obtaining the value indicated by the analog meter based on the angle of the needle of the analog meter that has been identified.

Images