Information processing system, information processing device, information processing method, and program

Abstract

To accurately detect a state of a lamp from an image including the lamp installed, the image being captured by an imaging apparatus.SOLUTION: An image processing system is configured to: clip a lamp part from an image including a lamp that can switch lighting state, the image being captured by an imaging apparatus; perform conversion processing on pixel values of pixels included in the clipped lamp part into a color space including hues; and determine a state of the lamp on the basis of values of the hues of an image of the lamp part.SELECTED DRAWING: Figure 4

Description

Technical Field

[0005]

[0001] The present invention relates to an information processing system, an information processing apparatus, an information processing method, and a program for determining the lighting state of a lamp from an image.

Background Art

[0002] Lamps such as LED (Light Emitting Diode) lamps, which are display devices, are installed and used in control panels, operation panels, instrumentation racks, etc. As technologies for determining the lighting or extinguishing state of a lamp from an image captured by a camera or the like, the technologies described in Patent Documents 1 and 2 are disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to accurately detect the state of a lamp from an image captured by an imaging device and including the installed lamp.

Means for Solving the Problems

[0005] The information processing system according to the present invention includes imaging means for imaging a lamp whose lighting state can be switched, extraction means for extracting a partial image including the lamp from the image captured by the imaging means by template matching, and based on coordinate information indicating a preset position of the lamp from the partial image including the lamp extracted by the extraction means, the The part occupied by the lamp.The device is characterized by comprising: an extraction means for cutting out a lamp portion; a conversion means for performing a conversion process on the pixel values ​​of each pixel included in the extracted lamp portion into a color space including hue; an average value calculation means for calculating the average value of the hue of the image of the lamp portion converted into a color space including hue; and a determination means for calculating the distance between a first reference value for lamp illumination and a second reference value for lamp deactivation, and the average value of the hue of the image of the lamp portion, and determining whether the lamp is illuminated or deactivated in association with the reference value with the smaller distance. [Effects of the Invention]

[0006] According to the present invention, it is possible to accurately detect the state of a lamp from an image including the installed lamp, which is captured by an imaging device. [Brief explanation of the drawing]

[0007] [Figure 1] This figure shows an example of the configuration of the information processing system in this embodiment. [Figure 2] This figure shows an example of the hardware configuration of the server device in this embodiment. [Figure 3] This figure shows an example of the functional configuration of the server device in this embodiment. [Figure 4] This flowchart shows an example of the lamp status detection process in this embodiment. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 shows an example of the configuration of an information processing system according to one embodiment of the present invention. Figure 1 shows an example in which the information processing system of this embodiment is applied to a remote monitoring system that enables remote equipment monitoring, etc. Note that the example shown in Figure 1 is just one example, and the information processing system of this embodiment is not limited to remote monitoring systems but can be applied to other systems as well.

[0009] The server device 101 is an information processing device that acquires data related to the status of a remotely monitored target based on images captured of the target. The server device 101 performs image analysis using the image of the control panel 106 captured by the imaging device 102 to acquire the status of the display components installed on the control panel 106. In this embodiment, the server device 101 acquires the status of lamps such as LED lamps installed on the control panel 106 that can be switched on or off (on / off, color when lit, etc.). In this example, the target is the control panel 106, but it may be other operation panels or instrumentation racks, and the target can be any lamp installed on any component that can be imaged by the imaging device 102.

[0010] The imaging device 102 is an imaging device such as a camera, and is installed in a position where it can image the control panel 106 (at least a position where the target lamp is included in the field of view). The imaging device 102 is connected to the server device 101 via a communication line 103 such as a carrier line. The imaging device 102 images the control panel 106 at predetermined time intervals, such as every minute or every five minutes, and transmits the image of the control panel 106 to the server device 101 via the communication line 103.

[0011] Furthermore, the imaging device 102 may be configured to perform imaging operations at any time in response to imaging instructions or other user commands. Also, the imaging device 102 and the server device 101 are not limited to being connected by a communication line 103; they may be connected via a network or the like, as long as the connection allows the imaging device 102 to transmit images it has captured to the server device 101. In addition, although Figure 1 illustrates one imaging device 102, multiple imaging devices may be connected to the server device 101.

[0012] Furthermore, in the information processing system of this embodiment, the server device 101 is connectable to terminal devices 104 used by users, etc., via a network 105 such as the Internet or a LAN (Local Area Network). The terminal devices 104 consist of, for example, a personal computer (PC), a tablet device, a smartphone, etc. The server device 101 provides, for example, data related to the status of the monitored object in response to a request from the terminal device 104. In addition, the server device 101 may be configured to notify the terminal device 104 if the acquired status of the monitored object meets predetermined conditions.

[0013] Figure 2 shows an example of the hardware configuration of the server device 101 in this embodiment. The server device 101 has a CPU (Central Processing Unit) 201, main memory 202, auxiliary storage 203, input I / F 204, output I / F 205, and communication I / F 206. The CPU 201, main memory 202, auxiliary storage 203, input I / F 204, output I / F 205, and communication I / F 206 are connected to each other via a system bus 207 so that they can communicate with one another.

[0014] The CPU 201 is a central processing unit that controls the server device 101. The main memory 202 is a storage device that functions as a work area for the CPU 201 and a temporary storage location for data. The main memory 202 is implemented using, for example, RAM (Random Access Memory). The auxiliary storage device 203 is a storage device that stores programs, image data, information related to various processes, etc. The auxiliary storage device 203 is implemented using, for example, ROM (Read Only Memory), hard disk drive (HDD), solid state drive (SSD), flash memory, etc.

[0015] The input I / F 204 is an interface used for inputting information from input devices such as a mouse and a keyboard. The output I / F 205 is an interface used for outputting information to output devices such as a display and a speaker. The communication I / F 206 is an interface used for communication with external devices via a network such as the Internet or a LAN, or for communication with an imaging device via a communication line such as a carrier line.

[0016] When the CPU 201 executes processing based on the program stored in the auxiliary storage device 203, each function of the server device 101 shown in FIG. 3 and each process shown in the flowchart of FIG. 4 are realized.

[0017] FIG. 3 is a diagram showing an example of the functional configuration of the server device 101 in the present embodiment. The server device 101 includes an image acquisition unit 301, an image storage unit 302, an image analysis unit 303, a determination unit 304, an information storage unit 305, and an output unit 306.

[0018] The image acquisition unit 301 acquires an image of the control panel 106 captured by the imaging device 102, specifically, an image of the control panel 106 including the target lamp. The image acquired by the image acquisition unit 301 is stored in the image storage unit 302.

[0019] The image analysis unit 303 performs image analysis on the image stored in the image storage unit 302 and outputs an analysis result. In the present embodiment, the image analysis unit 303 performs the lamp state detection process shown in FIG. 4 described later using the image stored in the image storage unit 302, and acquires the state (lighting / extinguishing, color when lit, etc.) of the target lamp in the image and outputs it as an analysis result. The image analysis unit 303 is an example of an extraction means, a cutting-out means, a conversion means, and a determination means.

[0020] The determination unit 304 determines whether the analysis result obtained by the image analysis in the image analysis unit 303 satisfies a predetermined condition. For example, the determination unit 304 determines whether the state of the lamp obtained as the analysis result is lighting in a predetermined color. The information storage unit 305 stores the analysis result obtained by the image analysis in the image analysis unit 303, the determination result by the determination unit 304, and the like.

[0021] The output unit 306 outputs the information obtained in the server device 101 and the like to the terminal device 104 and the like. For example, the output unit 306 provides the analysis result (lighting / extinguishing of the lamp, color at the time of lighting, etc.) obtained by the image analysis to the terminal device 104 in response to a request from the terminal device 104. Thereby, the terminal device 104 can display and present the analysis result to the user who uses it. Further, for example, when the determination unit 304 determines that a predetermined condition is satisfied, the output unit 306 may notify the terminal device 104 by e-mail, push notification, or the like. Thereby, it is possible to notify the user who uses the terminal device 104 that a predetermined condition is satisfied. Further, the output unit 306 may provide the image stored in the image storage unit 302 corresponding to the analysis result obtained by the image analysis to the terminal device 104.

[0022] FIG. 4 is a flowchart showing an example of the lamp state detection process performed by the server device 101 in the present embodiment. The lamp state detection process shown in FIG. It is executed every time the server device 101 receives an image including the control panel 106 captured by the imaging device 102.

[0023] In step S401, the image acquisition unit 301 acquires an image of the control panel 106 including the target lamp captured by the imaging device 102. The image of the control panel 106 acquired by the image acquisition unit 301 is stored in the image storage unit 302.

[0024] In step S402, the image analysis unit 303 extracts a partial image containing the target lamp from the image of the control panel 106 acquired in step S401 by template matching using a pre-registered template image. This partial image includes the lamp and its surrounding area.

[0025] In step S403, the image analysis unit 303 cuts out the lamp portion from the partial image extracted in step S402, based on coordinate information indicating the lamp's position. The coordinate information indicating the lamp's position is pre-set by a configuration file or the like.

[0026] In step S404, the image analysis unit 303 performs a color space conversion process on the image of the lamp portion extracted in step S403. Specifically, the image analysis unit 303 performs a conversion process on the pixel value of each pixel included in the lamp portion, from the RGB color space, which represents pixel values ​​using the elements red (R), green (G), and blue (B), to the HSV color space, which represents pixel values ​​using the elements hue (H), saturation (S), and lightness (V or B). By performing this color space conversion process, it becomes possible to determine the color based only on the hue (H) value, rather than on the combination of the values ​​of red (R), green (G), and blue (B).

[0027] In step S405, the image analysis unit 303 calculates the average value of hue (H) for the image of the lamp portion that was converted to the HSV color space in step S404.

[0028] In step S406, the image analysis unit 303 determines the state of the lamp based on the average value of the hue (H) of the image of the lamp portion calculated in step S405. The image analysis unit 303 determines whether the lamp is lit or unlit based on the average value of the hue (H). For example, the image analysis unit 303 compares the average value of the hue (H) with a first value indicating the lamp is lit and a second value indicating the lamp is unlit, and determines whether the lamp is lit or unlit depending on which value it is closer to. Furthermore, if the image analysis unit 303 determines that the lamp is lit, it may also determine the color of the lamp in question, for example, by setting the color corresponding to the average value of the hue (H) as the color of the lamp in question.

[0029] In step S407, the image analysis unit 303 stores the status of the target lamp (on / off, color when lit, etc.) acquired in step S406 in the information storage unit 305, and terminates the lamp status detection process.

[0030] As described above, according to this embodiment, a partial image containing the target lamp is extracted from the image of the control panel 106 captured by the imaging device 102 by template matching, and the lamp portion is cut out from the extracted partial image. Then, the pixel values ​​of each pixel included in the cut-out lamp portion are converted to the HSV color space, and the lamp state is determined based on the average value of the hue (H) of the converted image. In this way, by converting from a combination of primary colors to the HSV color space, which is a more intuitive combination of numerical values, it becomes possible to accurately detect the lamp state from the image captured by the imaging device 102.

[0031] Furthermore, since the status of the lamps can be detected from the image of the control panel 106 captured by the imaging device 102, the status of the lamps installed on the control panel 106 (on / off, color when lit, etc.) can be checked even from a location far from where the control panel 106 is located, making it possible to remotely check the status of the monitored object. Modification work on the control panel itself to enable remote checks may be subject to constraints such as cost and permit conditions. However, according to this embodiment, since it is only necessary to install an imaging device so that the control panel can be imaged without performing any modification work on the control panel, it is possible to address situations where there are constraints on modification work on the control panel, and it is possible to easily enable remote checks while suppressing the costs and work involved in the introduction.

[0032] In the above explanation, a server device 101, located away from the control panel 106 and the imaging device 102 that captures images of it, performs the process of detecting the lamp status from the captured images. However, this is not the only option. For example, another server device installed inside or near the imaging device 102 may be equipped with a function corresponding to the image analysis unit 303, and the process of detecting the lamp status on the side where the control panel 106 is installed may be performed, and the detected lamp status may be transmitted to the server device 101.

[0033] It should be noted that the embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features. [Explanation of Symbols]

[0034] 101 Server device (information processing device) 102 Imaging device 103 Communication lines 104 Terminal device 105 Network 106 Control Panel 201 CPU 202 Main storage 203 Auxiliary storage device 204 Input Interface 205 Output I / F 206 Communication I / F 207 System Bus 301 Image acquisition unit 302 Image storage unit 303 Image Analysis Department 304 Judgment section 305 Information storage section 306 Output section

Claims

1. An imaging means for capturing images of a lamp whose illumination state can be switched, An extraction means for extracting a partial image including the lamp from the image captured by the imaging means by template matching, From the partial image including the lamp extracted by the extraction means, a cutting means cuts out the lamp portion, which is the portion occupied by the lamp, based on coordinate information indicating the position of the lamp that has been set in advance. A conversion means that performs a conversion process on the pixel values ​​of each pixel included in the extracted lamp portion into a color space including hue, An average value calculation means for calculating the average value of the hue of the image of the lamp portion converted to a color space including hue, An information processing system characterized by having a determination means that calculates the distance between a first reference value for determining whether a lamp is lit and a second reference value for determining whether a lamp is off, and the average value of the hue of the image of the lamp portion, and determines whether the lamp is lit or off by associating it with the reference value with the smaller distance.

2. The determination means is If the distance between the average value of the hue of the image of the lamp portion and the first reference value is smaller than the distance between the average value of the hue of the image of the lamp portion and the second reference value, it is determined that the lamp is lit, and the color of the lamp is determined based on the average value of the hue of the image of the lamp portion. The information processing system according to claim 1, characterized in that if the distance between the average value of the hue of the image of the lamp portion and the second reference value is smaller than the distance between the average value of the hue of the image of the lamp portion and the first reference value, it is determined that the lamp is off.

3. The information processing system according to claim 1 or 2, characterized in that the color space including hue is an HSV color space.

4. Image acquisition means that acquires an image captured by an imaging means that images a lamp whose illumination state can be switched, An extraction means for extracting a partial image including the lamp from the image acquired by the image acquisition means by template matching, From the partial image including the lamp extracted by the extraction means, a cutting means cuts out the lamp portion, which is the portion occupied by the lamp, based on coordinate information indicating the position of the lamp that has been set in advance. A conversion means that performs a conversion process on the pixel values ​​of each pixel included in the extracted lamp portion into a color space including hue, An average value calculation means for calculating the average value of the hue of the image of the lamp portion converted to a color space including hue, An information processing device characterized by having a determination means that calculates the distance between a first reference value for determining whether a lamp is lit and a second reference value for determining whether a lamp is off, and the average value of the hue of the image of the lamp portion, and determines whether the lamp is lit or off by associating it with the reference value with the smaller distance.

5. An image acquisition step in which an image is acquired by an imaging means that images a lamp whose illumination state can be switched, An extraction step is performed to extract a partial image including the lamp from the image acquired in the image acquisition step by template matching, From the partial image including the lamp extracted in the extraction step, a cutting step is performed to cut out the lamp portion, which is the portion occupied by the lamp, based on coordinate information indicating the position of the lamp that has been set in advance. A conversion step which involves performing a conversion process on the pixel values ​​of each pixel included in the extracted lamp portion to a color space that includes hue, An average value calculation step for calculating the average value of the hue of the image of the lamp portion converted to a color space including hue, An information processing method characterized by comprising a determination step of calculating the distance between a first reference value for determining whether a lamp is lit and a second reference value for determining whether a lamp is off, and the average value of the hue of the image of the lamp portion, and determining whether the lamp is lit or off by associating it with the reference value with the smaller distance.

6. An image acquisition step involves acquiring an image captured by an imaging means that images a lamp whose illumination state can be switched, and An extraction step is performed to extract a partial image containing the lamp from the image acquired in the image acquisition step by template matching, From the partial image including the lamp extracted in the extraction step, a cutting step is performed to cut out the lamp portion, which is the portion occupied by the lamp, based on coordinate information indicating the position of the lamp that has been set in advance. A conversion step which involves performing a conversion process on the pixel values ​​of each pixel included in the extracted lamp portion to a color space that includes hue, An average value calculation step for calculating the average value of the hue of the image of the lamp portion converted to a color space including hue, A program for causing a computer to perform a determination step of calculating the distance between a first reference value for determining whether a lamp is lit and a second reference value for determining whether a lamp is off, and the average value of the hue of the image of the lamp portion, and determining whether the lamp is lit or off in association with the reference value with the smaller distance.

Images