Image monitoring device, method and program

Abstract

To provide an image monitoring device, method, and program which can automatically detect that display on a display has not been changed due to abnormality of a device for display on the display.SOLUTION: An image monitoring device 1 relating to the present disclosure comprises: an acquisition unit 2 which acquires periodically captured images of a display for use in a signage system; a specification unit 3 which specifies display images included in the acquired images and displayed on the display; and a determination unit 4 which determines a display state of the display on the basis of a result of comparison between the display images specified by the specification unit 3 with respect to each of the plurality of images captured at different times.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an image monitoring device, method, and program. [Background technology]

[0002] In recent years, digital signage has been used as a new form of advertising and in-store display. Digital signage can display multiple contents and videos on a single display by displaying images on the screen, and can be used in a wider range of applications than display methods that use media such as paper. However, problems can occur due to display malfunctions, such as the images you want to display not being displayed. Malfunctions can occur due to a variety of factors, including malfunctions of the display, the terminal device that controls the display, or network equipment.

[0003] Not only does the inability to display images or the like on a display reduce the value of the advertisement using the display, but when multiple displays are used to create a single display format, for example, even if only one display is not displaying, the appearance value of the advertisement is significantly reduced, etc. For this reason, it is desirable to automatically detect the malfunction as soon as possible and notify the administrator.

[0004] Patent Document 1 discloses a technique for detecting freezes and black images in a video by extracting images at predetermined intervals from the video. Patent Document 2 discloses a technique for monitoring an image displayed on a display device to determine whether the image is actually being displayed. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2019-212970 A [Patent Document 1] JP 2011-135477 A Summary of the Invention [Problem to be solved by the invention]

[0006] According to a method of detecting changes in the actual display by extracting images from a video at a predetermined interval, that is, by using captured images, the actual display and the captured image do not necessarily match. This is because the screen is captured by the device performing the display processing, and the image is not necessarily displayed correctly on the display, and if the display is broken, the actual display and the captured image may not match. Furthermore, if an abnormality occurs on the screen due to a breakdown in the device performing the display processing, a captured image cannot be generated.

[0007] The present disclosure has been made to solve such problems, and aims to provide an image monitoring device, method, and program that can automatically detect when the display no longer changes due to an abnormality in the equipment used to display the information on the display. [Means for solving the problem]

[0008] The image monitoring device according to the present disclosure comprises an acquisition unit that acquires images periodically captured of a display used in a signage system, an identification unit that identifies a display image contained in the acquired images and displayed on the display, and a determination unit that determines the display state of the display based on the results of comparing the display images identified by the identification unit in each of a plurality of images captured at different times.

[0009] The image monitoring method disclosed herein comprises the steps of acquiring images periodically captured of a display used in a signage system, identifying a display image contained in the acquired images and displayed on the display, and determining the display state of the display based on the results of comparing the identified display images for each of a number of images captured at different times.

[0010] The image monitoring program of the present disclosure is an image monitoring program that causes a computer to execute a process of acquiring images that are periodically photographed of a display used in a signage system, a process of identifying a display image that is displayed on the display and is included in the acquired images, and a process of determining the display state of the display based on the results of comparing the identified display images in each of a number of images that were taken at different times. Effect of the Invention

[0011] According to the present disclosure, it is possible to provide an image monitoring device, method, and program that can automatically detect when a display no longer changes due to an abnormality in the equipment used to display the image on the display. [Brief description of the drawings]

[0012] [Figure 1] 1 is a configuration diagram of an image monitoring device according to a first embodiment of the present disclosure. [Diagram 2] FIG. 11 is a configuration diagram of a display abnormality detection device according to a second embodiment of the present disclosure. [Diagram 3] FIG. 11 is a diagram showing an outline of an image generated by performing mask processing on an image acquired by a camera according to a second embodiment of the present disclosure. [Figure 4] FIG. 11 is a diagram showing an image of calculating a difference value using a comparison image obtained by multiplexing a mask image according to the second embodiment of the present disclosure. [Diagram 5] FIG. 11 is a diagram showing an image of installation of a camera for photographing a display according to a second embodiment of the present disclosure. [Figure 6] FIG. 11 is a flow diagram of an abnormality determination process performed by an abnormality determination unit according to the second embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the embodiments will be described with reference to the drawings. Note that the drawings are simplified, and the technical scope of the embodiments should not be interpreted narrowly based on the description in the drawings. Also, the same elements are given the same reference numerals, and duplicated explanations will be omitted.

[0014] <Embodiment 1> The configuration of an image monitoring device 1 in this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram of an image monitoring device 1 according to this embodiment. The image monitoring device 1 comprises an acquisition unit 2, an identification unit 3, and a determination unit 4.

[0015] The acquisition unit 2 acquires images captured periodically of a display used in the signage system. "Periodic" also includes images not captured at regular time intervals. The identification unit 3 identifies a display image that is included in the images acquired by the acquisition unit 2 and is displayed on the display. The judgment unit 4 judges the display state of the display based on the results of comparing the display images identified by the identification unit 3 for each of a plurality of images captured at different times.

[0016] According to the image monitoring device 1 of this embodiment, it is possible to automatically detect when no changes occur in the display due to an abnormality in the device for displaying information on the display.

[0017] <Embodiment 2> The configuration of the display abnormality detection device 10 in this embodiment will be described with reference to Fig. 2. Fig. 2 is a configuration diagram of the display abnormality detection device 10 in this embodiment. The display abnormality detection device 10 includes an image acquisition unit 11, an image storage unit 12, a difference calculation unit 13, an abnormality determination unit 14, and a reporting unit 15, and acquires images captured by a camera 20. The display abnormality detection device 10 automatically detects when a change no longer occurs in the image on a display 16 used in a signage system.

[0018] The image acquisition unit 11 acquires images captured by the camera 20 periodically on the display 16. "Periodically" may be, for example, the number of seconds at which the still images are switched when still images are switched on the display 16 every few seconds, or may be set arbitrarily. Furthermore, "periodically" also includes not being at a fixed time interval. It is preferable to set the timing at which the image acquisition unit 11 acquires images from the camera 20 so that the same still image is not acquired more than twice.

[0019] In order to eliminate the influence of an image captured from outside the display area of ​​display 16, image acquisition unit 11 identifies the display image displayed on display 16 that is included in the acquired image. The identification method may involve performing mask processing on the image acquired from camera 20. Image acquisition unit 11 superimposes a frame that surrounds an area including the display image displayed on display 16 on the image of display 16 captured by camera 20, and masks the peripheral portion of the image of display 16 from the frame.

[0020] Here, an overview of an image generated by performing mask processing on an image acquired from camera 20 will be described with reference to FIG. 3. FIG. 3 is a diagram showing an overview of an image generated by performing mask processing on an image acquired from camera 20. As shown in FIG. 3, image acquisition unit 11 generates a mask image by filling in the outside of a trapezoidal frame formed by connecting four preset points with straight lines with black. Note that the mask image is not limited to a shape connecting four points with straight lines, and various shapes such as polygons may be used. The mask image generated in image acquisition unit 11 is used as a comparison image in difference calculation unit 13, as described later.

[0021] The image storage unit 12 stores the image acquired by the image acquisition unit 11 and displayed on the display 16 as a comparison image. The image storage unit 12 may store a comparison image obtained by masking the image displayed on the display 16. The image storage unit 12 stores two images, the acquired comparison image and the comparison image acquired immediately before that. When a new image is saved, the image storage unit 12 may erase the older of the two stored images.

[0022] The difference calculation unit 13 compares the display images on the display 16 included in each of the two images stored in the image storage unit 12, and calculates a difference value. The difference value is a total value of differences in pixel values ​​obtained by calculating component values ​​corresponding to different colors in pixels of the two display images. The pixel values ​​may use each color component of RGB. When RGB values ​​range from 0 to 255, for example, white is expressed as RGB=(255, 255, 255). The difference value may be calculated based on the difference between each pixel of the two images using the pixel values ​​of each color component of RGB. The difference value may be calculated from all pixels in the comparison image, or may be calculated from only pixels extracted at equal intervals, for example.

[0023] Here, an overview of the process performed by the difference calculation unit 13 to calculate a difference value using a comparison image that has been subjected to mask processing will be described with reference to Fig. 4. Fig. 4 is a diagram showing an image of calculating a difference value using a comparison image that has been subjected to mask processing. As shown in Fig. 4, a comparison can be made to calculate a difference value for unmasked areas of the image acquired at the previous timing and the image acquired this time. This is because the difference value of the masked area is 0.

[0024] The difference calculation unit 13 can calculate the difference value only for the unmasked area included in the previous acquired image and the current acquired image. The difference value calculated by the difference calculation unit 13 is a value that excludes the influence of reflection of the image captured by the camera 20 onto the image displayed on the display 16.

[0025] The abnormality determination unit 14 determines whether or not the screen displayed on the display 16 is changing normally based on the difference value calculated by the difference calculation unit 13. The abnormality determination unit 14 notifies the notification unit 15 of the determination result. Details of the abnormality determination process performed by the abnormality determination unit 14 will be described later.

[0026] The reporting unit 15 receives a notification from the abnormality determination unit 14 of the result of determining whether the screen displayed on the display 16 has changed normally, and reports the result. Various means can be used for reporting, such as a textual means such as an email to the administrator, a voice means, and a display means.

[0027] An image of the installation of the camera 20 on the display 16 is shown in FIG. 5. As shown in FIG. 5, the camera 20 is installed facing the display 16 and can capture an image displayed on the display 16. As shown in FIG. 5, the camera 20 captures an image displayed on the display 16 directly from diagonally above the display 16, but it is only necessary that the camera 20 can capture images of the display 16 from various directions, not limited to diagonally above. Depending on the installation location of the display 16, a situation in which people pass in front of the display 16 is also conceivable. Therefore, in this embodiment, the camera 20 is fixed to the upper part of the display 16 and is installed at an angle that looks down on the display 16 from diagonally above. The camera 20 may also be provided with a polarizing filter on the lens in order to reduce reflection of lighting in the shooting environment on the display 16.

[0028] Here, the abnormality determination process performed by the abnormality determination unit 14 will be described with reference to Fig. 6. Fig. 6 is a flow diagram of the abnormality determination process performed by the abnormality determination unit 14. α is a threshold value of the difference value for determining whether or not the screen displayed on the display 16 is changing normally. β is a threshold value of the abnormal count number, which is the number of times when the difference value is the threshold value α, i.e., the number of times when it is estimated that the image has not changed. The threshold values ​​α and β can be set arbitrarily.

[0029] The abnormality determination unit 14 acquires the difference value calculated by the difference calculation unit 13 (step 101). When the difference value between the previously acquired image and the currently acquired image exceeds the threshold value α, the abnormality determination unit 14 determines that the screen has changed normally (NO in step 102), and clears the abnormality count number to 0 (step 103). When the calculated difference value is equal to or less than the threshold value α (YES in step 102), the abnormality determination unit 14 increments the abnormality count number by 1, assuming that the screen has not changed (step 104). When the abnormality count number is less than the threshold value β (NO in step 105), the abnormality determination unit 14 holds the current abnormality count number and waits until the next difference value is input. When the abnormality count number is equal to or more than the threshold value β (YES in step 105), the abnormality determination unit 14 determines that an abnormality has occurred on the screen displayed on the display 16, notifies the notification unit 15 of the occurrence of the abnormality (step 106), and ends the process.

[0030] As described above, the display abnormality detection device 10 detects a change in the image displayed on the display 16 based on an image captured by the camera 20 installed outside the display 16. Therefore, it is possible to suppress the processing of the display abnormality detection device 10 from affecting the display processing of the display 16 performed by other devices. In addition, the display abnormality detection device 10 detects an abnormality based on the difference between the images captured by the camera 20. Therefore, the display abnormality detection device 10 does not need to store the moving images or still images displayed on the display 16 in advance, and does not require the effort of storing a huge number of images in advance to compare with the image actually displayed. In addition, the camera 20 does not need to have a high resolution and does not need to have a function such as distortion correction.

[0031] According to the display abnormality detection device 10 of the present embodiment, it is possible to automatically detect when no change occurs in the display 16 due to an abnormality in the device for displaying on the display 16.

[0032] <Embodiment 3> The difference calculation unit 13 of the display abnormality detection device 10 in this embodiment divides an image into a plurality of regions, and for each divided region, compares the image acquired by the image acquisition unit 11 with the image stored in the image storage unit 12 to calculate a difference value. The abnormality determination unit 14 determines the presence or absence of a display abnormality on the display 16 based on the difference value calculated by the difference calculation unit 13.

[0033] Therefore, the abnormality determination unit 14 compares the calculated difference value with a difference threshold value α set for each corresponding region in the previously acquired image and the currently acquired image. The abnormality determination unit 14 determines that the image displayed on the display 16 has changed normally only when the differences for all regions included in each image exceed the threshold value α.

[0034] By configuring in this manner, the display abnormality detection device 10 in this embodiment can detect an abnormality even when an abnormality occurs in only a part of the image displayed on the display 16.

[0035] <Other embodiments> The image monitoring device 1 in the present disclosure includes, for example, an embodiment as an image monitoring method. That is, the image monitoring method includes a step of acquiring images periodically captured of a display used in a signage system, a step of identifying a display image included in the acquired images and displayed on the display, and a step of determining a display state of the display based on a result of comparing the identified display images in each of a plurality of the images captured at different times.

[0036] In the above example, the program can be stored and provided to the computer using various types of non-transitory computer readable media. The non-transitory computer readable media include various types of tangible storage media. The non-transitory computer readable media include, for example, magnetic recording media, optical magnetic recording media, CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory. The semiconductor memory is, for example, a mask ROM, PROM (Programmable ROM), EPROM, flash ROM, RAM (Random Access Memory), etc. The program may also be provided to the computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include an electric signal, an optical signal, and an electromagnetic wave. The transitory computer readable medium can provide the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

[0037] The above program is an image monitoring program that causes a computer to execute the following processes: acquiring images periodically captured of a display used in a signage system; identifying a display image contained in the acquired images and displayed on the display; and judging the display state of the display based on the results of comparing the identified display images in each of a number of images captured at different times.

[0038] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0039] 1. Image monitoring equipment 2 Acquisition part 3 Specific part 4 Judgment section 10. Display abnormality detection device 11 Image acquisition section 12 Image storage section 13 Difference calculation part 14 Abnormality determination section 15 Reporting Department 16 Display 20 Camera

Claims

1. an acquisition unit that acquires images of a display periodically using a camera installed above the display at an angle that faces the direction of the display used in the signage system and looks down on the display from diagonally above; an identification unit that identifies a display image displayed on the display and that is included in the acquired image; a determination unit that determines a display state of the display based on a result of comparing the display images identified by the identification unit in each of the plurality of images captured at different times; Equipped with the identification unit superimposes a trapezoidal frame having an upper base longer than a lower base, surrounding an area including the display image displayed on the display, on the captured image of the display, and performs mask processing on a peripheral portion of the image of the display from the frame. Image surveillance equipment.

2. the determination unit compares the display images included in the two images captured at different times to calculate a difference value, and determines the display state based on the calculated difference value.

2. The image monitoring device according to claim 1.

3. The determination unit calculates the number of times that the difference value is equal to or less than a threshold, and determines that a display abnormality exists in the display when the number of times is equal to or greater than the threshold.

3. An image monitoring device according to claim 2.

4. the determination unit calculates, as a difference value, a total value of differences between pixel values ​​obtained by calculating component values ​​corresponding to different colors in pixels in areas corresponding to the two display images; 4. An image monitoring device according to claim 2 or 3.

5. the determination unit divides each of the two display images into a plurality of regions, and determines the presence or absence of a display abnormality on the display by calculating the difference value for each corresponding region. An image monitoring device according to any one of claims 2 to 4.

6. A step of periodically acquiring images of a display used in a signage system using a camera installed above the display at an angle looking down on the display from above so as to face the direction of the display; Identifying a display image displayed on the display included in the acquired image; determining a display state of the display based on a result of comparing the identified display images in each of the plurality of images captured at different times; Equipped with In the step of specifying, a trapezoidal frame having an upper base longer than a lower base and surrounding an area including the display image displayed on the display is superimposed on the captured image of the display, and a peripheral portion of the image of the display is masked from the frame. Image monitoring method.

7. A process of acquiring images of a display periodically taken by a camera installed above the display at an angle facing the direction of the display used in the signage system and looking down on the display from above; A process of identifying a display image displayed on the display included in the acquired image; determining a display state of the display based on a result of comparing the identified display images in each of the plurality of images captured at different times; Run the following on your computer: In the process of specifying, a trapezoidal frame having an upper base longer than a lower base and surrounding an area including the display image displayed on the display is superimposed on the captured image of the display, and a peripheral portion of the image of the display is masked from the frame. Image surveillance program.

Images