Video monitoring device, method, and program

Abstract

This determination unit determines the necessity of monitoring a received video. A display unit displays the received video in time series for each elapsed time, and displays the video displayed for each elapsed time in a different mode on the basis of the monitoring necessity.

Description

Video monitoring device, method and program

[0001] The present disclosure relates to a video monitoring device, a video monitoring method, and a video monitoring program for monitoring video.

[0002] It is common to monitor video footage captured in real time and take appropriate action. For example, real-time video footage from an autonomous vehicle is displayed on a surveillance monitor, and a monitor closely watches the video footage, judging any danger and issuing instructions on how to deal with the situation.

[0003] For example, Patent Document 1 describes a method for displaying images used for remotely controlling a vehicle. In the method described in Patent Document 1, when remotely controlling a vehicle, video data captured in various directions relative to the vehicle is received and displayed.

[0004] Japanese Patent Application Laid-Open No. 2020-161039

[0005] Dangerous situations generally change depending on the surrounding environment. Therefore, when monitoring using captured video, it is preferable to be able to determine the need for monitoring not only from a temporary perspective but also from a continuous perspective.

[0006] Therefore, an object of the present disclosure is to provide a video monitoring device, a video monitoring method, and a video monitoring program that can determine the need to monitor captured video from a continuous perspective.

[0007] The video monitoring device according to the present disclosure is characterized by having a determination unit that determines the need for monitoring of the received video, and a display unit that displays the received video in chronological order according to the elapsed time, and displays the video displayed according to the elapsed time in different ways based on the need for monitoring.

[0008] The video monitoring method disclosed herein determines the need for monitoring received video, displays the received video in chronological order by elapsed time, and displays the video displayed by elapsed time in different ways based on the need for monitoring.

[0009] The video monitoring program according to the present disclosure is characterized in that it causes a computer to execute a determination process for determining the need for monitoring of received video, and a display process for displaying the received video in chronological order according to elapsed time, and displaying the video displayed according to elapsed time in different ways based on the need for monitoring.

[0010] The present disclosure allows for continuous monitoring of dangerous situations on video.

[0011] It is a block diagram showing an example of the configuration of an embodiment of a video monitoring device according to the present disclosure.It is a flowchart showing an example of the operation of a video monitoring device, which is an explanatory diagram showing an example of a mode of displaying video.It is a block diagram showing an overview of a video monitoring device according to the present disclosure.It is a schematic block diagram showing the configuration of a computer according to at least one embodiment.

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0013] 1 is a block diagram showing an example of the configuration of an embodiment of a video monitoring device according to the present disclosure. The video monitoring device 100 according to this embodiment includes a storage unit 10, a receiving unit 20, a determining unit 30, and a display unit 40. The video monitoring device 100 according to this embodiment is connected to a mobile object 200 so as to be able to communicate with the mobile object 200.

[0014] The mobile object 200 captures video using a camera or the like mounted on the mobile object 200 and transmits the captured video to the video monitoring device 100. The video captured by the mobile object 200 is arbitrary, and may be, for example, video of the surroundings of the mobile object 200. In addition, the mobile object 200 may transmit location information indicating its own position and attribute information of its own to the video monitoring device 100 as information accompanying the video. Furthermore, if the mobile object 200 can detect an abnormality by itself, the mobile object 200 may transmit abnormality information indicating the detected abnormality to the video monitoring device 100.

[0015] In the following description, a case where the moving body 200 is a vehicle will be exemplified. However, the moving body 200 is not limited to a vehicle and may be, for example, a drone. Furthermore, although the example shown in FIG. 1 exemplifies a case where there are two moving bodies 200, the number of moving bodies 200 may be one, or three or more.

[0016] The storage unit 10 stores data required when the video monitoring device 100 of this embodiment performs various processes. The storage unit 10 of this embodiment also stores various types of information transmitted from a mobile object. The various types of information transmitted from a mobile object include, for example, video, location information, attribute information, and abnormality information, as described above. The storage unit 10 is realized, for example, by a magnetic disk or the like.

[0017] The receiving unit 20 receives various information from the mobile object and stores it in the storage unit 10 .

[0018] The determination unit 30 determines whether or not monitoring of the video received from the moving object is necessary. The viewpoint of the necessity of monitoring is arbitrary, and various viewpoints can be taken into consideration. The viewpoint of the necessity of monitoring can be, for example, the degree of danger (hereinafter referred to as the danger level). For example, the sudden approach of another object, poor visibility, etc.

[0019] Another aspect of the need for monitoring is the degree to which something differs from the normal state (hereinafter referred to as the degree of abnormality). For example, an object that normally does not exist is discovered, or something that normally exists has disappeared. Another aspect of the need for monitoring is the degree to which the state changes (hereinafter referred to as the degree of change). For example, changes in traffic conditions or speed changes.

[0020] Any method can be used to determine indicators such as the level of danger, the level of abnormality, and the degree of change. For example, an indicator for received video may be determined using a trained model (video AI: Artificial Intelligence) for determining these indicators from video. Note that methods for determining the level of danger, the level of abnormality, and the degree of change from video are widely known, and therefore detailed description thereof will be omitted here.

[0021] The determination unit 30 may determine that monitoring is necessary when any of the indicators exceeds a predetermined threshold. Furthermore, the determination unit 30 may determine the necessity of monitoring in stages, rather than just determining whether monitoring is necessary or not. Examples of the stages of monitoring necessity include "no monitoring necessity," "low monitoring necessity," and "high monitoring necessity (visual confirmation required)" according to the indicator.

[0022] Furthermore, the determining unit 30 may use the above-described index itself as the necessity of monitoring.

[0023] The display unit 40 displays the received video in chronological order for each elapsed time. Furthermore, the display unit 40 displays the video displayed for each elapsed time in different formats based on the necessity for monitoring. It is preferable that the display unit 40 displays video with a higher monitoring necessity in a format that attracts more attention.

[0024] 2 is an explanatory diagram showing an example of a manner in which images are displayed. In the example shown in FIG. 2, the display unit 40 displays images captured by a plurality of moving objects in chronological order for each moving object.

[0025] Specifically, Figure 2 shows that images for which the monitoring necessity is "no monitoring necessity" are displayed normally (displayed as white in Figure 2), images for which the monitoring necessity is "low" are displayed in a manner that attracts attention (displayed as lightly shaded in Figure 2), and images for which the monitoring necessity is "high" are displayed in a manner that attracts attention even more (displayed as darkly shaded in Figure 2).

[0026] 2, the image types are distinguished by black and white shading, but the display unit 40 may distinguish the image types by color display. For example, in the above example, an image with "low monitoring necessity" may be displayed in yellow, and an image with "high monitoring necessity" may be displayed in red.

[0027] Furthermore, when the necessity of monitoring is expressed by the above-mentioned index, the display unit 40 may display the image in a manner corresponding to the index.

[0028] When displaying images received from multiple moving objects, it is preferable that the display unit 40 display the images in a list sorted by elapsed time, as shown in Fig. 2. Displaying images in this manner improves visibility.

[0029] Furthermore, the display unit 40 may receive a designation from the monitor regarding the displayed video. In this case, the display unit 40 may enlarge and display the designated video, or may display the designated video on a display device (not shown) other than the video monitoring device 100.

[0030] Furthermore, the display unit 40 may register, in the storage unit 10, information that was attached to the designated video when it was received. More specifically, the display unit 40 may register, in the storage unit 10, location information that was attached to the designated video when it was received.

[0031] When such information is registered, the determination unit 30 may determine that video accompanied by information matching the registered information is in greater need of monitoring. Since video designated by a monitor is determined to be in greater need of monitoring, when video matching the information accompanying the designated video is received, it can be determined that the video should be monitored more closely. Note that a match here does not only mean an exact match, but also includes cases where the information is close (difference within a predetermined range).

[0032] For example, if the specified video is accompanied by location information, the display unit 40 registers the location information as caution point location information in the storage unit 10. It can be determined that the video of the location indicated by the registered caution point location information should be monitored more, even if it is a different video. Therefore, the determination unit 30 may determine that the video accompanied by information matching the registered location information is more likely to be monitored.

[0033] Furthermore, the determination unit 30 may determine that a video accompanied by information exceeding a predetermined number of registered items is in higher need of monitoring, because the more designated the video, the higher the need for monitoring.

[0034] The receiving unit 20, the determining unit 30, and the display unit 40 are realized by a processor (for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)) of a computer that operates according to a program (video monitoring program).

[0035] For example, the program may be stored in the storage unit 10 of the video monitoring device 100, and the processor may read the program and operate as the receiving unit 20, the determining unit 30, and the display unit 40 in accordance with the program. Also, the functions of the video monitoring device 100 may be provided in a SaaS (Software as a Service) format.

[0036] The receiving unit 20, the determining unit 30, and the display unit 40 may each be realized by dedicated hardware. Some or all of the components of each device may be realized by general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and a program.

[0037] Furthermore, when some or all of the components of the video monitoring device 100 are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or decentralized. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in a form in which each is connected via a communication network.

[0038] Next, the operation of the video monitoring device 100 of this embodiment will be described. Fig. 3 is a flowchart showing an example of the operation of the video monitoring device 100 of this embodiment. The determination unit 30 determines whether monitoring of the received video is necessary (step S11). The display unit 40 then displays the received video in chronological order by elapsed time, and displays the video displayed for each elapsed time in different formats based on the necessity of monitoring (step S12).

[0039] As described above, in this embodiment, the determination unit 30 determines whether or not the received video needs to be monitored. The display unit 40 then displays the received video in chronological order according to the elapsed time, and displays the video displayed for each elapsed time in a different format based on the need for monitoring. This makes it possible to determine the need for monitoring the captured video from a continuous perspective.

[0040] Next, an overview of the present disclosure will be described. Fig. 4 is a block diagram showing an overview of a video monitoring device according to the present disclosure. A video monitoring device 80 according to the present disclosure (e.g., video monitoring device 100) includes a determination unit 81 (e.g., determination unit 30) that determines whether or not monitoring of received video is necessary, and a display unit 82 (e.g., display unit 40) that displays the received video in chronological order by elapsed time and displays the video displayed for each elapsed time in different formats based on the necessity of monitoring.

[0041] Such a configuration allows for a continuous perspective to determine the need to monitor the captured footage.

[0042] The display unit 82 may also display the images received from a plurality of moving bodies in a time-elapsed order.

[0043] The display unit 82 may also receive a designation from the observer regarding the displayed image, and enlarge and display the designated image.

[0044] The display unit 82 may also enlarge and display the designated image on another display device.

[0045] In addition, the display unit 82 may register the information that accompanied the specified video when it was received in a memory unit (e.g., memory unit 10), and the judgment unit 81 may judge that video accompanied by information that matches the information is more in need of monitoring.

[0046] Specifically, the display unit 82 may register the location information that was attached when the specified video was received in the memory unit, and the judgment unit 81 may judge that the video that is attached with information that matches the location information is more in need of monitoring.

[0047] Furthermore, the determining unit 81 may determine that a video accompanied by information exceeding a predetermined number of registered items is in higher need of monitoring.

[0048] Furthermore, the display unit 82 may display an image that is more likely to be watched, the more likely it is to be watched.

[0049] Furthermore, the determining unit 81 may determine that monitoring is necessary when the degree of danger, which indicates the degree of danger, exceeds a predetermined threshold value.

[0050] Furthermore, the determining unit 81 may determine that monitoring is necessary when the degree of abnormality, which is the degree of difference from the normal state, exceeds a predetermined threshold value.

[0051] Furthermore, the determining unit 81 may determine that monitoring is necessary when the degree of change, which is the degree to which the state changes, exceeds a predetermined threshold value.

[0052] 5 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. The computer 1000 includes a processor 1001, a main memory device 1002, an auxiliary memory device 1003, and an interface 1004. The computer 1000 may also be connected to a computer that executes a mathematical programming solver, an annealing machine, a simulator, or the like.

[0053] The above-described video monitoring device 80 is implemented in a computer 1000. The operations of the above-described processing units are stored in the form of a program (video monitoring program) in an auxiliary storage device 1003. The processor 1001 reads the program from the auxiliary storage device 1003, loads it into the main storage device 1002, and executes the above-described processing in accordance with the program.

[0054] In at least one embodiment, the auxiliary storage device 1003 is an example of a non-transitory tangible medium. Other examples of non-transitory tangible media include a magnetic disk, a magneto-optical disk, a CD-ROM (Compact Disc Read-only memory), a DVD-ROM (Read-only memory), and a semiconductor memory connected via the interface 1004. In addition, when this program is distributed to the computer 1000 via a communication line, the computer 1000 that receives the program may load the program into the main storage device 1002 and execute the above-described processing.

[0055] The program may also be a program for realizing part of the above-described functions. Furthermore, the program may be a so-called differential file (differential program) that realizes the above-described functions in combination with another program already stored in the auxiliary storage device 1003.

[0056] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0057] (Supplementary Note 1) A video monitoring device comprising: a determination unit that determines the necessity of monitoring received video; and a display unit that displays the received video in chronological order according to elapsed time, and displays the video displayed according to elapsed time in different ways based on the necessity of monitoring.

[0058] (Supplementary Note 2) The video monitoring device according to Supplementary Note 1, wherein the display unit displays the videos received from the plurality of moving objects in a time-elapsed order.

[0059] (Supplementary Note 3) The video monitoring device according to Supplementary Note 1 or Supplementary Note 2, wherein the display unit receives a designation from the monitor regarding the displayed video, and enlarges and displays the designated video.

[0060] (Supplementary Note 4) The video monitoring device according to any one of Supplementary Note 1 to Supplementary Note 3, wherein the display unit displays the designated video in an enlarged scale on another display device.

[0061] (Appendix 5) A video surveillance device described in any one of Appendices 1 to 4, wherein the display unit registers information that was attached to the specified video when it was received in the memory unit, and the determination unit determines that video that is accompanied by information that matches the information is more in need of monitoring.

[0062] (Appendix 6) A video surveillance device as described in any one of Appendices 1 to 5, wherein the display unit registers in the memory unit location information that was attached when the specified video was received, and the determination unit determines that video accompanied by information that matches the location information is more in need of monitoring.

[0063] (Supplementary Note 7) The video monitoring device according to Supplementary Note 5 or Supplementary Note 6, wherein the determining unit determines that video accompanied by information exceeding a predetermined number of registered items is in higher need of monitoring.

[0064] (Supplementary Note 8) The video monitoring device according to any one of Supplementary Notes 1 to 7, wherein the display unit displays video in a manner that attracts more attention as the need for monitoring increases.

[0065] (Supplementary Note 9) A video monitoring device described in any one of Supplementary Notes 1 to 8, wherein the judgment unit judges that monitoring is necessary when the degree of danger, which indicates the degree of danger, exceeds a predetermined threshold.

[0066] (Supplementary Note 10) A video monitoring device described in any one of Supplementary Note 1 to Supplementary Note 9, wherein the judgment unit determines that monitoring is necessary when the degree of abnormality, which indicates the degree of difference from a normal state, exceeds a predetermined threshold.

[0067] (Supplementary Note 11) A video monitoring device described in any one of Supplementary Note 1 to Supplementary Note 10, wherein the judgment unit judges that monitoring is necessary when the degree of change, which is the degree to which the state changes, exceeds a predetermined threshold.

[0068] (Appendix 12) A video monitoring method characterized by determining the necessity of monitoring received video, displaying the received video in chronological order for each elapsed time, and displaying the video displayed for each elapsed time in a different manner based on the necessity of monitoring.

[0069] (Supplementary Note 13) The video monitoring method according to Supplementary Note 12, wherein the display unit displays the videos received from the plurality of moving objects in a time-elapsed order.

[0070] (Supplementary Note 14) A storage medium storing a video monitoring program for causing a computer to execute a determination process for determining the need for monitoring received video, and a display process for displaying the received video in chronological order at each elapsed time, and displaying the video displayed at each elapsed time in a different manner based on the need for monitoring.

[0071] (Supplementary Note 15) A storage medium according to Supplementary Note 14, which stores a video monitoring program for displaying, in a computer, images received from a plurality of moving objects in a time-elapsed order.

[0072] Although the present invention has been described above with reference to the embodiments and examples, the present invention is not limited to the above-described embodiments and examples. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0073] 10 Storage unit 20 Receiving unit 30 Determination unit 40 Display unit 100 Video monitoring device 200 Mobile object

Claims

1. A determination unit that determines the necessity of monitoring the received video, The system includes a display unit that displays received video in chronological order according to elapsed time, and displays the video displayed according to elapsed time in different ways based on the monitoring requirement. A video surveillance device characterized by the following features.

2. The display unit displays video received from multiple moving objects, aligned according to elapsed time. The video surveillance device according to claim 1.

3. The display unit receives requests from the monitor regarding the displayed video and enlarges the specified video. The video surveillance device according to claim 1 or claim 2.

4. The display unit enlarges and displays the specified image on another display device. The video surveillance device according to claim 1 or claim 2.

5. The display unit registers the information associated with the specified video in the storage unit. The determination unit determines that video footage accompanied by information matching the aforementioned information has a higher need for monitoring. The video surveillance device according to claim 1 or claim 2.

6. The display unit registers the location information associated with the specified video in the storage unit. The determination unit determines that video footage accompanied by information matching the location information has a higher need for monitoring. The video surveillance device according to claim 1 or claim 2.

7. The judgment unit determines that video footage with information exceeding a predetermined number of registered items requires a higher level of monitoring. The video surveillance device according to claim 5.

8. The display unit shows images that require more monitoring in a manner that is more likely to attract attention. The video surveillance device according to claim 1 or claim 2.

9. Determine the need to monitor the received video, The received video is displayed chronologically according to the elapsed time, and the video displayed according to the elapsed time is displayed in different ways based on the monitoring necessity. A video surveillance method characterized by the following features.

10. On the computer, A determination process to determine the necessity of monitoring the received video, and Display process that displays the received video in chronological order according to elapsed time, and displays the video displayed according to elapsed time in different ways based on the monitoring necessity. A video surveillance program to execute this.