Information processing system, information processing method, and information processing program

Abstract

To find an abnormal condition in rear of a shield.SOLUTION: An information processing system comprises: detection means which detects the entrance of a target in the rear side of a shield and disappearance of the target in an image, in the image obtained by imaging the shield and the target; prediction means which predicts an appearance timing of the target out of the rear side of the shield; and notification means which gives notification when the target does not appear out of the rear side of the shield even when a predetermined period passes after the predicted appearance timing.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and an information processing program. [Background technology]

[0002] In the above technical fields, Patent Document 1 discloses a technology for assigning the same identification number to a tracked object when it is hidden behind an obstruction and then reappears. Patent Document 2 discloses a technology for displaying the predicted position of a tracked vehicle on a monitor when the vehicle is hidden behind an obstruction. Furthermore, Patent Document 3 discloses a technology for predicting the danger of an object appearing in the driver's blind spot and sounding an alarm. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-334631 [Patent Document 2] Japanese Patent Application Laid-Open No. 2009-193135 [Patent Document 3] Japanese Patent Application Laid-Open No. 2013-97480 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in the above patent document was unable to detect abnormal situations behind an object.

[0005] An object of the present invention is to provide a technique for solving the above-mentioned problems. [Means for solving the problem]

[0006] In order to achieve the above object, an information processing system according to the present invention comprises: a detection means for detecting, in a captured image of an obstruction and an object, that the object has entered behind the obstruction and is no longer visible in the image; a prediction means for predicting the timing at which the target object appears from behind the obstruction; a notification means for notifying the user when the target object does not appear from behind the obstruction even after a predetermined time has elapsed since the predicted appearance time; It is an information processing system equipped with the above.

[0007] In order to achieve the above object, an information processing method according to the present invention comprises: a detection step in which a detection means detects that an object has entered behind the obstruction and is no longer visible in an image captured of the obstruction and the object; a prediction step in which a prediction means predicts a timing at which the target object will appear from behind the obstruction; a notification step in which a notification means notifies when the target object does not appear from behind the obstruction even after a predetermined time has elapsed since the predicted appearance time; The information processing method includes:

[0008] In order to achieve the above object, an information processing program according to the present invention comprises: a detection step of detecting, in a captured image of an obstruction and an object, that the object has entered behind the obstruction and is no longer visible in the image; a prediction step of predicting a timing at which the target object will appear from behind the obstruction; a notification step of notifying the user when the target object does not appear from behind the obstruction even after a predetermined time has elapsed since the predicted appearance time; It is an information processing program that causes a computer to execute the above. [Effects of the Invention]

[0009] According to the present invention, abnormal situations behind an object can be discovered. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing the configuration of a monitoring system according to a first embodiment of the present invention. [Figure 2] FIG. 10 is a diagram illustrating an outline of the operation of a monitoring system according to a second embodiment of the present invention. [Figure 3] FIG. 10 is a block diagram showing the configuration of a monitoring system according to a second embodiment of the present invention. [Figure 4A] FIG. 10 is a diagram showing the configuration of a prediction table included in the monitoring system according to the second embodiment of the present invention. [Figure 4B] FIG. 10 is a diagram showing the configuration of an obstruction table included in a monitoring system according to a second embodiment of the present invention. [Figure 4C] FIG. 10 is a diagram showing the configuration of a notification table included in the monitoring system according to the second embodiment of the present invention. [Figure 5] FIG. 10 is a diagram illustrating a hardware configuration of a monitoring system according to a second embodiment of the present invention. [Figure 6] 10 is a flowchart showing a processing procedure of a monitoring system according to a second embodiment of the present invention. [Figure 7] FIG. 10 is a diagram illustrating an outline of the operation of a monitoring system according to a third embodiment of the present invention. [Figure 8] FIG. 10 is a block diagram showing the configuration of a monitoring system according to a third embodiment of the present invention. [Figure 9] 10 is a flowchart showing a processing procedure of a monitoring system according to a third embodiment of the present invention. [Figure 10] FIG. 10 is a diagram illustrating an outline of the operation of a monitoring system according to a fourth embodiment of the present invention. [Figure 11] FIG. 10 is a block diagram showing the configuration of a monitoring system according to a fourth embodiment of the present invention. [Figure 12] 10 is a flowchart showing a processing procedure of a monitoring system according to a fourth embodiment of the present invention. [Figure 13] FIG. 10 is a diagram illustrating an outline of the operation of the monitoring system according to the fifth embodiment of the present invention. [Figure 14]FIG. 10 is a block diagram showing the configuration of a monitoring system according to a fifth embodiment of the present invention. [Figure 15] 13 is a flowchart showing a processing procedure of a monitoring system according to a fifth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the components described in the following embodiments are merely examples and are not intended to limit the technical scope of the present invention.

[0012] [First embodiment] A monitoring system 100 according to a first embodiment of the present invention will be described with reference to Fig. 1. The monitoring system 100 is a system that detects an abnormal situation behind an object that has entered a blind spot and does not emerge from the blind spot even after a predetermined time has passed.

[0013] 1, the monitoring system 100 includes an image acquisition unit 101, a detection unit 103, and a notification unit 104. The notification unit 104 further includes a first unit 141. The image acquisition unit 101 acquires an image.

[0014] The detection unit 103 detects the intrusion of an object into a blind spot created by an obstruction in the video and the emergence of an object from the blind spot. The notification unit 104 notifies the observer 130 or the like if the object does not emerge from the blind spot even after a first time 141 has elapsed since the object entered the blind spot.

[0015] According to this embodiment, an abnormal situation behind an object can be discovered.

[0016] [Second embodiment] Next, a monitoring system according to a second embodiment of the present invention will be described with reference to FIGS. 2 to 6. FIG. 2 is a diagram for explaining an outline of the operation of the monitoring system according to this embodiment. As shown in FIG. 2, an object 210 is moving from the left side to the right side of the figure at a speed V. An obstruction 220 is present in the destination of the object 210, and the presence of the obstruction 220 creates a blind spot behind the obstruction 220. Thereafter, when the object 210 enters the blind spot created by the obstruction 220 and becomes invisible, it becomes impossible to track the object 210.

[0017] The monitoring system 200 predicts the timing at which the object 210 will reappear, based on the speed (V) of the object 210 when it entered the blind spot. If the object 210 does not reappear at the predicted timing or after a predetermined time has elapsed since the predicted timing, the monitoring system 200 issues an alert to, for example, a monitor.

[0018] For example, if the predicted appearance timing is three seconds later, and if the object 210 does not reappear after [3 seconds + α seconds] has elapsed, it is determined that an abnormal situation has occurred in the blind spot behind the shield 220, and an alert is issued to the monitor or the like. Conversely, if the object 210 appears within [3 seconds + α seconds], the monitoring system 200 determines that no abnormal situation has occurred behind the shield 220, and does not issue an alert or the like. This makes it possible to easily detect abnormal situations behind the shield 220, such as a person falling or an accident, or criminal activity including drug trafficking.

[0019] 3 is a block diagram showing the configuration of a monitoring system 200 according to this embodiment. The monitoring system 200 includes a video acquisition unit 301, a determination unit 302, a speed detection unit 303, a prediction unit 304, a detection unit 305, and a notification unit 306.

[0020] Image acquisition unit 301 acquires images from camera 320. Note that the number of cameras 320 is not limited to one, and may be multiple. Determination unit 302 determines whether a blind spot caused by an obstruction exists in the image acquired by image acquisition unit 301. Determination unit 302 may determine the existence of a blind spot by two-dimensionally analyzing the image from one camera, or by three-dimensionally analyzing the images from multiple cameras. When performing two-dimensional analysis, determination unit 302 may determine that a blind spot exists directly in the area occupied by the obstruction in the image.

[0021] The speed detection unit 303 detects the intrusion speed of an object when it intrudes into a blind spot created by an obstruction. The prediction unit 304 then predicts the timing at which the object will emerge from the blind spot based on the intrusion speed of the object detected by the speed detection unit 303. The detection unit 305 detects the appearance of the object from the blind spot. The notification unit 306 notifies the monitor 330 or the like if the object does not emerge from the blind spot even after a predetermined time has elapsed since the appearance timing predicted by the prediction unit 304.

[0022] FIG. 4A is a diagram showing an example of the configuration of a prediction table 401 included in the monitoring system 200 according to this embodiment. The prediction table 401 stores a predicted appearance time and a predetermined time, which is the elapsed time from the predicted appearance time, in association with a combination of an object speed and a blind spot size. The prediction unit 304 may detect the size of an obstruction and derive the appearance timing from the object's intrusion speed and the detected size of the obstruction. Alternatively, the prediction unit 304 may identify the appearance timing using the prediction table 401 shown in FIG. 4A.

[0023] 4B is a diagram showing an example of the configuration of the obstruction table 402 included in the monitoring system 200 according to this embodiment. The obstruction table 402 stores the positions of blind spots created by the obstructions in association with the positions and sizes of the obstructions. The monitoring system 200 may detect the size of the obstruction for the blind spot created by the obstruction and derive the position of the blind spot based on the detected size. Alternatively, the monitoring system 200 may identify the position of the blind spot using the obstruction table 402 shown in FIG. 4B.

[0024] FIG. 4C is a diagram showing an example of the configuration of the notification table 403 included in the monitoring system 200 according to this embodiment. The notification table 403 stores an alert level and a notification destination in association with an appearance delay time. For example, if the delay time is "T4", the alert level is "1" and the notification destination is "the monitor". The alert level may be changed, for example, according to the time elapsed from the predicted appearance timing (appearance delay time). For example, the alert level may be changed by gradually increasing the volume of the alarm sound, or by blinking or gradually increasing the display on the screen viewed by the monitor.

[0025] 5 is a diagram showing the hardware configuration of the monitoring system 200. The monitoring system 200 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a storage 504, and a communication control unit 505.

[0026] The CPU 501 is a processor for arithmetic processing, and executes programs to realize each functional component of the monitoring system 200. The number of CPUs 501 is not limited to one, but may be multiple, and may include a GPU (Graphics Processing Unit) for image processing. The ROM 502 is a read-only memory, and stores programs such as firmware.

[0027] The communication control unit 505 communicates with other devices via a network. The communication control unit 505 may also include a CPU independent of the CPU 501, and may write or read transmitted / received data to or from the RAM 503.

[0028] The RAM 503 is a random access memory used by the CPU 501 as a work area for temporary storage. The RAM 503 has an area for storing data necessary for implementing this embodiment. As such data, the monitoring system 200 temporarily stores a predicted appearance time 531, a delay time 532, notification content 533, and obstructing object information 534. The RAM 503 also has an application execution area 535 for executing various application modules.

[0029] The storage 504 is a storage device that stores programs, databases, etc. necessary to realize this embodiment. The storage 504 stores a prediction table 401, an obstructing object table 402, a notification table 403, a monitoring program 541, and a control program 545.

[0030] The monitoring program 541 stores a determination module 542, a speed detection module 543, and a prediction module 544. These modules 542 to 544 are read into the application execution area 535 by the CPU 501 and executed. The control program 545 is a program for controlling the entire monitoring system 200. It is also desirable to provide a DMAC (Direct Memory Access Controller) (not shown) for transferring data between the RAM 503 and the storage 504.

[0031] 5 does not include programs or data related to the general-purpose functions of the monitoring system 200 or other feasible functions. The hardware configuration of the monitoring system 200 described here is merely an example, and the monitoring system is not limited to this hardware configuration, and various hardware configurations are possible.

[0032] FIG. 6 is a flowchart illustrating the processing steps of the monitoring system 200. In step S601, the monitoring system 200 acquires video from a camera. In step S603, the monitoring system 200 recognizes the presence of an obstruction in the acquired video and determines whether a blind spot exists due to the obstruction. In step S605, the monitoring system 200 detects the intrusion speed of an object that has entered the blind spot behind the obstruction and predicts the timing of the object's appearance from the blind spot. In step S607, the monitoring system 200 determines whether the object has appeared from the blind spot at the predicted appearance timing (predicted appearance time) or within a predetermined time from this appearance timing. If the object has not appeared from the blind spot at the predicted appearance timing or within a predetermined time from this appearance timing, the monitoring system 200 issues an alert in step S609.

[0033] According to this embodiment, the timing of the appearance of an object is predicted, making it possible to discover abnormal situations such as a person falling or an accident behind an obstruction, or criminal activity including drug dealing.

[0034] [Third embodiment] Next, a monitoring system 700 according to a third embodiment of the present invention will be described with reference to FIGS. 7 to 9. FIG. 7 is a diagram for explaining an outline of the operation of the monitoring system 700 according to this embodiment. The monitoring system 700 according to this embodiment differs from the second embodiment in that it includes a direction detection unit. Other configurations and operations are similar to those of the second embodiment, and therefore the same configurations and operations are denoted by the same reference numerals and detailed description thereof will be omitted.

[0035] The monitoring system 700 detects the direction of intrusion of the object 210 into the blind spot created by the shielding object 220. Based on the detected intrusion direction and speed, the timing and position at which the object 210 will appear are predicted.

[0036] 8 is a block diagram showing the configuration of monitoring system 700. Monitoring system 700 further includes a direction detection unit 801. Direction detection unit 801 detects the direction of an object entering a blind spot created by an obstruction. Based on the intrusion direction and intrusion speed, monitoring system 700 predicts the appearance timing and position of the object.

[0037] 9 is a flowchart illustrating the processing procedure of the monitoring system 700. Note that the same steps as those in FIG. 6 are assigned the same step numbers and their explanations are omitted. In step S901, the monitoring system 700 further detects the direction of entry of the object into the blind spot, and predicts the position at which the object will emerge from the blind spot based on the detected direction of entry. In step S903, the monitoring system 700 determines whether the object has appeared at the predicted appearance position at the predicted appearance time.

[0038] According to this embodiment, the appearance position is predicted taking into account the direction of entry of the object into the blind spot created by the obstruction, making it possible to detect abnormal situations such as a person falling or having an accident behind the obstruction, or criminal activity including drug trafficking.

[0039] [Fourth embodiment] Next, a monitoring system 1000 according to a fourth embodiment of the present invention will be described with reference to Figs. 10 to 12. Fig. 10 is a diagram for explaining an outline of the operation of the monitoring system 1000 according to this embodiment. The monitoring system 1000 according to this embodiment differs from the third embodiment in that it includes a video generation unit. Other configurations and operations are similar to those of the third embodiment, and therefore the same configurations and operations are denoted by the same reference numerals and detailed description thereof will be omitted.

[0040] When the object 1010 enters a blind spot created by the shielding object 1020, the monitoring system 1000 displays a predicted position mark 1011 indicating the predicted position of the object 1010 in the blind spot created by the shielding object 1020, based on the intrusion speed at which the object 1010 entered the blind spot. Furthermore, the monitoring system 1000 displays a predicted movement trajectory 1030 along with the predicted position mark 1011. The monitoring system 1000 may also display the predicted position mark 1011 and the predicted movement trajectory 1030 taking into consideration the intrusion direction in addition to the intrusion speed of the object 1010 into the blind spot. Furthermore, the monitoring system 1000 may predict the emergence position of the object 1010 that has entered the blind spot, and display the predicted emergence position. For example, the monitoring system 1000 may display the appearance position as an object 210 shown by a dotted line in FIG. 2, an object 210 shown by a dotted line in FIG. 7 (or an object 1310 shown by a dotted line in FIG. 13 described later), or may display the appearance position as an area.

[0041] 11 is a block diagram showing the configuration of the monitoring system 1000. The monitoring system 1000 further includes an image generation unit 1101. The image generation unit 1101 generates an image by superimposing, on the image acquired by the image acquisition unit 301, a trajectory of an object generated based on the prediction by the prediction unit 304. The image generation unit 1101 then shows the generated image to an observer 1130 or the like.

[0042] 12 is a flowchart illustrating the processing procedure of the monitoring system 1000. Note that the same steps as those in FIG. 6 are assigned the same step numbers, and their explanations will be omitted. In step S1201, the monitoring system 1000 generates an image in which the trajectory of the movement of the object, generated based on the prediction, is superimposed on the acquired image. The monitoring system 1000 then shows the generated superimposed image to the monitor 1130 or the like.

[0043] According to this embodiment, a predicted position mark and a predicted movement trajectory are displayed, making it possible to detect abnormal situations such as a person falling or an accident behind an obstruction, or criminal activity including drug trafficking, and furthermore, the observer can visually recognize the timing of the appearance of the target object.

[0044] [Fifth embodiment] Next, a monitoring system 1300 according to a fifth embodiment of the present invention will be described with reference to Figs. 13 to 15. Fig. 13 is a diagram for explaining an outline of the operation of the monitoring system 1300 according to this embodiment. The monitoring system 1300 according to this embodiment differs from the third embodiment in that the prediction unit takes relative velocity into account. Other configurations and operations are similar to those of the second embodiment, and therefore the same configurations and operations are denoted by the same reference numerals and detailed description thereof will be omitted.

[0045] 13 is a diagram illustrating an outline of the operation of the monitoring system 1300 when an object 1310 and an obstructing object 1320 are moving. The object 1310 is moving from left to right in the drawing at a speed V1. A vehicle such as a bus, which is the obstructing object 1320, is moving from right to left in the drawing at a speed V2. In this case, the monitoring system 1300 predicts the appearance timing of the object 1310 from the relative speeds of the object 1310 and the obstructing object 1320.

[0046] 14 is a block diagram showing the configuration of a monitoring system 1300. The monitoring system 1300 includes a relative speed calculation unit 1401 and a prediction unit 1402. The relative speed calculation unit 1401 calculates the relative speed between a moving object 1310 and a moving obstruction 1320. The prediction unit 1402 then predicts the appearance timing of the object 1310 based on the relative speed calculated by the relative speed calculation unit 1401.

[0047] 15 is a flowchart illustrating the processing procedure of the monitoring system 1300. Note that the same steps as those in FIG. 6 are assigned the same step numbers and their explanations will be omitted. In step S1501, the monitoring system 1300 calculates the relative speed between the moving object 1310 and the moving obstruction 1320. In step S1503, the monitoring system 1300 predicts the timing at which the object 1310 will appear from the blind spot based on the calculated relative speed.

[0048] According to this embodiment, prediction is made taking relative speed into consideration, so that abnormal situations such as a person falling or an accident behind an obstacle, or criminal activity including drug dealing, can be discovered.

[0049] [Other embodiments] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that are understandable to those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. Furthermore, systems or devices that combine the separate features included in each embodiment in any manner are also included in the scope of the present invention.

[0050] The present invention may also be applied to a system consisting of multiple devices or to a single device. Furthermore, the present invention is also applicable when an information processing program that realizes the functions of the embodiments is supplied directly or remotely to a system or device. Therefore, the scope of the present invention also includes a program installed on a computer to realize the functions of the present invention, a medium storing the program, and a WWW (World Wide Web) server from which the program can be downloaded. In particular, the scope of the present invention includes at least a non-transitory computer-readable medium storing a program that causes a computer to execute the processing steps included in the above-described embodiments.

[0051] This application claims priority based on Japanese Patent Application No. 2015-058279, filed March 20, 2015, the disclosure of which is incorporated herein in its entirety.

Claims

1. a detection means for detecting, in a captured image of an obstruction and an object, that the object has entered behind the obstruction and is no longer visible in the image; a prediction means for predicting the timing at which the target object appears from behind the obstruction; a notification means for notifying the user when the target object does not appear from behind the obstruction even after a predetermined time has elapsed since the predicted appearance time; An information processing system comprising:

2. The information processing system according to claim 1 , wherein the prediction means predicts the timing of the appearance of the object based on the speed at which the object approaches the other side of the obstruction.

3. The information processing system according to claim 1, wherein the notification means is a display control means that superimposes a message on the image to notify the user that the object will not appear from behind the obstruction even after a predetermined time has passed since the predicted appearance time.

4. 2. The information processing system according to claim 1, further comprising a display control means for superimposing on the obstruction a mark indicating the position of the object estimated based on the approach speed of the object to the other side of the obstruction.

5. a detection step in which a detection means detects that an object has entered behind the obstruction and is no longer visible in an image captured of the obstruction and the object; a prediction step in which a prediction means predicts a timing at which the target object will appear from behind the obstruction; a notification step in which a notification means notifies when the target object does not appear from behind the obstruction even after a predetermined time has elapsed since the predicted appearance time; An information processing method including:

6. a detection step of detecting, in a captured image of an obstruction and an object, that the object has entered behind the obstruction and is no longer visible in the image; a prediction step of predicting a timing at which the target object will appear from behind the obstruction; a notification step of notifying the user when the target object does not appear from behind the obstruction even after a predetermined time has elapsed since the predicted appearance time; An information processing program that causes a computer to execute the above.

Images