Surveillance system and method for surveillance

Abstract

To provide a surveillance system and a method for surveillance that can efficiently collect images and other information useful for searches for missing persons and investigations of incidents and accidents.SOLUTION: As a surveillance center that conducts surveillance of a preset region, the system includes: a drive recorder for capturing images of surroundings of a mounted vehicle and storing the captured images, or a communication unit for communicating with the drive recorder and a communication terminal capable of transmitting and receiving data with the drive recorder; a location information processing unit for collecting a travel trajectory or a position of the communication terminal or the drive recorder; and an analysis unit for, when a surveillance image capturing a surveillance target location, mobile object, or person within the region is required, determining, based on the travel trajectory or the position obtained by the location information processing unit, the drive recorder of a vehicle that is likely to provide the surveillance image, and collecting, via the communication unit, the captured images stored in the determined drive recorder, analyzing the collected images, and thereby obtaining information regarding the surveillance target location, mobile object, or person.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a monitoring system and a monitoring method. [Background technology]

[0002] With the recent advancement of information technology, various technologies have been developed for searching for people to be watched over, such as elderly people and children. For example, Patent Document 1 describes a local watching system that, when a missing person occurs, requests cooperation in the search from the terminals of participants in a watching community who are estimated to have a high probability of encountering the person to be watched over. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-116980 Summary of the Invention [Problem to be solved by the invention]

[0004] The community monitoring system described in Patent Document 1, when a missing person is reported, determines the destination of a search request among the devices of participants in the monitoring community based on the predicted behavior of the missing person, and then makes the search request. Therefore, with the technology described in Patent Document 1, a request for cooperation in the search cannot be made unless the participant's device has a monitoring community participation app installed. Furthermore, with the technology described in Patent Document 1, the search for the missing person itself is left to the participants' manual efforts, so it cannot be said to be an efficient monitoring system.

[0005] In addition to searching for missing elderly people or children, there is also a need for efficient collection of eyewitness information on incidents such as theft, snatching, and assault, as well as traffic accidents, and tracking information for cases in which the perpetrator or assailant escapes. For example, when a traffic accident occurs at a specific intersection, eyewitness information on the accident can be useful for the investigation by investigative authorities, but until now, it has been difficult to efficiently collect such eyewitness information.

[0006] The present invention has been made in view of the above, and aims to provide a monitoring system and a monitoring method that can efficiently collect information useful for searching for missing persons and investigating incidents and accidents. [Means for solving the problem]

[0007] In order to solve the above problems, for example, the configurations described in the claims are adopted. The present application includes multiple means for solving the above-mentioned problems, and one example thereof is a monitoring system including a monitoring center configured with an information processing device and monitoring a predetermined area. The monitoring center includes a drive recorder that captures images of the surroundings of the vehicle in which it is installed and stores the captured images, or a communication unit that communicates with a communication terminal that can send and receive data to and from the drive recorder, a location information processing unit that collects the movement trajectory or position of the communication terminal or drive recorder, and an analysis unit that, when monitoring images capturing images of a location, moving object, or person to be monitored within the area are needed, determines, based on the movement trajectory or position obtained by the location information processing unit, a drive recorder installed in a vehicle that may be able to obtain the monitoring images, and collects and analyzes the captured images stored by the determined drive recorder via the communication unit to obtain information about the location, moving object, or person to be monitored. [Effects of the Invention]

[0008] According to the present invention, the monitoring center can collect images of the object to be monitored taken by a drive recorder, and can appropriately monitor the location, moving object, or person to be monitored. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of the configuration of devices that make up a monitoring system according to an embodiment of the present invention. [Figure 2] 1 is a configuration diagram showing an example of a monitoring system according to an embodiment of the present invention. [Figure 3] FIG. 2 is a diagram showing an example of a monitoring process performed by a monitoring system according to an embodiment of the present invention. [Figure 4] 1 is a diagram showing an outline of processing performed by a monitoring system according to an embodiment of the present invention. [Figure 5] 1 is a flowchart showing the overall processing flow of a monitoring system according to an embodiment of the present invention. [Figure 6] 10 is a flowchart showing the flow of an initial setting process of a monitoring system according to an embodiment of the present invention. [Figure 7] 1 is a flowchart showing the flow of processing when an incident or accident occurs in a monitoring system according to an embodiment of the present invention. [Figure 8] 10 is a flowchart showing the flow of a patrol member process of the monitoring system according to an embodiment of the present invention. [Figure 9] 10 is a flowchart showing the flow of part-time patrol member processing in the monitoring system according to an embodiment of the present invention. [Figure 10] 10 is a flowchart showing the flow of information collection from part-time patrol members and the process of soliciting members in a monitoring system according to an embodiment of the present invention. [Figure 11] FIG. 10 is a diagram showing an example of a collection process of movement trajectories of a monitoring system according to an embodiment of the present invention. [Figure 12] FIG. 1 is a diagram illustrating an example of a parking monitoring function of a monitoring system according to an embodiment of the present invention. [Figure 13]FIG. 10 is a diagram showing an example of a notification to a monitoring patrol member of a monitoring system according to an embodiment of the present invention. [Figure 14] FIG. 10 is a diagram showing an example of a notification to a monitoring patrol member of a monitoring system according to an embodiment of the present invention. [Figure 15] FIG. 10 is a diagram showing an example of a notification to a monitoring patrol member of a monitoring system according to an embodiment of the present invention. [Figure 16] 10A and 10B are diagrams illustrating an example of a process for acquiring a recorded image of a drive recorder according to an embodiment of the present invention. [Figure 17] FIG. 10 is a diagram illustrating an example of an investigation target automatic cooperation tracking process of a monitoring system according to an embodiment of the present invention. [Figure 18] FIG. 10 is a diagram showing an example of an image of a monitoring system according to an embodiment of the present invention detecting and tracking a witnessed vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0010] A monitoring system and a monitoring method according to an embodiment of the present invention (hereinafter referred to as "this example") will be described below with reference to the accompanying drawings.

[0011] [Configuration of monitoring system] FIG. 1 shows an example of the configuration of each device that constitutes the monitoring system of this example. As shown in FIG. 1, the watching system of this example is made up of a first terminal 1, a second terminal 2, a watching center 3, and a third terminal 4. 1 shows one each of the first terminal 1, second terminal 2, and third terminal 4, but in reality, multiple terminals are provided. A regional monitoring application, which will be described later, is implemented in these terminals 1, 2, and 4 (or devices that control each terminal), and by executing the regional monitoring application, terminals 1, 2, and 4 function as communication terminals for the monitoring system of this example. However, the implementation of the local watch application is just one example, and there may be cases where a request is made to obtain images from the local watch center 3 for a terminal on which the local watch application is not implemented.

[0012] The first terminal 1 is a communication terminal called a smartphone. The first terminal 1 as a smartphone is configured as an information processing device and includes a processing unit 11, a main memory unit 12, an auxiliary memory unit 13, a communication unit 14, an input unit 15, and a positioning unit 16, which are connected to each other via a bus 17 so that data can be transferred between them. The first terminal 1 also includes a display unit (not shown). The processing unit 11 is an arithmetic processing unit called a CPU (Central Processing Unit) that reads the program code of the software (application) that realizes the functions performed by the first terminal 1 from the main memory unit 12 or the auxiliary memory unit 13 and executes it in the main memory unit 12.

[0013] The processing unit 11 reads out program code from the main memory unit 12 or the auxiliary memory unit 13 and executes arithmetic processing in the work area of ​​the main memory unit 12, thereby configuring various processing function units in the main memory unit 12. For example, the main memory unit 12 has a function of acquiring and storing the movement trajectory of the first terminal 1 based on information from the positioning unit 16.

[0014] A large-capacity information storage medium such as a non-volatile memory or a memory card is used as the auxiliary storage unit 13. The auxiliary storage unit 13 stores software that realizes the functions of the first terminal 1 and data obtained by executing the program.

[0015] The communication unit 14 communicates with the outside via a predetermined wireless communication network (a network for wireless telephones). One of the parties with which the communication unit 14 of the first terminal 1 communicates is the local watch center 3. The communication unit 14 of the first terminal 1 can also communicate with the second terminal 2. This communication between the first terminal 1 and the second terminal 2 is performed by, for example, a wireless LAN or a short-range wireless communication method called Bluetooth (registered trademark).

[0016] The input unit 15 receives operation information from the holder of the first terminal 1. As the input unit 15, for example, a touch panel that is arranged integrally with a display panel that constitutes the display unit is used. The positioning unit 16 measures the current position using a satellite positioning system called GPS (Global Positioning System). Using GPS to measure position is just one example, and the positioning unit 16 may use a positioning method other than GPS. Note that the first terminal 1 shown in Fig. 1 only shows the components necessary for the monitoring system of this example. In reality, the first terminal 1 as a smartphone has general components that a smartphone has, such as a display unit, a speaker for calls, a microphone, a camera, and an acceleration sensor, in addition to the components shown in Fig. 1.

[0017] The second terminal 2 is a terminal called a drive recorder, and is mounted on a vehicle (moving object) such as an automobile. The second terminal 2 as a drive recorder includes a processing unit 21, a main memory unit 22, an auxiliary memory unit 23, a communication unit 24, and a camera 25, which are connected to each other via a bus 26 so that data can be transferred between them.

[0018] Similar to the processing unit 11 of the first terminal 1, the processing unit 21 is a calculation processing unit called a CPU, and various processing function units are configured in the main memory unit 22 by the processing unit 21 performing calculation processing in the work area of ​​the main memory unit 22. The processing function unit configured in the main memory unit 22 executes processing as a drive recorder under the control of the processing unit 21. For example, under the control of the processing unit 21, an image captured by the camera 25 is stored in the auxiliary memory unit 23.

[0019] The communication unit 24 performs wireless communication with the first terminal 1 using a short-range wireless communication method. Alternatively, the communication unit 24 may perform direct communication with the local watching center 3 via a wireless telephone network. The second terminal 2 may also be provided with a positioning unit similar to that of the first terminal 1 as a drive recorder to measure the current position.

[0020] The regional monitoring center 3 is a device installed in the police, security companies, local governments, etc. The regional monitoring center 3 is composed of an information processing device (computer) and includes a communication unit 31, a processing unit 32, a main memory unit 33, an auxiliary memory unit 34, and a display unit 35, each of which is connected to each other via a bus 36 so that data can be transferred between them. The communication unit 31 performs communication processing with the outside via a wireless communication network. For example, the communication unit 31 communicates directly or indirectly with the first terminal 1, the second terminal 2, and the third terminal 4.

[0021] The processing unit 32 is an arithmetic processing unit known as a CPU. The processing unit 32 reads out the program code of software that realizes the functions performed by the local watching center 3 from the main memory unit 33 or the auxiliary memory unit 34, and executes arithmetic processing in the work area of ​​the main memory unit 33, thereby configuring processing function units in the main memory unit 33, such as a processing unit for processing location information (location information processing unit) and a processing unit for performing analysis processing (analysis unit). The auxiliary storage unit 34 stores route information, location information, monitoring information, etc. of each terminal. The display unit 35 displays a screen on which an operator operating the local monitoring center 3 monitors the monitoring status and the like.

[0022] The third terminal 4 is a terminal configured as a surveillance camera. The third terminal 4 is a surveillance camera installed on a road or at the entrance of a building, and includes a camera 41 and an auxiliary storage unit 42 that stores images (surveillance images) captured by the camera 41. The third terminal 4 also includes a communication unit, so that it can transmit surveillance images to the local watch center 3. In the following description, the first terminal 1, the second terminal 2, and the third terminal 4 may also be referred to as a smartphone, a drive recorder, and a surveillance camera, respectively.

[0023] [Overview of monitoring performed by the monitoring system] FIG. 2 shows an overview of the processing performed in the monitoring system of this example. The example of FIG. 2 is an image diagram in which the occurrence of an accident A1 at a specific intersection is confirmed on the display unit 35 of the local watch center 3. At this time, as shown in Fig. 2, a map of the vicinity of the intersection where the relevant accident A1 occurred is displayed on the display unit 35. Then, on the map displayed on the display unit 35, terminals held or possessed by persons registered as patrol members who have been requested in advance to cooperate in monitoring the relevant intersection are displayed.

[0024] In the example of Figure 2, the display unit 35 displays a drive recorder 2a mounted on a vehicle 101 traveling on a road, a smartphone 1a carried by a person riding in the vehicle 101, and a smartphone 1b carried by a person walking on the road. Furthermore, the display unit 35 displays a drive recorder 2b mounted on a vehicle 102 parked in a parking lot facing a nearby road, a surveillance camera 4a installed on a house 301, and a surveillance camera 4b installed on a utility pole 302.

[0025] The local monitoring center 3 then acquires images taken by the smartphones 1a, 1b, drive recorders 2a, 2b, and surveillance cameras 4a, 4b displayed on these display units 35, analyzes the acquired images, and obtains monitoring information related to the accident A1. Note that the fact that the local monitoring center 3 obtains monitoring information related to accidents is just one example, and as explained in the following Figure 3, the local monitoring center 3 can obtain various types of monitoring information other than monitoring information related to accidents.

[0026] 3 shows an example of the monitoring process performed by the local monitoring center 3 of this example. That is, when an incident such as a hit-and-run, an accident, a robbery, an assault, a fight, or a theft occurs, the local monitoring center 3 of this example can acquire images of the neighborhood before and after the incident. As a result, the local watching center 3 in this example can obtain watching information that is useful for solving the incident by analyzing the acquired images of the neighborhood before and after the incident occurred. In addition to accidents and incidents, the local watch center 3 of this example can also obtain watch information that is useful for searching for lost or missing people and wandering elderly people.

[0027] FIG. 4 shows an example of an actual operational situation using the local monitoring center 3. As shown in Figure 4, when an accident A1 occurs at a specific intersection, for example, a party involved in the accident A1 or a witness notifies the police station and security company 5. Upon receiving the notification, the police station and security company 5 rushes to the scene of the accident A1.

[0028] The police station / security company 5 that received the notification then requests support or backup from the local watch center 3. The local watch center 3 that received this request for support or backup displays information such as the location of the incident, the patrol members who were in the vicinity at the time of the incident, or the devices held by the patrol members on the display unit 35. The local watch center 3 then acquires images from the devices of those registered as patrol members who were in the vicinity of the location of the incident at the time of the incident, as explained in FIG.

[0029] When searching for a lost or missing person or a wandering elderly person, the community watch center 3 may send facial images P1 and P2 of the missing person, sent by the search requester to the police station or security company 5, to the smartphones of patrol officers in the vicinity of the incident. Alternatively, the facial images P1 and P2 of the missing person may be searched for using AI object detection and facial recognition technology among the recorded images collected by the community watch center 3. Furthermore, the missing person's full-body image from the day of the incident can be extracted and automatically searched for using AI technology (object detection, facial recognition technology) among other recorded images, thereby enabling automatic tracking of the person. In this case, the missing person may be found and automatically tracked using automatic analysis using AI technology among recorded images collected from around the time of the incident in the vicinity of the location where the incident requiring monitoring occurred. Even if the object being monitored is a moving object other than a person, it can be found and automatically tracked using automatic analysis using AI technology from recorded images collected from around the time of the incident or accident. This allows the system to automatically find clues for investigation that humans might overlook, and by informing a human (a center operator or a missing person's next of kin) and prompting them to confirm the information, it becomes possible to efficiently locate the person being monitored. Furthermore, rather than delivering facial photographs P1 and P2 of the person being searched for from the person requesting the search, when the facial image is found from a recorded image, the full-body image of the missing person reflects the clothing and appearance of the missing person on the day, and by newly analyzing the (full-body) image of the person in question as a monitoring image, a more efficient search can be carried out.

[0030] [Patrol member registration process] FIG. 5 is a flowchart showing the flow of the patrol member registration process. First, the local watch center 3 distributes a local watch application to terminals such as smartphones and allows each smartphone to download the application (step S11).

[0031] Next, the terminal that has downloaded the community watching app determines whether or not the initial settings have been completed (step S12). If the initial settings have not been completed in step S12 (NO in step S12), the smartphone performs an initial setting process (step S13). Details of the initial setting process will be explained in FIG. 6. Once the initial setting process has been completed in step S13, the terminal that has downloaded the community watching app will be able to receive commands from the community watching center 3.

[0032] If the initial setting process is completed in step S13, or if the initial setting is completed in step S12 (YES in step S12), the corresponding terminal (smartphone) starts processing as a patrol member and records movement information (step S14). The movement information is recorded at regular intervals, such as every 10 seconds, in step S14. Here, the patrol member also includes a person who owns and manages a surveillance camera that is basically fixed, such as the third terminal 4, and in this case, movement information is not recorded.

[0033] Thereafter, the corresponding terminal determines whether or not it has received an emergency command (step S15). When it has received an emergency command in step S15 (YES in step S15), the terminal immediately uploads the movement information recorded in step S14 to the local watching center 3 (step S16).

[0034] Furthermore, if no emergency command is received in step S15 (NO in step S15), the terminal uploads the movement information recorded in step S14 to the local watching center 3 at regular intervals (step S17). The movement information is preferably uploaded every few hours, for example, every six hours.

[0035] The terminal then determines whether or not the patrol member has been unsubscribed (step S18). If no unsubscribe processing has been performed in step S18 (NO in step S18), the terminal returns to the movement information recording processing in step S14. Furthermore, if there is a withdrawal process in step S18 (YES in step S18), the process of recording and uploading movement information at the corresponding terminal is terminated.

[0036] FIG. 6 is a flowchart showing an example of the flow of the initial setting process for a patrol member. When the initial setup process for the patrol member begins on the terminal (smartphone), personal information is first registered (step S21). This personal information may include, for example, name, address, email address, and telephone number.

[0037] Next, the registrant is asked whether or not he or she owns a car or a surveillance camera, and if so, the car's license plate number, whether or not it has a drive recorder, and even the location where the vehicle is parked or the location where the surveillance camera is installed are registered (step S22). Furthermore, if the presence of a drive recorder is registered in step S22, details of the drive recorder, such as the model of the drive recorder, the memory card installed, whether the parking monitoring function is used, etc., are registered (step S23), and the initial setting process is completed.

[0038] [Handling of incidents and accidents reported at the local watch center] FIG. 7 is a flowchart showing the flow of processing when the local watch center 3 receives a report of an incident or accident from authorities such as the police. First, the local watch center 3 receives a report of an incident or accident as a request for assistance from the authorities (step S31). The local watch center 3 that has received the report of an incident or accident determines whether the incident or accident is an emergency (step S32). This determination of whether it is an emergency is made based on, for example, information included in the request for assistance from the authorities.

[0039] When determining that an emergency exists in step S32 (YES in step S32), the local watch center 3 first calculates the range of information to be collected in response to an emergency command (step S33). The calculation method involves calculating the time lag (e.g., 5 minutes) between the actual incident or accident occurrence time and the issuance of the emergency command, determining the travelable distance for a normal vehicle (e.g., 60 km / h * 5 minutes / 60 minutes = 5 km), and issuing an emergency command to the terminals of patrol members who are still within this distance from the incident or accident location (step S34). After issuing the emergency command in step S34, the local watch center 3 waits for a short time (e.g., 1 minute) until the information upload from the terminal is completed (step S35). If it is not determined to be an emergency in step S32 (NO in step S32), the local watch center 3 waits for a certain period of time (for example, a maximum of 6 hours) until the next regular information upload from the terminal (step S40). Then, the local watch center 3 collects and analyzes the movement trajectories of the terminals registered as patrol members (step S36). The process of collecting the movement trajectories of the terminals is executed, for example, by a location information processing unit configured inside the information processing device serving as the local watch center 3. Similarly, the analysis of the movement trajectories is executed by an analysis unit configured inside the information processing device serving as the local watch center 3 (FIG. 1). Other processing and analysis related to location information described below are also executed by a location information processing unit and analysis unit configured inside the information processing device serving as the local watch center 3.

[0040] When analyzing the movement trajectory in step S36, the analysis also includes the positions of the monitoring cameras 4a and 4b and the position of the drive recorder 2 mounted on the parked vehicle with the parking monitoring function (parking mode) activated. Through this analysis, the local watch center 3 determines a terminal that may be able to obtain images of the perpetrator of the incident or accident (the vehicle that caused the accident) or other subject of investigation related to the incident or accident.

[0041] Next, the community monitoring center 3 collects and analyzes image information recorded on devices such as drive recorders that may have images related to the incident or accident as a result of the analysis in step S36 (step S37). The analysis process may be automated using object detection AI technology or may be performed by humans. For example, not only may facial photos P1 and P2 of wandering elderly people, lost children, or missing persons be identified using AI object detection and facial recognition technology, but tracking processing may also be automated. The community monitoring center 3 then determines whether the perpetrator of the incident or accident, or other target of investigation, has been identified (step S38). In step S38, if the subject of investigation, such as the perpetrator of the incident or accident, can be identified (YES in step S38), the processing in the local watching center 3 ends.

[0042] In step S38, if the perpetrator of the incident or accident or other subject of investigation cannot be identified (NO in step S38), the local watch center 3 proceeds to patrol member processing (step S39). When the patrol member processing is completed, the local watch center 3 returns to the judgment in step S38 again.

[0043] FIG. 8 is a flowchart showing the flow of the patrol member process (the process of step S39 in FIG. 7). 7, the local watch center 3 extracts moving objects (automobiles, motorcycles, etc.) in the vicinity of the scene of the incident or accident (step S41). Then, the local watch center 3 estimates the moving direction and moving speed of the moving objects extracted in step S41 from the analysis results (step S42). After the estimation in step S42, the local watch center 3 searches for the movement trajectories of the patrol members near the new point obtained using the estimated movement direction and movement speed, and searches for the terminals of the patrol members near the new point (step S43). A specific example will be described later with reference to FIG. 17.

[0044] Then, the local watch center 3 collects and analyzes recorded images from the drive recorders of the patrol members at the location searched for in step S43 (step S44). Thereafter, the local watch center 3 determines whether or not it is impossible to track the moving object any further (step S45). If it is not impossible to track the moving object in step S45 (NO in step S45), the local watch center 3 returns to the process of step S42 and continues to estimate the moving direction and moving speed of the moving object. In step S45, if tracking of the moving object is not possible (YES in step S45), the local watching center 3 proceeds to part-time patrol member processing (step S46).

[0045] FIG. 9 is a flowchart showing the flow of the part-time patrol member process in step S46 of FIG. Here, a part-time patrol officer refers to a person who possesses a device (such as a drive recorder) that has not been registered in advance as a patrol officer as explained in Figure 5. In other words, this refers to an ordinary resident, who is different from a patrol officer who has previously declared that he or she is willing to participate in neighborhood watch. First, the local watch center 3 analyzes the collected images from each location to identify the vehicle that was at the scene of the incident or accident shown in the image (step S51).The local watch center 3 then reads the number printed on the license plate of the identified vehicle from the image (step S52).

[0046] The local watch center 3 then identifies the owner of the vehicle with the license plate number read in step S52 from the database of the Land Transport Bureau that registers license plates (step S53).The local watch center 3 then contacts the owner identified in step S53 by email, chat screen, telephone, etc., and collects eyewitness information about the incident or accident (step S54).This eyewitness information may be collected, for example, by automatically sending data in email or chat format through processing at the local watch center 3, or by having an operator make a telephone call.

[0047] After collecting the sighting information in step S54, the local watch center 3 determines whether or not the vehicle identified in step S53 is equipped with a drive recorder (step S55). If it is determined in step S55 that the vehicle is not equipped with a drive recorder (NO in step S55), the local watch center 3 returns to the process of step S52 and reads the license plate number of another vehicle.

[0048] If a drive recorder is installed in step S55 (YES in step S55), the local watch center 3 collects and analyzes information such as images recorded by the drive recorder (step S56). At this time, the local watch center 3 also invites owners of vehicles equipped with drive recorders to become patrol members.

[0049] Fig. 10 is a flowchart showing the flow of another example of the part-time patrol member processing in step S46 of Fig. 8. A specific example will be described later with reference to Fig. 18. The local watch center 3 analyzes images taken approximately one minute before and after the time of the incident or accident (step S61). From this analysis, the local watch center 3 determines whether or not a vehicle passed by before the accident (step S62). If a vehicle passed by before the accident in step S62 (YES in step S62), the local watch center 3 analyzes video footage of vehicles passing by in the opposite lane on the way to the scene of the incident or accident (step S63).

[0050] If no vehicle has passed before the accident in step S62 (NO in step S62), and after analyzing the video in step S63, the local monitoring center 3 determines whether any vehicle has passed after the accident (step S64). In step S64, if there is a vehicle that has passed after the accident (YES in step S64), the local watch center 3 analyzes an image showing a vehicle passing in the opposite lane after passing the scene of the incident or accident (step S65).

[0051] Then, the regional watch center 3 determines the license plate number of the oncoming vehicle from the results of the analyses in steps S63 and S65, and analyzes the image of the vehicle (step S66).From the results of the analysis in step S66, the regional watch center 3 determines whether the vehicle is equipped with a drive recorder (step S67).

[0052] If it is determined in step S67 that a drive recorder is installed (YES in step S67), the local watch center 3 requests the owner of the vehicle to provide images recorded by the drive recorder and invites the owner to become a patrol member (step S68). On the other hand, if it is determined in step S67 that a drive recorder is not installed (NO in step S67), the local watch center 3 requests eyewitness information about incidents and accidents and invites the owner to become a patrol member (step S69). The requests in steps S68 and S69 may be made automatically by email or the like, or by telephone by an operator.

[0053] [Example of movement trajectory collection by a monitoring system] FIG. 11 shows an example of the process of collecting movement trajectories in the local watch center 3 of this example. The example in Figure 11 shows an example in which the local monitoring center 3 acquires data on the date and time, latitude, longitude, movement speed, and movement direction at regular intervals from smartphones 1a, 1b, and 1c owned by Mr. A, Mr. B, and Mr. C, and stores the data in a movement trajectory database.

[0054] Here, person A has a drive recorder 2a installed in his car, and person C has a drive recorder 2b installed in his car. Person B does not own a car and uses a motorcycle for transportation. The local monitoring center 3 acquires the necessary recorded images from the drive recorder based on the information stored in the movement trajectory database. Alternatively, the recorded images may be acquired from Mr. B's smartphone.

[0055] 11, the local monitoring center 3 also has a fixed surveillance camera information database. That is, the local monitoring center 3 has, as the fixed surveillance camera information database, information such as the camera ID (fixed camera ID), latitude and longitude of the installation location, whether or not there is an internet connection, the direction of shooting, and the contact telephone number of the administrator for each surveillance camera 4a installed in a house or the like and surveillance camera 4b installed on a road. Then, the local monitoring center 3 acquires recorded images from the necessary monitoring cameras.

[0056] FIG. 12 is a diagram showing an example in which drive recorders 2a to 2h of parked vehicles with the parking monitoring function (parking mode) activated record images on the road. In other words, by having the drive recorders 2a-2h of vehicles parked near roads register their respective parking locations in advance, the local monitoring center 3 can use the drive recorders 2a-2h of parked vehicles to search for incidents, accidents, and the like. However, the drive recorders 2a-2h that can be used are limited to those that were registered as patrol members to record while parked. The parking monitoring function (parking mode) covers (1) continuous recording (always records while parked): frame-by-frame recording at 1-5 fps (time-lapse recording function). In addition to this, other methods include (2) G-sensor recording (detecting and recording impacts) and (3) motion detection (detecting and recording motion).

[0057] In this way, if the drive recorders 2a to 2h of parked vehicles can be treated as surveillance cameras, in the monitoring system of this example, a large number of cameras will be placed to monitor the area, and the local monitoring center 3 will be able to appropriately collect eyewitness information on incidents, accidents, etc.

[0058] Figures 13, 14 and 15 are display screens on a patrol officer's smartphone 1, showing an example in which a request for eyewitness information on an incident, accident, etc. sent from the community monitoring center 3 is displayed in chat format on the patrol officer's smartphone 1. As shown in FIG. 13, upon receiving the request from the local watch center 3, the smartphone 1 displays a request message saying, "Looking for witnesses. XX Police Station..." At this time, three answer options are displayed on Smartphone 1: "I saw it," "I don't remember much," and "I didn't see it," prompting the owner of Smartphone 1 to answer. In addition, if the owner of Smartphone 1 is equipped with a dashcam, a message urging the owner to send images recorded by the dashcam is also added to the request.

[0059] FIG. 14 shows the location and date and time of the incident or accident, which are displayed together with the request text, and the movement trajectory of the smartphone 1 (or the movement trajectory of the drive recorder 2). This movement trajectory shows the location of smartphone 1 at the time of the incident or accident ("Person A's location around the time of the accident" in the figure). Furthermore, smartphone 1 displays a street view image of the location of the incident or accident, sent from the local monitoring center 3. Here, the street view image is an image of the relevant location viewed from multiple directions on the road. The local watch center 3 can acquire appropriate eyewitness information from the patrol members by making requests for eyewitness information as shown in FIGS. 13 and 14 to the patrol members.

[0060] Fig. 15 shows an example of a display when a request for eyewitness information on an incident, accident, etc. is sent from the local watch center 3 to the smartphone 1 of a patrol officer. Fig. 15 shows an example of a display in chat format on the smartphone 1. As explained in Figs. 8 and 17, the patrol officer is the owner of a vehicle whose image indicates a high possibility that he or she is a witness to the subject of the investigation.

[0061] As shown in Figure 15, upon receiving the request from the local monitoring center 3, the smartphone 1 displays a request to send the recorded images, saying, "We have a request for you, who are thought to have been a likely witness. Please send us the data from the images recorded by your car's dashcam on XX / XX / 2020, from approximately XX:00 PM to YY:00 PM..."

[0062] Smartphone 1 then displays three options for response: "We will cooperate," "We cannot cooperate," and "We will provide the information, but please come and collect it," prompting the owner of Smartphone 1 to respond. If cooperation is possible, the owner of Smartphone 1 will also be shown customized instructions for uploading recorded images based on the model of the drive recorder registered during the initial setup process. This allows the local watch center 3 to efficiently collect eyewitness information on incidents, accidents, etc. from drive recorders that are not registered as patrol members.

[0063] FIG. 16 shows the flow of the regional watching center 3 of this example collecting recorded images from the drive recorder and objective information based on the movement trajectory. As shown in Figure 16, the local watch center 3 acquires the movement trajectories of patrol officers and obtains analysis results from the acquired movement trajectories. For example, if the time T when an incident or accident occurred is known accurately to the nearest 10 seconds (with an error of ±10 seconds), it determines that there is a high possibility that a patrol officer whose movement trajectory is recorded within a radius of, say, 50 meters (because a distance of more than 50 meters exceeds the visual (recording resolution) limit) + 167 meters (10 seconds × average speed of 60 km / h) = 217 meters from the scene point X witnessed the incident. Based on the analysis results, the local watch center 3 then requests the collection of recorded images from the drive recorder and interviews the patrol officers for witness information. For example, the local monitoring center 3 collects drive recorder information such as recorded images from the drive recorder 2a of person A and the drive recorder 2b of person C. If the drive recorder 2b of person C records rear images in addition to forward images, the local monitoring center 3 also collects the rear images.

[0064] In addition, the local monitoring center 3 can also collect eyewitness information from Mr. B, who does not have a drive recorder, and from Mr. B's smartphone 1b. This enables the monitoring system in this example to collect images and eyewitness information related to appropriate incidents and accidents based on the movement trajectories of patrol officers.

[0065] FIG. 17 shows an example in which the local watching center 3 of this example tracks the target of investigation (hereinafter, assumed to be a car carrying a person X who is likely to be the criminal) based on the movement trajectory. In the example of Fig. 17, for example, the movement trajectory of person A overlaps with the scene of a hit-and-run accident, as shown in the upper left of Fig. 17. In such a case, the local watching center 3 can determine from the drive recorder image provided by person A that a car carrying person X, who is likely the perpetrator of the crime, is going straight through a certain intersection. However, as shown in the lower left of Figure 17, suppose that Mr. A's car turns right onto the road and loses sight of the car carrying Person X.

[0066] At this time, it is assumed that the local monitoring center 3 discovers a car carrying person X, photographed by person A, in the recorded image of person B's drive recorder, who came from the opposite direction at the same time. Thereafter, the local watching center 3 takes over the processing of the recorded images from Mr. B's drive recorder and tracks the car carrying person X. The upper right corner of FIG. In this way, the local monitoring center 3 can link together the drive recorder information of Mr. C, Mr. D, etc., and track down the person X, who is likely the culprit of the crime, to the vicinity of his home or hideout.

[0067] FIG. 18 shows an example of the process for recognizing a vehicle passing in the opposite lane around the time of the occurrence of an incident or accident, as explained in the flowchart of FIG. As shown in the upper left of Figure 18, for example, at the time when incident / accident A2 occurs, a patrol vehicle M1 is about one minute away from arriving at the scene or 100 m away, and the situation at the time of incident / accident A2 is too far away to be seen from vehicle M1. In such a situation, for example, as shown in the upper right corner of the upper row of Figure 18, suppose that 30 seconds after the incident / accident A2 occurred or 50 meters from the incident site, the patrol officer's vehicle M1 passes oncoming vehicles M2, M3, and M4 in succession. Even 30 seconds later or 50 meters before the incident / accident A2, the patrol officer is still far from the incident / accident A2, so it is unlikely that the patrol officer himself witnessed the incident / accident A2 or that the incident / accident A2 was recorded on his dashcam. However, it is highly likely that oncoming vehicles M2, M3, and M4 passed the incident / accident A2 exactly 30 seconds before or 50 meters before the incident / accident A2. In such a case, the local watch center 3 reads the license plate numbers of each oncoming vehicle, confirms whether or not the owner of each vehicle has a dashcam, and then requests recorded images or eyewitness information from the vicinity of the incident / accident site (the time of the incident / accident) to be provided. This makes it possible to obtain recorded images or eyewitness information from the time of the incident / accident.

[0068] Similarly, as shown in the lower right of Figure 18, for example, the time when incident / accident A2 occurs is approximately one minute or 100 meters after a patrol vehicle K1 passes the scene, and the situation at the time of incident / accident A2 cannot be seen from vehicle K1. As shown in the lower left of the bottom row of Figure 18, assume that 30 seconds after the incident / accident A2 occurred, the patrol vehicle K1 passed oncoming vehicles K2, K3, and K4 in succession. Even if the patrol vehicle K1 was far away from the incident / accident scene 30 seconds before or 50 meters after passing the scene, it is unlikely that the patrol vehicle witnessed the incident / accident A2 or that the incident / accident A2 was recorded on its dashcam. However, it is highly likely that the oncoming vehicles K2, K3, and K4 encountered the scene of the incident / accident A2 exactly 30 seconds later or 50 meters after passing the scene. In such a case, the local monitoring center 3 reads the license plate numbers of each oncoming vehicle, confirms whether or not the owners of each vehicle have a dashcam, and then requests recorded images or eyewitness information from the vicinity of the incident / accident scene (the time of the incident / accident) to be provided. This makes it possible to obtain recorded images or eyewitness information from the time of the incident / accident.

[0069] In this way, even if appropriate eyewitness information about an incident or accident cannot be obtained from the patrol officer's drive recorder, the necessary images can be obtained by requesting oncoming vehicles identified from recorded images.

[0070] [Variations] The embodiment examples described so far have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. For example, the processing described in Figures 13 to 18 is intended to obtain images of the occurrence of an incident or accident, and tracking images of the perpetrator, but the present invention can also be applied when searching for witnesses to an incident or accident, missing persons, etc.

[0071] In addition, the configuration diagram shown in Figure 1 shows only the control lines and information lines that are considered necessary for explanation, and does not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. Furthermore, the processing flows shown in the flowcharts of FIGS. 5 to 10 are also examples, and as long as the processing results are the same, the order of some of the processing may be changed or multiple processes may be executed simultaneously.

[0072] In the above-described embodiment, the terminal as a smartphone is equipped with a positioning unit, and the regional monitoring center 3 acquires a movement trajectory from the smartphone and determines the movement trajectory of the drive recorder registered together with the smartphone. On the other hand, if the drive recorder has a built-in positioning unit, the regional monitoring center 3 may acquire a movement trajectory from the terminal as the drive recorder. Furthermore, images may be acquired from the drive recorder either via another terminal such as a smartphone or directly from the drive recorder.

[0073] Furthermore, the local monitoring center 3 may perform similar control processing by installing a program that executes the processing described in the above-described embodiment in, for example, an existing computer (information processing device). In this case, the program to be installed in the computer may be stored in the storage or memory of the computer, or may be stored in and transferred from an external memory, an IC card, an SD card, an optical disk, or other recording medium. Furthermore, the storage unit for storing the movement trajectories and images at the local watch center 3 may be provided in a device separate from the local watch center 3. For example, a server connected to the Internet may perform the process of storing the movement trajectories and images, and the local watch center 3 may acquire the data stored in the server. In this case, processing such as analysis may also be performed on the server side. [Explanation of symbols]

[0074] DESCRIPTION OF SYMBOLS 1, 1a, 1b...First terminal (smartphone), 2, 2a, 3b...Second terminal (drive recorder), 3...Communication center, 4, 4a, 4b...Third terminal (surveillance camera), 5...Police station / security company, 11...Processing unit, 12...Main memory unit, 13...Auxiliary memory unit, 14...Communication unit, 15...Input unit, 16...Positioning unit, 17...Bus, 21...Processing unit, 22...Main memory unit, 23...Auxiliary memory unit, 24...Communication unit, 25...Camera, 26...Bus, 31...Communication unit, 32...Processing unit, 32a...Location information processing unit, 32b...Analysis unit, 33...Main memory unit, 34...Auxiliary memory unit, 35...Display unit, 36...Bus, 41...Camera, 42...Auxiliary memory unit, 101, 102...Vehicle, 301...House, 302...Electric pole

Claims

1. A monitoring system including a monitoring center that monitors a predetermined area and is configured with an information processing device, The monitoring center a communication unit that communicates with a drive recorder that captures images of the surroundings of the vehicle in which the drive recorder is installed and stores the captured images, or a communication terminal that can transmit and receive data to and from the drive recorder; a location information processing unit that collects movement trajectories or locations of the communication terminal or the drive recorder; and an analysis unit that, when a monitoring image capturing a location, moving object, or person to be monitored within the area is required, determines the drive recorder mounted on a vehicle from which the monitoring image may be obtained based on the movement trajectory or position obtained by the position information processing unit, and collects and analyzes the captured images stored in the determined drive recorder via the communication unit to obtain information about the location, moving object, or person to be monitored. Monitoring system.

2. The location information processing unit further identifies a vehicle from which the monitoring image may be obtained from the images collected from the drive recorder or the images from a monitoring camera installed at a fixed location, The communication unit requests the communication terminal of the identified vehicle owner to collect the captured images stored in the drive recorder. The monitoring system according to claim 1 .

3. the location information processing unit determines a parking position of the vehicle equipped with the drive recorder, and determines the drive recorder from which the monitoring image may be obtained based on the determined parking position; The analysis unit collects and analyzes the captured images stored by the drive recorder of the determined parking position via the communication unit. The monitoring system according to claim 1 .

4. When a monitoring image of a location to be monitored within the area at a specific time is required, the location information processing unit determines a vehicle that has passed through the location to be monitored before the specific time, The communication unit acquires a captured image stored in the drive recorder mounted on the determined vehicle, The analysis unit determines, from the acquired photographed images, passing vehicles that may pass through the location to be monitored at the specific time and after that time, and collects and analyzes photographed images stored in the drive recorder of the determined passing vehicles via the communication unit. The monitoring system according to claim 1 .

5. The communication unit delivers a facial image of the person to be monitored to the communication terminal. The monitoring system according to claim 1 .

6. The monitoring image is an image obtained by searching for a face image of the person to be monitored by face recognition processing from among the photographed images acquired by the monitoring center. The monitoring system according to claim 1 .

7. The analysis unit automatically analyzes and finds the moving object or person to be monitored from the recorded images collected from around the time when the incident requiring monitoring occurred, and automatically tracks the moving object or person to be monitored. The monitoring system according to claim 1 .

8. If the drive recorder is not installed in a vehicle from which the monitoring image may be obtained, the communication unit sends a request for witness information to a communication terminal of the owner of the vehicle. The monitoring system according to claim 1 .

9. A monitoring method in which a monitoring center configured with an information processing device monitors a predetermined area, The process performed by the monitoring center is as follows: a communication process for communicating with a drive recorder that captures images of the surroundings of the vehicle and stores the captured images, or a communication terminal that can transmit and receive data to and from the drive recorder; A location information process for collecting a movement trajectory or a position of the communication terminal or the drive recorder; When a monitoring image capturing a location, moving object, or person to be monitored within the area is required, the method determines the drive recorder mounted on the vehicle from which the monitoring image can be obtained based on the movement trajectory or position obtained by the position information processing, and collects and analyzes the captured images stored in the determined drive recorder by the communication processing to obtain information about the location, moving object, or person to be monitored. How to monitor.

Images