Monitoring device, monitoring system, monitoring method, and monitoring program

Abstract

A monitoring device (10) is provided with: a camera image acquisition unit (11) that acquires, from a camera located at a first point, a camera image obtained by capturing a road at the first point; a traffic information acquisition unit (12) that analyzes the camera image to acquire traffic information indicating a traffic condition of the road at the first point; a road information acquisition unit (13) that acquires road information indicating a traffic obstruction condition at a second point leading to the first point; and an abnormality detection unit (14) that detects an abnormality in the camera image on the basis of the traffic information, statistical information of the traffic information, and the road information.

Description

Technical Field

[0001] The present disclosure relates to a monitoring device, a monitoring system, a monitoring method, and a non - temporary computer - readable medium.

Background Art

[0002] There is a traffic monitoring system that installs cameras on roads and monitors traffic conditions and the like based on the captured images. In such a traffic monitoring system, since cameras are increasingly installed in various locations, it is desired to improve the efficiency of maintenance work including anomaly detection after installation.

[0003] As a technology related to anomaly detection in a monitoring system, for example, Patent Document 1 is known. In Patent Document 1, anomalies in a learning model are detected by comparing the output result of the learning model that performs monitoring processing in the monitoring system with the statistical information that is its expected value.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In a traffic monitoring system, since monitoring is performed by analyzing camera images captured by cameras installed on roads, it is necessary to detect anomalies in the camera images for maintenance. However, in related technologies such as Patent Document 1, application to anomaly detection of camera images in a traffic monitoring system is not considered. For this reason, in related technologies, anomalies in camera images captured by cameras installed on roads may not be accurately detected.

[0006] In view of these challenges, this disclosure aims to provide a monitoring device, monitoring system, monitoring method, and non-temporary computer-readable medium that can accurately detect abnormalities in camera images. [Means for solving the problem]

[0007] The monitoring device relating to this disclosure comprises: camera image acquisition means for acquiring camera images of the road at a first location from a camera positioned at the first location; traffic information acquisition means for analyzing the camera images and acquiring traffic information indicating the traffic conditions on the road at the first location; road information acquisition means for acquiring road information indicating the traffic obstruction situation at a second location leading to the first location; and anomaly detection means for detecting anomalies in the camera images based on the traffic information, statistical information of the traffic information, and the road information.

[0008] Audit regarding this disclosure View The system comprises a camera positioned at a first location and a monitoring device, the monitoring device comprising: camera image acquisition means for acquiring camera images of the road at the first location from the camera; traffic information acquisition means for analyzing the camera images and acquiring traffic information indicating the traffic conditions on the road at the first location; road information acquisition means for acquiring road information indicating traffic obstruction conditions at a second location leading to the first location; and anomaly detection means for detecting anomalies in the camera images based on the traffic information, statistical information of the traffic information, and the road information.

[0009] The monitoring method relating to this disclosure involves acquiring camera footage of the road at a first location from a camera placed at the first location, analyzing the camera footage, acquiring traffic information indicating the traffic conditions on the road at the first location, acquiring road information indicating the traffic obstruction situation at a second location leading to the first location, and detecting abnormalities in the camera footage based on the traffic information, statistical information on the traffic information, and the road information.

[0010] The non-temporary computer-readable medium relating to this disclosure is a non-temporary computer-readable medium that stores a monitoring program for causing a computer to execute a process that acquires camera footage of the road at a first location from a camera placed at the first location, analyzes the camera footage, acquires traffic information indicating the traffic conditions on the road at the first location, acquires road information indicating the traffic obstruction situation at a second location leading to the first location, and detects abnormalities in the camera footage based on the traffic information, statistical information of the traffic information, and the road information. [Effects of the Invention]

[0011] This disclosure provides a monitoring device, monitoring system, monitoring method, and non-temporary computer-readable medium capable of accurately detecting abnormalities in camera images. [Brief explanation of the drawing]

[0012] [Figure 1] This is a configuration diagram showing an overview of the monitoring device according to the embodiment. [Figure 2] This is a configuration diagram showing an example of the configuration of the monitoring system according to Embodiment 1. [Figure 3] This flowchart shows an example of the operation of the statistical information generation process according to Embodiment 1. [Figure 4] This flowchart shows an example of the operation of the camera anomaly detection process according to Embodiment 1. [Figure 5] This flowchart shows an example of the operation of the camera abnormality detection process according to Embodiment 1. [Figure 6] This is a configuration diagram showing an example of the configuration of the monitoring system according to Embodiment 2. [Figure 7] This is a configuration diagram showing an overview of the computer hardware according to the embodiment. [Modes for carrying out the invention]

[0013] The embodiments will be described below with reference to the drawings. In each drawing, the same elements are denoted by the same reference numerals, and redundant explanations will be omitted where necessary.

[0014] (Summary of the embodiment) When considering systems that monitor vehicles traveling on roads, such as traffic monitoring systems, road traffic conditions are not always constant. For example, if an accident occurs on the monitored road, traffic volume at other locations may decrease from normal levels. In such cases, simply comparing the detection results of a learning model with its statistical information, as in Patent Document 1, will result in a discrepancy, or divergence, between the detection results and the statistical information, leading to the system being judged as abnormal.

[0015] Specifically, the traffic monitoring system installs cameras at multiple points along the road and obtains traffic information, including traffic volume, from the camera footage captured at each point. For example, if an accident occurs at another point connected to a camera location, such as upstream, and the traffic volume at the camera location changes, a discrepancy may arise between the detection results and the statistical information, which could be judged as an anomaly and trigger an alert.

[0016] In this embodiment, the abnormalities in the camera image detected are those that may necessitate maintenance work on the installed camera. That is, an abnormality in the camera image may be an abnormality that occurs in all or part of the image due to a camera malfunction or dirt on the camera lens, or it may be an abnormality in the image caused by a camera malfunction. For example, if the camera malfunctions, the entire image becomes unrecognizable, and if snow or other debris adheres to the camera lens, part of the image becomes unrecognizable. Unrecognizable states include, for example, the image becoming black, blurry, or nothing being captured. Therefore, abnormalities that occur when an accident occurs as described above are not abnormalities caused by a camera malfunction, and thus should not be detected as abnormalities in the camera image.

[0017] Therefore, in the embodiment, by considering the road conditions at other locations connected to the location where the camera is installed, it becomes possible to accurately detect anomalies in the camera video.

[0018] FIG. 1 shows an overview of the monitoring device according to the embodiment. As shown in FIG. 1, the monitoring device 10 according to the embodiment includes a camera video acquisition unit 11, a traffic information acquisition unit 12, a road information acquisition unit 13, and an anomaly detection unit 14.

[0019] The camera video acquisition unit 11 acquires a camera video obtained by shooting the road at the first location from a camera disposed at the first location. The traffic information acquisition unit 12 analyzes the camera video acquired by the camera video acquisition unit 11 and acquires traffic information indicating the traffic conditions of the road at the first location. The road information acquisition unit 13 acquires road information indicating the traffic obstruction conditions of the road at the second location leading to the first location. The anomaly detection unit 14 detects an anomaly in the camera video acquired by the camera video acquisition unit 11 based on the traffic information acquired by the traffic information acquisition unit 12, the statistical information of the traffic information, and the road information acquired by the road information acquisition unit 13.

[0020] Thus, in the embodiment, in addition to the traffic information and the statistical information of the traffic information obtained from the camera video at the first location, based on the road information at the second location leading to the first location, an anomaly in the camera video at the first location is detected. Thereby, by considering the road information at the second location, it is possible to determine an anomaly in the camera video at the first location, suppress the generation of unnecessary alerts, and accurately detect an anomaly in the camera video.

[0021] (Embodiment 1) Hereinafter, Embodiment 1 will be described with reference to the drawings. FIG. 2 shows a configuration example of the monitoring system according to the present embodiment. The monitoring system 1 according to the present embodiment is a system that monitors the traffic conditions of a road, that is, the traffic flow, and is also a system that detects or monitors anomalies in camera videos captured by cameras installed or disposed on the road.

[0022] As shown in Figure 2, the monitoring system 1 according to this embodiment includes a central server 100, a plurality of cameras 200 (e.g., 200a, 200b), and an edge processing unit 300. The cameras 200 and the edge processing unit 300 are connected wirelessly or via wired connections. The edge processing unit 300 and the central server 100 are also connected wirelessly or via wired connections. Each device may be directly connected or connected via any network. The cameras 200 and the central server 100 may also be connected in a way that allows for communication.

[0023] Each camera 200 is a surveillance camera installed or positioned at various points along the road 400, capturing images of the road 400 at each point. The cameras 200 may continuously capture images of the road 400, or they may capture images at regular intervals or triggered by user actions. The cameras 200 transmit the captured camera images to the edge processing unit 300. The camera images may be a so-called video streaming service, or a series of still images captured at predetermined intervals, such as periodically. In this example, the camera images are transmitted from the cameras 200 to the central server 100 via the edge processing unit 300. Alternatively, the camera images may be transmitted directly from the cameras 200 to the central server 100 without going through the edge processing unit 300.

[0024] For example, each camera 200 is installed at each intersection of the road 400 to be monitored, and captures video including the road at each intersection. However, cameras may be installed at locations other than intersections, as long as they can capture images of various points along the road 400. In this example, camera 200a is installed at intersection 401a (the first point), and camera 200b is installed at intersection 401b (the second point). For example, camera 200a is fixed to traffic light 402a installed at intersection 401a, and camera 200b is fixed to traffic light 402b installed at intersection 401b. Cameras may be fixed at locations other than traffic lights, as long as they can capture images of the intersections that are the monitoring points.

[0025] Camera 200a can monitor traffic conditions at intersection 401a, and camera 200b can monitor traffic conditions at intersection 401b. In this example, camera 200a at intersection 401a is used for anomaly detection of its camera image, and camera 200b at intersection 401b is used to acquire road information used in the anomaly detection process of camera image 200a. Intersection 401b is a point that is connected to intersection 401a by at least road 400. For example, intersection 401b is an intersection upstream of intersection 401a, but it may also be an intersection downstream of intersection 401a. For example, intersection 401b is an intersection adjacent to intersection 401a, but it may also be an intersection further adjacent. In addition, the camera 200b for acquiring road information is not limited to one unit, but may be multiple cameras installed at multiple locations.

[0026] The edge processing unit 300 is a server installed or located on the edge side of the system, and is, for example, a MEC (Multi-access Edge Computing) device. The edge processing unit 300 includes a VMS (Video Management System) 301. The VMS 301 is a video management means that manages the camera images of each camera 200. The VMS 301 acquires the camera images transmitted from multiple cameras 200 (200a and 200b) and transmits the acquired camera images to the central server 100. The VMS 301 may change the video format and bitrate as needed. In addition, multiple edge processing units 300 may be deployed, and the camera images of multiple cameras 200 may be transmitted from multiple edge processing units 300 to the central server 100.

[0027] The central server 100 is a server installed or located on the central side of the system, and is, for example, a cloud server built on the cloud. In the monitoring system 1, the central server 100 is a device that performs traffic condition monitoring processing and is also a device that performs anomaly detection processing of camera images.

[0028] The center server 100 includes a camera image acquisition unit 101, a traffic information recognition unit 102, a traffic information database 103, a statistical information generation unit 104, a statistical information storage unit 105, a road information recognition unit 106, an anomaly detection unit 107, and an output unit 108. Note that the configuration of the center server 100 is just one example, and other configurations are acceptable as long as they enable the operation described in this embodiment. Each function of the center server 100 may be implemented in a single device or in multiple devices. Some functions of the center server 100 may be placed in external devices or edge processing devices 300.

[0029] The camera image acquisition unit 101 acquires camera images captured by each camera 200. In this example, the camera image of intersection 401a captured by camera 200a and the camera image of intersection 401b captured by camera 200b are received and acquired via the edge processing unit 300.

[0030] The traffic information recognition unit 102 is a traffic information acquisition unit that recognizes and acquires traffic information for each location from the camera images of each camera 200 acquired by the camera image acquisition unit 101. In this example, the traffic information recognition unit 102 recognizes road traffic information at intersection 401a from the camera image of intersection 401a taken by camera 200a. In addition, traffic information for multiple locations may be recognized from the camera images of multiple cameras as needed. For example, road traffic information at intersection 401b may be recognized from the camera image of camera 200b, or traffic information for other locations may be recognized from the camera images of other cameras 200.

[0031] Traffic information refers to road traffic conditions, specifically information indicating traffic flow. For example, traffic information includes the volume, speed, and type of traffic on the road. Traffic includes not only vehicles but also people, such as pedestrians. In the case of vehicle traffic information, it includes the volume of vehicles, the speed of the vehicles that passed, and the type of vehicles that passed, and may also include the volume and speed for each type of vehicle. Vehicle volume includes the number of vehicles that passed during a specified period. Vehicle types include passenger cars, trucks, buses, motorcycles, bicycles, etc. In the case of pedestrian traffic information, it includes the volume of pedestrians, the speed of pedestrians, and pedestrian attributes, and may also include the volume and speed for each attribute. Pedestrian volume includes the number of people that passed during a specified period. Pedestrian attributes include gender, age, etc.

[0032] The traffic information recognition unit 102 may recognize the traffic situation of vehicles and people in the camera image using an AI (Artificial Intelligence) engine (a learning model using machine learning) for traffic information recognition. The AI ​​engine may be a CNN (Convolutional Neural Network) or any other neural network. For example, by machine learning the features of images of vehicles and pedestrians and labels of vehicle types and pedestrian attributes, it is possible to recognize the vehicle types and pedestrian attributes in the image and obtain the traffic volume and speed of the recognized vehicles and pedestrians.

[0033] The traffic information recognition unit 102 stores the traffic information recognized from the camera footage in the traffic information DB 103, along with the date and time the camera footage was taken, for traffic monitoring or statistical information generation. For example, the camera footage has the date and time it was taken by camera 200 set. If the camera footage does not have a date and time set, the date and time the center server 100 received the camera footage may be used. In addition, the traffic information recognition unit 102 outputs the traffic information recognized from the camera footage to the anomaly detection unit 107 for anomaly detection of the camera footage.

[0034] The traffic information DB 103 is a traffic information storage unit, such as a database, that stores and accumulates traffic information for each location recognized by the traffic information recognition unit 102 from camera images. The traffic information DB 103 may store not only traffic information for each location, but also camera images for each location. For example, the traffic information DB 103 may be a non-volatile memory such as flash memory or a hard disk drive.

[0035] The statistical information generation unit 104 generates statistical information on traffic information for each location stored in the traffic information DB 103. In this example, the statistical information generation unit 104 generates statistical information on traffic information for intersection 401a recognized by the traffic information recognition unit 102. In addition, statistical information on traffic information for multiple locations may be generated as needed. For example, statistical information on traffic information for intersection 401b may be generated, or statistical information on traffic information for other locations may be generated.

[0036] The statistical information generation unit 104 calculates statistical information for traffic information at each location based on the aggregated results of traffic information over a predetermined period. For example, the statistical information may be the average or total value over the predetermined period, but it may also be other statistical values ​​such as variance or median. The statistical information may be an aggregated statistical value for the entire predetermined period, or an aggregated statistical value for each time period, day of the week, month, etc. For example, it may be an aggregated statistical value for each time period on each day of the week. Also, if the traffic information includes vehicle traffic information and pedestrian traffic information, statistical information for vehicles and statistical information for pedestrians may be obtained. For example, statistical information for each type of vehicle and statistical information for each attribute of a person may be obtained. Statistical information for each type of vehicle may be the average or total traffic volume, average speed, etc. Statistical information for each attribute of a person may be the average or total traffic volume, total speed, etc. The statistical information generation unit 104 stores the generated statistical information for each location in the statistical information storage unit 105.

[0037] The statistical information storage unit 105 stores the traffic information statistics for each location generated by the statistical information generation unit 104. For example, similar to the traffic information DB 103, it may be a non-volatile memory such as flash memory or a hard disk drive.

[0038] The road information recognition unit 106 is a road information acquisition unit that recognizes and acquires road information from camera images acquired by the camera image acquisition unit 101. In this example, the road information recognition unit 106 recognizes road information for the road at intersection 401b from the camera image of intersection 401b taken by camera 200b. In addition, if necessary, road information for multiple locations may be recognized from camera images of multiple cameras. For example, road information for other locations may be recognized from camera images of other cameras 200, or road information for the road at intersection 401a may be recognized from the camera image of camera 200a.

[0039] Road information is information that indicates the extent of traffic disruptions on roads. For example, road information may include traffic information at a given point, or information about external factors that affect camera footage. For instance, road information may include information about road closures due to accidents, construction, landslides, etc., or information based on these. Road information may indicate whether or not an event is causing traffic disruptions on the road, or it may indicate the level of traffic disruption. For example, it may indicate whether or not there is a traffic disruption depending on whether or not there is a road closure. It may also indicate the level of traffic disruption depending on the number of restricted lanes, or it may indicate the level of traffic disruption depending on whether or not people or vehicle types can pass.

[0040] The road information recognition unit 106 may recognize the condition of road obstructions in the camera image using an AI engine for road information recognition. The AI ​​engine may be a CNN or other neural network. For example, by machine learning the features of road closure images and labels indicating the presence or absence of obstructions or the level of obstruction, the presence or absence of road obstructions or the level of obstruction in the image can be recognized. The road information recognition unit 106 outputs the road information recognized from the camera image to the anomaly detection unit 107.

[0041] The anomaly detection unit 107 detects anomalies in the camera image acquired by the camera image acquisition unit 101. It can also be said that the anomaly detection unit 107 detects anomalies in the camera image caused by a camera malfunction. In this example, the anomaly detection unit 107 detects anomalies in the camera image captured by camera 200a at intersection 401a. Alternatively, the same detection method may be used to detect anomalies in the camera image captured by camera 200b at intersection 401b, or to detect anomalies in the camera image captured by other cameras 200 at other locations.

[0042] The anomaly detection unit 107 detects anomalies in the camera image of camera 200a at intersection 401a based on road information of intersection 401b acquired from the camera image of camera 200b, traffic information of intersection 401a acquired from the camera image of camera 200a, and statistical information of traffic information stored in the statistical information storage unit 105. For example, the anomaly detection unit 107 determines whether the camera image of camera 200a is anomaly by using the traffic information and statistical information of traffic information recognized from the camera image of camera 200a, according to the road information of intersection 401b. Note that the anomaly determination of the camera image of camera 200a may also be performed according to road information from multiple locations.

[0043] For example, the anomaly detection unit 107 performs an anomaly detection on the camera image of camera 200a if the road information indicates that there is no obstruction to traffic at intersection 401b, or if the road information includes information indicating that there is no obstruction to traffic. However, if the road information indicates that there is an obstruction to traffic at intersection 401b, or if the road information includes information indicating that there is an obstruction to traffic, the unit does not detect an anomaly in the camera image of camera 200a. For example, if there is an obstruction to traffic at intersection 401b, it is not necessary to perform an anomaly detection on the camera image of camera 200a. In this case, it is also possible not to compare the traffic information with the statistical information, or not to judge an anomaly in the camera image even if there is a large discrepancy between the traffic information and the statistical information. Furthermore, in the anomaly detection of the camera image, an anomaly in the camera image of camera 200a is detected based on the discrepancy between the traffic information and the statistical information at intersection 401a. The anomaly detection unit 107 may detect whether or not there is an anomaly in the camera image based on the discrepancy between the traffic information and statistical information of intersection 401a, or it may detect the level of anomaly in the camera image according to the degree of discrepancy. Alternatively, it may detect whether or not there is an anomaly in the camera image and the level of anomaly by considering the duration of time during which a large discrepancy between the traffic information and statistical information of intersection 401a persists.

[0044] If there is no traffic disruption at intersection 401b, i.e., no external factors, but the current traffic information based on the camera footage from camera 200a deviates significantly from the statistical information that indicates normal conditions, there is a high probability that the video is abnormal due to a camera malfunction in camera 200a. Therefore, if the road information at intersection 401b indicates that there is no traffic disruption, and there is a large discrepancy between the current traffic information and the statistical information, it is possible to detect an abnormality in the camera footage from camera 200a. In other words, it is possible to detect the possibility of a video abnormality due to camera malfunction, such as camera failure or dirt on the camera lens, for example, depending on the presence or level of the malfunction. To put it another way, it is possible to detect the possibility of a camera malfunction. Furthermore, if the camera malfunctions, the large discrepancy between the current traffic information and the statistical information will continue for a long time, so the possibility of a camera malfunction can be reliably detected by the duration of this large discrepancy.

[0045] The output unit 108 outputs the abnormality detection result of the camera image detected by the abnormality detection unit 107. For example, the output unit 108 is a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The output unit 108 is not limited to a display device and may also include an output device such as an audio output device. The output unit 108 displays an alarm when an abnormality is detected in the camera image and also displays information according to the detected abnormality level. By displaying the detection results of the camera image of each camera 200, the monitor can monitor camera malfunctions of each camera. In addition, the output unit 108 may display the camera image of each camera 200, traffic information, road information, etc. For example, by displaying traffic information and road information along with the camera image of each camera 200, the monitor can monitor traffic at each location.

[0046] Next, the operation of the center server 100 according to this embodiment will be explained using Figures 3 to 5. Figure 3 shows an example of the operation of the statistical information generation process in the center server 100, illustrating the process flow from acquiring camera images to generating traffic information statistics. For example, traffic information is acquired and stored from camera images continuously captured by the camera, and statistical information is generated at a predetermined timing. Note that the process in Figure 3 is executed at least before the camera image anomaly detection process.

[0047] As shown in Figure 3, the central server 100 acquires the first camera image captured by the first camera set at the first location (S101). The camera 200a (first camera) installed at intersection 401a (first location) continuously photographs the road at intersection 401a, for example, and transmits the captured camera image (first camera image) to the edge processing unit 300. The edge processing unit 300 receives the camera image transmitted from camera 200a and transmits the received camera image to the central server 100. In the central server 100, the camera image acquisition unit 101 receives and acquires the camera image from camera 200a transmitted from the edge processing unit 300. The camera image acquisition unit 101 may display the acquired camera image from camera 200a on the display of the output unit 108.

[0048] Next, the center server 100 acquires traffic information from the acquired first camera image (S102). The traffic information recognition unit 102 analyzes the camera image from camera 200a acquired by the camera image acquisition unit 101 and acquires road traffic information at intersection 401a. The traffic information recognition unit 102 inputs the camera image from camera 200a into the AI ​​engine for traffic information recognition to acquire road traffic information at intersection 401a. The traffic information recognition unit 102 may display the acquired traffic information for intersection 401a together with the camera image from camera 200a on the display of the output unit 108.

[0049] Next, the central server 100 stores the acquired traffic information (S103). The traffic information recognition unit 102 stores the traffic information for intersection 401a acquired from the camera image of camera 200a in the traffic information DB 103. For example, the traffic information recognition unit 102 sequentially stores and accumulates traffic information acquired from the camera image that camera 200a continuously captures in the traffic information DB 103 along with the date and time of capture.

[0050] Next, the central server 100 generates statistical information of the accumulated traffic information (S104). The statistical information generation unit 104 refers to the traffic information DB 103 and generates statistical information of the accumulated traffic information at intersection 401a. For example, the statistical information generation unit 104 aggregates traffic information over a predetermined period at a predetermined statistical information generation timing to obtain statistical information. The statistical information generation timing may be at regular intervals or before the start of the anomaly detection process. The statistical information generation unit 104 stores the obtained statistical information of traffic information at intersection 401a in the statistical information storage unit 105.

[0051] Figure 4 shows an example of the operation of the camera anomaly detection process in the central server 100, illustrating the process flow from acquiring the current camera image to detecting anomalies in the acquired camera image.

[0052] As shown in Figure 4, the central server 100 acquires the first camera image from the first camera installed at the first location (S111). The camera image acquisition unit 101 acquires the camera image from the camera at the first location at the timing of an anomaly detection in the camera image. The anomaly detection timing may be at regular intervals or at a timing specified by the monitor. The camera image acquisition unit 101 acquires the camera image (first camera image) from camera 200a (first camera) installed at intersection 401a (first location), which is the target of anomaly detection, in the same way as S101 in Figure 3.

[0053] Next, the center server 100 acquires traffic information from the first camera image acquired (S112). The traffic information recognition unit 102 acquires traffic information at intersection 401a from the camera image of camera 200a acquired by the camera image acquisition unit 101, similar to S102 in Figure 3.

[0054] Next, the center server 100 acquires the second camera image from the second camera installed at the second location (S113). The camera image acquisition unit 101, similar to S111, acquires the camera image (second camera image) from camera 200b (second camera) installed at intersection 401b (second location), which is used for acquiring road information.

[0055] Next, the center server 100 acquires road information from the acquired second camera image (S114). The road information recognition unit 106 analyzes the camera image from camera 200b acquired by the camera image acquisition unit 101 and acquires road information for the road at intersection 401b. The road information recognition unit 106 inputs the camera image from camera 200b into the AI ​​engine for road information recognition to acquire road information for the road at intersection 401b.

[0056] Next, the center server 100 performs camera anomaly detection processing on the first camera image (S115). Figure 5 shows a specific example of the camera anomaly detection processing (S115) in Figure 4.

[0057] As shown in Figure 5, the anomaly detection unit 107 determines whether or not there is a traffic obstruction at the second location (S121). The anomaly detection unit 107 determines whether or not there is a traffic obstruction at intersection 401b (the second location) based on the road information at intersection 401b (the second location) acquired by the road information recognition unit 106 from the camera image of camera 200b. If road information from multiple locations is used, the presence or absence of a traffic obstruction may be determined based on the road information from multiple locations.

[0058] If the anomaly detection unit 107 determines that there is no disruption to traffic at the second location (S121 / YES), it compares the traffic information and statistical information at the first location (S122). If it determines that there is a disruption to traffic at the second location (S121 / NO), it terminates processing without performing an anomaly detection on the camera image. In other words, in this case, the anomaly detection unit 107 does not detect an anomaly in the camera image. Furthermore, if the acquired road information indicates that there is no disruption to traffic at intersection 401b, the anomaly detection unit 107 compares the current traffic information acquired by the traffic information recognition unit 102 from the camera image of camera 200a with the statistical information of the traffic information stored in the statistical information storage unit 105. The current traffic information and statistical information may also be compared if the road information indicates that there is no disruption to all traffic, i.e., all traffic at one or more locations, or if it indicates that there is no disruption to some traffic, i.e., some traffic at one or more locations. Processing may be terminated if it indicates that all traffic is affected, or if it indicates that some traffic is affected. Alternatively, if the level of traffic disruption is less than a predetermined level, the current traffic information may be compared with statistical information, and processing may be terminated if the level of traffic disruption is greater than the predetermined level.

[0059] For example, if the statistical information includes data from multiple locations, the system selects the traffic statistics for intersection 401a (camera 200a) and compares the selected statistics with the current traffic information. Furthermore, if the statistical information includes statistics for specific times of day or days of the week, the system selects the statistics corresponding to the time and day of the current traffic information (camera 200a footage) and compares the selected statistics with the current statistics.

[0060] Next, the anomaly detection unit 107 determines the magnitude of the discrepancy between the traffic information and statistical information at the first location (S123). If the discrepancy is large (S123 / YES), it determines that there is an anomaly in the camera image (S124). If the discrepancy is small (S123 / NO), it determines that there is no anomaly in the camera image (S125). The output unit 108 outputs the anomaly detection result of the camera image to a display or the like.

[0061] For example, the anomaly detection unit 107 calculates the discrepancy, or difference, between the current traffic information and statistical information obtained from the camera image of camera 200a, and detects an anomaly in the camera image of camera 200a based on the comparison result between the calculated discrepancy and a predetermined threshold. In other words, if the calculated discrepancy is greater than the predetermined threshold, the anomaly detection unit 107 determines that there is an anomaly in the camera image of camera 200a, and if the discrepancy is less than the predetermined threshold, it determines that there is no anomaly in the camera image of camera 200a.

[0062] A predetermined threshold for determining whether or not there is an abnormality in the camera image may be set in advance. For example, if the traffic information and statistical information includes traffic information and statistical information for vehicles and people, a threshold for vehicles and a threshold for people may be set, or a threshold for each type of vehicle and a threshold for each attribute of a person may be set. In this case, the abnormality in the camera image may be determined based on the comparison result of any one threshold or all of the thresholds, or the abnormality in the camera image may be determined for each threshold. In addition, the predetermined threshold may be set according to road information, etc. For example, a predetermined threshold may be set according to the level of traffic disruption at intersection 401b indicated by the road information. The greater the level of traffic disruption, the greater the predetermined threshold may be set, and the less the level of traffic disruption, the smaller the predetermined threshold may be set.

[0063] Alternatively, abnormalities in the camera image may be detected based on the duration of a large discrepancy between the current traffic information and the statistical information. For example, the abnormality detection unit 107 may determine that there is an abnormality in the camera image if the discrepancy remains above a predetermined threshold for a predetermined time, and determine that there is no abnormality in the camera image if the discrepancy does not remain above a predetermined threshold for a predetermined time.

[0064] Furthermore, the anomaly detection unit 107 may determine the discrepancy between the current traffic information and statistical information obtained from the camera image of camera 200a, and determine the anomaly level of the camera image of camera 200a according to the degree or magnitude of the discrepancy. For example, the anomaly level may be determined in three stages: high level, medium level, and low level, or it may be determined in any number of levels. For example, if the degree of discrepancy is large and the anomaly level is determined to be high, for example, if the number of passing vehicles remains at 0 and does not change, there is a possibility that the camera itself is malfunctioning. Also, if the degree of discrepancy is small and the anomaly level is determined to be low, for example, if the number of passing vehicles remains low compared to the statistical information, there is a possibility that the camera or lens is in an abnormal state due to external factors such as dirt or snow accumulation.

[0065] The criteria or range for determining the abnormality level of camera footage may be predetermined, similar to the specified thresholds mentioned above, or they may be set according to road information, etc. For example, criteria for determining the abnormality level may be set for each type of vehicle or for each person's attributes. Criteria for determining the abnormality level may be set according to the level of traffic disruption at intersection 401b indicated by the road information. The abnormality level of camera footage may also be determined based on the degree of deviation estimated from the level of traffic disruption at intersection 401b indicated by the road information, and the degree of deviation from the traffic information and statistical information.

[0066] For example, the abnormality level of the camera image may be determined based on the duration of the discrepancy between the current traffic information and the statistical information. For instance, the abnormality detection unit 107 may determine that the abnormality level is high if the discrepancy is greater than a predetermined threshold and that condition persists for a predetermined time, and that the abnormality level is low if the discrepancy is greater than a predetermined threshold and that condition does not persist for a predetermined time.

[0067] As described above, in this embodiment, in a monitoring system for monitoring road traffic conditions, abnormality detection of camera images from cameras installed on the road is performed based on road information from other points on the road. For example, if there is a traffic disruption at another point due to a road closure or the like, external factors at other points are likely to affect the camera images, so abnormality detection of the camera images is not performed. However, if there is no traffic disruption at other points, it is likely that factors related to the camera itself are affecting the camera images, so abnormality detection of the camera images is performed. This prevents misidentification of image abnormalities caused by external factors at other points as abnormalities in the camera images, thereby improving the accuracy of camera image abnormality detection. Furthermore, by not performing abnormality detection, the processing load can be reduced.

[0068] (Embodiment 2) Embodiment 2 will now be described with reference to the drawings. Figure 6 shows an example of the system configuration according to this embodiment.

[0069] As shown in Figure 6, the monitoring system 1 according to this embodiment includes a road management server 500 in addition to the configuration of Embodiment 1. The center server 100 and the road management server 500 are connected in a communication manner.

[0070] The road management server 500 is a management server that manages road information, such as a VICS (Vehicle Information and Communication System) server. The road management server 500 collects and manages road information, including road closures and restrictions due to accidents, construction, etc., based on various sensors installed at various points along the road and input information. For example, the road management server 500 manages road information for intersection 401b on road 400.

[0071] Furthermore, the center server 100 includes a road information acquisition unit 109 instead of the road information recognition unit 106 of Embodiment 1. The road information acquisition unit 109 acquires road information for intersection 401b from the road management server 500. The road information acquisition unit 109 may also acquire road information for intersection 401b based on various sensors installed at intersection 401b and input information.

[0072] In this embodiment, the anomaly detection unit 107 detects anomalies in the camera image of camera 200a based on the traffic information of intersection 401a recognized by the traffic information recognition unit 102 from the camera image of camera 200a, the statistical information stored in the statistical information storage unit 105, and the road information of intersection 401b acquired by the road information acquisition unit 109. The other configurations and anomaly detection methods are the same as in Embodiment 1. In this way, even when road information is acquired from a road management server or the like that manages road information, the accuracy of anomaly detection in the camera image can be improved, as in Embodiment 1.

[0073] This disclosure is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. For example, in the embodiments described above, the abnormality of the camera image at the first location was determined by considering the road information at the second location, but the abnormality of the camera image at the first location may be determined by considering the road information at the first location.

[0074] Each configuration in the above-described embodiment may consist of hardware, software, or both, and may consist of one piece of hardware or software, or multiple pieces of hardware or software. Each device and each function (process) may be realized by a computer 20 having a processor 21 such as a CPU (Central Processing Unit) and a memory 22 as a storage device, as shown in Figure 7. For example, a program for performing the method in the embodiment (monitoring method or anomaly detection method) may be stored in the memory 22, and each function may be realized by executing the program stored in the memory 22 with the processor 21.

[0075] These programs, when loaded into a computer, include a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The programs may be stored on non-temporary computer-readable media or tangible storage media. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drives (SSDs), or other memory technologies, CD-ROMs, digital versatile discs (DVDs), Blu-ray® discs, or other optical disc storage, magnetic cassettes, magnetic tapes, magnetic disk storage, or other magnetic storage devices. The programs may be transmitted over temporary computer-readable media or communication media. Examples, but not limited to, include electrical, optical, acoustic, or other forms of propagating signals.

[0076] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be understood by those skilled in the art within the scope of the present disclosure.

[0077] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) A camera image acquisition means that acquires camera images of the road at the first location from a camera positioned at the first location, Traffic information acquisition means for analyzing the camera images and acquiring traffic information indicating the traffic conditions on the road at the first point, A means for acquiring road information that acquires road information indicating the traffic obstruction situation at a second point leading to the first point, An anomaly detection means for detecting anomalies in the camera image based on the traffic information, the statistical information of the traffic information, and the road information, A monitoring device equipped with the following features. (Note 2) The anomaly detection means performs an anomaly determination of the camera image using the traffic information and statistical information in accordance with the road information. The monitoring device described in Appendix 1. (Note 3) The anomaly detection means performs an anomaly determination of the camera image if the road information includes information indicating that there is no obstruction to traffic at the second point. The monitoring device described in Appendix 2. (Note 4) The anomaly detection means detects anomalies in the camera image based on the discrepancy between the traffic information and the statistical information when determining anomalies in the camera image. The monitoring device described in Appendix 3. (Note 5) The anomaly detection means detects an anomaly in the camera image based on the result of comparing the deviation with a predetermined threshold. The monitoring device described in Appendix 4. (Note 6) The anomaly detection means sets a predetermined threshold according to the level of traffic disruption at the second point indicated by the road information. The monitoring device described in Appendix 5. (Note 7) The anomaly detection means does not detect an anomaly in the camera image if the road information includes information indicating that there is a traffic obstruction at the second point. A monitoring device as described in any one of the items 2 to 5 of the appendix. (Note 8) The anomaly detection means determines the level of anomaly in the camera image according to the degree of discrepancy between the traffic information and the statistical information. A monitoring device as described in any one of the items 1 to 7 of the appendix. (Note 9) The anomaly detection means determines the anomaly level of the camera image based on the degree of deviation estimated from the level of traffic disruption at the second point indicated by the road information, and the degree of deviation between the traffic information and the statistical information. The monitoring device described in Appendix 8. (Note 10) The anomaly detection means detects anomalies in the camera image caused by a camera malfunction. A monitoring device as described in any one of the items 1 to 9 of the appendix. (Note 11) The road information acquisition means acquires the road information from camera images captured by a camera positioned at the second location. A monitoring device as described in any one of the appendices 1 to 10. (Note 12) The road information acquisition means acquires road information at the second location from a management device that manages the road information. A monitoring device as described in any one of the appendices 1 to 10. (Note 13) The road information acquisition means acquires a plurality of road information at a plurality of second points, The anomaly detection means detects anomalies in the camera image based on a plurality of road information. A monitoring device as described in any one of the items 1 to 12 of the appendix. (Note 14) The aforementioned road information includes information on road closures or restrictions on the aforementioned road. A monitoring device as described in any one of the items 1 to 13 of the appendix. (Note 15) The aforementioned traffic information includes the volume, speed, or type of traffic of the objects traveling on the aforementioned road. A monitoring device as described in any one of the appendices 1 to 14. (Note 16) The aforementioned objects of passage include vehicles or people. The monitoring device described in Appendix 15. (Note 17) The system includes statistical information storage means for storing statistical information of the aforementioned traffic information, The anomaly detection means detects anomalies in the camera image based on the stored statistical information. A monitoring device as described in any one of the items 1 to 16 of the appendix. (Note 18) The system includes statistical information generation means that generates the statistical information based on the aggregated results of the traffic information over a predetermined period. A monitoring device as described in any one of the items 1 to 17 of the appendix. (Note 19) The aforementioned statistical information includes statistical information broken down by time of day, day of the week, or month. A monitoring device as described in any one of the items 1 to 18 of the appendix. (Note 20) Equipped with a camera and monitoring device located at the first point, The aforementioned monitoring device is A camera image acquisition means that acquires camera images of the road at the first location from the aforementioned camera, Traffic information acquisition means for analyzing the camera images and acquiring traffic information indicating the traffic conditions on the road at the first point, A means for acquiring road information that acquires road information indicating the traffic obstruction situation at a second point leading to the first point, An anomaly detection means for detecting anomalies in the camera image based on the traffic information, the statistical information of the traffic information, and the road information, A monitoring system equipped with the following features. (Note 21) The anomaly detection means performs an anomaly determination of the camera image using the traffic information and statistical information in accordance with the road information. The monitoring system described in Appendix 20. (Note 22) Camera footage of the road at the first location is acquired from a camera positioned at the first location. The camera footage is analyzed to obtain traffic information indicating the road traffic conditions at the first location. Obtain road information indicating the traffic disruption situation at a second point leading to the first point, Based on the traffic information, the statistical information of the traffic information, and the road information, an anomaly in the camera image is detected. Monitoring method. (Note 23) In detecting the aforementioned anomaly, an anomaly determination of the camera image is performed using the traffic information and statistical information in accordance with the road information. The monitoring method described in Appendix 22. (Note 24) Camera footage of the road at the first location is acquired from a camera positioned at the first location. The camera footage is analyzed to obtain traffic information indicating the road traffic conditions at the first location. Obtain road information indicating the traffic disruption situation at a second point leading to the first point, Based on the traffic information, the statistical information of the traffic information, and the road information, an anomaly in the camera image is detected. A non-temporary, computer-readable medium containing a monitoring program that causes a computer to execute a process. (Note 25) In detecting the aforementioned anomaly, an anomaly determination of the camera image is performed using the traffic information and statistical information in accordance with the road information. Non-temporary computer-readable media as described in Appendix 24. [Explanation of Symbols]

[0078] 1. Monitoring System 10 Monitoring equipment 11. Camera image acquisition unit 12 Traffic information acquisition department 13 Road information acquisition department 14 Anomaly detection unit 20 Computers 21 processors 22 memory 100 Center Servers 101 Camera Image Acquisition Unit 102 Traffic information recognition section 103 Traffic information DB 104 Statistical information generation section 105 Statistical information storage unit 106 Road information recognition unit 107 Anomaly detection unit 108 Output section 109 Road information acquisition department 200, 200a, 200b cameras 300 Edge Processing Units 400 road Intersections 401, 401a, and 401b 402, 402a, 402b traffic lights 500 Road Management Servers

Claims

1. A camera image acquisition means that acquires camera images of the road at the first location from a camera positioned at the first location, Traffic information acquisition means for analyzing the camera images and acquiring traffic information indicating the traffic conditions on the road at the first point, A statistical information generation means that generates statistical information on traffic information based on the aggregated results of the traffic information over a predetermined period, A road information acquisition means for acquiring road information indicating the traffic obstruction situation at a second point leading to the first point, If the road information includes information indicating that there is no traffic obstruction at the second point, an anomaly detection means for detecting an anomaly in the camera image based on the discrepancy between the traffic information and the statistical information, A monitoring device equipped with the following features.

2. The anomaly detection means detects an anomaly in the camera image based on the result of comparing the deviation with a predetermined threshold. The monitoring device according to claim 1.

3. The anomaly detection means sets a predetermined threshold according to the level of traffic disruption at the second point indicated by the road information. The monitoring device according to claim 2.

4. The anomaly detection means does not detect an anomaly in the camera image if the road information includes information indicating that there is a traffic obstruction at the second point. The monitoring device according to any one of claims 1 to 3.

5. Equipped with a camera and monitoring device located at the first location, The aforementioned monitoring device is A camera image acquisition means that acquires camera images of the road at the first location from the aforementioned camera, Traffic information acquisition means for analyzing the camera images and acquiring traffic information indicating the traffic conditions on the road at the first point, A statistical information generation means that generates statistical information on traffic information based on the aggregated results of the traffic information over a predetermined period, A road information acquisition means for acquiring road information indicating the traffic obstruction situation at a second point leading to the first point, If the road information includes information indicating that there is no traffic obstruction at the second point, an anomaly detection means for detecting an anomaly in the camera image based on the discrepancy between the traffic information and the statistical information, A monitoring system equipped with the following features.

6. Camera footage of the road at the first location is acquired from a camera positioned at the first location. The camera footage is analyzed to obtain traffic information indicating the traffic conditions on the road at the first location. Based on the aggregated results of the traffic information over a predetermined period, statistical information on the traffic information is generated. Obtain road information indicating the traffic disruption situation at a second point leading to the first point, If the road information includes information indicating that there is no traffic obstruction at the second point, an anomaly in the camera image is detected based on the discrepancy between the traffic information and the statistical information. Monitoring method.

7. Camera footage of the road at the first location is acquired from a camera positioned at the first location. The camera footage is analyzed to obtain traffic information indicating the traffic conditions on the road at the first location. Based on the aggregated results of the traffic information over a predetermined period, statistical information on the traffic information is generated. Obtain road information indicating the traffic disruption situation at a second point leading to the first point, If the road information includes information indicating that there is no traffic obstruction at the second point, an anomaly in the camera image is detected based on the discrepancy between the traffic information and the statistical information. A monitoring program that causes a computer to execute a process.

Images