Monitoring device, monitoring system, monitoring method, and monitoring program

Abstract

A monitoring device (10) is provided with: a camera image acquisition unit (11) that acquires, from a camera located at a first point, a camera image obtained by capturing a road at the first point; a traffic information acquisition unit (12) that analyzes the camera image to acquire traffic information indicating a traffic condition of the road at the first point; a road information acquisition unit (13) that acquires road information indicating a traffic obstruction condition at a second point leading to the first point; and an abnormality detection unit (14) that detects an abnormality in the camera image on the basis of the traffic information, statistical information of the traffic information, and the road information.

Description

Monitoring device, monitoring system, monitoring method, and non-transitory computer-readable medium

[0001] The present disclosure relates to a monitoring device, a monitoring system, a monitoring method, and a non-transitory computer-readable medium.

[0002] Traffic monitoring systems are used in which cameras are installed on roads to monitor traffic conditions, etc., using the captured images. In such traffic monitoring systems, cameras are increasingly being installed in various locations, so there is a demand for more efficient maintenance work, including abnormality detection after installation.

[0003] A known technology related to anomaly detection in a monitoring system is, for example, that disclosed in Patent Document 1. In Patent Document 1, an anomaly in a learning model is detected by comparing the output result of the learning model that performs monitoring processing in the monitoring system with statistical information that is its expected value.

[0004] Japanese Patent Application Laid-Open No. 2020-113119

[0005] In traffic monitoring systems, monitoring is performed by analyzing camera images captured by cameras installed on roads, and therefore, it is necessary to detect abnormalities in the camera images for maintenance purposes. However, related technologies such as those disclosed in Patent Document 1 do not take into consideration application to the detection of abnormalities in camera images in traffic monitoring systems. As a result, the related technologies may not be able to accurately detect abnormalities in camera images captured by cameras installed on roads.

[0006] In view of such problems, the present disclosure aims to provide a monitoring device, a monitoring system, a monitoring method, and a non-transitory computer-readable medium that are capable of accurately detecting abnormalities in camera footage.

[0007] The monitoring device according to the present disclosure comprises a camera image acquisition means for acquiring camera image of a road at a first location from a camera placed at the first location, a traffic information acquisition means for analyzing the camera image and acquiring traffic information indicating the traffic conditions on the road at the first location, a road information acquisition means for acquiring road information indicating the traffic disruption conditions at a second location leading to the first location, and an abnormality detection means for detecting an abnormality in the camera image based on the traffic information, statistical information on the traffic information, and the road information.

[0008] The surveillance system according to the present disclosure comprises a camera disposed at a first location and a surveillance device, the surveillance device comprising: a camera image acquisition means for acquiring camera image of a road at the first location from the camera; a traffic information acquisition means for analyzing the camera image and acquiring traffic information indicating the traffic conditions of the road at the first location; a road information acquisition means for acquiring road information indicating the traffic disruption conditions at a second location leading to the first location; and an abnormality detection means for detecting an abnormality in the camera image based on the traffic information, statistical information on the traffic information, and the road information.

[0009] The monitoring method according to the present disclosure involves obtaining camera footage of a road at a first location from a camera placed at the first location, analyzing the camera footage, obtaining traffic information indicating the traffic conditions on the road at the first location, obtaining road information indicating traffic disruptions at a second location leading to the first location, and detecting abnormalities in the camera footage based on the traffic information, statistical information on the traffic information, and the road information.

[0010] The non-transitory computer-readable medium of the present disclosure is a non-transitory computer-readable medium that stores a monitoring program for causing a computer to execute a process of acquiring camera footage of a road at a first location from a camera placed at the first location, analyzing the camera footage, acquiring traffic information indicating traffic conditions on the road at the first location, acquiring road information indicating traffic disruptions at a second location leading to the first location, and detecting abnormalities in the camera footage based on the traffic information, statistical information on the traffic information, and the road information.

[0011] According to the present disclosure, it is possible to provide a monitoring device, a monitoring system, a monitoring method, and a non-transitory computer-readable medium that are capable of accurately detecting abnormalities in camera footage.

[0012] Fig. 1 is a configuration diagram showing an overview of a monitoring device according to an embodiment. Fig. 2 is a configuration diagram showing an example configuration of a monitoring system according to embodiment 1. Fig. 3 is a flowchart showing an example operation of statistical information generation processing according to embodiment 1. Fig. 4 is a flowchart showing an example operation of camera abnormality detection processing according to embodiment 1. Fig. 5 is a flowchart showing an example operation of camera abnormality determination processing according to embodiment 1. Fig. 6 is a configuration diagram showing an example configuration of a monitoring system according to embodiment 2. Fig. 7 is a configuration diagram showing an overview of hardware of a computer according to an embodiment.

[0013] Hereinafter, embodiments will be described with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and redundant description will be omitted as necessary.

[0014] (Overview of the embodiment) When considering a system that monitors vehicles traveling on a road, such as a traffic monitoring system, the traffic conditions on the road are not always constant. For example, if an accident or the like occurs on a monitored road, the traffic volume at other points may decrease from normal levels. In this case, if the detection results of a learning model are simply compared with the statistical information thereof, as in Patent Document 1, a discrepancy, i.e., a divergence, occurs between the detection results and the statistical information, and the result is determined to be an abnormality.

[0015] Specifically, in a traffic monitoring system, cameras are installed at multiple points on a road, and traffic information, including traffic volume, is obtained from the images captured by the cameras at each point. For example, if an accident occurs at another point connected to the point where a camera is installed, such as upstream, and the traffic volume at the point where the camera is installed changes, a discrepancy will occur between the detection result and the statistical information, which may be judged as an abnormality and an alert may be issued.

[0016] Here, the abnormality in the camera image detected in the embodiment is an abnormality that may require maintenance work on the installed camera. In other words, the abnormality in the camera image may be an abnormality that occurs in all or part of the image due to a camera malfunction, such as a camera failure or dirt on the camera lens, or an abnormality in the image caused by a camera malfunction. For example, if a camera malfunctions, the entire image becomes unrecognizable, and if snow or other particles adhere to the camera lens, part of the image becomes unrecognizable. Examples of an unrecognizable state include the image becoming black, blurry, or not showing anything. Therefore, the abnormality that occurs when an accident occurs, as described above, is not an abnormality caused by a camera malfunction, and therefore an abnormality in the camera image should not be detected.

[0017] Therefore, in the embodiment, by taking into consideration the road conditions at other points connected to the point where the camera is installed, it becomes possible to accurately detect abnormalities in the camera image.

[0018] 1 shows an overview of a monitoring device according to an embodiment. As shown in FIG. 1, a monitoring device 10 according to an embodiment includes a camera image acquisition unit 11, a traffic information acquisition unit 12, a road information acquisition unit 13, and an abnormality detection unit 14.

[0019] The camera image acquisition unit 11 acquires camera images of a road at a first location from a camera installed at the first location. The traffic information acquisition unit 12 analyzes the camera images acquired by the camera image acquisition unit 11 and acquires traffic information indicating the traffic conditions of the road at the first location. The road information acquisition unit 13 acquires road information indicating the traffic disruption conditions of a road at a second location leading to the first location. The abnormality detection unit 14 detects abnormalities in the camera images acquired by the camera image acquisition unit 11 based on the traffic information acquired by the traffic information acquisition unit 12, statistical information on the traffic information, and the road information acquired by the road information acquisition unit 13.

[0020] In this manner, in the embodiment, an abnormality in the camera image at the first location is detected based on the traffic information and statistical information of the traffic information obtained from the camera image at the first location, as well as road information at the second location leading to the first location. This makes it possible to determine an abnormality in the camera image at the first location while taking into account the road information at the second location, thereby suppressing the generation of unnecessary alerts and enabling accurate detection of an abnormality in the camera image.

[0021] (Embodiment 1) Hereinafter, embodiment 1 will be described with reference to the drawings. Fig. 2 shows an example of the configuration of a monitoring system according to this embodiment. The monitoring system 1 according to this embodiment is a system that monitors road traffic conditions, i.e., traffic flow, and also a system that detects or monitors abnormalities in camera images captured by cameras installed or positioned on roads.

[0022] As shown in Fig. 2, the monitoring system 1 according to this embodiment includes a center server 100, a plurality of cameras 200 (e.g., 200a, 200b), and an edge processing device 300. The cameras 200 and the edge processing device 300 are connected to each other wirelessly or via a wired connection so that they can communicate with each other. The edge processing device 300 and the center server 100 are also connected to each other wirelessly or via a wired connection so that they can communicate with each other. The devices may be connected directly to each other or via any network. The cameras 200 and the center server 100 may also be connected to each other so that they can communicate with each other.

[0023] Each camera 200 is a surveillance camera installed or placed at a different point on the road 400 and captures the road 400 at each point. The camera 200 may capture video of the road 400 continuously, or may capture video at regular intervals or when some trigger, such as a user operation, is detected. The camera 200 transmits the captured camera video to the edge processing device 300. The camera video may be so-called video streaming or a sequence of still images captured at predetermined intervals, such as periodically. In this example, the camera video is transmitted from the camera 200 to the center server 100 via the edge processing device 300. Note that the camera video may also be transmitted from the camera 200 to the center server 100 without passing through the edge processing device 300.

[0024] For example, each camera 200 is installed at each intersection of road 400 to be monitored, and captures video including the road at each intersection. Note that the cameras may be installed at other locations, not just intersections, as long as they can capture video of each point on road 400. In this example, camera 200a is installed at intersection 401a (first location), and camera 200b is installed at intersection 401b (second location). For example, camera 200a is fixed to traffic light 402a installed at intersection 401a, and camera 200b is fixed to traffic light 402b installed at intersection 401b. As long as they can capture video of the intersections that are monitoring points, the cameras may be fixed at other locations, not just traffic lights.

[0025] Camera 200a can monitor the traffic conditions at intersection 401a, and camera 200b can monitor the traffic conditions at intersection 401b. In this example, camera 200a at intersection 401a is the camera targeted for detecting anomalies in its camera image, and camera 200b at intersection 401b is the camera for acquiring road information used in the anomaly detection process for the camera image from camera 200a. Intersection 401b is a point connected to intersection 401a by at least road 400. For example, intersection 401b is an intersection upstream of intersection 401a, but it may also be an intersection downstream of intersection 401a. For example, intersection 401b is an intersection adjacent to intersection 401a, but it may also be an intersection further adjacent to intersection 401a. Furthermore, the number of camera 200b for acquiring road information is not limited to one, and may be multiple cameras installed at multiple points.

[0026] The edge processing device 300 is a server installed or placed on the edge side of the system, such as a multi-access edge computing (MEC) device. The edge processing device 300 includes a video management system (VMS) 301. The VMS 301 is a video management unit that manages the camera images of each camera 200. The VMS 301 acquires camera images transmitted from multiple cameras 200 (200a and 200b) and transmits the acquired camera images to the center server 100. The VMS 301 may change the format or bit rate of the image as necessary. Note that multiple edge processing devices 300 may be installed, and camera images of multiple cameras 200 may be transmitted from the multiple edge processing devices 300 to the center server 100.

[0027] The center server 100 is a server installed or located on the center side of the system, such as a cloud server built on a cloud. The center server 100 is a device that performs traffic condition monitoring processing in the monitoring system 1, and also a device that performs anomaly detection processing on camera images.

[0028] The center server 100 includes a camera image acquisition unit 101, a traffic information recognition unit 102, a traffic information DB 103, a statistical information generation unit 104, a statistical information storage unit 105, a road information recognition unit 106, an abnormality detection unit 107, and an output unit 108. Note that the configuration of the center server 100 is an example, and other configurations may be used as long as the operations according to this embodiment are possible. The functions of the center server 100 may be realized by one device or by multiple devices. Some of the functions of the center server 100 may be arranged in an external device or an edge processing device 300.

[0029] The camera image acquisition unit 101 acquires camera images captured by each camera 200. In this example, the camera image of the intersection 401a captured by the camera 200a and the camera image of the intersection 401b captured by the camera 200b are received and acquired via the edge processing device 300.

[0030] The traffic information recognition unit 102 is a traffic information acquisition unit that recognizes and acquires traffic information for each location from the camera images of each camera 200 acquired by the camera image acquisition unit 101. In this example, the traffic information recognition unit 102 recognizes traffic information for the road at intersection 401a from the camera image of intersection 401a captured by camera 200a. Furthermore, traffic information for multiple locations may be recognized from camera images of multiple cameras, as necessary. For example, traffic information for the road at intersection 401b may be recognized from the camera image of camera 200b, or traffic information for other locations may be recognized from the camera images of other cameras 200.

[0031] Traffic information is information that indicates the traffic conditions of a road, i.e., traffic flow. For example, traffic information includes the traffic volume, speed, type, etc. of passing objects traveling on a road. Passing objects are not limited to vehicles but also include people such as pedestrians. In the case of vehicle traffic information, the information includes the traffic volume of vehicles, the speed of passing vehicles, and the type of passing vehicles, and may be, for example, the traffic volume and speed for each type of vehicle. Vehicle traffic information includes the number of vehicles passing in a specified period. Vehicle types include passenger cars, trucks, buses, motorcycles, bicycles, etc. In the case of human traffic information, the information includes the traffic volume of pedestrians, pedestrian speed, and pedestrian attributes, and may be, for example, the traffic volume and speed for each attribute. Pedestrian traffic includes the number of people passing in a specified period. Pedestrian attributes include gender, age, etc.

[0032] The traffic information recognition unit 102 may recognize the traffic conditions of vehicles and people in the camera video using an AI (Artificial Intelligence) engine (a learning model using machine learning) for traffic information recognition. The AI ​​engine may be a CNN (Convolutional Neural Network) or other neural networks. For example, by performing machine learning on the features of images of vehicles and pedestrians and the labels of the vehicle models and pedestrian attributes, the traffic information recognition unit 102 can recognize the vehicle models and pedestrian attributes in the video and obtain the traffic volume and speed of the recognized vehicles and pedestrians.

[0033] The traffic information recognition unit 102 stores the traffic information recognized from the camera footage in the traffic information DB 103 together with the date and time when the camera footage was captured, for traffic monitoring or generating statistical information. For example, the date and time when the camera 200 captured the footage is set in the camera footage. If the date and time when the camera footage was captured is not set in the camera footage, the date and time when the center server 100 received the camera footage may be used. Furthermore, the traffic information recognition unit 102 outputs the traffic information recognized from the camera footage to the anomaly detection unit 107 in order to detect anomalies in the camera footage.

[0034] The traffic information DB 103 is a traffic information storage unit such as a database that stores and accumulates traffic information for each location recognized from camera images by the traffic information recognition unit 102. The traffic information DB 103 is not limited to storing traffic information for each location, and may also store camera images for each location. For example, the traffic information DB 103 is a non-volatile memory such as a flash memory, a hard disk drive, or the like.

[0035] The statistical information generating unit 104 generates statistical information on traffic information for each point stored in the traffic information DB 103. In this example, the statistical information generating unit 104 generates statistical information on traffic information for the intersection 401a recognized by the traffic information recognizing unit 102. Furthermore, statistical information on traffic information for multiple points may be generated as needed. For example, statistical information on traffic information for the intersection 401b may be generated, or statistical information on traffic information for other points may be generated.

[0036] The statistical information generating unit 104 calculates statistical information about the traffic information for each location based on the results of the traffic information aggregation for a predetermined period. For example, the statistical information may be an average value or a total value for the predetermined period, but may also be other statistical values ​​such as variance or a median. The statistical information may be a statistical value aggregated for the entire predetermined period, or may be a statistical value aggregated by time period, day of the week, month, etc. For example, it may be a statistical value for each time period on each day of the week. Furthermore, if the traffic information includes vehicle traffic information and pedestrian traffic information, statistical information about vehicles and statistical information about pedestrians may be calculated. For example, statistical information for each vehicle type and statistical information for each pedestrian attribute may be calculated. The statistical information for each vehicle type may be an average or total traffic volume, an average speed, etc. The statistical information for each pedestrian attribute may be an average or total traffic volume, a total speed, etc. The statistical information generating unit 104 stores the generated statistical information for each location in the statistical information storage unit 105.

[0037] The statistical information storage unit 105 stores the statistical information of the traffic information at each point generated by the statistical information generation unit 104. For example, similar to the traffic information DB 103, it is a nonvolatile memory such as a flash memory or a hard disk drive.

[0038] The road information recognition unit 106 is a road information acquisition unit that recognizes and acquires road information from the camera images acquired by the camera image acquisition unit 101. In this example, the road information recognition unit 106 recognizes road information of the road at the intersection 401b from the camera image of the intersection 401b captured by the camera 200b. Furthermore, the road information recognition unit 106 may recognize road information of multiple points from the camera images of multiple cameras, as necessary. For example, the road information of other points may be recognized from the camera images of other cameras 200, or the road information of the road at the intersection 401a may be recognized from the camera image of the camera 200a.

[0039] The road information is information indicating a traffic disruption status on a road. For example, the road information may be traffic information at a first location or information regarding external factors affecting camera footage. For example, the road information may include information regarding road closures due to accidents, construction work, landslides, etc., or information regarding road restrictions, or may be information based on these. The road information may indicate whether an event that disrupts traffic on a road has occurred, or may indicate the level of traffic disruption. For example, the presence or absence of a traffic disruption may be indicated depending on whether a road is closed. Furthermore, the traffic disruption level may be indicated depending on the number of restricted lanes, or depending on whether each person or vehicle type can pass.

[0040] The road information recognition unit 106 may recognize road obstruction conditions in the camera image using an AI engine for road information recognition. The AI ​​engine may be a CNN or other neural network. For example, by machine learning the characteristics of an image of a road closure and a label indicating the presence or absence of an obstruction or the obstruction level, the presence or absence or the obstruction level of the road in the image can be recognized. The road information recognition unit 106 outputs the road information recognized from the camera image to the anomaly detection unit 107.

[0041] The abnormality detection unit 107 detects abnormalities in the camera images acquired by the camera image acquisition unit 101. It can also be said that the abnormality detection unit 107 detects abnormalities in the camera images caused by camera failure. In this example, the abnormality detection unit 107 detects abnormalities in the camera images captured by the camera 200a at the intersection 401a. Note that a similar detection method may be used to detect abnormalities in the camera images captured by the camera 200b at the intersection 401b, or to detect abnormalities in the camera images captured by other cameras 200 at other points.

[0042] The abnormality detection unit 107 detects an abnormality in the camera image of the camera 200a at the intersection 401a based on road information of the intersection 401b acquired from the camera image of the camera 200b, traffic information of the intersection 401a acquired from the camera image of the camera 200a, and statistical information on the traffic information stored in the statistical information storage unit 105. For example, the abnormality detection unit 107 determines an abnormality in the camera image of the camera 200a based on the road information of the intersection 401b, using traffic information recognized from the camera image of the camera 200a and statistical information on the traffic information. Note that the abnormality determination in the camera image of the camera 200a may also be based on road information from multiple points.

[0043] For example, if the road information indicates that there is no traffic disruption at the intersection 401b or includes information indicating that there is no traffic disruption, the anomaly detection unit 107 determines whether there is an anomaly in the camera image from the camera 200a. Furthermore, if the road information indicates that there is a traffic disruption at the intersection 401b or includes information indicating that there is a traffic disruption, the anomaly detection unit 107 does not detect an anomaly in the camera image from the camera 200a. For example, if there is a traffic disruption at the intersection 401b, it is not necessary to determine whether there is an anomaly in the camera image from the camera 200a. In this case, the traffic information may not be compared with statistical information, or an anomaly in the camera image may not be determined even if there is a large discrepancy between the traffic information and statistical information. Furthermore, in determining whether there is an anomaly in the camera image, an anomaly in the camera image from the camera 200a is detected based on the discrepancy between the traffic information and statistical information for the intersection 401a. The abnormality detection unit 107 may detect the presence or absence of an abnormality in the camera image based on the discrepancy between the traffic information and statistical information for the intersection 401a, or may detect the level of abnormality in the camera image according to the degree of discrepancy. Furthermore, the presence or absence of an abnormality or the level of abnormality in the camera image may be detected taking into account the duration of time during which a state in which there is a large discrepancy between the traffic information and statistical information for the intersection 401a continues.

[0044] If the current traffic information based on the camera image from camera 200a deviates significantly from the statistical information indicating normal traffic conditions even when there is no traffic disruption at intersection 401b, i.e., no external factors, there is a high possibility of an image abnormality due to a camera malfunction of camera 200a. Therefore, if the road information for intersection 401b indicates that there is no traffic disruption and there is a large deviation between the current traffic information and the statistical information, it is possible to detect an abnormality in the camera image from camera 200a. That is, it is possible to detect the possibility of a camera malfunction, such as a camera malfunction or dirt on the camera lens, for example, an image abnormality due to the presence or absence of a malfunction and the level of the malfunction. In other words, it is possible to detect the possibility of a camera malfunction. Furthermore, if the camera malfunctions, the large deviation between the current traffic information and the statistical information continues for a long period of time, so the possibility of a camera malfunction can be reliably detected based on the duration of this large deviation.

[0045] The output unit 108 outputs the abnormality detection results of the camera images detected by the abnormality detection unit 107. For example, the output unit 108 is a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The output unit 108 is not limited to a display device, but may also include an output device such as an audio output device. The output unit 108 displays an alarm when an abnormality is detected in the camera images, and also displays information according to the detected abnormality level. By displaying the detection results of the camera images from each camera 200, a monitor can monitor camera failures at each camera. The output unit 108 may also display the camera images from each camera 200, traffic information, road information, etc. For example, by displaying traffic information and road information along with the camera images from each camera 200, a monitor can monitor traffic at each location.

[0046] Next, the operation of the center server 100 according to this embodiment will be described with reference to Figures 3 to 5. Figure 3 shows an example of the operation of the statistical information generation process in the center server 100, and shows the flow of processing from acquiring camera images to generating statistical information on traffic information. For example, traffic information is constantly acquired from camera images captured by a camera, accumulated, and statistical information is generated at a predetermined timing. Note that the processing in Figure 3 is executed at least before the process of detecting anomalies in the camera images.

[0047] 3 , the center server 100 acquires a first camera image captured by a first camera set at a first location (S101). A camera 200a (first camera) installed at an intersection 401a (first location) constantly captures, for example, an image of a road at the intersection 401a and transmits the captured camera image (first camera image) to the edge processing device 300. The edge processing device 300 receives the camera image transmitted from the camera 200a and transmits the received camera image to the center server 100. In the center server 100, the camera image acquisition unit 101 receives and acquires the camera image of the camera 200a transmitted from the edge processing device 300. The camera image acquisition unit 101 may display the acquired camera image of the camera 200a on the display of the output unit 108.

[0048] Next, the center server 100 acquires traffic information from the acquired first camera image (S102). The traffic information recognition unit 102 analyzes the camera image from the camera 200a acquired by the camera image acquisition unit 101, and acquires traffic information about the road at the intersection 401a. The traffic information recognition unit 102 inputs the camera image from the camera 200a into an AI engine for traffic information recognition, thereby acquiring traffic information about the road at the intersection 401a. The traffic information recognition unit 102 may display the acquired traffic information about the intersection 401a on the display of the output unit 108 together with the camera image from the camera 200a.

[0049] Next, the center server 100 accumulates the acquired traffic information (S103). The traffic information recognition unit 102 stores the traffic information of the intersection 401a acquired from the camera image of the camera 200a in the traffic information DB 103. For example, the traffic information recognition unit 102 sequentially stores and accumulates the traffic information acquired from the camera image constantly captured by the camera 200a in the traffic information DB 103 together with the date and time of the image capture.

[0050] Next, the center server 100 generates statistical information of the accumulated traffic information (S104). The statistical information generation unit 104 references the traffic information DB 103 and generates statistical information of the accumulated traffic information at the intersection 401a. For example, the statistical information generation unit 104 calculates statistical information by aggregating traffic information for a predetermined period at a predetermined statistical information generation timing. The statistical information generation timing may be periodic or may be before the start of the abnormality detection process. The statistical information generation unit 104 stores the calculated statistical information of the traffic information at the intersection 401a in the statistical information storage unit 105.

[0051] FIG. 4 shows an example of the operation of the camera abnormality detection process in the center server 100, and shows the flow of the process from acquiring the current camera image to detecting an abnormality in the acquired camera image.

[0052] As shown in Fig. 4, the center server 100 acquires a first camera image from a first camera installed at a first location (S111). The camera image acquisition unit 101 acquires the camera image from the camera at the first location at an abnormality detection timing in the camera image. The abnormality detection timing may be periodic or may be a timing specified by a monitor. As in S101 of Fig. 3, the camera image acquisition unit 101 acquires the camera image (first camera image) from the camera 200a (first camera) installed at the intersection 401a (first location) that is the target of abnormality detection.

[0053] Next, the center server 100 acquires traffic information from the acquired first camera image (S112). The traffic information recognition unit 102 acquires traffic information at the intersection 401a from the camera image of the camera 200a acquired by the camera image acquisition unit 101, similar to S102 in FIG. 3 .

[0054] Next, the center server 100 acquires a second camera image from a second camera installed at a second location (S113). As in S111, the camera image acquisition unit 101 acquires a camera image (second camera image) from a camera 200b (second camera) installed at an intersection 401b (second location) for acquiring road information.

[0055] Next, the center server 100 acquires road information from the acquired second camera image (S114). The road information recognition unit 106 analyzes the camera image from the camera 200b acquired by the camera image acquisition unit 101, and acquires road information about the road at the intersection 401b. The road information recognition unit 106 inputs the camera image from the camera 200b into an AI engine for road information recognition, thereby acquiring road information about the road at the intersection 401b.

[0056] Next, the center server 100 executes a camera abnormality determination process for the first camera image (S115). Fig. 5 shows a specific example of the camera abnormality determination process (S115) of Fig. 4 .

[0057] 5 , the abnormality detection unit 107 determines whether or not there is a traffic disruption at the second location (S121). The abnormality detection unit 107 determines whether or not there is a traffic disruption at the intersection 401b based on road information for the intersection 401b (second location) acquired from the camera image of the camera 200b by the road information recognition unit 106. When road information for multiple locations is used, the presence or absence of a traffic disruption may be determined based on the road information for the multiple locations.

[0058] If the abnormality detection unit 107 determines that there is no traffic disruption at the second location (YES in S121), it compares the traffic information at the first location with the statistical information (S122). If the abnormality detection unit 107 determines that there is a traffic disruption at the second location (NO in S121), it terminates the process without determining an abnormality in the camera image. In other words, in this case, the abnormality detection unit 107 does not detect an abnormality in the camera image. If the acquired road information indicates that there is no traffic disruption at the intersection 401b, the abnormality detection unit 107 compares the current traffic information acquired by the traffic information recognition unit 102 from the camera image of the camera 200a with the statistical information of the traffic information stored in the statistical information storage unit 105. The current traffic information may be compared with the statistical information when the road information indicates that all traffic, i.e., all traffic at one or multiple locations, is undisturbed, or when the road information indicates that only some traffic, i.e., only some traffic at one or multiple locations, is undisturbed. The process may be terminated when the traffic congestion level is greater than the predetermined level by comparing the current traffic information with the statistical information, or when the traffic congestion level is less than the predetermined level by comparing the current traffic information with the statistical information.

[0059] For example, if the statistical information includes statistical information for multiple points, the statistical information for traffic information at intersection 401a (camera 200a) is selected and the selected statistical information is compared with the current traffic information. Also, if the statistical information includes statistical information for each time period or day of the week, the statistical information corresponding to the time or day of the week of the current traffic information (image from camera 200a) is selected and the selected statistical information is compared with the current statistical information.

[0060] Next, the abnormality detection unit 107 determines the magnitude of the deviation between the traffic information and the statistical information at the first location (S123), and if the deviation is large (S123 / YES), determines that there is an abnormality in the camera image (S124), and if the deviation is small (S123 / NO), determines that there is no abnormality in the camera image (S125). The output unit 108 outputs the abnormality detection result of the camera image from a display or the like.

[0061] For example, the abnormality detection unit 107 calculates the deviation, i.e., the difference, between the current traffic information and the statistical information obtained from the camera image of the camera 200a, and detects an abnormality in the camera image of the camera 200a based on the result of comparing the calculated deviation with a predetermined threshold. That is, if the calculated deviation is larger than the predetermined threshold, the abnormality detection unit 107 determines that there is an abnormality in the camera image of the camera 200a, and if the deviation is smaller than the predetermined threshold, the abnormality detection unit 107 determines that there is no abnormality in the camera image of the camera 200a.

[0062] The predetermined threshold for determining whether or not there is an abnormality in the camera image may be set in advance. For example, if the traffic information and statistical information include traffic information and statistical information on vehicles and people, a threshold for vehicles and a threshold for people may be set, or a threshold for each vehicle type and each person attribute may be set. In this case, an abnormality in the camera image may be determined based on a comparison result of any or all of the thresholds, or an abnormality in the camera image may be determined for each threshold. The predetermined threshold may also be set according to road information, etc. For example, the predetermined threshold may be set according to the traffic disruption level at the intersection 401b indicated by the road information. The higher the traffic disruption level, the higher the predetermined threshold may be set, and the lower the traffic disruption level, the lower the predetermined threshold may be set.

[0063] Furthermore, an abnormality in the camera image may be detected based on the duration of time during which a large deviation between the current traffic information and the statistical information continues. For example, the abnormality detection unit 107 may determine that there is an abnormality in the camera image if the deviation is larger than a predetermined threshold value for a predetermined period of time, and may determine that there is no abnormality in the camera image if the deviation is not larger than the predetermined threshold value for a predetermined period of time.

[0064] The abnormality detection unit 107 may also determine the deviation between the current traffic information acquired from the camera image of the camera 200a and the statistical information, and determine the abnormality level of the camera image of the camera 200a based on the degree or magnitude of the deviation. For example, the abnormality level may be determined as one of three levels: high, medium, and low, or any number of levels. For example, if the degree of deviation is large and the abnormality level is determined to be high, for example, if the number of passing vehicles remains unchanged at zero, the camera itself may be malfunctioning. If the degree of deviation is small and the abnormality level is determined to be low, for example, if the number of passing vehicles continues to be lower than the statistical information, the abnormality may be due to external factors such as dirt or snow on the camera or lens.

[0065] The criteria or range for determining the abnormality level of the camera image may be set in advance, similar to the above-mentioned predetermined threshold, or may be set according to road information, etc. For example, the criteria for determining the abnormality level may be set for each vehicle type or each person attribute. The criteria for determining the abnormality level may be set according to the traffic disruption level at the intersection 401b indicated by the road information. The abnormality level of the camera image may be determined based on the degree of deviation estimated from the traffic disruption level at the intersection 401b indicated by the road information and the degree of deviation from the traffic information and statistical information.

[0066] For example, the abnormality level of the camera image may be determined according to the duration of the discrepancy between the current traffic information and the statistical information. For example, the abnormality detection unit 107 may determine that the abnormality level is high if the discrepancy is greater than a predetermined threshold and this state continues for a predetermined time, and may determine that the abnormality level is low if the discrepancy is greater than the predetermined threshold and this state does not continue for the predetermined time.

[0067] As described above, in this embodiment, in a monitoring system that monitors road traffic conditions, an abnormality determination is performed on the camera image captured by a camera installed on a road based on road information from other points on the road. For example, if traffic is disrupted at another point due to a road closure or the like, an external factor at that point is likely to affect the camera image, so an abnormality determination is not performed on the camera image. However, if traffic is not disrupted at that point, an abnormality determination is performed on the camera image, so that an abnormality determination is performed on the camera image, because factors within the camera itself are likely to affect the camera image. This prevents image abnormalities caused by external factors at other points from being determined to be abnormal in the camera image, thereby improving the accuracy of abnormality detection in the camera image. Furthermore, not performing an abnormality determination reduces the processing load.

[0068] Second Embodiment Hereinafter, a second embodiment will be described with reference to the drawings. Fig. 6 shows an example of the configuration of a system according to this embodiment.

[0069] 6, the monitoring system 1 according to this embodiment includes a road management server 500 in addition to the configuration of the first embodiment. The center server 100 and the road management server 500 are connected so as to be able to communicate with each other.

[0070] The road management server 500 is a management server that manages road information, such as a VICS (registered trademark) (Vehicle Information and Communication System) server. The road management server 500 collects and manages road information, including road closures and restrictions due to accidents, construction, etc., based on various sensors installed at various points on the road and input information. For example, the road management server 500 manages road information for intersection 401b on road 400.

[0071] Furthermore, the center server 100 includes a road information acquisition unit 109 instead of the road information recognition unit 106 of the first embodiment. The road information acquisition unit 109 acquires road information about the intersection 401b from the road management server 500. Note that the road information acquisition unit 109 may acquire the road information about the intersection 401b based on various sensors installed at the intersection 401b, input information, and the like.

[0072] In this embodiment, the anomaly detection unit 107 detects anomalies in the camera image from the camera 200a based on the traffic information for the intersection 401a recognized by the traffic information recognition unit 102 from the camera image from the camera 200a, the statistical information stored in the statistical information storage unit 105, and the road information for the intersection 401b acquired by the road information acquisition unit 109. The other configurations and anomaly detection methods are the same as those in the first embodiment. In this way, even when road information is acquired from a road management server or the like that manages road information, the accuracy of detecting anomalies in the camera image can be improved, as in the first embodiment.

[0073] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present disclosure. For example, in the above-described embodiment, an abnormality in the camera image at the first location is determined taking into account road information at the second location, but an abnormality in the camera image at the first location may also be determined taking into account road information at the first location.

[0074] Each component in the above-described embodiments may be configured with hardware or software, or both, and may be configured with a single piece of hardware or software, or may be configured with multiple pieces of hardware or software. Each device and each function (process) may be realized by a computer 20 having a processor 21 such as a CPU (Central Processing Unit) and a memory 22 serving as a storage device, as shown in FIG. 7. For example, a program for performing the method in the embodiment (monitoring method or anomaly detection method) may be stored in the memory 22, and each function may be realized by the processor 21 executing the program stored in the memory 22.

[0075] These programs include instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The programs may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The programs may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0076] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0077] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. (Supplementary Note 1) A monitoring device comprising: a camera image acquisition means for acquiring camera image of a road at a first location from a camera disposed at the first location; a traffic information acquisition means for analyzing the camera image and acquiring traffic information indicating traffic conditions on the road at the first location; a road information acquisition means for acquiring road information indicating traffic disruptions at a second location leading to the first location; and an abnormality detection means for detecting an abnormality in the camera image based on the traffic information, statistical information about the traffic information, and the road information. (Supplementary Note 2) The monitoring device according to Supplementary Note 1, wherein the abnormality detection means determines an abnormality in the camera image using the traffic information and the statistical information in accordance with the road information. (Supplementary Note 3) The monitoring device according to Supplementary Note 2, wherein the abnormality detection means determines an abnormality in the camera image when the road information includes information indicating no traffic disruptions at the second location. (Supplementary Note 4) The monitoring device according to Supplementary Note 3, wherein the abnormality detection means detects an abnormality in the camera image based on a discrepancy between the traffic information and the statistical information when determining an abnormality in the camera image. (Supplementary Note 5) The monitoring device according to Supplementary Note 4, wherein the abnormality detection means detects an abnormality in the camera image based on a comparison result between the discrepancy and a predetermined threshold. (Supplementary Note 6) The monitoring device according to Supplementary Note 5, wherein the abnormality detection means sets the predetermined threshold according to a traffic disruption level at the second location indicated by the road information. (Supplementary Note 7) The monitoring device according to any one of Supplements 2 to 5, wherein the abnormality detection means does not detect an abnormality in the camera image when the road information includes information indicating a traffic disruption at the second location. (Supplementary Note 8) The monitoring device according to any one of Supplements 1 to 7, wherein the abnormality detection means determines the abnormality level of the camera image according to a degree of discrepancy between the traffic information and the statistical information. (Supplementary Note 9) The monitoring device according to Supplementary Note 8, wherein the abnormality detection means determines the abnormality level of the camera image based on the degree of deviation estimated from the traffic disruption level at the second location indicated by the road information and the degree of deviation of the traffic information and the statistical information.(Supplementary Note 10) The monitoring device according to any one of Supplements 1 to 9, wherein the abnormality detection means detects an abnormality in the camera image caused by a camera failure. (Supplementary Note 11) The monitoring device according to any one of Supplements 1 to 10, wherein the road information acquisition means acquires the road information from camera images captured by a camera arranged at the second location. (Supplementary Note 12) The monitoring device according to any one of Supplements 1 to 10, wherein the road information acquisition means acquires road information for the second location from a management device that manages the road information. (Supplementary Note 13) The monitoring device according to any one of Supplements 1 to 12, wherein the road information acquisition means acquires a plurality of pieces of road information for a plurality of the second locations, and the abnormality detection means detects an abnormality in the camera image based on the plurality of pieces of road information. (Supplementary Note 14) The monitoring device according to any one of Supplements 1 to 13, wherein the road information includes road closure information for the road or regulation information for the road. (Supplementary Note 15) The monitoring device according to any one of Supplements 1 to 14, wherein the traffic information includes traffic volume, speed, or type of vehicular objects traveling on the road. (Supplementary Note 16) The monitoring device according to Supplementary Note 15, wherein the vehicular objects include vehicles or people. (Supplementary Note 17) The monitoring device according to any one of Supplements 1 to 16, comprising statistical information storage means for storing statistical information of the traffic information, and wherein the abnormality detection means detects an abnormality in the camera image based on the stored statistical information. (Supplementary Note 18) The monitoring device according to any one of Supplements 1 to 17, comprising statistical information generation means for generating the statistical information based on a compilation result of the traffic information for a predetermined period of time. (Supplementary Note 19) The monitoring device according to any one of Supplements 1 to 18, wherein the statistical information includes statistical information for each time period, each day of the week, or each month.(Supplementary Note 20) A monitoring system comprising: a camera arranged at a first location and a monitoring device, wherein the monitoring device comprises: camera image acquisition means for acquiring camera image of a road at the first location from the camera, traffic information acquisition means for analyzing the camera image and acquiring traffic information indicating traffic conditions on the road at the first location, road information acquisition means for acquiring road information indicating traffic disruptions at a second location leading to the first location, and abnormality detection means for detecting an abnormality in the camera image based on the traffic information, statistical information on the traffic information, and the road information. (Supplementary Note 21) The monitoring system according to Supplementary Note 20, wherein the abnormality detection means determines an abnormality in the camera image using the traffic information and the statistical information in accordance with the road information. (Supplementary Note 22) A monitoring method comprising: acquiring camera video of a road at a first point from a camera placed at the first point; analyzing the camera video to acquire traffic information indicating traffic conditions on the road at the first point; acquiring road information indicating traffic disruptions at a second point leading to the first point; and detecting an abnormality in the camera video based on the traffic information, statistical information on the traffic information, and the road information. (Supplementary Note 23) The monitoring method according to Supplementary Note 22, wherein the detection of the abnormality comprises determining an abnormality in the camera video using the traffic information and the statistical information in accordance with the road information. (Supplementary Note 24) A non-transitory computer-readable medium storing a monitoring program for causing a computer to execute the following processes: acquiring camera video of a road at a first location from a camera placed at the first location, analyzing the camera video to acquire traffic information indicating traffic conditions on the road at the first location, acquiring road information indicating traffic disruptions at a second location leading to the first location, and detecting an abnormality in the camera video based on the traffic information, statistical information on the traffic information, and the road information. (Supplementary Note 25) The non-transitory computer-readable medium according to Supplementary Note 24, wherein the detection of the abnormality involves determining an abnormality in the camera video using the traffic information and the statistical information in accordance with the road information.

[0078] REFERENCE SIGNS LIST 1 Monitoring system 10 Monitoring device 11 Camera image acquisition unit 12 Traffic information acquisition unit 13 Road information acquisition unit 14 Abnormality detection unit 20 Computer 21 Processor 22 Memory 100 Center server 101 Camera image acquisition unit 102 Traffic information recognition unit 103 Traffic information DB 104 Statistical information generation unit 105 Statistical information storage unit 106 Road information recognition unit 107 Abnormality detection unit 108 Output unit 109 Road information acquisition unit 200, 200a, 200b Camera 300 Edge processing device 400 Road 401, 401a, 401b Intersection 402, 402a, 402b Traffic light 500 Road management server

Claims

1. a camera image acquisition means for acquiring a camera image of a road at a first location from a camera disposed at the first location; a traffic information acquisition means for analyzing the camera image and acquiring traffic information indicating a traffic condition of a road at the first location; a road information acquisition means for acquiring road information indicating a traffic disruption state at a second point leading to the first point; an abnormality detection means for detecting an abnormality in the camera image based on the traffic information, statistical information of the traffic information, and the road information; A monitoring device comprising:

2. the abnormality detection means performs an abnormality determination for the camera image using the traffic information and the statistical information in response to the road information; The monitoring device according to claim 1.

3. the abnormality detection means performs an abnormality determination on the camera image when the road information includes information indicating that there is no traffic disruption at the second point. The monitoring device according to claim 2.

4. the abnormality detection means detects an abnormality in the camera image based on a deviation between the traffic information and the statistical information in determining an abnormality in the camera image. The monitoring device according to claim 3.

5. the abnormality detection means detects an abnormality in the camera image based on a comparison result between the deviation and a predetermined threshold value.

5. The monitoring device according to claim 4.

6. the abnormality detection means sets the predetermined threshold value according to a traffic disruption level at the second location indicated by the road information.

6. The monitoring device according to claim 5.

7. the abnormality detection means does not detect an abnormality in the camera image when the road information includes information indicating that there is a traffic disruption at a second point; A monitoring device according to any one of claims 2 to 5.

8. A camera disposed at a first location and a monitoring device, The monitoring device includes: a camera image acquisition means for acquiring a camera image of a road at the first location from the camera; A traffic information acquisition means for analyzing the camera image and acquiring traffic information indicating a traffic condition of a road at the first location; a road information acquisition means for acquiring road information indicating a traffic disruption state at a second point leading to the first point; an abnormality detection means for detecting an abnormality in the camera image based on the traffic information, statistical information of the traffic information, and the road information; A surveillance system comprising:

9. A camera image of a road at a first location is acquired from a camera disposed at the first location; Analyzing the camera image and acquiring traffic information indicating a traffic condition of a road at the first location; acquiring road information indicating a traffic disruption state at a second point leading to the first point; Detecting an abnormality in the camera image based on the traffic information, statistical information of the traffic information, and the road information. Monitoring method.

10. A camera image of a road at a first location is acquired from a camera disposed at the first location; Analyzing the camera image and acquiring traffic information indicating a traffic condition of a road at the first location; acquiring road information indicating a traffic disruption state at a second point leading to the first point; Detecting an abnormality in the camera image based on the traffic information, statistical information of the traffic information, and the road information. A monitoring program that causes a computer to execute a process.