Traffic video monitoring system and method and storage medium

Abstract

The invention discloses a traffic video monitoring system and method and a storage medium. The system comprises a video sensing module used for providing video sensing data; the video analysis module is used for providing a plurality of video analysis services which are set in a grading manner; the traffic monitoring application module is used for analyzing a business scene demand to obtain a real-time video analysis demand, the video analysis demand comprises a target video data ID and a target video analysis service, and the video analysis demand is sent to the scheduling module; and the scheduling module is used for scheduling a target video analysis service to carry out video analysis on the target video data to obtain video analysis data, and sending the video analysis data to the traffic monitoring application module. According to the invention, the scheduling module is constructed, the corresponding camera device and the video analysis service are called as required based on the real-time video analysis requirement, intelligent scheduling is realized, optimization of resource and computing power distribution of video perception and video analysis is facilitated, and the efficiency of traffic monitoring and management is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a traffic video monitoring system, method and storage medium. Background Art

[0002] Video monitoring is a basic means of traffic operation monitoring. By accessing different video surveillance images, monitoring personnel can keep abreast of traffic operation conditions, detect abnormal traffic events in a timely manner and deal with them. In recent years, with the encrypted deployment of video acquisition equipment on expressways and ordinary roads, and the continuous development of technologies such as artificial intelligence, AI analysis of traffic videos and images can automatically identify various abnormal events in the video, such as vehicle violations, pedestrian intrusions, traffic congestion, etc., greatly reducing the workload of monitoring personnel.

[0003] However, in terms of the business needs of highway operation monitoring and management, the current video monitoring methods still have some shortcomings. First, AI video analysis services are not mature enough to independently undertake traffic video monitoring work. Secondly, the AI ​​video monitoring system is a relatively closed system, with low integration and synergy with other traffic monitoring systems, and cannot be differentiated and integrated and called by business systems, which to a certain extent limits the effectiveness of the AI ​​video monitoring system. Finally, indiscriminate video analysis of all traffic videos is not only inefficient and difficult to respond in real time, but also wastes unnecessary video computing resources. Summary of the invention

[0004] The embodiments of the present application provide a traffic video monitoring system, method and storage medium to at least solve the technical problem in the related art that it is difficult to call the video analysis service on demand.

[0005] According to one aspect of an embodiment of the present application, a traffic video monitoring system is provided, including:

[0006] A video perception module, used to provide video perception data, wherein the video perception module includes a plurality of camera devices;

[0007] A video analysis module, which is in communication with the video perception module, and is used to provide a plurality of video analysis services arranged in a hierarchical manner;

[0008] Traffic monitoring application module, used to analyze business scenario requirements, obtain real-time video analysis requirements, the video analysis requirements include target video data ID and target video analysis service, and send the video analysis requirements to the scheduling module;

[0009] The scheduling module is used to schedule the target video analysis service to perform video analysis on the target video data, obtain video analysis data, and send the video analysis data to the traffic monitoring application module.

[0010] According to one aspect of an embodiment of the present application, a traffic video monitoring method is provided, comprising:

[0011] Providing video perception data based on a video perception module, wherein the video perception module includes a plurality of camera devices;

[0012] Providing multiple video analysis services with hierarchical settings based on video analysis modules;

[0013] Based on the traffic monitoring application module, the business scenario is analyzed to obtain the real-time video analysis requirements, and the video analysis requirements are sent to the scheduling module;

[0014] The target video analysis service is scheduled based on the scheduling module to perform video analysis on the target video data to obtain video analysis data, and the video analysis data is sent to the traffic monitoring application module.

[0015] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned traffic video monitoring method when running.

[0016] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0017] The traffic video monitoring system provided by the embodiment of the present application includes a video perception module, a video analysis module, a traffic monitoring application module, and a scheduling module. The video analysis capability is separated into software and hardware, which can realize independent upgrades of software and hardware, expand the source of video analysis services, and improve the reliability of video analysis capabilities. The system design allows more cameras and video analysis services to be added as needed, which provides convenience for future technical upgrades and functional expansions. And by constructing a scheduling module, based on real-time video analysis needs, the corresponding video analysis service is called on demand to perform video analysis on the target video data, and the video analysis data required by the application system is obtained. Through intelligent scheduling of video data and video analysis services, it helps to optimize resource allocation, improve analysis efficiency, reduce unnecessary data processing, and improve the efficiency of traffic monitoring and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a schematic diagram of a traffic video monitoring system according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of another traffic video monitoring system according to an embodiment of the present application;

[0021] Figure 3 is a functional structure diagram of a traffic incident handling application system according to an embodiment of the present application;

[0022] Figure 4 is a functional structure diagram of a vehicle video tracking application system according to an embodiment of the present application;

[0023] Figure 5 It is a functional structure diagram of a traffic enhancement application system according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] At present, there are still some shortcomings in AI-based video monitoring methods, which are mainly reflected in the following aspects:

[0027] (1) There is still a gap in the accuracy and scene generalization capabilities of AI video recognition. The false positives and missed positives of abnormal event detection cannot be ignored, and the workload of manual video inspection cannot be substantially reduced.

[0028] (2) The AI ​​video monitoring system is a relatively closed system. It has low integration and coordination with other traffic monitoring systems and cannot be integrated and called by business systems in a differentiated manner. This limits the effectiveness of the AI ​​video monitoring system to a certain extent.

[0029] (3) AI video surveillance systems are computationally intensive and energy intensive. It is not economical to deploy AI video analysis capabilities indiscriminately over a large area, and it does not meet the requirements of high-quality development. We need AI analysis capabilities that are classified and supplied on demand.

[0030] (4) Although the hardware level of video cameras is constantly improving and integrating more AI analysis capabilities, from the perspective of overall coordination and high-quality development, AI video monitoring systems should promote the separation of software and hardware. On the one hand, they should be compatible with different levels of video camera hardware, and on the other hand, they should enhance the scalability of the video monitoring system.

[0031] Based on this, the embodiment of the present application provides a new traffic video monitoring application framework, especially for traffic video monitoring application framework on highways. In the traffic operation monitoring application scenario, the hierarchical classification and on-demand call of traffic video AI analysis capabilities are realized, and the data of other traffic monitoring systems are integrated to improve the practicality of AI video recognition, significantly improve the software and hardware performance of the current AI video monitoring system, and promote the high-quality development of traffic operation monitoring business.

[0032] The following is combined with Figure 1-5 The traffic video monitoring system of an embodiment of the present application is introduced in detail.

[0033] like Figure 1 As shown, the traffic video monitoring system provided by the embodiment of the present application includes:

[0034] A video perception module, used to provide video perception data, wherein the video perception module includes a plurality of camera devices;

[0035] A video analysis module, which is in communication with the video perception module, and is used to provide a plurality of video analysis services arranged in a hierarchical manner;

[0036] The traffic monitoring application module is used to analyze the business scenario requirements, obtain the video analysis requirements, and send the video analysis requirements to the scheduling module, wherein the video data collected by each camera device has a unique ID identification;

[0037] The scheduling module is used to schedule the target video analysis service to perform video analysis on the target video data, obtain video analysis data, and send the video analysis data to the traffic monitoring application module.

[0038] In one embodiment, the traffic video monitoring system includes two parts: a terminal side and a cloud side, wherein a video perception module is deployed on the terminal side. The video perception module includes multiple cameras, which capture video data and store it on the terminal side or send it to the cloud.

[0039] In one embodiment, the video perception module is in communication connection with a video data storage module located in the cloud, wherein the video data storage module is used to store video data collected by the camera device and video analysis data after video analysis using the primary video analysis service. It can be understood that, on the one hand, the video perception module sends the collected video data to the cloud video data storage module through the gateway for storage, and on the other hand, the video perception module can call its own primary video analysis service to analyze the collected video data, obtain the first video analysis data after the primary video analysis service analysis, and send it to the cloud video data storage module for storage through the gateway, and the first video analysis data is part of the traffic perception data.

[0040] In one embodiment, a video analysis module is used to provide multiple hierarchical video analysis services. According to different computing power requirements, the video analysis services can be divided into multiple levels, such as level one, level two, and level three. The video analysis service can be deployed on the video perception module on the terminal side or on the cloud side, that is, the video analysis service is separated from the video perception module in terms of hardware and software. On the one hand, independent upgrades of hardware and software can be achieved, and on the other hand, flexible scheduling of video analysis service capabilities and coordination between various video analysis service capabilities can be achieved, thereby improving the reliability of video analysis service capabilities.

[0041] In one embodiment, the video analysis service provided by the video analysis module includes a primary video analysis service and a secondary video analysis service.

[0042] The first-level video analysis service includes at least one of flow calculation, speed calculation, and object tracking, wherein the first-level video analysis service is set on the cloud service end or the video perception module; the second-level video analysis service includes at least one or more of license plate recognition, vehicle face recognition, and event detection. In an exemplary scenario, event detection includes meteorological event detection, traffic accident detection, abnormal parking, traffic control, and spilled object detection. These complex video analysis services are set as second-level video analysis services. Among them, the second-level video analysis service is set on the cloud service end. When the traffic monitoring application module obtains the business scenario requirements such as traffic accident detection, abnormal parking, traffic control, etc., the scheduling module can call the corresponding second-level video analysis service from the cloud to analyze the video data of the corresponding time and space range.

[0043] It can be understood that the target video analysis service corresponding to the video analysis requirement in the above implementation manner may include both a primary video analysis service and a secondary video analysis service.

[0044] In one embodiment, each target video analysis service has a unique identifier, including a video analysis service type identifier, a level identifier and a location identifier, wherein the type identifier refers to the functional type of the video analysis service: such as traffic calculation, object tracking, and license plate recognition; the level identifier refers to the level of the video service, such as level one, level two, and level three; the location identifier refers to the location where the video analysis service is deployed. If it is deployed on the terminal side, the location identifier is used to indicate the camera device where the video analysis service is located. If it is deployed on the cloud side, the location identifier is used to indicate the cloud address where the video analysis service is located or the spatial range that can be served.

[0045] It can be understood that in this implementation, traffic video AI analysis capabilities are divided into two categories based on the differences in scene complexity, maturity and frequency of use: one category is video AI analysis capabilities with low complexity, high maturity and high frequency of use, recorded as first-level video analysis capabilities, such as traffic calculation, speed calculation, object tracking, etc.; the other category is video AI analysis capabilities with high complexity, low maturity and low frequency of use, recorded as second-level video analysis capabilities, such as license plate recognition, vehicle face recognition, abnormal event detection, etc.

[0046] Based on the implementation methods of the present application, by distinguishing between primary and secondary AI video analysis capabilities, and deploying video analysis capabilities in different levels and regions, video analysis computing resources can be reasonably allocated and scheduled according to the complexity and maturity of the task, ensuring efficient coordination and effective utilization of video analysis resources. The primary AI video analysis capability supports both end-side and cloud-side deployment modes, which improves deployment flexibility and allows the most appropriate deployment method to be selected based on the hardware capabilities of the camera; the secondary AI video analysis capability is called on the cloud side on demand, and resource allocation can be flexibly adjusted according to actual needs to achieve on-demand service. This hierarchical classification and on-demand deployment model helps to achieve efficient utilization of resources, improve system performance and scalability, while reducing costs and improving user experience.

[0047] like Figure 2 As shown, in one embodiment, the traffic video monitoring system further includes: a data fusion module, which is used to receive traffic sensing data and perform data fusion calculation on the traffic sensing data, and send the data fusion calculation result to the traffic monitoring application module;

[0048] Among them, traffic perception data includes at least one of the flow analysis data of the ETC gantry module, abnormal behavior analysis data based on floating vehicles, collision detector data, and video perception data. The abnormal behavior analysis data based on floating vehicles such as sudden acceleration, sudden deceleration, speeding and other "three emergencies and one speed" data, the traffic perception data is stored in the cloud-side database.

[0049] In one embodiment of the present application, the data fusion module receives traffic perception data from other traffic monitoring systems, such as flow analysis data based on the ETC gantry system, abnormal behavior analysis data based on floating vehicles, collision detector data, etc. The data fusion module provides data fusion services based on multi-source traffic perception data to obtain perception fusion data, wherein the original traffic perception data and the perception fusion data are both stored in the cloud-side database.

[0050] In one embodiment, the data fusion module can be used to identify clues of abnormal traffic operation. Specifically, the clues of abnormal traffic operation can be identified based on the fusion calculation results of traffic perception data, and the clues of abnormal traffic operation can be sent to the traffic monitoring application module. It can be understood that the clues of abnormal traffic operation are traffic monitoring requirements under the current business scenario. The traffic monitoring application module parses the traffic business scenario requirement of "abnormal traffic operation" to obtain real-time video analysis requirements, including the video ID of the set of road sections affected by abnormal traffic operation (i.e., the target video data ID) and the analysis service corresponding to the abnormal traffic operation analysis tracking (i.e., the target video analysis service); wherein the clues of abnormal traffic operation are obtained by fusion calculation of one or more of the flow analysis data of the ETC gantry module, the abnormal behavior analysis data based on floating vehicles, the collision detector data, and the video perception data.

[0051] Traffic perception data fusion calculation, in detail, based on the electronic map, the observation field of view of different video cameras needs to be mapped with the electronic map sections; different ETC gantries also need to be mapped with the electronic map sections, and the same is true for collision detectors; floating vehicles go through the Map Matching process, and their data can also be mapped with the electronic map sections. Through data fusion calculation, on the one hand, traffic perception data anomalies discovered by different sensors can be verified against each other in the same spatiotemporal framework; on the other hand, different sensors complement each other in space, especially for flow monitoring scenarios, ETC gantry perception data and video camera perception data can complement each other, in fact encrypting flow sensors, which is beneficial for applications that rely on flow calculation results.

[0052] For example, traffic analysis based on ETC gantry perception data shows that there is currently traffic congestion on a certain road section. The monitoring application module obtains the traffic anomaly clue of the current traffic congestion on a certain road section through the perception fusion module, and retrieves the video data of the corresponding road section for verification, which can improve the accuracy of the recognition result.

[0053] In one embodiment of the present application, when the target video analysis service is dispatched to perform video analysis on the target video data, the traffic perception fusion data of the corresponding time and space range can be referred to to perform video analysis on the target video data to obtain video analysis data. That is, when performing video analysis, abnormal traffic clues such as flow rate and congestion obtained by referring to multi-source traffic perception data and fusion calculations can be used to improve the efficiency and accuracy of video analysis.

[0054] In one implementation, the first-level AI video analysis capability is integrated with high-speed ETC traffic perception technology, floating car-based abnormal behavior analysis technology, collision detector data, etc. to achieve ubiquitous perception of traffic parameters such as traffic and congestion. At the same time, based on the perception fusion results, rapid discovery and cross-verification of abnormal clues are achieved, thereby improving the recall rate and precision rate of traffic anomaly clue discovery (especially the recall rate, to ensure that no clues are missed). Through this monitoring, only video verification of the discovered abnormal traffic clues is required, forming a "man in the loop" video monitoring workflow, which not only improves the reliability of video monitoring work, but also can substantially reduce the workload of manual video inspections.

[0055] In one embodiment of the present application, after determining the business scenario, the business scenario requirements are analyzed to obtain real-time video analysis requirements, and the specific implementation method is as follows: analyze the business scenario requirements, obtain the space-time range and the space-time range of video data ID corresponding to the business scenario requirements, wherein the video data collected by each camera device has a unique ID identification; determine the target video analysis service that meets the business scenario requirements based on the camera device configuration information, wherein the camera device configuration information includes the operating status and capability attributes of the camera device.

[0056] In the above implementation, the service capability attributes include: whether it can rotate, the field of view after rotation, whether it can focus, video clarity, and the video analysis service carried by the camera device; the operating status refers to whether the camera device is turned on and whether it is operating normally.

[0057] In one implementation, according to the camera configuration information, a target video analysis service corresponding to the business scenario requirement is determined, specifically in the following manner: according to the capability attributes and operating status of the camera, a set of video analysis services corresponding to the business scenario is determined, wherein each video analysis service information includes a video analysis service type identifier, a level identifier, and a location identifier;

[0058] The video analysis service set is determined as the target video service ID.

[0059] In one implementation of the present application, the business scenario requirements are derived from traffic application system requirements or abnormal traffic operation clues.

[0060] In one embodiment, parsing business scenario requirements to obtain real-time video analysis requirements includes: when the business scenario requirements are derived from abnormal traffic operation clues, determining corresponding video analysis requirements based on the abnormal traffic operation clues. For example, based on the data fusion module, abnormal traffic operation clues such as traffic congestion or vehicle collision on the target road section are obtained, and the abnormal clues are sent to the traffic application module, and the road section to be analyzed and the event to be analyzed are determined according to the abnormal clues, that is, the video analysis requirements corresponding to the abnormal traffic operation clues.

[0061] Alternatively, when the business scenario requirements are derived from the application system requirements, such as the application system requirements are business scenario development requirements, based on the business scenario development requirements, the real-time video analysis requirements related to the business scenario development requirements are determined. In specific business scenarios, based on business development needs, such as when the business scenario is a traffic incident, based on the development requirements of the traffic incident, dynamic video analysis requirements are continuously generated.

[0062] In one embodiment, the traffic monitoring application module sends the target video data ID obtained by parsing the video analysis requirements of the business scenario to the scheduling module, and the scheduling module calls the camera device video analysis service corresponding to the target video data ID as needed to perform video analysis, obtain video analysis data, and send the video analysis data to the traffic monitoring application module.

[0063] Specifically, the scheduling module obtains the time and space range and video analysis service type corresponding to the video analysis demand; obtains the camera device information within the corresponding time and space range, and determines the video analysis service that meets the video analysis demand based on the camera device information, wherein the camera device information includes the operating status and service capability of the camera device; the service capability attributes include: whether it can be rotated, the field of view after rotation, whether it can focus, video clarity, and the video analysis service carried by the camera device. The video analysis service information that meets the video analysis demand includes the video analysis service type and the location information of the video analysis service.

[0064] The video analysis service at the corresponding position is dispatched to perform video analysis on the video data collected by the camera device to obtain a video analysis result; and the video analysis result is sent to the traffic monitoring application module.

[0065] In one embodiment, the scheduling module pre-stores the location attributes and capability attributes of the camera device; obtains the camera device information within the corresponding time and space range, and determines the video analysis service that meets the video analysis demand based on the camera device information, including: determining the camera device within the time and space range corresponding to the video analysis demand based on the location attributes of the camera device; some cameras have the ability to provide first-level video analysis services, and determining the video analysis service that meets the video analysis demand based on the capability attributes of the camera device, and the service information that meets the video analysis demand includes the video analysis service type and the location information of the video analysis service.

[0066] In another embodiment, the data fusion module is also used to store traffic perception fusion data; the traffic monitoring application module parses the video analysis requirements, obtains the video data ID and video analysis service corresponding to the video analysis requirements, and each video data ID corresponds to a camera device; schedules the video data collected by the camera device in the corresponding time and space range and the traffic perception fusion data in the corresponding time and space range; schedules the corresponding video analysis service, refers to the traffic perception fusion data, performs video analysis on the video data, and obtains video analysis data; and sends the video analysis data to the traffic monitoring application module.

[0067] In one embodiment, the traffic monitoring application module includes a video streaming demand analysis module, which is used to pull the target video stream data in real time according to the target video data ID corresponding to the video analysis demand, and send it to the scheduling module; the scheduling module schedules the target video analysis service to perform video analysis on the target video data.

[0068] In one embodiment, when the video streaming demand analysis module pulls the corresponding video data according to the video analysis demand, it can directly pull it from the camera device, or in some cases, it can also pull it from the cloud data database. The scheduling module calls the target video analysis service to perform video analysis on the pulled video data.

[0069] The following is an explanation combining three commonly used traffic monitoring application modules.

[0070] like Figure 3 The figure shows the process of the traffic incident handling application system. The data fusion module generates abnormal analysis clues by analyzing the flow transfer relationship of adjacent flow sensors, the "three emergencies and one speed" analysis based on floating vehicles, or the analysis results of collision sensors. The clues include a set of road sections that may be affected. Based on the abnormal clues, the video analysis requirements are obtained. The video analysis requirements include the video ID of the affected road section set and the corresponding video analysis service, where the video ID is the ID of the camera, and the video analysis requirements are sent to the scheduling module.

[0071] In this embodiment, the traffic monitoring application module includes a video streaming demand analysis module, and the video analysis services obtained by parsing the video analysis demand include event analysis and event tracking. On the one hand, the scheduling module pulls the video streaming demand analysis module to the manual verification interface according to the video ID corresponding to the video analysis demand, and on the other hand, calls the second-class AI video analysis service corresponding to the video analysis demand to analyze and track the event. It should be pointed out here that with the occurrence and development of abnormal events, the video data ID and video analysis service corresponding to the video analysis demand are also constantly changing. The video streaming demand analysis module should dynamically adjust the video ID set of the streaming, such as continuously pulling the upstream camera video during the process of congestion diffusion caused by the event, and continuously pulling the downstream camera video during the process of congestion dissipation to meet the needs of tracking the event. In other words, the on-demand call of the target video analysis service (second-class AI video analysis service) is based on the cooperation of the video streaming demand analysis module and the scheduling module to generate video streaming requirements that meet the business scenario. It should be noted that the above three urgent and one fast refers to sudden braking, sudden acceleration, sharp turns, and speeding.

[0072] Furthermore, after the abnormal clues are generated, the results of the second-category AI analysis services can be verified based on the manual review process, that is, the manual labeling of the second-category AI analysis services can be realized. This labeling process can be used as a data set for subsequent AI training, further promoting the maturity of the second-category AI analysis services.

[0073] like Figure 4 As shown, the traffic monitoring application module is a vehicle video tracking application system. The application system performs video streaming demand analysis, parses business scenario requirements, and obtains real-time video analysis requirements. The video analysis requirements include the camera ID corresponding to the vehicle's driving route. Under the premise of specifying the vehicle's initial video ID, the video streaming demand analysis module searches for the next one or more camera IDs that the vehicle may pass through based on the vehicle's driving route, as well as license plate recognition, vehicle face recognition and other video analysis services, and sends them to the scheduling module. In an embodiment of the present application, the scheduling module schedules the corresponding camera to shoot video, on the one hand, streams it to the manual tracking interface, and on the other hand, calls the license plate recognition or vehicle face recognition services of the second category of AI analysis services for video analysis, and feeds back the recognition results to the vehicle video tracking application system and the manual tracking interface.

[0074] It is also important to point out that the traffic monitoring application module needs to be linked with the dispatching module. When a vehicle is identified in the current camera, the video demand analysis in the traffic monitoring application module should promptly adjust the subsequent camera IDs so that the camera is dynamically aligned with the preset lane position to achieve continuous tracking services.

[0075] like Figure 5As shown, the traffic monitoring application module is a traffic enhancement application system. The application system performs vehicle flow detection. Based on current business needs, the application system analyzes business scenario requirements and obtains real-time video analysis requirements. The video analysis requirements include the camera ID corresponding to the traffic section to be detected and the video analysis service for vehicle flow calculation. The video analysis requirements are sent to the scheduling module. The scheduling module schedules the video data collected by the camera device in the corresponding time and space range and the traffic perception fusion data in the corresponding time and space range, and schedules the vehicle flow calculation video analysis service to perform video analysis to obtain the calculated vehicle flow. The vehicle flow calculated by the video analysis is fused and verified with the vehicle flow of the traffic perception fusion data to obtain the final flow analysis result. At this time, the traffic perception fusion data includes the vehicle flow data collected by the ETC gantry.

[0076] Therefore, in high-speed scenarios, video cameras and ETC gantries will complement each other in space, which in fact forms the effect of encrypting or filling blind spots of highway traffic sensors. In other words, the distance between two adjacent sensors is smaller, the time for traffic transmission is shorter, and the characteristics are more obvious. This will significantly enhance the accuracy of high-speed traffic prediction and the timeliness of traffic anomaly detection, and improve the efficiency of traffic incident handling.

[0077] The traffic video monitoring system of the present application has at least the following beneficial effects:

[0078] (1) This application classifies video analysis services, and sets the first-level video analysis service on the cloud server or video perception module, and sets the second-level video analysis service on the cloud server. By classifying and differentially deploying the analysis capabilities of video AI, the resource integration and utilization of different video cameras can be achieved, and the energy efficiency of video AI analysis in the traffic system can be improved;

[0079] (2) This application can set up the video analysis service on the cloud service end, separate the software and hardware of the video AI analysis capability, realize the independent upgrade of software and hardware, expand the source of AI analysis services, and improve the reliability of video AI analysis capabilities;

[0080] (3) This application also includes a data fusion module, which can fuse and analyze the abnormal behavior analysis data of floating vehicles, collision detector data, and video perception data, deeply integrate the video AI analysis capabilities with other perception technologies and business systems, and realize the mutual empowerment of video AI analysis and other perception technologies and business systems, thereby improving the capabilities of the overall traffic operation monitoring system;

[0081] (4) This application also constructs a scheduling module, which calls the corresponding camera devices and video analysis services on demand based on real-time video analysis needs. Through intelligent scheduling, it helps to optimize resource allocation, improve analysis efficiency, reduce unnecessary data processing, and improve the efficiency of traffic operation monitoring and management.

[0082] According to another aspect of the embodiment of the present application, a traffic video monitoring method is also provided, including:

[0083] Providing video perception data based on a video perception module, wherein the video perception module includes a plurality of camera devices;

[0084] Provide multiple video analysis services based on the video analysis module;

[0085] Based on the traffic monitoring application module, the business scenario requirements are analyzed to obtain real-time video analysis requirements, which include target video data ID and target video analysis service, and the video analysis requirements are sent to the scheduling module, wherein the video data collected by each camera device has a unique ID identification;

[0086] The scheduling module obtains the video analysis requirements, schedules the corresponding camera devices and video analysis services to perform video analysis according to the video analysis requirements, obtains video analysis data, and sends the video analysis data to the traffic monitoring application module.

[0087] It should be noted that the traffic video monitoring system and the traffic video monitoring method provided in the above embodiments belong to the same concept, and their implementation process is detailed in the system embodiment, which will not be repeated here.

[0088] According to another aspect of the embodiments of the present application, a computer-readable storage medium corresponding to the traffic video monitoring method provided in the aforementioned embodiments is also provided, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the traffic video monitoring method provided in any of the aforementioned embodiments.

[0089] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0090] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the traffic video monitoring method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0091] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

Claims

1. A traffic video monitoring system, characterized in that: include: A video perception module, used to provide video perception data, wherein the video perception module includes a plurality of camera devices; A video analysis module, which is in communication with the video perception module, and is used to provide a plurality of video analysis services arranged in a hierarchical manner; Traffic monitoring application module, used to analyze business scenario requirements, obtain real-time video analysis requirements, the video analysis requirements include target video data ID and target video analysis service, and send the video analysis requirements to the scheduling module; The scheduling module is used to schedule the target video analysis service to perform video analysis on the target video data, obtain video analysis data, and send the video analysis data to the traffic monitoring application module.

2. The system according to claim 1, characterized in that The video analysis service includes a primary video analysis service and a secondary video analysis service; The first-level video analysis service includes at least one or more of flow calculation, speed calculation, and object tracking, wherein the first-level video analysis service is provided in a cloud service end or a video perception module; The secondary video analysis service includes at least one or more of license plate recognition, vehicle face recognition, and event detection, wherein the secondary video analysis service is set on a cloud service end.

3. The system according to claim 2, characterized in that The video perception module is communicatively connected with a video data storage module located in the cloud; The video data storage module is used to store video data collected by the camera device and video analysis data after video analysis using a first-level video analysis service.

4. The system according to claim 1, characterized in that Also includes: A data fusion module is used to receive traffic sensing data and perform data fusion calculation, and send the data fusion calculation result to the traffic monitoring application module; Among them, the traffic perception data includes at least one of the flow analysis data of the ETC gantry module, abnormal behavior analysis data based on floating vehicles, collision detector data, and video perception data.

5. The system according to claim 1, characterized in that The analysis of business scenario requirements to obtain real-time video analysis requirements includes: Parse the business scenario requirements, and obtain the space-time range and video data ID within the space-time range corresponding to the business scenario requirements, wherein the video data collected by each camera device has a unique ID identification; According to the camera configuration information, a target video analysis service that meets the business scenario requirements is determined, wherein the camera configuration information includes an operating state and capability attributes of the camera.

6. The system according to claim 5, characterized in that Determining a target video analysis service that meets the business scenario requirements according to the camera device configuration information includes: Determine a target video analysis service set that meets the business scenario requirements according to the capability attributes and operating status of the camera device, wherein each target video analysis service information includes a video analysis service type identifier, a level identifier, and a location identifier; The video analysis service set is determined as the target video analysis service ID.

7. The system according to claim 1, characterized in that The traffic monitoring application module includes a video streaming demand analysis module; The video streaming demand analysis module is used to pull the target video stream data in real time according to the target video data ID and send it to the scheduling module; The scheduling module schedules the target video analysis service to perform video analysis on the target video data.

8. The system according to claim 4, characterized in that The data fusion module is also used to store traffic perception data and data fusion calculation results; The scheduling target video analysis service performs video analysis on the target video data, and the obtained video analysis data includes: The target video analysis service is dispatched, and the traffic perception fusion data in the corresponding time and space range is referred to to perform video analysis on the target video data to obtain video analysis data.

9. A traffic video monitoring method, characterized in that: include: Providing video perception data based on a video perception module, wherein the video perception module includes a plurality of camera devices; Providing multiple video analysis services with hierarchical settings based on video analysis modules; Based on the traffic monitoring application module, the business scenario requirements are analyzed to obtain real-time video analysis requirements, which include the target video data ID and the target video analysis service ID, and the video analysis requirements are sent to the scheduling module; The target video analysis service is scheduled based on the scheduling module to perform video analysis on the target video data to obtain video analysis data, and the video analysis data is sent to the traffic monitoring application module.

10. A computer-readable medium, characterized in that Computer-readable instructions are stored thereon, and the computer-readable instructions are executed by a processor to implement a traffic video monitoring method as claimed in claim 9.

Images