Highway intelligent monitoring method and system based on video AI

Abstract

The invention discloses an expressway intelligent monitoring method and system based on video AI, and relates to the technical field of expressway monitoring, and the method comprises the steps: employing a plurality of video collection devices to collect a plurality of monitoring video streams of a plurality of monitoring areas of an expressway in real time; according to the position relationship of the plurality of monitoring areas, splicing the plurality of monitoring video streams to obtain a spliced video stream; performing filtering processing on the three-dimensional point cloud data of the spliced video stream to obtain de-noised three-dimensional point cloud data; road surface dynamic feature data are extracted from the denoised three-dimensional point cloud data; according to the target object feature data and the target behavior feature data, using a Kalman filter to predict and update the state of the target object and the state of the target behavior to obtain a state prediction update result; and on the basis of the state prediction updating result, AI analysis is performed on the plurality of monitoring video streams, so that highway abnormal events can be identified and early warning can be performed.

Description

Technical Field

[0001] The present application relates to the field of highway monitoring technology, and more specifically, to a method and system for intelligent highway monitoring based on video AI. Background Art

[0002] With the rapid development of society and the acceleration of urbanization, the traffic volume on highways is growing, and real-time monitoring and management of highway traffic conditions are becoming increasingly important. As an important hub of urban transportation, the smoothness of highways is directly related to the economic development of cities and the daily travel of residents. Highways adapt to the development of industrialization and urbanization. Cities are places where industries and populations gather. The growth of cars in cities is much faster than that in rural areas, making them automobile gathering centers. Highway construction often starts from urban ring roads, radial roads and busy traffic sections, and gradually becomes urban transportation with highways as the backbone.

[0003] Although existing highway monitoring systems have achieved full-course video information transmission, allowing traffic regulators to obtain real-time traffic conditions on highways, these systems can only provide basic video information and lack in-depth analysis and prediction capabilities of video information. This means that when abnormal conditions occur on highways, traffic regulators are usually unable to quickly understand the actual situation and make effective emergency responses.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method and system for intelligent highway monitoring based on video AI to solve the above-mentioned technical problems.

[0006] This application provides a method for intelligent highway monitoring based on video AI, including:

[0007] Using multiple video acquisition devices to collect multiple monitoring video streams of multiple monitoring areas of the highway in real time; wherein the multiple video acquisition devices include a time-of-flight camera and a structured light camera;

[0008] splicing the multiple monitoring video streams according to the positional relationship of the multiple monitoring areas to obtain a spliced ​​video stream;

[0009] Performing filtering processing on the three-dimensional point cloud data of the spliced ​​video stream to obtain denoised three-dimensional point cloud data;

[0010] Extracting road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data and obstacle feature data;

[0011] According to the target object feature data and the target behavior feature data, a Kalman filter is used to predict and update the state of the target object and the state of the target behavior to obtain a state prediction update result;

[0012] Based on the status prediction update results, AI analysis is performed on the multiple monitoring video streams to identify abnormal events on the highway and issue early warnings.

[0013] Furthermore, the multiple monitoring video streams are spliced ​​according to the positional relationship of the multiple monitoring areas to obtain a spliced ​​video stream; and the three-dimensional point cloud data of the spliced ​​video stream is filtered to obtain denoised three-dimensional point cloud data, including:

[0014] Performing distortion correction on the multiple surveillance video streams to obtain multiple corrected video streams;

[0015] Determining overlapping areas and overlapping relationships of the multiple corrected video streams according to the positional relationship of the multiple monitoring areas;

[0016] Based on the SLIC algorithm, superpixel segmentation is performed on the multiple corrected video streams according to the overlapping areas and the overlapping relationships to obtain contrast information of each pixel in the superpixel;

[0017] splicing the multiple corrected video streams according to contrast information of each pixel in the superpixel to obtain the spliced ​​video stream;

[0018] Converting the three-dimensional point cloud data of the spliced ​​video stream into a three-dimensional point cloud matrix;

[0019] The three-dimensional point cloud matrix is ​​filtered using a Laplace filter operator to obtain denoised three-dimensional point cloud data.

[0020] Furthermore, extracting road surface dynamic feature data from the denoised three-dimensional point cloud data includes:

[0021] The ground plane filtering algorithm is used to separate the ground point cloud data and the non-ground point cloud data from the denoised three-dimensional point cloud data;

[0022] Calculating the geometric features of each point in the ground point cloud data; wherein the geometric features include normals and curvatures;

[0023] Selecting multiple seed points in the road surface area of ​​the expressway as starting points for region growth;

[0024] Defining a region growing rule; wherein the region growing rule includes a normal angle threshold and a curvature threshold;

[0025] Starting from the plurality of seed points, diffusing to the surrounding areas according to the regional growth rule, and adding points satisfying the regional growth rule into the region;

[0026] Extracting pavement static characteristic data based on the area obtained by growth;

[0027] The depth value of each pixel in the non-ground point cloud data is calculated and processed based on the road surface static feature data to extract the target object feature data and the target behavior feature data.

[0028] Further, the state of the target object and the state of the target behavior are predicted and updated using a Kalman filter according to the target object feature data and the target behavior feature data to obtain a state prediction update result; based on the state prediction update result, the multiple monitoring video streams are subjected to AI analysis to identify abnormal events on the highway and issue an early warning, including:

[0029] Setting a first initial parameter of the state of the target object and a second initial parameter of the state of the target behavior; wherein the first initial parameter includes a first initial state value, a first initial covariance matrix, and a first constraint condition; and the second initial parameter includes a second initial state value, a second initial covariance matrix, and a second constraint condition;

[0030] Calculating a priori estimated state and a Kalman gain matrix according to the first initial parameter of the state of the target object and the second initial parameter of the state of the target behavior;

[0031] Calculating a corrected estimated state based on the prior estimated state and the Kalman gain matrix;

[0032] Predicting and updating the state of the target object and the state of the target behavior according to the corrected estimated state to obtain the state prediction update result;

[0033] Identifying the state prediction update result based on a gated recurrent unit to obtain a target state factor;

[0034] If the target state factor belongs to the preset state interval, it is determined that the highway abnormal event is identified and an early warning prompt is issued.

[0035] Furthermore, the highway abnormal events include vehicle speeding events, illegal parking events, overtime traffic jam events, wrong-way events, spilled objects events, collision events, and vehicle out-of-control events;

[0036] The vehicle characteristic data includes vehicle flow characteristic data, vehicle speed characteristic data and vehicle type characteristic data.

[0037] Furthermore, the normal angle threshold is used to determine whether the points are on the same plane, and the curvature threshold is used to determine whether the points are within the same curvature variation range.

[0038] The present application provides a highway intelligent monitoring system based on video AI, including a monitoring video stream acquisition unit, a road surface dynamic feature data extraction unit, a state prediction update result determination unit, and an abnormal event recognition and warning unit; wherein,

[0039] The monitoring video stream acquisition unit is used to use multiple video acquisition devices to collect multiple monitoring video streams of multiple monitoring areas of the highway in real time; wherein the multiple video acquisition devices include a time-of-flight camera and a structured light camera;

[0040] The road surface dynamic feature data extraction unit is used to splice the multiple monitoring video streams according to the positional relationship of the multiple monitoring areas to obtain a spliced ​​video stream; filter the three-dimensional point cloud data of the spliced ​​video stream to obtain denoised three-dimensional point cloud data; extract the road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data and obstacle feature data;

[0041] The state prediction update result determination unit is used to predict and update the state of the target object and the state of the target behavior using a Kalman filter according to the target object feature data and the target behavior feature data to obtain a state prediction update result;

[0042] The abnormal event identification and early warning unit is used to perform AI analysis on the multiple monitoring video streams based on the state prediction update result to identify abnormal events on the highway and issue early warnings.

[0043] Based on the technical solution of the present application, multiple video acquisition devices are used to collect multiple monitoring video streams of multiple monitoring areas of a highway in real time; wherein the multiple video acquisition devices include a time-of-flight camera and a structured light camera; according to the positional relationship of the multiple monitoring areas, the multiple monitoring video streams are spliced ​​to obtain a spliced ​​video stream; the three-dimensional point cloud data of the spliced ​​video stream is filtered to obtain denoised three-dimensional point cloud data; the road surface dynamic feature data is extracted from the denoised three-dimensional point cloud data; wherein the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data and obstacle feature data; according to the target object feature data and target behavior feature data, a Kalman filter is used to predict and update the state of the target object and the state of the target behavior to obtain a state prediction update result; based on the state prediction update result, AI analysis is performed on multiple monitoring video streams to identify abnormal events on the highway and issue an early warning. Thus, automatic monitoring, automatic judgment and automatic early warning of abnormal events on the highway based on video AI are realized, the risk management efficiency and risk response efficiency of highway managers are improved, and the all-weather and all-section highway traffic order monitoring and management are realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0045] Figure 1 is a flowchart of an optional video AI-based intelligent highway monitoring method according to an embodiment of the present application;

[0046] Figure 2 This is a structural diagram of an optional video AI-based intelligent highway monitoring system according to an embodiment of the present application.

[0047] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0049] Alternatively, if Figure 1 As shown, the present application provides a method for intelligent highway monitoring based on video AI, comprising:

[0050] S101, using multiple video acquisition devices to collect multiple monitoring video streams of multiple monitoring areas of a highway in real time; wherein the multiple video acquisition devices include a time-of-flight camera and a structured light camera;

[0051] S102, splicing multiple monitoring video streams according to the positional relationship of multiple monitoring areas to obtain a spliced ​​video stream;

[0052] S103, filtering the three-dimensional point cloud data of the spliced ​​video stream to obtain denoised three-dimensional point cloud data;

[0053] S104, extracting road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data and obstacle feature data;

[0054] S105, predicting and updating the state of the target object and the state of the target behavior using a Kalman filter according to the target object feature data and the target behavior feature data, and obtaining a state prediction update result;

[0055] S106, based on the status prediction update results, AI analysis is performed on multiple monitoring video streams to identify abnormal events on the highway and issue warnings.

[0056] Based on the embodiment provided by the present application, multiple video acquisition devices are used to collect multiple monitoring video streams of multiple monitoring areas of a highway in real time; wherein the multiple video acquisition devices include a time-of-flight camera and a structured light camera; according to the positional relationship of the multiple monitoring areas, the multiple monitoring video streams are spliced ​​to obtain a spliced ​​video stream; the three-dimensional point cloud data of the spliced ​​video stream is filtered to obtain the denoised three-dimensional point cloud data; the road surface dynamic feature data is extracted from the denoised three-dimensional point cloud data; wherein the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data and obstacle feature data; according to the target object feature data and target behavior feature data, the Kalman filter is used to predict and update the state of the target object and the state of the target behavior to obtain a state prediction update result; based on the state prediction update result, AI analysis is performed on multiple monitoring video streams to identify abnormal events on the highway and issue an early warning. Thus, automatic monitoring, automatic judgment and automatic early warning of abnormal events on the highway based on video AI are realized, the risk management level and risk response efficiency of the highway operation management team are improved, and the all-weather and all-section highway traffic order monitoring and management are realized.

[0057] Further, according to the positional relationship of the multiple monitoring areas, the multiple monitoring video streams are spliced ​​to obtain a spliced ​​video stream; the three-dimensional point cloud data of the spliced ​​video stream is filtered to obtain denoised three-dimensional point cloud data, including:

[0058] Performing distortion correction on multiple surveillance video streams to obtain multiple corrected video streams;

[0059] Determine overlapping areas and overlapping relationships of multiple corrected video streams according to positional relationships of multiple monitoring areas;

[0060] Based on the SLIC algorithm, multiple corrected video streams are segmented into superpixels according to the overlapping areas and overlapping relationships to obtain the contrast information of each pixel in the superpixel.

[0061] splicing multiple corrected video streams according to contrast information of each pixel in the superpixel to obtain a spliced ​​video stream;

[0062] Convert the three-dimensional point cloud data of the spliced ​​video stream into a three-dimensional point cloud matrix;

[0063] The Laplace filter operator is used to filter the three-dimensional point cloud matrix to obtain denoised three-dimensional point cloud data.

[0064] Furthermore, the road surface dynamic feature data is extracted from the denoised 3D point cloud data, including:

[0065] The ground plane filtering algorithm is used to separate the ground point cloud data and the non-ground point cloud data from the denoised three-dimensional point cloud data;

[0066] Calculate the geometric features of each point in the ground point cloud data; the geometric features include normal and curvature;

[0067] Select multiple seed points in the road surface area of ​​the highway as the starting point of region growth;

[0068] Define a region growing rule; wherein the region growing rule includes a normal angle threshold and a curvature threshold;

[0069] Starting from multiple seed points, they spread outward according to the regional growth rule, and add the points that meet the regional growth rule into the region;

[0070] Extracting pavement static characteristic data based on the area obtained by growth;

[0071] The depth value of each pixel in the non-ground point cloud data is calculated and processed based on the static feature data of the road surface to extract the target object feature data and the target behavior feature data.

[0072] Furthermore, according to the target object feature data and the target behavior feature data, the Kalman filter is used to predict and update the state of the target object and the state of the target behavior to obtain the state prediction update result; based on the state prediction update result, AI analysis is performed on multiple monitoring video streams to identify abnormal events on the highway and issue early warnings, including:

[0073] Setting a first initial parameter of the state of the target object and a second initial parameter of the state of the target behavior; wherein the first initial parameter includes a first initial state value, a first initial covariance matrix, and a first constraint condition; and the second initial parameter includes a second initial state value, a second initial covariance matrix, and a second constraint condition;

[0074] Calculating a priori estimated state and a Kalman gain matrix according to a first initial parameter of the state of the target object and a second initial parameter of the state of the target behavior;

[0075]

[0076] Among them, M Kal represents the Kalman gain matrix; M Cov1 represents the first initial covariance matrix; M Cov2 represents the second initial covariance matrix; M prev_cov is the non-deterministic matrix of the prior estimated state; M prev_cov T is the transpose of the non-deterministic matrix of the prior estimate state; M Q is the process noise covariance matrix;

[0077] Based on the prior estimated state and the Kalman gain matrix, a corrected estimated state is calculated;

[0078] Predict and update the state of the target object and the state of the target behavior according to the corrected estimated state to obtain a state prediction update result;

[0079] Based on the gated recurrent unit, the state prediction update result is identified to obtain the target state factor;

[0080] If the target state factor belongs to the preset state range, it is determined that an abnormal event on the highway is identified and an early warning prompt is issued.

[0081] Furthermore, M prev_cov =M prev_sta ×P cov ×M prev_sta T +M Q

[0082] M prev_cov is the non-deterministic matrix of the prior estimated state; M prev_stais the state transfer matrix of the prior estimated state; M prev_sta T is the transpose of the state transfer matrix of the prior estimated state; P cov The estimated covariance matrix of the corrected estimated state; M Q is the process noise covariance matrix.

[0083] Furthermore, the gated recurrent unit includes an update gate Z t Based on the following formula:

[0084] Z t =σ[W z ×(X t ,H t-1 )+b z ]

[0085] Where σ is the sigmoid activation function; W z It is the update gate Z t The weight matrix of t is the state prediction update result of the current time step input; H t-1 is the hidden state of the previous time step; b z It is the update gate Z t The bias term.

[0086] Furthermore, the highway abnormal events include vehicle speeding events, illegal parking events, overtime traffic jam events, wrong-way events, spilled objects events, collision events, and vehicle loss of control events;

[0087] The vehicle characteristic data includes vehicle flow characteristic data, vehicle speed characteristic data and vehicle type characteristic data.

[0088] Furthermore, the normal angle threshold is used to determine whether the points are on the same plane, and the curvature threshold is used to determine whether the points are within the same curvature variation range.

[0089] Alternatively, if Figure 2 As shown, the present application provides a highway intelligent monitoring system based on video AI, including a monitoring video stream acquisition unit 201, a road surface dynamic feature data extraction unit 202, a state prediction update result determination unit 203, and an abnormal event recognition and warning unit 204; wherein,

[0090] The monitoring video stream acquisition unit 201 is used to use multiple video acquisition devices to collect multiple monitoring video streams of multiple monitoring areas of the highway in real time; wherein the multiple video acquisition devices include a time-of-flight camera and a structured light camera;

[0091] The road surface dynamic feature data extraction unit 202 is used to splice multiple monitoring video streams according to the positional relationship of multiple monitoring areas to obtain a spliced ​​video stream; filter the three-dimensional point cloud data of the spliced ​​video stream to obtain denoised three-dimensional point cloud data; extract road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data and obstacle feature data;

[0092] The state prediction update result determination unit 203 is used to predict and update the state of the target object and the state of the target behavior using a Kalman filter according to the target object feature data and the target behavior feature data to obtain a state prediction update result;

[0093] The abnormal event identification and warning unit 204 is used to perform AI analysis on multiple monitoring video streams based on the state prediction update results to identify abnormal events on the highway and issue warnings.

[0094] Furthermore, the road surface dynamic feature data extraction unit splices multiple monitoring video streams according to the positional relationship of multiple monitoring areas to obtain a spliced ​​video stream; and performs filtering processing on the three-dimensional point cloud data of the spliced ​​video stream to obtain denoised three-dimensional point cloud data, which is configured as follows:

[0095] Performing distortion correction on multiple surveillance video streams to obtain multiple corrected video streams;

[0096] Determine overlapping areas and overlapping relationships of multiple corrected video streams according to positional relationships of multiple monitoring areas;

[0097] Based on the SLIC algorithm, multiple corrected video streams are segmented into superpixels according to the overlapping areas and overlapping relationships to obtain the contrast information of each pixel in the superpixel.

[0098] splicing multiple corrected video streams according to contrast information of each pixel in the superpixel to obtain a spliced ​​video stream;

[0099] Convert the three-dimensional point cloud data of the spliced ​​video stream into a three-dimensional point cloud matrix;

[0100] The Laplace filter operator is used to filter the three-dimensional point cloud matrix to obtain denoised three-dimensional point cloud data.

[0101] It should be noted that in the present application, the embodiments implemented on the highway intelligent monitoring system side based on video AI can be cross-referenced with the embodiments implemented on the highway intelligent monitoring method side based on video AI, and this application will not go into details one by one.

[0102] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for intelligent highway monitoring based on video AI, characterized in that: include: Using multiple video acquisition devices to collect multiple monitoring video streams of multiple monitoring areas of the highway in real time; wherein the multiple video acquisition devices include a time-of-flight camera and a structured light camera; splicing the multiple monitoring video streams according to the positional relationship of the multiple monitoring areas to obtain a spliced ​​video stream; Performing filtering processing on the three-dimensional point cloud data of the spliced ​​video stream to obtain denoised three-dimensional point cloud data; Extracting road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data and obstacle feature data; According to the target object feature data and the target behavior feature data, a Kalman filter is used to predict and update the state of the target object and the state of the target behavior to obtain a state prediction update result; Based on the status prediction update results, AI analysis is performed on the multiple monitoring video streams to identify abnormal events on the highway and issue early warnings.

2. The method for intelligent highway monitoring based on video AI according to claim 1 is characterized in that: The multiple monitoring video streams are spliced ​​according to the positional relationship of the multiple monitoring areas to obtain a spliced ​​video stream; The three-dimensional point cloud data of the spliced ​​video stream is filtered to obtain denoised three-dimensional point cloud data, including: Performing distortion correction on the multiple surveillance video streams to obtain multiple corrected video streams; Determining overlapping areas and overlapping relationships of the multiple corrected video streams according to the positional relationship of the multiple monitoring areas; Based on the SLIC algorithm, superpixel segmentation is performed on the multiple corrected video streams according to the overlapping areas and the overlapping relationships to obtain contrast information of each pixel in the superpixel.

3. The method for intelligent highway monitoring based on video AI according to claim 2 is characterized in that: The multiple monitoring video streams are spliced ​​according to the positional relationship of the multiple monitoring areas to obtain a spliced ​​video stream; The three-dimensional point cloud data of the spliced ​​video stream is filtered to obtain denoised three-dimensional point cloud data, including: splicing the multiple corrected video streams according to contrast information of each pixel in the superpixel to obtain the spliced ​​video stream; Converting the three-dimensional point cloud data of the spliced ​​video stream into a three-dimensional point cloud matrix; The three-dimensional point cloud matrix is ​​filtered using a Laplace filter operator to obtain denoised three-dimensional point cloud data.

4. The method for intelligent highway monitoring based on video AI according to claim 3 is characterized in that: The step of extracting road surface dynamic feature data from the denoised three-dimensional point cloud data includes: The ground plane filtering algorithm is used to separate the ground point cloud data and the non-ground point cloud data from the denoised three-dimensional point cloud data; Calculating the geometric features of each point in the ground point cloud data; wherein the geometric features include normals and curvatures; A plurality of seed points are selected in the road surface area of ​​the expressway as starting points for region growth.

5. The method for intelligent highway monitoring based on video AI according to claim 4 is characterized in that: The step of extracting road surface dynamic feature data from the denoised three-dimensional point cloud data includes: Defining a region growing rule; wherein the region growing rule includes a normal angle threshold and a curvature threshold; Starting from the plurality of seed points, diffusing to the surrounding areas according to the regional growth rule, and adding points satisfying the regional growth rule to the region; Extracting pavement static characteristic data based on the area obtained by growth; The depth value of each pixel in the non-ground point cloud data is calculated and processed based on the road surface static feature data to extract the target object feature data and the target behavior feature data.

6. The method for intelligent highway monitoring based on video AI according to claim 5 is characterized in that: The state of the target object and the state of the target behavior are predicted and updated using a Kalman filter according to the target object feature data and the target behavior feature data to obtain a state prediction update result; Based on the state prediction update result, AI analysis is performed on the multiple monitoring video streams to identify abnormal events on the highway and issue warnings, including: Setting a first initial parameter of the state of the target object and a second initial parameter of the state of the target behavior; wherein the first initial parameter includes a first initial state value, a first initial covariance matrix, and a first constraint condition; and the second initial parameter includes a second initial state value, a second initial covariance matrix, and a second constraint condition; A priori estimated state and a Kalman gain matrix are calculated based on the first initial parameters of the state of the target object and the second initial parameters of the state of the target behavior.

7. The method for intelligent highway monitoring based on video AI according to claim 6 is characterized in that: The state of the target object and the state of the target behavior are predicted and updated using a Kalman filter according to the target object feature data and the target behavior feature data to obtain a state prediction update result; Based on the state prediction update result, AI analysis is performed on the multiple monitoring video streams to identify abnormal events on the highway and issue warnings, including: Calculating a corrected estimated state based on the prior estimated state and the Kalman gain matrix; Predicting and updating the state of the target object and the state of the target behavior according to the corrected estimated state to obtain the state prediction update result; Identifying the state prediction update result based on a gated recurrent unit to obtain a target state factor; If the target state factor belongs to the preset state interval, it is determined that the highway abnormal event is identified and an early warning prompt is issued.

8. The method for intelligent highway monitoring based on video AI according to claim 1 is characterized in that: The abnormal highway events include speeding events, illegal parking events, overtime traffic jam events, wrong-way events, spilled objects events, collision events and vehicle out-of-control events; The vehicle characteristic data includes vehicle flow characteristic data, vehicle speed characteristic data and vehicle type characteristic data.

9. The method for intelligent highway monitoring based on video AI according to claim 5 is characterized in that: The normal angle threshold is used to determine whether the points are on the same plane, and the curvature threshold is used to determine whether the points are within the same curvature variation range.

10. A highway intelligent monitoring system based on video AI, the system implements the method as claimed in claim 1, characterized in that: It includes a monitoring video stream acquisition unit, a road surface dynamic feature data extraction unit, a state prediction update result determination unit, and an abnormal event recognition and warning unit; wherein, The monitoring video stream acquisition unit is used to use multiple video acquisition devices to collect multiple monitoring video streams of multiple monitoring areas of the highway in real time; wherein the multiple video acquisition devices include a time-of-flight camera and a structured light camera; The road surface dynamic feature data extraction unit is used to splice the multiple monitoring video streams according to the positional relationship of the multiple monitoring areas to obtain a spliced ​​video stream; filter the three-dimensional point cloud data of the spliced ​​video stream to obtain denoised three-dimensional point cloud data; extract the road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data and obstacle feature data; The state prediction update result determination unit is used to predict and update the state of the target object and the state of the target behavior using a Kalman filter according to the target object feature data and the target behavior feature data to obtain a state prediction update result; The abnormal event identification and early warning unit is used to perform AI analysis on the multiple monitoring video streams based on the state prediction update result to identify abnormal events on the highway and issue early warnings.

Images