An intelligent monitoring method and system for expressways based on video AI

Abstract

The present invention discloses a method and system for intelligent monitoring of expressways based on video AI, which relates to the technical field of expressway monitoring. The method includes: using a plurality of video acquisition devices to collect a plurality of monitoring video streams of a plurality of monitoring areas on the expressway in real time; splicing the plurality of monitoring video streams according to the positional relationship of the plurality of monitoring areas to obtain a spliced video stream; performing filtering processing on the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data; extracting road surface dynamic feature data from the denoised three-dimensional point cloud data; using a Kalman filter to predict and update the state of the target object and the state of the target behavior according to the target object feature data and the target behavior feature data to obtain a state prediction and update result; based on the state prediction and update result, performing AI analysis on the plurality of monitoring video streams to identify abnormal events on the expressway and give early warnings.

Description

Technical Field

[0001] This application relates to the technical field of highway monitoring, and more specifically, to an intelligent highway monitoring method and system based on video AI. Background Art

[0002] With the rapid development of society and the acceleration of the urbanization process, the traffic flow on highways is constantly increasing, and the real-time monitoring and management of highway traffic conditions have become increasingly important. As an important hub of urban traffic, the smoothness of highways is directly related to the economic development of cities and the daily travel of residents. Highways adapt to the development of industrialization and urbanization. Cities are places where industries and populations gather, and the growth of cars in cities is much faster than that in rural areas, becoming a center for car aggregation. Highway construction often starts from urban ring roads, radial roads, and busy traffic sections, and gradually becomes an urban traffic backbone based on highways.

[0003] Although existing highway monitoring systems have achieved full-process video information transmission, enabling traffic supervision departments to obtain real-time highway traffic conditions, these systems often only provide basic video information and lack the ability to deeply analyze and predict video information. This means that when abnormal conditions occur on the highway, traffic supervision departments usually cannot quickly understand the actual situation and make effective emergency responses.

[0004] For the above problems, no effective solutions have been proposed yet. Summary of the Invention

[0005] Embodiments of this application provide an intelligent highway monitoring method and system based on video AI to solve the above technical problems.

[0006] This application provides an intelligent highway monitoring method based on video AI, including:

[0007] Using a plurality of video acquisition devices to collect a plurality of monitoring video streams of a plurality of monitoring areas on the highway in real time; wherein, the plurality of video acquisition devices include time-of-flight cameras and structured light cameras;

[0008] According to the positional relationship of the plurality of monitoring areas, splicing the plurality of monitoring video streams to obtain a spliced video stream;

[0009] Performing filtering processing on the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data;

[0010] Extracting road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein, the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data, and obstacle feature data;

[0011] According to the target object feature data and the target behavior feature data, use a Kalman filter to predict and update the state of the target object and the state of the target behavior, and obtain a state prediction and update result;

[0012] Based on the state prediction and update result, perform AI analysis on the multiple monitoring video streams to identify highway abnormal events and give early warnings.

[0013] Further, according to the positional relationship of the multiple monitoring areas, splice the multiple monitoring video streams to obtain a spliced video stream; perform filtering processing on the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data, including:

[0014] Perform distortion correction on the multiple monitoring video streams to obtain multiple corrected video streams;

[0015] According to the positional relationship of the multiple monitoring areas, determine the overlapping areas and overlapping relationships of the multiple corrected video streams;

[0016] Based on the SLIC algorithm, perform superpixel segmentation on the multiple corrected video streams according to the overlapping areas and the overlapping relationships to obtain the contrast information of each pixel point in the superpixels;

[0017] Splice the multiple corrected video streams according to the contrast information of each pixel point in the superpixels to obtain the spliced video stream;

[0018] Convert the three-dimensional point cloud data of the spliced video stream into a three-dimensional point cloud matrix;

[0019] Use a Laplace filter operator to perform filtering processing on the three-dimensional point cloud matrix to obtain denoised three-dimensional point cloud data.

[0020] Further, extracting the road surface dynamic feature data from the denoised three-dimensional point cloud data includes:

[0021] Use a ground plane filtering algorithm to separate the ground point cloud data and the non-ground point cloud data from the denoised three-dimensional point cloud data;

[0022] Calculate the geometric features of each point in the ground point cloud data; wherein, the geometric features include normal vectors and curvatures;

[0023] Select multiple seed points as the starting points of region growing within the road surface area of the highway;

[0024] Define region growing rules; wherein, the region growing rules include a normal vector angle threshold and a curvature threshold;

[0025] Starting from the multiple seed points, diffusing around according to the region growing rule, and adding the points that meet the region growing rule to the region;

[0026] Extracting road surface static feature data based on the region obtained by growth;

[0027] Based on the road surface static feature data, calculating and processing the depth values of each pixel point in the non-ground point cloud data to extract the target object feature data and the target behavior feature data.

[0028] Further, according to the target object feature data and the target behavior feature data, using a Kalman filter to predict and update the state of the target object and the state of the target behavior, obtaining a state prediction update result; based on the state prediction update result, performing AI analysis on the multiple monitoring video streams to identify highway abnormal events and give early warnings, including:

[0029] Setting a first initial parameter for the state of the target object and a second initial parameter for the state of the target behavior; wherein, the first initial parameter includes a first initial state value, a first initial covariance matrix, and a first constraint condition; the second initial parameter includes a second initial state value, a second initial covariance matrix, and a second constraint condition;

[0030] According to the first initial parameter of the state of the target object and the second initial parameter of the state of the target behavior, calculating a prior estimated state and a Kalman gain matrix;

[0031] Based on the prior estimated state and the Kalman gain matrix, calculating a corrected estimated state;

[0032] According to the corrected estimated state, predicting and updating the state of the target object and the state of the target behavior, obtaining the state prediction update result;

[0033] Based on a gated recurrent unit, identifying the state prediction update result to obtain a target state factor;

[0034] If the target state factor belongs to a preset state interval, it is determined that the highway abnormal event is identified and an early warning prompt is given.

[0035] Further, the highway abnormal events include vehicle speeding events, illegal parking events, overtime traffic jam events, reverse driving events, spillage events, car crash events, and vehicle out-of-control events; ]>

[0036] The vehicle feature data includes vehicle flow feature data, vehicle speed feature data, and vehicle type feature data.

[0037] Further, the normal angle threshold is used to determine whether points are on the same plane, and the curvature threshold is used to determine whether points are within the same curvature change range.

[0038] This application provides an intelligent highway monitoring system based on video AI, including a monitoring video stream acquisition unit, a road surface dynamic feature data extraction unit, a state prediction update result determination unit, and an abnormal event recognition and warning unit; wherein,

[0039] The monitoring video stream acquisition unit is used to use a plurality of video acquisition devices to collect a plurality of monitoring video streams of a plurality of monitoring areas on the highway in real time; wherein, the plurality of video acquisition devices include a time-of-flight camera and a structured light camera;

[0040] The road surface dynamic feature data extraction unit is used to splice the plurality of monitoring video streams according to the positional relationship of the plurality of monitoring areas to obtain a spliced video stream; perform filtering processing on the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data; extract road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein, the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data, and obstacle feature data;

[0041] The state prediction update result determination unit is used to predict and update the state of the target object and the state of the target behavior using a Kalman filter according to the target object feature data and the target behavior feature data to obtain a state prediction update result;

[0042] The abnormal event recognition and warning unit is used to perform AI analysis on the plurality of monitoring video streams based on the state prediction update result to identify and warn of highway abnormal events.

[0043] Based on the technical solution of the present application, multiple monitoring video streams of multiple monitoring areas on a highway are collected in real time by using multiple video acquisition devices; wherein, the multiple video acquisition devices include a time-of-flight camera and a structured light camera; according to the positional relationship of the multiple monitoring areas, the multiple monitoring video streams are spliced to obtain a spliced video stream; the three-dimensional point cloud data of the spliced video stream is filtered to obtain denoised three-dimensional point cloud data; road surface dynamic feature data is extracted from the denoised three-dimensional point cloud data; wherein, the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data, and obstacle feature data; according to the target object feature data and the target behavior feature data, a Kalman filter is used to predict and update the state of the target object and the state of the target behavior to obtain a state prediction and update result; based on the state prediction and update result, AI analysis is performed on the multiple monitoring video streams to identify highway abnormal events and give early warnings. Thus, automatic monitoring, automatic judgment, and automatic early warning of highway abnormal events based on video AI are realized, the risk management efficiency and risk response efficiency of highway management personnel are improved, and all-weather and full-section highway traffic order monitoring and management are realized. Brief Description of the Drawings

[0044] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, form a part of this application, and the illustrative embodiments and descriptions thereof are used to explain this application and do not constitute an improper limitation of this application. In the drawings:

[0045] Figure 1 is a flowchart of an optional method for intelligent monitoring of a highway based on video AI according to an embodiment of the present application;

[0046] Figure 2 is a structural diagram of an optional intelligent monitoring system for a highway based on video AI according to an embodiment of the present application.

[0047] The realization of the purpose of the present invention, functional features, and advantages will be further described with reference to the embodiments and the drawings. Detailed Embodiments

[0048] In order to enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the drawings in the embodiments of this application.

[0049] Optionally, as Figure 1 shown, this application provides a method for intelligent monitoring of a highway based on video AI, including:

[0050] S101. Use multiple video acquisition devices to collect multiple surveillance video streams of multiple surveillance areas on the highway in real time. Among them, the multiple video acquisition devices include a time-of-flight camera and a structured light camera.

[0051] S102. Stitch the multiple surveillance video streams according to the positional relationship of the multiple surveillance areas to obtain a stitched video stream.

[0052] S103. Perform filtering processing on the three-dimensional point cloud data of the stitched video stream to obtain denoised three-dimensional point cloud data.

[0053] S104. Extract road surface dynamic feature data from the denoised three-dimensional point cloud data. Among them, the road surface dynamic feature data includes target object feature data and target behavior feature data. The target object feature data includes vehicle feature data, pedestrian feature data, and obstacle feature data.

[0054] S105. According to the target object feature data and the target behavior feature data, use a Kalman filter to predict and update the state of the target object and the state of the target behavior to obtain a state prediction and update result.

[0055] S106. Based on the state prediction and update result, perform AI analysis on the multiple surveillance video streams to identify highway abnormal events and issue early warnings.

[0056] Based on the embodiments provided in this application, use multiple video acquisition devices to collect multiple surveillance video streams of multiple surveillance areas on the highway in real time. Among them, the multiple video acquisition devices include a time-of-flight camera and a structured light camera. Stitch the multiple surveillance video streams according to the positional relationship of the multiple surveillance areas to obtain a stitched video stream. Perform filtering processing on the three-dimensional point cloud data of the stitched video stream to obtain denoised three-dimensional point cloud data. Extract road surface dynamic feature data from the denoised three-dimensional point cloud data. Among them, the road surface dynamic feature data includes target object feature data and target behavior feature data. The target object feature data includes vehicle feature data, pedestrian feature data, and obstacle feature data. According to the target object feature data and the target behavior feature data, use a Kalman filter to predict and update the state of the target object and the state of the target behavior to obtain a state prediction and update result. Based on the state prediction and update result, perform AI analysis on the multiple surveillance video streams to identify highway abnormal events and issue early warnings. Thus, it realizes the automatic monitoring, automatic judgment, and automatic early warning of highway abnormal events based on video AI, improves the risk management level and risk response efficiency of the highway operation management team, and realizes the all-weather and full-section monitoring and management of highway traffic order.

[0057] Further, according to the positional relationship of multiple monitoring areas, splice multiple monitoring video streams to obtain a spliced video stream; perform filtering processing on the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data, including:

[0058] Perform distortion correction on multiple monitoring video streams to obtain multiple corrected video streams;

[0059] According to the positional relationship of multiple monitoring areas, determine the overlapping areas and overlapping relationships of multiple corrected video streams;

[0060] Based on the SLIC algorithm, perform superpixel segmentation on multiple corrected video streams according to the overlapping areas and overlapping relationships to obtain the contrast information of each pixel point in the superpixels;

[0061] Splice multiple corrected video streams according to the contrast information of each pixel point in the superpixels to obtain a spliced video stream;

[0062] Convert the three-dimensional point cloud data of the spliced video stream into a three-dimensional point cloud matrix;

[0063] Use a Laplacian filter operator to perform filtering processing on the three-dimensional point cloud matrix to obtain denoised three-dimensional point cloud data.

[0064] Further, extract road surface dynamic feature data from the denoised three-dimensional point cloud data, including:

[0065] Use a ground plane filtering algorithm to separate ground point cloud data and non-ground point cloud data from the denoised three-dimensional point cloud data;

[0066] Calculate the geometric features of each point in the ground point cloud data; among them, the geometric features include normal vectors and curvatures;

[0067] Select multiple seed points within the road surface area of the highway as the starting points for region growing;

[0068] Define region growing rules; among them, the region growing rules include a normal vector angle threshold and a curvature threshold;

[0069] Starting from multiple seed points, spread around according to the region growing rules, and add the points that meet the region growing rules to the region;

[0070] Extract road surface static feature data based on the grown regions;

[0071] Based on the road surface static feature data, perform calculation processing on the depth values of each pixel point in the non-ground point cloud data to extract target object feature data and target behavior feature data.

[0072] Further, based on the target object feature data and the target behavior feature data, use the Kalman filter to predict and update the state of the target object and the state of the target behavior, and obtain the state prediction update result; based on the state prediction update result, perform AI analysis on multiple monitoring video streams to identify highway abnormal events and give early warnings, including:

[0073] Set the first initial parameter of the state of the target object and the second initial parameter of the state of the target behavior; wherein, the first initial parameter includes the first initial state value, the first initial covariance matrix, and the first constraint condition; the second initial parameter includes the second initial state value, the second initial covariance matrix, and the second constraint condition;

[0074] According to the first initial parameter of the state of the target object and the second initial parameter of the state of the target behavior, calculate the prior estimated state and the Kalman gain matrix;

[0075]

[0076] Among them, M Kal represents the Kalman gain matrix; M Cov1 represents the first initial covariance matrix; M Cov2 represents the second initial covariance matrix; M prev_cov is the non-deterministic matrix of the prior estimated state; M prev_cov T is the transpose of the non-deterministic matrix of the prior estimated state; M Q is the process noise covariance matrix;

[0077] Based on the prior estimated state and the Kalman gain matrix, calculate the corrected estimated state;

[0078] According to the corrected estimated state, predict and update the state of the target object and the state of the target behavior, and obtain the state prediction update result;

[0079] Based on the gated recurrent unit, identify the state prediction update result to obtain the target state factor;

[0080] If the target state factor belongs to the preset state interval, it is determined that a highway abnormal event is identified and a warning prompt is given.

[0081] Further, M prev_cov = M prev_sta × P cov × M prev_sta T + M Q

[0082] M prev_cov is the non-deterministic matrix of the prior estimated state; M prev_stais the state transition matrix of the prior estimated state; M prev_sta T is the transpose of the state transition matrix of the prior estimated state; P cov is the estimated covariance matrix of the corrected estimated state; M Q is the process noise covariance matrix.

[0083] Further, the gated recurrent unit includes an update gate Z t which is obtained based on the following formula:

[0084] Z t = σ[W z ×(X t , H t-1 ) + b z

[0085] where σ is the sigmoid activation function; W z is the weight matrix of the update gate Z t ; X t is the state prediction update result input at the current time step; H t-1 is the hidden state at the previous time step; b z is the bias term of the update gate Z t .

[0086] Further, highway abnormal events include vehicle speeding events, illegal parking events, overtime traffic jam events, reverse driving events, spillage events, car crash events, and vehicle out-of-control events;

[0087] Vehicle feature data includes vehicle flow feature data, vehicle speed feature data, and vehicle type feature data.

[0088] Further, the normal angle threshold is used to determine whether points are on the same plane, and the curvature threshold is used to determine whether points are within the same curvature change range.

[0089] Optionally, as Figure 2 shown, the present application provides a highway intelligent monitoring system based on video AI, including a monitoring video stream acquisition unit 201, a road surface dynamic feature data extraction unit 202, a state prediction update result determination unit 203, and an abnormal event identification and warning unit 204; wherein,

[0090] The monitoring video stream acquisition unit 201 is configured to use a plurality of video acquisition devices to collect a plurality of monitoring video streams of a plurality of monitoring areas on the highway in real time; wherein, the plurality of video acquisition devices include a time-of-flight camera and a structured light camera;

[0091] ​The road surface dynamic feature data extraction unit 202 is configured to splice multiple monitoring video streams according to the positional relationships of multiple monitoring areas to obtain a spliced video stream; filter the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data; extract road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein, the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data, and obstacle feature data;

[0092] The state prediction and update result determination unit 203 is configured to predict and update the states of the target object and the target behavior using a Kalman filter according to the target object feature data and the target behavior feature data to obtain a state prediction and update result;

[0093] The abnormal event recognition and warning unit 204 is configured to perform AI analysis on multiple monitoring video streams based on the state prediction and update result to identify and warn of highway abnormal events.

[0094] Further, the road surface dynamic feature data extraction unit is configured to splice multiple monitoring video streams according to the positional relationships of multiple monitoring areas to obtain a spliced video stream; filter the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data, and is configured as follows:

[0095] Perform distortion correction on multiple monitoring video streams to obtain multiple corrected video streams;

[0096] Determine the overlapping areas and overlapping relationships of multiple corrected video streams according to the positional relationships of multiple monitoring areas;

[0097] Based on the SLIC algorithm, perform superpixel segmentation on multiple corrected video streams according to the overlapping areas and overlapping relationships to obtain the contrast information of each pixel point in the superpixels;

[0098] Splice multiple corrected video streams according to the contrast information of each pixel point in the superpixels to obtain a spliced video stream;

[0099] Convert the three-dimensional point cloud data of the spliced video stream into a three-dimensional point cloud matrix;

[0100] Use a Laplacian filter operator to filter the three-dimensional point cloud matrix to obtain denoised three-dimensional point cloud data.

[0101] It should be noted that in this application, the embodiments implemented on the side of the highway intelligent monitoring system based on video AI can be mutually referred to the embodiments implemented on the side of the highway intelligent monitoring method based on video AI, and this application will not elaborate one by one.

[0102] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be similarly included in the patent protection scope of the present invention.

Claims

1. An intelligent monitoring method for highways based on video AI, characterized in that, Including: Using multiple video acquisition devices to collect multiple surveillance video streams of multiple surveillance areas on the highway in real time; wherein, the multiple video acquisition devices include a time-of-flight camera and a structured light camera; According to the positional relationship of the multiple surveillance areas, splicing the multiple surveillance video streams to obtain a spliced video stream; Performing filtering processing on the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data; Extracting road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein, the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data, and obstacle feature data; According to the target object feature data and the target behavior feature data, using a Kalman filter to predict and update the state of the target object and the state of the target behavior to obtain a state prediction update result; Based on the state prediction update result, performing AI analysis on the multiple surveillance video streams to identify highway abnormal events and give early warnings; The extracting the road surface dynamic feature data from the denoised three-dimensional point cloud data includes: Using a ground plane filtering algorithm to separate ground point cloud data and non-ground point cloud data from the denoised three-dimensional point cloud data; Calculating the geometric features of each point in the ground point cloud data; wherein, the geometric features include normal vectors and curvatures; Selecting multiple seed points within the road surface area of the highway as the starting points for region growing; The extracting the road surface dynamic feature data from the denoised three-dimensional point cloud data includes: Defining region growing rules; wherein, the region growing rules include a normal vector angle threshold and a curvature threshold; Starting from the multiple seed points, spreading around according to the region growing rules, and adding the points that meet the region growing rules to the region; Extracting road surface static feature data based on the grown region; Based on the road surface static feature data, performing calculation processing on the depth values of each pixel point in the non-ground point cloud data to extract the target object feature data and the target behavior feature data.

2. The intelligent highway monitoring method based on video AI according to claim 1, characterized in that, The splicing the multiple surveillance video streams according to the positional relationship of the multiple surveillance areas to obtain a spliced video stream; The performing filtering processing on the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data includes: Performing distortion correction on the multiple surveillance video streams to obtain multiple corrected video streams; According to the positional relationship of the multiple surveillance areas, determining the overlapping areas and overlapping relationships of the multiple corrected video streams; Based on the SLIC algorithm, performing superpixel segmentation on the multiple corrected video streams according to the overlapping areas and the overlapping relationships to obtain the contrast information of each pixel point in the superpixels.

3. The method for intelligent monitoring of highways based on video AI according to claim 2, wherein, The splicing the multiple surveillance video streams according to the positional relationship of the multiple surveillance areas to obtain a spliced video stream; The performing filtering processing on the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data includes: Stitch the multiple corrected video streams according to the contrast information of each pixel point in the superpixel to obtain the stitched video stream; Convert the three-dimensional point cloud data of the stitched video stream into a three-dimensional point cloud matrix; Perform filtering processing on the three-dimensional point cloud matrix using a Laplacian filter operator to obtain denoised three-dimensional point cloud data.

4. The intelligent highway monitoring method based on video AI according to claim 1, characterized in that, According to the target object feature data and the target behavior feature data, use a Kalman filter to predict and update the state of the target object and the state of the target behavior to obtain a state prediction update result; Based on the state prediction update result, perform AI analysis on the multiple monitoring video streams to identify highway abnormal events and give early warnings, including: Set a first initial parameter for the state of the target object and a second initial parameter for the state of the target behavior; wherein, the first initial parameter includes a first initial state value, a first initial covariance matrix, and a first constraint condition; the second initial parameter includes a second initial state value, a second initial covariance matrix, and a second constraint condition; Calculate a prior estimated state and a Kalman gain matrix according to the first initial parameter of the state of the target object and the second initial parameter of the state of the target behavior.

5. The method for intelligent monitoring of expressways based on video AI according to claim 4, characterized in that, According to the target object feature data and the target behavior feature data, use a Kalman filter to predict and update the state of the target object and the state of the target behavior to obtain a state prediction update result; Based on the state prediction update result, perform AI analysis on the multiple monitoring video streams to identify highway abnormal events and give early warnings, including: Calculate a corrected estimated state based on the prior estimated state and the Kalman gain matrix; Predict and update the state of the target object and the state of the target behavior according to the corrected estimated state to obtain the state prediction update result; Identify the state prediction update result based on a gated recurrent unit to obtain a target state factor; If the target state factor belongs to a preset state interval, it is determined that the highway abnormal event is identified and a warning prompt is given.

6. The method for intelligent highway monitoring based on video AI according to claim 1, wherein The highway abnormal events include vehicle speeding events, illegal parking events, overtime traffic jam events, reverse driving events, spillage events, car crash events, and vehicle out-of-control events; The vehicle feature data includes vehicle flow feature data, vehicle speed feature data, and vehicle type feature data.

7. The method for intelligent highway monitoring based on video AI according to claim 1, wherein The normal angle threshold is used to determine whether points are on the same plane, and the curvature threshold is used to determine whether points are within the same curvature change range.

8. An intelligent highway monitoring system based on video AI, which implements the method described in claim 1, characterized in that, It includes a monitoring video stream acquisition unit, a road surface dynamic feature data extraction unit, a state prediction update result determination unit, and an abnormal event identification and warning unit; wherein The monitoring video stream acquisition unit is used to collect multiple monitoring video streams of multiple monitoring areas on the highway in real time by using multiple video acquisition devices; wherein, the multiple video acquisition devices include a time-of-flight camera and a structured light camera; The road surface dynamic feature data extraction unit is used to splice the multiple monitoring video streams according to the positional relationship of the multiple monitoring areas to obtain a spliced video stream; filter the three-dimensional point cloud data of the spliced video stream to obtain denoised three-dimensional point cloud data; extract road surface dynamic feature data from the denoised three-dimensional point cloud data; wherein, the road surface dynamic feature data includes target object feature data and target behavior feature data; the target object feature data includes vehicle feature data, pedestrian feature data, and obstacle feature data; The state prediction and update result determination unit is used to predict and update the state of the target object and the state of the target behavior by using a Kalman filter according to the target object feature data and the target behavior feature data to obtain a state prediction and update result; The abnormal event recognition and early warning unit is used to perform AI analysis on the multiple monitoring video streams based on the state prediction and update result to identify and early warn of highway abnormal events.

Images