A radar and video information fusion coding method for vehicle overload warning

Abstract

The present invention discloses a radar and video information fusion coding method for vehicle overload warning, the steps include: obtaining three-dimensional point cloud data through a radar module; measuring the length, width and height of the target vehicle in real time; obtaining vehicle image data using a video information acquisition module, the video data identifies the license plate number and vehicle appearance features through an improved YOLO deep learning algorithm, and assists in judging whether the vehicle is overloaded; the three-dimensional dimensions of the vehicle measured by the radar are compared with the standard dimensions in the database through a data processing module, and a threshold method is used to judge whether the vehicle is overloaded, and the radar data and the video data are effectively fused through an algorithm, and the fused data is compressed. The present invention utilizes radar and video information fusion technology, combined with an efficient coding scheme, to achieve a high-precision and high-efficiency vehicle overload warning function, and adapt to complex road environments and changeable traffic scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of video coding and information fusion, and in particular to a radar and video information fusion coding method for vehicle overload warning. Background Art

[0002] With the rapid development of intelligent transportation systems (ITS) and road safety monitoring technology, vehicle overload detection has become an important task in the field of traffic management. Overloaded vehicles (such as over-height, over-width, over-length, etc.) not only cause damage to road facilities, but may also endanger public safety. Therefore, accurate and timely detection and warning of overloaded vehicles are of great significance to maintaining road safety and reducing traffic accidents.

[0003] Most existing overload detection systems rely on a single technology, such as radar detection technology or video surveillance technology. Radar technology mainly determines whether a target vehicle is overloaded by measuring the distance, speed and three-dimensional dimensions (such as height, width and length) of the target vehicle, but the accuracy and detection ability of radar are greatly affected by environmental interference, especially in complex road scenes, and its accuracy is low. On the other hand, video surveillance technology uses image recognition to perform license plate recognition and appearance feature extraction. Although it is more sophisticated in visual effects, the accuracy of vehicle recognition will be greatly reduced in complex backgrounds or low light conditions. Summary of the invention

[0004] In view of the above-mentioned technical deficiencies, the present invention proposes a radar and video information fusion coding method, which aims to solve the problems of low data processing efficiency, poor real-time performance, high bandwidth requirement, etc. in the existing vehicle overload detection system through efficient data fusion and coding technology.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] The present invention provides a radar and video information fusion coding method for vehicle overload warning, comprising the following steps:

[0007] Step 1: Scan the vehicle in real time through the radar module to obtain the three-dimensional size data of the vehicle;

[0008] Step 2: Use the visible light camera module to capture vehicle images and extract license plate information by improving YOLO's deep learning license plate recognition algorithm;

[0009] Step 3, the data processing module retrieves the standard size in the database according to the license plate number, and calculates the difference between the actual size of the vehicle and the standard size;

[0010] Step 4: By comparing the vehicle size difference values ​​with the threshold, determine whether the vehicle exceeds the limit;

[0011] Step 5. If the vehicle is determined to be over-limit, an alarm is triggered and the vehicle information is recorded, including the license plate number, actual three-dimensional dimensions, over-limit dimensions and magnitude, and detection time and location;

[0012] Step 6: The radar data and video data are effectively fused through the adaptive registration algorithm and weighted fusion algorithm, and the fused data is encoded using H.265 / HEVC video coding.

[0013] Furthermore, the radar module in Step 1 specifically includes the following contents:

[0014] (1) LiDAR data collection: Use a drone equipped with a mechanical LiDAR to fly around the vehicle under test, and the LiDAR scans to obtain point cloud data of the side and rear of the vehicle;

[0015] (2) Data format conversion: The original data format is the point cloud data obtained by laser radar scanning in PCAP format. The conversion tool is in the Ubuntu system, using the relevant instructions in the robot operating system, using the rosbag command to convert the PCAP file to a bag file, and then using the bag_to_pcd command to convert the bag file to a PCD file. The PCD file supports the n-dimensional type extension mechanism and can store and process ordered point cloud data, which is convenient for subsequent point cloud processing;

[0016] (3) Point cloud data preprocessing: Use the conditional filter in the point cloud library to set the conditional thresholds of multiple fields, remove obvious outliers that are far away from the target vehicle point cloud, calculate the average distance from each point to all its k-neighborhood neighboring points, set the distance threshold, and remove outliers. Set the neighborhood radius threshold (Eps) and the minimum number of points to form a cluster (MinPts), cluster the point cloud data, and obtain complete vehicle point cloud data;

[0017] (4) Polar coordinates to Cartesian coordinates: The point cloud data obtained by LiDAR scanning is converted from polar coordinates to Cartesian coordinates. The conversion formula is as follows:

[0018] ;

[0019] Among them, r is the distance between the measured point and the center of the laser radar, θ is the horizontal rotation angle, is the vertical angle;

[0020] (5) Principal component analysis (PCA): Calculate the centroid m of the point cloud data using the following formula:

[0021] ;

[0022] in, is the point in the point cloud dataset, and n is the number of points;

[0023] Decentralize the n points in the point cloud set and construct the covariance matrix C. The formula is as follows:

[0024] ;

[0025] Solve the eigenvalue λ and the corresponding eigenvector v of the covariance matrix; orthogonally normalize the eigenvector to obtain the orthogonal matrix A; transform the point cloud data to the vicinity of the origin of the coordinate system, and the three main directions of the point cloud data are the directions of the new coordinate system;

[0026] (6) Minimum bounding box construction: Call the pcl::getMinMax3D function in the PCL library to traverse all the point cloud data in the minimum bounding box to obtain the coordinates of the eight vertices of the bounding box. Based on the coordinates of the bounding box vertices, calculate the length L, width W, and height H of the vehicle. The formula is as follows:

[0027] ;

[0028] Furthermore, the license plate recognition algorithm of the visible light camera module in Step 2 adopts an improved YOLO deep learning model, which specifically includes the following contents:

[0029] (1) Add a multi-scale feature fusion module: Add a multi-scale feature fusion module to the network, use high-resolution feature maps to enhance the detail features of the small license plate area, and fuse the context information in the low-resolution feature map to improve the detection ability of the small license plate area. In the implementation process, the detection model with FPN (feature pyramid network) is trained through the PyTorch or ONNX framework, combined with the video stream real-time processing module, to support multi-resolution input;

[0030] (2) Introducing the attention mechanism: Use the channel attention mechanism (such as the SE module) and the spatial attention mechanism (such as the CBAM module) to optimize the weight distribution of the feature map and suppress the influence of irrelevant areas in the complex background. For example, for the license plate detection task in the video stream, the attention mechanism will strengthen the features of the license plate area and filter the background through real-time point cloud data; call the visualization function of the detection result and output the features after attention optimization as a highlighted rectangular box of the license plate area;

[0031] (3) Use pre-trained models for transfer learning: Use ImageNet or Common Objects in Context pre-trained weights to adapt to license plate styles from different countries through transfer learning, including different fonts, colors, and character arrangements; transfer learning can maintain a recognition accuracy of more than 99% in complex environments (such as uneven lighting and blurred images) by fine-tuning the weights of high-level detection heads;

[0032] (4) Thread-based real-time detection and result drawing: Implement a multi-threaded license plate detection framework, manage the detection thread through the QThread module of PyQt5, and ensure efficient parallel execution of license plate detection and drawing tasks: QThreadModelPredict handles the detection thread, including reading frame data from the video stream, calling the detection model for inference, and drawing the detection results;

[0033] The signal mechanism ensures that the UI is updated immediately after the detection is completed to display the detection results, including the license plate number, license plate confidence, and positioning frame coordinates;

[0034] Use the cnn_predict_image_drawn_sig signal to present the drawn detection results in real time through the UI module;

[0035] (5) Optimize video streaming and real-time interaction: Enhance compatibility with multiple image sources, including image files, video streams, and real-time camera input. In video mode, use the cross-platform computer vision library VideoCapture to manage frame capture. Call the detection algorithm in a multi-threaded environment through cnn_infer_thread to achieve real-time dynamic update of detection results, including license plate number, license plate color, and detection confidence. Call show_infer_res and show_image functions to display the detection results to ensure that the detection frame accurately corresponds to the license plate information.

[0036] (6) Data accuracy verification and dynamic threshold adjustment: Supports dynamic adjustment of confidence and IoU (Intersection over Union) thresholds to adapt to detection requirements in different environments.

[0037] Furthermore, the data processing module in Step 4 is combined with a dynamic threshold adjustment algorithm to optimize the over-limit judgment. The size difference and threshold adjustment formula are as follows:

[0038] ;

[0039] ;

[0040] in, is the standard size, v is the vehicle speed, and To adjust the parameters, it is used to balance the errors of static size and dynamic size detection.

[0041] Furthermore, in Step 6, the radar data and the video data are effectively fused by using an adaptive registration algorithm and a weighted fusion algorithm, and the fused data is encoded by using H.265 / HEVC video coding:

[0042] (1) Data preprocessing: The data obtained by the radar is processed by denoising and filtering to ensure the accuracy of its three-dimensional measurement. The video data is processed by image enhancement, denoising and other technologies to improve image quality, especially visibility in low-light environments or complex backgrounds. The denoising and filtering formulas are as follows:

[0043] ;

[0044] ;

[0045] in It is a low-pass filter to remove high-frequency noise in radar data; It is a Gaussian filtering operation, which removes noise from the image and enhances the image quality.

[0046] (2) Adaptive registration algorithm: After data preprocessing is completed, the adaptive registration algorithm is used to spatially align radar data and video data. Adaptive registration automatically adjusts the spatial relationship between different data sources to ensure that radar data and video data are fused in the same coordinate system, avoiding errors caused by factors such as acquisition angle and viewing distance differences;

[0047] (3) For key tasks such as license plate recognition, video data may provide more accurate information, while radar data may have more advantages in terms of the accuracy of the vehicle's three-dimensional dimensions. The weighted fusion algorithm will take these factors into account and combine the information from both to generate the final fused data. The weighted formula is:

[0048] ;

[0049] in is the fusion weight coefficient, ∈[0,1], radar data can be given a higher weight, and video data is weighted according to image quality and license plate recognition results;

[0050] (4) Fusion data encoding: After the fusion data is generated, the fusion data is compressed and encoded using H.265 / HEVC video encoding. The encoding formula is:

[0051] ;

[0052] in For H.265 / HEVC encoder, is the fused data;

[0053] After encoding, the integrated data is easy to store and transmit, and can be transmitted to the monitoring platform in real time via the network for further vehicle overload analysis and early warning.

[0054] The beneficial effects of the present invention are as follows: the present invention accurately fuses radar and video data through an adaptive registration algorithm and a weighted fusion algorithm, and efficiently compresses the fused data in combination with H.265 / HEVC video coding technology, thereby significantly reducing the data transmission bandwidth requirement, and being able to efficiently and in real time complete overload vehicle detection and warning tasks in various complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 It is a schematic diagram of the process of the present invention;

[0057] Figure 2 This is a schematic diagram of the process of measuring the length, width and height of a vehicle by a radar module in an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of the process of recognizing license plate numbers using the improved YOLO deep learning algorithm used in an embodiment of the present invention;

[0059] Figure 4 The figure is a schematic diagram of the process of information fusion and encoding of radar data and video data in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Example 1

[0062] like Figure 1 As shown, a radar and video information fusion coding method for vehicle overload warning includes the following steps:

[0063] Step 1: Scan the vehicle in real time through the radar module to obtain the three-dimensional size data of the vehicle;

[0064] Step 2: Use the visible light camera module to capture vehicle images and extract license plate information by improving YOLO's deep learning license plate recognition algorithm;

[0065] Step 3, the data processing module retrieves the standard size in the database according to the license plate number, and calculates the difference between the actual size of the vehicle and the standard size;

[0066] Step 4: By comparing the vehicle size difference values ​​with the threshold, determine whether the vehicle exceeds the limit;

[0067] Step 5. If the vehicle is determined to be over-limit, an alarm is triggered and the vehicle information is recorded, including the license plate number, actual three-dimensional dimensions, over-limit dimensions and magnitude, and detection time and location;

[0068] Step 6: The radar data and video data are effectively fused through the adaptive registration algorithm and weighted fusion algorithm, and the fused data is encoded using H.265 / HEVC video coding.

[0069] Furthermore, if Figure 2 As shown, the process of measuring the length, width and height of the vehicle by the radar module in Step 1 specifically includes the following contents:

[0070] (1) LiDAR data acquisition: A YHY62 quad-rotor drone equipped with a 16-line mechanical LiDAR (WLR-720) was used. The flight altitude was set to 6 meters and the flight speeds were set to 0 m / s, 2.5 m / s, and 5 m / s. The drone flew around the vehicle under test and the LiDAR scanned the side and tail point cloud data of the vehicle.

[0071] (2) Data format conversion: The original data format is the point cloud data obtained by laser radar scanning in PCAP format. The conversion tool is in the Ubuntu system, using the relevant instructions in the robot operating system, using the rosbag command to convert the PCAP file to a bag file, and then using the bag_to_pcd command to convert the bag file to a PCD file. The PCD file supports the n-dimensional type extension mechanism and can store and process ordered point cloud data, which is convenient for subsequent point cloud processing;

[0072] (3) Point cloud data preprocessing: Use the conditional filter in the point cloud library to set conditional thresholds for multiple fields, remove obvious outliers that are far away from the target vehicle point cloud, calculate the average distance from each point to all its k-neighboring points, set the distance threshold, remove outliers, set the neighborhood radius threshold (Eps) and the minimum number of points to form a cluster (MinPts), cluster the point cloud data, and obtain complete vehicle point cloud data;

[0073] (4) Polar coordinates to Cartesian coordinates: The point cloud data obtained by LiDAR scanning is converted from polar coordinates to Cartesian coordinates. The conversion formula is as follows:

[0074] ;

[0075] Among them, r is the distance between the measured point and the center of the laser radar, θ is the horizontal rotation angle, is the vertical angle;

[0076] (5) Principal component analysis (PCA): Calculate the centroid m of the point cloud data using the following formula:

[0077] ;

[0078] in, is the point in the point cloud dataset, and n is the number of points;

[0079] Decentralize the n points in the point cloud set and construct the covariance matrix C. The formula is as follows:

[0080] ;

[0081] Solve the eigenvalue λ and the corresponding eigenvector v of the covariance matrix; orthogonally normalize the eigenvector to obtain the orthogonal matrix A; transform the point cloud data to the vicinity of the origin of the coordinate system, and the three main directions of the point cloud data are the directions of the new coordinate system;

[0082] (6) Minimum bounding box construction: Call the pcl::getMinMax3D function in the PCL library to traverse all the point cloud data in the minimum bounding box to obtain the coordinates of the eight vertices of the bounding box. Based on the coordinates of the bounding box vertices, calculate the length L, width W, and height H of the vehicle. The formula is as follows:

[0083]

[0084] Furthermore, the process of the visible light camera module recognizing the license plate in Step 2 is as follows: Figure 3 As shown in the figure, the algorithm adopts the improved YOLO deep learning model, which specifically includes the following contents:

[0085] (1) Adding a multi-scale feature fusion module: Add a multi-scale feature fusion module to the network, use high-resolution feature maps to enhance the detail features of the small license plate area, and fuse the context information in the low-resolution feature map to improve the detection capability of the small license plate area. In the implementation process, the detection model with FPN (feature pyramid network) is trained through PyTorch or ONNX framework, combined with the real-time video stream processing module to support multi-resolution input;

[0086] (2) Introducing the attention mechanism: Use the channel attention mechanism (such as the SE module) and the spatial attention mechanism (such as the CBAM module) to optimize the weight distribution of the feature map and suppress the influence of irrelevant areas in the complex background. For example, for the license plate detection task in the video stream, the attention mechanism will enhance the features of the license plate area, and at the same time filter the background through real-time point cloud data, call the visualization function of the detection results, and output the features after attention optimization as a highlighted rectangular box of the license plate area;

[0087] (3) Use pre-trained models for transfer learning: Use ImageNet or Common Objects in Context pre-trained weights to adapt to license plate styles from different countries, including different fonts, colors, and character arrangements. Transfer learning can maintain a recognition accuracy of more than 99% in complex environments (such as uneven lighting and blurred images) by fine-tuning the weights of high-level detection heads.

[0088] (4) Thread-based real-time detection and result drawing: Implement a multi-threaded license plate detection framework, manage the detection thread through the QThread module of PyQt5, and ensure efficient parallel execution of license plate detection and drawing tasks: QThreadModelPredict handles the detection thread, including reading frame data from the video stream, calling the detection model for inference, and drawing the detection results;

[0089] The signal mechanism ensures that the UI is updated immediately after the detection is completed to display the detection results, including the license plate number, license plate confidence, and positioning frame coordinates;

[0090] Use the cnn_predict_image_drawn_sig signal to present the drawn detection results in real time through the UI module;

[0091] (5) Optimize video streaming and real-time interaction: Enhance compatibility with multiple image sources, including image files, video streams, and real-time camera input. In video mode, use the cross-platform computer vision library VideoCapture to manage frame capture, call the detection algorithm in a multi-threaded environment through cnn_infer_thread, and realize real-time dynamic update of detection results, including license plate number, license plate color, and detection confidence. Call show_infer_res and show_image functions to display the detection results to ensure that the detection frame accurately corresponds to the license plate information;

[0092] (6) Data accuracy verification and dynamic threshold adjustment: Supports dynamic adjustment of confidence and IoU (Intersection over Union) thresholds to adapt to detection requirements in different environments.

[0093] Furthermore, the data processing module in Step 4 is combined with a dynamic threshold adjustment algorithm to optimize the over-limit judgment. The size difference and threshold adjustment formula are as follows:

[0094] ;

[0095] ;

[0096] in, is the standard size, v is the vehicle speed, and To adjust the parameters, it is used to balance the errors of static size and dynamic size detection.

[0097] Furthermore, if Figure 4 As shown, in Step 6, the radar data and the video data are effectively fused by an adaptive registration algorithm and a weighted fusion algorithm, and the fused data is encoded using H.265 / HEVC video coding. The specific steps are as follows:

[0098] (1) Data preprocessing: The data obtained by the radar is processed by denoising and filtering to ensure that its three-dimensional measurement is accurate. The video data is processed by image enhancement, denoising and other technologies to improve image quality, especially visibility in low-light environments or complex backgrounds. The denoising and filtering formulas are as follows:

[0099] ;

[0100] ;

[0101] in It is a low-pass filter to remove high-frequency noise in radar data; It is a Gaussian filtering operation to remove noise from the image and enhance the image quality;

[0102] (2) Adaptive registration algorithm: After data preprocessing is completed, the radar data and video data are spatially aligned using an adaptive registration algorithm. Adaptive registration automatically adjusts the spatial relationship between different data sources to ensure that the radar data and video data are fused in the same coordinate system, avoiding errors caused by factors such as acquisition angle and viewing distance.

[0103] (3) For key tasks such as license plate recognition, video data may provide more accurate information, while radar data may have more advantages in terms of the accuracy of the vehicle's three-dimensional dimensions. The weighted fusion algorithm will take these factors into account and combine the information from both to generate the final fused data. The weighted formula is:

[0104] ;

[0105] in is the fusion weight coefficient, ∈[0,1], radar data can be given a higher weight, and video data is weighted according to image quality and license plate recognition results;

[0106] (4) Fusion data encoding: After the fusion data is generated, the fusion data is compressed and encoded using H.265 / HEVC video encoding. The encoding formula is:

[0107] ;

[0108] in For H.265 / HEVC encoder, is the fused data;

[0109] After encoding, the integrated data is easy to store and transmit, and can be transmitted to the monitoring platform in real time via the network for further vehicle overload analysis and early warning.

[0110] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A radar and video information fusion coding method for vehicle overload warning, characterized in that: The steps include: Step 1. Scan the vehicle in real time through the radar module to obtain the three-dimensional size data of the vehicle; Step 2. Use the visible light camera module to capture vehicle images and extract license plate information by improving YOLO's deep learning license plate recognition algorithm; Step 3, the data processing module retrieves the standard size in the database according to the license plate number and calculates the difference between the actual size of the vehicle and the standard size; Step 4: Compare the vehicle size difference values ​​with the threshold value to determine whether the vehicle exceeds the limit; Step 5. If the vehicle is determined to be over-limited, an alarm is triggered and the vehicle information is recorded, including the license plate number, actual three-dimensional dimensions, over-limit dimensions and magnitude, and detection time and location; Step 6: Preprocess the radar data and video data, use the adaptive registration algorithm to spatially align the radar data and video data, use the weighted fusion algorithm to combine the license plate recognition and the accuracy information of the vehicle's three-dimensional size to generate the final fused data, and use H.265 / HEVC video coding to encode the fused data.

2. A radar and video information fusion coding method for vehicle overload warning as claimed in claim 1, characterized in that: Step 1 specifically includes: Step 1.1, LiDAR data collection: Use a drone equipped with a mechanical LiDAR to fly around the vehicle under test, and the LiDAR scans to obtain the side and tail point cloud data of the vehicle; Step 1.2, data format conversion: In the Ubuntu system, use the relevant instructions in the robot operating system to convert the PCAP file into a bag file through the rosbag instruction, and then use the bag_to_pcd instruction to convert the bag file into a PCD file; Step 1.3, point cloud data preprocessing: Use the conditional filter in the point cloud library, set the conditional thresholds of multiple fields, remove obvious outliers that are far away from the target vehicle point cloud, calculate the average distance from each point to all its k-neighborhood neighboring points, set the distance threshold, remove outliers, set the neighborhood radius threshold and the number of minimum points to form a cluster, cluster the point cloud data, and obtain complete vehicle point cloud data; Step 1.4, polar coordinates to Cartesian coordinates: Convert the point cloud data obtained by the LiDAR scan from polar coordinates to Cartesian coordinates. The conversion formula is as follows: ； Among them, r is the distance between the measured point and the center of the laser radar, θ is the horizontal rotation angle, is the vertical angle; Step 1.5, principal component analysis: Calculate the centroid m of the point cloud data. The calculation formula is: ； in, is the point in the point cloud dataset, and n is the number of points; Decentralize the n points in the point cloud set and construct the covariance matrix C. The formula is as follows: ； Solve the eigenvalue λ and the corresponding eigenvector v of the covariance matrix; orthogonally normalize the eigenvector to obtain the orthogonal matrix A; transform the point cloud data to the vicinity of the origin of the coordinate system, and the three main directions of the point cloud data are the directions of the new coordinate system; Step 1.6, minimum bounding box construction: call the pcl::getMinMax3D function in the PCL library, and get the coordinates of the eight vertices of the bounding box by traversing all the point cloud data in the minimum bounding box. According to the coordinates of the bounding box vertices, calculate the length L, width W and height H of the vehicle. The formula is as follows: ； , , Respectively represent the minimum coordinates of the point cloud data in the x, y, and z directions, , , Represents the maximum coordinates of the point cloud data in the x, y, and z directions respectively.

3. The radar and video information fusion coding method for vehicle overload warning as claimed in claim 1 is characterized in that: The license plate recognition algorithm of the visible light camera module in Step 2 adopts an improved YOLO deep learning model, and its optimization includes the following contents: a. Add a multi-scale feature fusion module: Add a multi-scale feature fusion module to the network, use high-resolution feature maps to enhance the detail features of the small license plate area, and fuse the context information in the low-resolution feature map. Use the PyTorch or ONNX framework to train the detection model with a feature pyramid network, combined with the video stream real-time processing module, to support multi-resolution input; b. Introduce the attention mechanism: Use the channel attention mechanism and the spatial attention mechanism to optimize the weight distribution of the feature map and suppress the influence of irrelevant areas in the complex background; call the visualization function of the detection results to output the attention-optimized features as a highlighted rectangular frame of the license plate area; c. Use pre-trained models for transfer learning: Use ImageNet or Common Objects in Context pre-trained weights to adapt to license plate styles from different countries through transfer learning, including different fonts, colors, and character arrangements; d. Thread-based real-time detection and result drawing: The detection thread is managed through the QThread module of PyQt5 to ensure efficient parallel execution of license plate detection and drawing tasks: QThreadModelPredict handles the detection thread, including reading frame data from the video stream, calling the detection model for reasoning, and drawing the detection results; The signal mechanism ensures that the UI is updated immediately after the detection is completed to display the detection results, including the license plate number, license plate confidence, and positioning frame coordinates; Use the cnn_predict_image_drawn_sig signal to present the drawn detection results in real time through the UI module; e. Optimize video streaming and real-time interaction: Enhance compatibility with multiple image sources, including image files, video streams, and real-time camera input. In video mode, use the cross-platform computer vision library VideoCapture to manage frame capture, call the detection algorithm in a multi-threaded environment through cnn_infer_thread, and realize real-time dynamic update of detection results, including license plate number, license plate color, and detection confidence. Call show_infer_res and show_image functions to display the detection results to ensure that the detection frame accurately corresponds to the license plate information; f. Data accuracy verification and dynamic threshold adjustment: Support dynamic adjustment of confidence and IoU thresholds.

4. The radar and video information fusion coding method for vehicle overload warning according to claim 1 is characterized in that: In Step 4, the over-limit judgment is optimized by combining the data processing module with the dynamic threshold adjustment algorithm. The size difference and threshold adjustment formula are as follows: ； ； in, is the standard size, v is the vehicle speed, and To adjust the parameters, it is used to balance the errors of static size and dynamic size detection.

5. The radar and video information fusion coding method for vehicle overload warning as claimed in claim 1 is characterized in that: Step 6 specifically includes: Step 6.1, Data preprocessing: The data obtained by the radar is processed by denoising and filtering to ensure the accuracy of its three-dimensional measurement. The video data is processed by image enhancement and denoising. The denoising and filtering formulas are as follows: ； ； in It is a low-pass filter to remove high-frequency noise in radar data; It is a Gaussian filtering operation to remove noise from the image. Represents radar data, represents the radar data after noise removal, Represents video data, Represents the video data after noise removal; Step 6.2, Adaptive registration algorithm: After data preprocessing is completed, the radar data and video data are spatially aligned using an adaptive registration algorithm, which automatically adjusts the spatial relationship between different data sources; Step 6.3: Use a weighted fusion algorithm to combine the license plate recognition and the vehicle's three-dimensional size accuracy information to generate the final fusion data. The weighted formula is: ； in is the fusion weight coefficient, ∈[0,1], radar data can be given a higher weight, and video data is weighted according to image quality and license plate recognition results; Step 6.4, fusion data encoding: After the fusion data is generated, the fusion data is compressed and encoded using H.265 / HEVC video encoding. The encoding formula is: ； in For H.265 / HEVC encoder, is the fused data.

Images