A method and system for warning of trucks speeding on curves based on radar-visual fusion sensing technology

Abstract

This invention relates to the field of truck traffic control technology, specifically to a method and system for warning trucks about speeding on curves based on radar-visual fusion perception technology. The invention obtains multiple truck marker boxes in each frame of an image, identifies the license plate area within each truck marker box, and determines the target truck based on the tonal characteristics of each license plate area. Based on the morphological characteristics of lane lines in each frame of the image, as well as the positional distribution and radar speed of the target truck, the corrected speed of the target truck in each frame is obtained. Multiple sub-videos of the vehicle's curve driving video are obtained, and a real-time curve speeding threshold is obtained based on the corrected speed of the target truck in all frames within different sub-videos, the number of target trucks, and pixel grayscale characteristics. Speeding warnings are then issued to the target truck. This invention improves the accuracy of speeding warnings by obtaining an accurate curve speeding threshold for trucks driving on curves.

Description

Technical Field

[0001] This invention relates to the field of truck traffic control technology, specifically to a method and system for warning of trucks speeding on curves based on radar-visual fusion perception technology. Background Technology

[0002] Due to their large size and heavy weight, trucks have a longer braking distance and a higher risk of rollover when speeding, especially when driving on curves. Speeding significantly increases the possibility of losing control and can lead to serious traffic accidents. Therefore, it is important to provide warnings for trucks speeding on curves.

[0003] In existing technologies, installing a radar-visual integrated machine on curves integrates radar speed measurement and video monitoring functions to monitor the speed of each passing truck in real time. A fixed overspeed threshold is set based on experience to determine whether a truck is speeding, and the system automatically triggers a warning mechanism when speeding occurs. However, since radar measures the straight-line distance between the vehicle and the radar device, while the truck's trajectory is curved when turning, the radar may not be able to accurately capture the actual speed of the vehicle on the curve. Furthermore, the requirements for safe driving vary under different driving environments and conditions. For example, in severe weather and with heavy traffic, the driving risk increases, and the required overspeed threshold is relatively lower. Using only a fixed overspeed threshold cannot adapt to different changes, resulting in poor effectiveness of curve overspeed warnings. Summary of the Invention

[0004] To address the technical problems of radar measurement's inability to accurately capture vehicle speed on curves and the inability of fixed overspeed thresholds to adapt to changing driving environments and conditions, resulting in poor curve overspeed warning performance, this invention aims to provide a method and system for truck curve overspeed warning based on radar-visual fusion perception technology. The specific technical solution adopted is as follows:

[0005] This invention proposes a method for warning of trucks speeding on curves based on radar-visual fusion perception technology, the method comprising:

[0006] Obtain the radar speed of each vehicle in every frame of a historical video of a vehicle driving on a curve at a real time.

[0007] Multiple truck marker boxes are obtained in each frame of the image. The license plate area within each truck marker box is obtained. The target truck is obtained based on the hue characteristics of each license plate area. The corrected speed of the target truck in each frame of the image is obtained based on the morphological characteristics of the lane lines in each frame of the image, as well as the position distribution and radar speed of the target truck.

[0008] Multiple sub-videos of the vehicle driving on a curve are obtained. Based on the corrected speed of the target truck, the number of target trucks, and the pixel grayscale features in all frames of different sub-videos, the real-time curve overspeed threshold is obtained.

[0009] The target truck is given an overspeed warning based on the real-time curve overspeed threshold.

[0010] Furthermore, the method for acquiring the target truck includes:

[0011] The mean value of the most hue of each pixel in the HSV space within each license plate area is obtained as the area hue value;

[0012] If the color value of the area corresponding to each license plate area is within the preset color range, the truck in the corresponding license plate area will be used as the target truck.

[0013] Furthermore, the method for obtaining the correction speed includes:

[0014] Based on the morphological features of the lane lines in each frame of the image and the positional distribution of the target truck, the lane line curvature of the target truck in each frame of the image is obtained.

[0015] For each target truck, a negative correlation mapping is performed on the lane curvature as the first mapping coefficient; the product between the radar speed and lane curvature of each target truck is obtained and a positive correlation mapping is performed as the second mapping coefficient.

[0016] The product of the first and second mapping coefficients is obtained as the corrected speed for each target truck in each frame of the image.

[0017] Furthermore, the method for obtaining the lane line curvature includes:

[0018] Based on the Hough elliptic transform, each frame of the image is mapped to the Hough space. The intersection point with the most intersection lines in the Hough space is selected to obtain the elliptic parameters of the intersection point in the original space of each frame of the image, and then the elliptic function curve is constructed.

[0019] Obtain the center position of the truck marker box corresponding to each target truck, and use it as the position of each target truck;

[0020] The optimization algorithm is used to obtain the point on the elliptic curve that is closest to the position of the target truck, which is taken as the target point; the curvature of the target point on the elliptic curve is taken as the lane line curvature.

[0021] Furthermore, the method for obtaining the real-time curve overspeed threshold includes:

[0022] For each video segment, obtain the average corrected speed of the target truck in all frames as the speed level; obtain the sum of the number of target trucks in all frames as the traffic flow; obtain the average grayscale value of the pixels in all frames as the grayscale level.

[0023] For traffic flow or grayscale level corresponding to each video segment as the data to be analyzed, the data change rate of the data to be analyzed between adjacent videos is obtained based on the distribution of the data to be analyzed in different videos; the correlation coefficient of the sequence formed by the data to be analyzed and the speed level corresponding to all videos is obtained as the data influence coefficient of the data to be analyzed.

[0024] The product of the speed level corresponding to the first sub-video and the preset multiple is obtained as the initial curve overspeed threshold. According to the distribution order of the sub-videos, the initial curve overspeed threshold is updated and iterated based on the data change rate and data influence coefficient of the corresponding data to be analyzed between adjacent sub-videos to obtain the real-time curve overspeed threshold.

[0025] Furthermore, the method for obtaining the rate of change of the data to be analyzed includes:

[0026] For any data to be analyzed, obtain the difference between each sub-video and the corresponding data to be analyzed in the previous sub-video, and calculate the ratio between the difference result and the corresponding data to be analyzed in the previous sub-video as the data change rate.

[0027] Furthermore, the method for obtaining the real-time curve overspeed threshold includes:

[0028] Each sub-video is analyzed sequentially, and the product of the data change rate and data influence coefficient of the corresponding data to be analyzed between the next adjacent sub-video and each sub-video is used as the first product;

[0029] The sum of the first products of all data to be analyzed between the next adjacent sub-video and each sub-video is used as the first sum value; the sum of the positive integer 1 and the first sum value is used as the adjustment weight; the product between the adjustment weight and the initial curve overspeed threshold of each sub-video is used as the new initial curve overspeed threshold corresponding to the next adjacent sub-video.

[0030] Obtain the new initial curve overspeed threshold corresponding to the last segment video, and use it as the real-time curve overspeed threshold.

[0031] Furthermore, the method for obtaining the license plate area includes:

[0032] The CANNY algorithm is used to perform edge detection on each truck marker box, and the license plate rectangle contour detection is used to obtain the license plate region.

[0033] Furthermore, the correlation coefficient is the Pearson correlation coefficient.

[0034] The present invention also proposes a truck curve overspeed warning system based on radar-visual fusion perception technology, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the steps of the truck curve overspeed warning method based on radar-visual fusion perception technology.

[0035] The present invention has the following beneficial effects:

[0036] This invention obtains multiple truck marker boxes in each frame of an image, and the license plate area within each truck marker box. Based on the tonal characteristics of each license plate area, the target truck is identified, accurately identifying and marking the trucks in the video that need to be analyzed. Considering that radar measurements ignore the curved driving of vehicles on curves, the corrected speed of the target truck in each frame of the image is obtained based on the morphological characteristics of the lane lines, the position distribution of the target truck, and the radar speed. The morphology of the lane lines reflects the road conditions, and the position and radar speed of the truck reflect its driving state. Combining these factors allows for a more comprehensive assessment of the truck's actual speed. Multiple sub-videos of the vehicle's curved driving video are obtained. Based on the corrected speed of the target truck, the number of target trucks, and the pixel grayscale characteristics in all frames of different sub-videos, a real-time curve overspeed threshold is obtained. This helps the system dynamically adjust the curve overspeed threshold, making it more consistent with actual curve conditions and vehicle driving situations. Overspeed warnings are then issued for the target trucks. This invention improves the accuracy of overspeed warnings by obtaining accurate curve overspeed thresholds for trucks driving on curves. Attached Figure Description

[0037] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 A flowchart illustrating a method for warning of truck overspeeding on curves based on radar-visual fusion perception technology, as provided in an embodiment of the present invention;

[0039] Figure 2 A flowchart illustrating a method for obtaining lane curvature according to an embodiment of the present invention;

[0040] Figure 3 The flowchart illustrates a method for obtaining a real-time overspeed threshold on a curve, as provided in one embodiment of the present invention. Detailed Implementation

[0041] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a truck curve overspeed warning method and system based on radar-visual fusion perception technology proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0043] The following description, in conjunction with the accompanying drawings, details the specific scheme of the truck curve overspeed warning method and system based on radar-visual fusion perception technology provided by the present invention.

[0044] Please see Figure 1 The document illustrates a flowchart of a method for warning of trucks speeding on curves based on radar-visual fusion perception technology, according to an embodiment of the present invention. The method specifically includes:

[0045] Step S1: Obtain the radar speed of each vehicle in each frame of the historical video of vehicles driving on curves at real time.

[0046] In this embodiment of the invention, considering that radar measures the straight-line distance between the vehicle and the radar device, and that the truck's trajectory is curved when turning, radar speed measurement is inaccurate when turning. Therefore, a method combining radar measurement and video images is needed. Based on historical truck speed data on curves, the demand for truck speeding on curves under different environments and conditions is analyzed. First, a radar-view integrated device is installed above the curve to ensure overlapping of the radar and camera fields of view. Network time protocol is used to ensure that the timestamps of the radar and camera are completely consistent. Then, the Kalman tracking algorithm is used to track the movement of each vehicle in the video. The radar speed of each vehicle is determined by comparing the radar monitoring area and the vehicle's position in the video, obtaining the radar speed of each vehicle in each frame of the historical video of vehicles turning curves within the real-time range. The specific Kalman tracking algorithm is a well-known technique to those skilled in the art and will not be described in detail here.

[0047] It should be noted that, in one embodiment of the present invention, the radar speed and each frame of image are acquired at a frequency of 30 times per second, and the historical range of the real-time moment is the range formed by the real-time moment and the historical moment, which is 30 days, that is, the duration of the video of the vehicle driving on the curve is 30 days; in other embodiments of the present invention, the acquisition frequency and the historical range of the real-time moment can be specifically set according to the specific situation, and are not limited or described here.

[0048] In one embodiment of the present invention, in order to facilitate the subsequent image processing, median filtering is performed on each frame of the acquired image to reduce noise in the image and enhance the image quality. The specific median filtering is a well-known technique to those skilled in the art and will not be described in detail here.

[0049] It should be noted that, in the embodiments of the present invention, in order to facilitate subsequent data processing, the acquired data is standardized to eliminate the dimensions of the data, so that indicators of different units or orders of magnitude can be comprehensively analyzed and compared, ensuring the accuracy of subsequent analysis. Standardization can be achieved using existing methods such as Z-score standardization and max-min standardization. The specific means are well known to those skilled in the art and will not be elaborated here.

[0050] Step S2: Obtain multiple truck marker boxes in each frame image, obtain the license plate area within each truck marker box, and obtain the target truck based on the hue characteristics of each license plate area; obtain the corrected speed of the target truck in each frame image based on the morphological characteristics of the lane lines in each frame image, as well as the position distribution and radar speed of the target truck.

[0051] Obtaining multiple truck bounding boxes in each frame of an image allows for a visual understanding of the trucks' distribution and movement. It should be noted that, in this embodiment of the invention, the method for obtaining truck bounding boxes includes: preparing a large dataset of truck images, typically including truck images from different angles and under different lighting conditions, with each image containing annotation information, namely the truck's location and category, thereby training... The algorithm detects trucks and obtains multiple truck bounding boxes in each frame of the image; the specific techniques are well known to those skilled in the art and will not be described in detail here.

[0052] License plate color is an important identifier in traffic management, helping to quickly identify the type, purpose, and affiliated organization of a vehicle. Certain special-colored license plates represent vehicles with special purposes, enjoying special rights and priority on the road, and are therefore not included in the analysis. The license plate area within each truck's marked frame is obtained, and the target truck is identified based on the tonal characteristics of each license plate area.

[0053] It should be noted that, in one embodiment of the present invention, the method for obtaining the license plate area includes: using the CANNY algorithm to perform edge detection on each truck marker box, and using license plate rectangle contour detection processing to obtain the license plate area; the specific CANNY algorithm and license plate rectangle contour detection are technical means well known to those skilled in the art, and will not be described in detail here.

[0054] Preferably, considering that the commonly used color for ordinary truck license plates in most areas is blue or yellow; in one embodiment of the present invention, the method for obtaining the target truck includes:

[0055] The mean value of the most hue of each pixel in the HSV space within each license plate area is obtained as the area hue value;

[0056] If the color value of the area corresponding to each license plate area is within the preset color range, the truck in the corresponding license plate area will be used as the target truck.

[0057] It should be noted that, in one embodiment of the present invention, the preset color range includes the color range of yellow license plates and the color range of blue license plates, which can be obtained in advance by the implementer based on relevant experience and knowledge. For example, the standard color range of blue license plates is 120 degrees to 180 degrees, and the standard color range of yellow license plates is 30 degrees to 60 degrees.

[0058] When trucks drive on curves, they generally follow the lane markings. Analyzing the morphological characteristics of the lane markings helps identify the road direction and turning conditions, which is beneficial for determining the truck's trajectory. The truck's position within the lane reflects its driving status and further helps determine the curvature of the lane markings. Radar speed can reflect the truck's speed in real time, helping to obtain dynamic information about the truck promptly. However, since radar measures the straight-line distance between the vehicle and the radar device, and the truck's trajectory is curved when turning, the radar speed is corrected based on the morphological characteristics of the lane markings and the position distribution of the target truck to reflect the truck's true speed in real time. Therefore, based on the morphological characteristics of the lane markings in each frame of the image, as well as the position distribution of the target truck and the radar speed, the corrected speed of the target truck in each frame of the image is obtained.

[0059] Preferably, under normal circumstances, the ratio between the actual speed and the radar speed is close to the ratio between the corresponding travel length. Considering that the travel speed and travel length are positively correlated, in one embodiment of the present invention, the method for obtaining the corrected speed includes:

[0060] Based on the morphological features of the lane lines in each frame of the image and the positional distribution of the target truck, the lane line curvature of the target truck in each frame of the image is obtained.

[0061] Preferably, in one embodiment of the present invention, the method for obtaining the lane line curvature is described in [reference needed]. Figure 2 It illustrates a flowchart of a method for obtaining lane curvature, including:

[0062] Step S201: Based on the Hough elliptic transform, map each frame of the image to the Hough space, select the intersection point with the most intersection lines in the Hough space, obtain the elliptic parameters of the intersection point in the original space of each frame of the image, and construct the elliptic function curve.

[0063] In a curve, the lane lines take on an elliptical shape. The Hough elliptic transform is suitable for detecting elliptical shapes in images. By mapping each frame of the image to the Hough space and detecting the intersection point with the most intersections in the Hough space, the ellipse parameters can be accurately obtained, which can approximate the elliptical shape of the lane lines in the image.

[0064] Step S202: Obtain the center position of the truck marker box corresponding to each target truck, and use it as the position of each target truck;

[0065] Regardless of how the target truck changes, its center position is usually located at the geometric center of the bounding box, thus reducing the error in position estimation; using the center position as the representation of the target truck can ensure the consistency of data between different frames.

[0066] Step S203: Use an optimization algorithm to obtain the point on the elliptic curve that is closest to the position of the target truck, and use it as the target point; obtain the curvature of the target point on the elliptic curve as the lane line curvature.

[0067] Lane curvature reflects the geometry of the road, especially its degree of curvature. The greater the lane curvature and the greater the degree of curvature, the less accurately radar may be able to capture the vehicle's true speed.

[0068] It should be noted that in some embodiments of the present invention, the optimization algorithm may employ gradient descent or Newton's method to obtain the target point; the specific means are well known to those skilled in the art and will not be elaborated here.

[0069] For each target truck, a negative correlation mapping is performed on the lane curvature as the first mapping coefficient; the product between the radar speed and lane curvature of each target truck is obtained and a positive correlation mapping is performed as the second mapping coefficient.

[0070] The product of the first and second mapping coefficients is obtained as the corrected speed for each target truck in each frame of the image.

[0071] In one embodiment of the present invention, the formula for correcting the speed is expressed as:

[0072] ;

[0073] in, Indicates the first frame of each image Correction speed of the target truck; Indicates the first The lane curvature of the target truck; Indicates the first The radar speed of the target truck; ( ) represents the inverse trigonometric sine function.

[0074] In the formula for correcting the velocity, This indicates a negative correlation mapping of the lane curvature for each target truck, i.e., the first mapping coefficient. The larger the first mapping coefficient, the smaller the lane curvature, which means the curve is less curved, the radar speed is closer to the real speed, and the speed correction is smaller. The second mapping coefficient is represented by the inverse trigonometric sine function, which is used to perform a positive correlation mapping between the radar speed and lane curvature of each target truck. This reflects a measure of the speed measurement deviation caused by the lane curvature. The smaller the radar speed, the smaller the lane curvature, the smaller the speed measurement deviation, and the smaller the speed correction.

[0075] Step S3: Obtain multiple sub-videos of the vehicle driving on the curve. Based on the corrected speed of the target truck, the number of target trucks, and the pixel grayscale features in all frames of different sub-videos, obtain the real-time curve overspeed threshold.

[0076] Driving on curves is a high-risk area for traffic accidents. Segmenting the video into multiple shorter sub-videos, each covering a specific time period, can speed up processing, improve analysis efficiency, and capture more detailed information about the vehicle's movement on curves. Multiple sub-videos of the vehicle's curve driving are obtained. It should be noted that, in one embodiment of this invention, the duration of the vehicle curve driving video is 30 days, and the sub-videos are obtained by dividing the video into hourly segments according to time sequence. In other embodiments of this invention, the segmentation interval can be set according to specific circumstances, and is not limited or elaborated here.

[0077] Because the requirements for safe driving vary under different driving environments and conditions, the overspeed threshold for curves will change. Corrected speed provides a more accurate real-time vehicle speed. By analyzing the number of target trucks, the degree of congestion and driving safety on curves can be reflected. The more trucks there are, the more congested the traffic becomes, increasing the risk of speeding. Pixel grayscale features reflect the road environment and weather conditions. The larger the pixel grayscale features, the higher the brightness information of the image, the better the weather conditions, the clearer the visibility, and the higher the threshold for speeding. Therefore, based on the corrected speed of target trucks, the number of target trucks, and the pixel grayscale features in all frames of images within different video segments, the real-time overspeed threshold for curves can be obtained.

[0078] Preferably, in one embodiment of the present invention, the method for obtaining the real-time curve overspeed threshold is described in [reference needed]. Figure 3 It illustrates a flowchart of a method for obtaining the real-time curve overspeed threshold, including:

[0079] Step S301: For each sub-video, obtain the average corrected speed of the target truck in all frame images as the speed level; obtain the sum of the number of target trucks in all frame images as the traffic flow; obtain the average grayscale value of the pixels in all frame images as the grayscale level.

[0080] The data characteristics of each sub-video are quantified by taking the average value, reflecting the general level of roads within the sub-video.

[0081] Step S302: For the traffic flow or grayscale level corresponding to the sub-video as the data to be analyzed, according to the distribution of the data to be analyzed corresponding to different sub-videos, obtain the data change rate of the data to be analyzed between adjacent sub-videos; obtain the correlation coefficient of the sequence formed by the data to be analyzed and the speed level corresponding to all sub-videos, as the data influence coefficient of the data to be analyzed.

[0082] The distribution of the data to be analyzed can reflect the trend of data change. The closer the distribution, the smaller the data change between adjacent sub-videos and the more similar the performance characteristics. The correlation coefficient between the data to be analyzed and the speed level reflects the influence of the data to be analyzed on the speed level. The larger the correlation coefficient, the greater the influence coefficient of the data.

[0083] It should be noted that, in one embodiment of the present invention, the correlation coefficient is the Pearson correlation coefficient. In other embodiments of the present invention, the DTW distance can also be used to reflect the correlation coefficient. The larger the DTW distance, the smaller the correlation coefficient, and negative correlation mapping is required. The specific means are well known to those skilled in the art and will not be described in detail here.

[0084] Preferably, in one embodiment of the present invention, the method for obtaining the rate of change of the data to be analyzed includes:

[0085] For any data to be analyzed, obtain the difference between the next adjacent sub-video and the corresponding data to be analyzed in each sub-video, and calculate the ratio between the difference result and the corresponding data to be analyzed in each sub-video as the data change rate.

[0086] In one embodiment of the present invention, the formula for the rate of change of data is expressed as:

[0087] ;

[0088] in, Indicates the first The video and the first The rate of change of the corresponding data to be analyzed between individual videos; Indicates the first Data to be analyzed for each video segment; Indicates the first The data to be analyzed corresponds to each video segment.

[0089] In the formula for the rate of change of data, Indicates the first The video and the first The difference between the data to be analyzed in each video segment is the largest difference. The larger the difference, the greater the change in the data to be analyzed in the subsequent video segment relative to the previous video segment, and the greater the rate of data change.

[0090] Step S303: Obtain the product between the speed level corresponding to the first sub-video and the preset multiple, as the initial curve overspeed threshold; according to the distribution order of the sub-videos, update and iterate the initial curve overspeed threshold based on the data change rate and data influence coefficient of the corresponding data to be analyzed between adjacent sub-videos to obtain the real-time curve overspeed threshold.

[0091] Preferably, in one embodiment of the present invention, the method for obtaining the real-time curve overspeed threshold includes:

[0092] Each sub-video is analyzed sequentially, and the product of the data change rate and data influence coefficient of the corresponding data to be analyzed between the next adjacent sub-video and each sub-video is used as the first product;

[0093] The sum of the first products of all data to be analyzed between the next adjacent sub-video and each sub-video is used as the first sum value; the sum of the positive integer 1 and the first sum value is used as the adjustment weight; the product between the adjustment weight and the initial curve overspeed threshold of each sub-video is used as the new initial curve overspeed threshold corresponding to the next adjacent sub-video.

[0094] In one embodiment of the present invention, for the first Analyze each video segment, the first... The formula for the new initial curve overspeed threshold corresponding to each video segment is:

[0095] ;

[0096] in, Indicates the first Each video segment corresponds to a new initial curve overspeed threshold; Indicates the first The video and the first The rate of change of corresponding grayscale levels between individual videos; Indicates the first The video and the first The rate of change in traffic flow data between individual videos; The correlation coefficient between the gray level and the speed level corresponding to all sub-videos is used as the data influence coefficient of the gray level. The correlation coefficient represents the sequence of traffic flow and speed levels corresponding to all sub-videos, and is used as the data influence coefficient of traffic flow. Indicates the first The initial curve overspeed threshold corresponding to each video segment.

[0097] In the formula for the initial overspeed threshold on curves, This represents the sum of the first product of the corresponding gray level and the corresponding traffic flow, i.e., the first sum value; This represents the product of the data change rate and the data influence coefficient at the gray level. The larger the data change rate at the corresponding gray level, the brighter the later video segment is relative to the previous video segment, which means the weather is relatively better and the demand for vehicle speeding threshold is relatively greater. The larger the gray level, the greater the demand for vehicle speeding threshold. The larger the data influence coefficient of the gray level, the greater the positive impact on the speed level, and the greater the positive adjustment of the curve speeding threshold. This represents the product of the data change rate and the data influence coefficient under traffic flow. The larger the data change rate of traffic flow, the larger the traffic flow in the later video segment relative to the previous video segment, the smaller the vehicle spacing, and the smaller the vehicle overspeed threshold requirement. The larger the traffic flow, the smaller the vehicle overspeed threshold requirement, the smaller the data influence coefficient of traffic flow, the greater the negative impact on speed level, and the greater the negative adjustment of the curve overspeed threshold. This indicates that the weight is adjusted. The higher the grayscale level of the next video segment, the lower the traffic flow, and the larger the positive adjustment of the curve overspeed threshold. The smaller the negative adjustment of the curve overspeed threshold, the larger the adjustment weight and the larger the curve overspeed threshold.

[0098] It should be noted that, in one embodiment of the present invention, the preset multiple is 1.05; in other embodiments of the present invention, the preset multiple can be set according to specific circumstances, and will not be limited or elaborated here.

[0099] Based on this, the analysis starts from the first segment video, and the new initial cornering speeding threshold corresponding to the next segment video of each segment video is continuously obtained until the new initial cornering speeding threshold corresponding to the last segment video is obtained, which is used as the real-time cornering speeding threshold.

[0100] Step S4: Issue an overspeed warning to the target truck based on the real-time curve overspeed threshold.

[0101] The system analyzes driving conditions within a local video range to obtain real-time curve speeding thresholds, which helps in timely responses to different environments and conditions. If the real-time curve speeding threshold is reached, a warning should be promptly issued to the driver of the speeding vehicle via broadcast equipment, thereby reducing the risk of traffic accidents caused by speeding and ensuring the safety of every driver and passenger. It should be noted that to ensure the timeliness and accuracy of the curve speeding thresholds, the real-time curve speeding thresholds are acquired every hour.

[0102] In summary, this invention obtains multiple truck marker boxes in each frame of an image, obtains the license plate area within each truck marker box, and identifies the target truck based on the tonal characteristics of each license plate area. Based on the morphological characteristics of lane lines in each frame of the image, as well as the positional distribution and radar speed of the target truck, the corrected speed of the target truck in each frame of the image is obtained. Multiple sub-videos of the vehicle's curve driving video are obtained, and based on the corrected speed of the target truck in all frames within different sub-videos, the number of target trucks, and pixel grayscale characteristics, a real-time curve overspeed threshold is obtained. Overspeed warnings are then issued to the target truck. This invention improves the accuracy of overspeed warnings by obtaining an accurate curve overspeed threshold for trucks driving on curves.

[0103] This invention also proposes a truck curve overspeed warning system based on radar-visual fusion perception technology, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the steps of a truck curve overspeed warning method based on radar-visual fusion perception technology.

[0104] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0105] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A truck curve overspeed early warning method based on radar and visual fusion perception technology, characterized in that, The method includes: Obtain the radar speed of each vehicle in every frame of a historical video of a vehicle driving on a curve at a real-time moment. Multiple truck marker boxes are obtained in each frame of the image. The license plate area within each truck marker box is obtained. The target truck is obtained based on the hue characteristics of each license plate area. The corrected speed of the target truck in each frame of the image is obtained based on the morphological characteristics of the lane lines in each frame of the image, as well as the position distribution and radar speed of the target truck. Multiple sub-videos of the vehicle driving on a curve are obtained. Based on the corrected speed of the target truck, the number of target trucks, and the pixel grayscale features in all frames of different sub-videos, the real-time curve overspeed threshold is obtained. The method for obtaining the real-time curve overspeed threshold includes: For each video segment, obtain the average corrected speed of the target truck in all frames as the speed level; obtain the sum of the number of target trucks in all frames as the traffic flow; obtain the average grayscale value of the pixels in all frames as the grayscale level. For traffic flow or grayscale level corresponding to each video segment as the data to be analyzed, the data change rate of the data to be analyzed between adjacent videos is obtained based on the distribution of the data to be analyzed in different videos; the correlation coefficient of the sequence formed by the data to be analyzed and the speed level corresponding to all videos is obtained as the data influence coefficient of the data to be analyzed. The product between the speed level corresponding to the first sub-video and the preset multiple is obtained as the initial curve overspeed threshold. According to the distribution order of the sub-videos, the initial curve overspeed threshold is updated and iterated based on the data change rate and data influence coefficient of the corresponding data to be analyzed between adjacent sub-videos to obtain the real-time curve overspeed threshold. The methods for obtaining the rate of change of the data to be analyzed include: For any data to be analyzed, obtain the difference between each sub-video and the corresponding data to be analyzed in the previous sub-video, and calculate the ratio between the difference result and the corresponding data to be analyzed in the previous sub-video as the data change rate; The target truck is given an overspeed warning based on the real-time curve overspeed threshold.

2. The method for warning of truck overspeeding on curves based on radar-visual fusion perception technology according to claim 1, characterized in that, The method for obtaining the target truck includes: The mean value of the most hue of each pixel in the HSV space is obtained as the region hue value; If the color value of the area corresponding to each license plate area is within the preset color range, the truck in the corresponding license plate area will be used as the target truck.

3. The method for warning of truck overspeeding on curves based on radar-visual fusion perception technology according to claim 1, characterized in that, The method for obtaining the correction speed includes: Based on the morphological features of the lane lines in each frame of the image and the positional distribution of the target truck, the lane line curvature of the target truck in each frame of the image is obtained. For each target truck, a negative correlation mapping is performed on the lane curvature as the first mapping coefficient; the product between the radar speed and lane curvature of each target truck is obtained and a positive correlation mapping is performed as the second mapping coefficient. The product of the first and second mapping coefficients is obtained as the corrected speed of each target truck in each frame of the image.

4. The method for warning of truck overspeeding on curves based on radar-visual fusion perception technology according to claim 3, characterized in that, The method for obtaining the lane line curvature includes: Based on the Hough elliptic transform, each frame of the image is mapped to the Hough space. The intersection point with the most intersection lines in the Hough space is selected to obtain the elliptic parameters of the intersection point in the original space of each frame of the image, and then the elliptic function curve is constructed. Obtain the center position of the truck marker box corresponding to each target truck, and use it as the position of each target truck; The optimization algorithm is used to obtain the point on the elliptic curve that is closest to the position of the target truck, which is taken as the target point; the curvature of the target point on the elliptic curve is taken as the lane line curvature.

5. The method for warning of truck overspeeding on curves based on radar-visual fusion perception technology according to claim 1, characterized in that, The method for obtaining the real-time curve overspeed threshold includes: Each sub-video is analyzed sequentially, and the product of the data change rate and data influence coefficient of the corresponding data to be analyzed between the next adjacent sub-video and each sub-video is used as the first product; The sum of the first products of all data to be analyzed between the next adjacent sub-video and each sub-video is used as the first sum value; the sum of the positive integer 1 and the first sum value is used as the adjustment weight; the product between the adjustment weight and the initial curve overspeed threshold of each sub-video is used as the new initial curve overspeed threshold corresponding to the next adjacent sub-video. Obtain the new initial curve overspeed threshold corresponding to the last segment video, and use it as the real-time curve overspeed threshold.

6. The method for warning of truck overspeeding on curves based on radar-visual fusion perception technology according to claim 1, characterized in that, The method for obtaining the license plate area includes: The CANNY algorithm is used to perform edge detection on each truck marker box, and the license plate rectangle contour detection is used to obtain the license plate region.

7. A method for warning of truck overspeeding on curves based on radar-visual fusion perception technology according to claim 1, characterized in that, The correlation coefficient mentioned is the Pearson correlation coefficient.

8. A truck overspeed warning system based on radar-visual fusion perception technology, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the truck curve overspeed warning method based on radar-visual fusion perception technology as described in any one of claims 1 to 7.

Images