A highway intelligent supervision platform and radar data fusion method

Abstract

The present invention relates to the technical field of data fusion processing, and more specifically to a highway intelligent supervision platform and radar-visual data fusion method, comprising: obtaining speed data and vehicle position data corresponding to radar-visual data; obtaining lateral and longitudinal risk characteristic indicators corresponding to each type of driving data; obtaining a driving style characteristic indicator based on the changing trend of vehicle displacement of the target vehicle corresponding to the driving data under lane-changing driving behavior in historical data; determining a feature matching indicator for the driving data by combining the lateral and longitudinal risk characteristic indicators and the driving style characteristic indicator; performing data matching on the radar data and the video data based on the similarity between the feature matching indicators corresponding to the radar data and the feature matching indicators corresponding to the video data, combined with the trajectory differences between the radar data and the video data, and performing radar-visual data fusion based on the matching results. The present invention reduces the probability of mismatching and improves the accuracy of radar-visual data fusion.

Description

Technical Field

[0001] The present invention relates to the technical field of data fusion processing, and in particular to an intelligent highway supervision platform and a radar data fusion method. Background Art

[0002] The intelligent highway monitoring platform integrates advanced sensing and analysis technologies to monitor vehicle driving behavior in real time, providing strong data support for traffic management and effectively ensuring road safety and order. The platform's core recognition function is based on radar and video data fusion. It leverages the strengths of radar and video data, using algorithms to match, extract features, and fuse them together to achieve comprehensive perception of vehicle status.

[0003] Existing radar and video data fusion methods analyze the similarities between radar and video trajectory data, perform feature matching, and then use feature-level fusion algorithms to match radar and video target data. However, in complex high-speed environments, multiple vehicles may be traveling at similar speeds or at close distances, resulting in extremely similar features in the radar and video data. This can lead to confusion in trajectory similarity, affecting the matching results of the radar and video data, resulting in poor radar and video data fusion results. Summary of the Invention

[0004] In order to solve the technical problem that the vehicle trajectory similarity confusion in the existing method affects the matching results of the radar data and makes the radar data fusion results less effective, the purpose of the present invention is to provide an intelligent highway supervision platform and a radar data fusion method. The technical solutions adopted are as follows:

[0005] In a first aspect, the present invention provides a method for fusing radar and visual data for an intelligent highway supervision platform, comprising:

[0006] Acquire driving data of the vehicle in each time period during the driving process on the highway, including radar data and video data; obtain speed data and vehicle position data corresponding to each type of driving data based on the driving data acquired in the time period;

[0007] According to the speed data difference between the target vehicle and the surrounding vehicles in the longitudinal direction, and the vehicle position data difference between the target vehicle and the surrounding vehicles in the lateral direction corresponding to each driving data, a lateral risk characteristic index and a longitudinal risk characteristic index corresponding to each driving data are obtained respectively;

[0008] Obtaining a driving style characteristic index corresponding to each type of driving data based on a change trend in vehicle displacement of the target vehicle under lane-changing driving behavior in historical data corresponding to each type of driving data; and determining a characteristic matching index for each type of driving data by combining the lateral risk characteristic index, the longitudinal risk characteristic index, and the driving style characteristic index.

[0009] According to the similarity between the feature matching indicators corresponding to the radar data and the feature matching indicators corresponding to the video data, combined with the trajectory differences between the radar data and the video data, data matching is performed on the radar data and the video data is then fused based on the matching results.

[0010] Preferably, the lateral risk characteristic index and the longitudinal risk characteristic index corresponding to each type of driving data are obtained based on the speed data difference between the target vehicle and the surrounding vehicles in the longitudinal direction, and the vehicle position data difference between the target vehicle and the surrounding vehicles in the lateral direction, respectively, specifically including:

[0011] For any type of driving data, the longitudinal risk characteristic index corresponding to the driving data is obtained based on the difference between the speed data of the target vehicle within the time period corresponding to the driving data and the speed data of the adjacent vehicles in the same lane as the target vehicle;

[0012] Based on the difference between the vehicle position data of the target vehicle at each moment in the time period corresponding to the driving data and the vehicle position data corresponding to the vehicles in the adjacent lanes of the target vehicle, the distance changes between the target vehicle and other vehicles are analyzed to obtain the lateral risk characteristic index corresponding to the driving data.

[0013] Preferably, obtaining the longitudinal risk characteristic index corresponding to the driving data based on the difference between the speed data of the target vehicle within the time period corresponding to the driving data and the speed data corresponding to adjacent vehicles in the same lane as the target vehicle specifically includes:

[0014] The preceding vehicle in the same lane and in the driving direction of the target vehicle corresponding to the driving data is used as the preceding reference vehicle, and the following vehicle in the same lane and in the driving direction of the target vehicle corresponding to the driving data is used as the following reference vehicle;

[0015] Each vehicle corresponds to a speed data in a time period. For any time period, the normalized value of the difference between the speed data of the target vehicle and the previous reference vehicle in the same time period is calculated to obtain the forward risk characteristic index of the target vehicle's driving data corresponding to the current time period.

[0016] The normalized value of the difference between the speed data of the reference vehicle and the target vehicle in the same time period is calculated to obtain the backward risk characteristic index of the target vehicle's driving data corresponding to the current time period;

[0017] The longitudinal risk characteristic indicators include forward risk characteristic indicators and backward risk characteristic indicators.

[0018] Preferably, the step of analyzing the distance change between the target vehicle and other vehicles based on the difference between the vehicle position data of the target vehicle at each moment within the time period corresponding to the driving data and the vehicle position data corresponding to vehicles in adjacent lanes of the target vehicle to obtain the lateral risk characteristic index corresponding to the driving data specifically includes:

[0019] The vehicle closest to the target vehicle in the left lane adjacent to the target vehicle corresponding to the driving data is used as the left reference vehicle, and the vehicle closest to the target vehicle in the right lane adjacent to the target vehicle corresponding to the driving data is used as the right reference vehicle; each vehicle corresponds to a vehicle position data at each moment in a time period; the lateral risk characteristic index includes a left-hand risk characteristic index and a right-hand risk characteristic index;

[0020] The method for obtaining the leftward risk characteristic indicator includes:

[0021] For any time period, based on the difference between the horizontal coordinates of the vehicle position data of the target vehicle and the left reference vehicle at each moment in the time period, the lateral difference distance between the target vehicle and the left reference vehicle at each moment in the time period is determined; the lateral difference distances at all moments in the time period are accumulated and summed and negatively correlated to obtain a first difference coefficient; the difference between the speed data of the target vehicle and the left reference vehicle in the time period is negatively correlated to obtain a second difference coefficient; the normalized value of the product of the first difference coefficient and the second difference coefficient is used as the left-bound risk characteristic indicator.

[0022] Preferably, the method for obtaining the rightward risk characteristic indicator includes:

[0023] The longitudinal difference distances at all moments in the time period are accumulated and summed up and negatively correlated to obtain the third difference coefficient; the speed data between the target vehicle and the right reference vehicle in the time period are negatively correlated to obtain the fourth difference coefficient; the normalized value of the product of the third difference coefficient and the fourth difference coefficient is used as the right-bound risk characteristic indicator.

[0024] Preferably, obtaining the driving style characteristic index corresponding to each driving data according to the change trend of the vehicle displacement of the target vehicle under the lane change driving behavior in the historical data corresponding to each driving data specifically includes:

[0025] The lane-changing driving behavior of a vehicle on a ramp includes entering the ramp and exiting the ramp. Based on the difference in vehicle displacement data of the target vehicle when entering the ramp and the difference in vehicle displacement data of the target vehicle when exiting the ramp corresponding to each type of driving data, the driving style characteristic index corresponding to each type of driving data is obtained.

[0026] Preferably, obtaining the driving style characteristic index corresponding to each type of driving data based on the difference in vehicle displacement data of the target vehicle entering the ramp and the difference in vehicle displacement data of the target vehicle exiting the ramp corresponding to each type of driving data specifically includes:

[0027] For any type of driving data within any time period and any lane change on a ramp, the difference between the horizontal coordinates of the vehicle displacement data at each moment the target vehicle enters the ramp and the next adjacent moment corresponding to the driving data is recorded as the entry displacement data. The first displacement characteristic coefficient is calculated by calculating the ratio of the cumulative sum of all the entry displacement data of the target vehicle entering the ramp and the total duration of the ramp entry.

[0028] The difference between the horizontal coordinates of the vehicle displacement data at each moment when the target vehicle exits the ramp and the next adjacent moment is recorded as the exit displacement data, and the ratio of the cumulative sum of all the exit displacement data of the target vehicle exiting the ramp to the total duration of the exit ramp is calculated to obtain the second displacement characteristic coefficient;

[0029] The normalized result of the cumulative sum of the first displacement characteristic coefficient and the second displacement characteristic coefficient is used as the driving characteristic factor of the current driving behavior; the average value of the driving characteristic factors of the lane change driving behavior of the target vehicle on all secondary ramps is calculated to obtain the driving style characteristic index of the target vehicle corresponding to the driving data.

[0030] Preferably, the determining of the feature matching index for each type of driving data by combining the lateral risk feature index, the longitudinal risk feature index, and the driving style feature index specifically includes:

[0031] For any type of driving data in any time period, the calculation formula of the feature matching index is:

[0032]

[0033] in, Represents the feature matching index of the target vehicle corresponding to the driving data in the t-th time period, represents the forward risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, represents the backward risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, represents the left-bound risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, represents the right-bound risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, is the preset adjustment coefficient, , Indicates the driving style characteristic index of the target vehicle corresponding to the driving data.

[0034] Preferably, the data matching of the radar data and the video data based on the similarity between the feature matching index corresponding to the radar data and the feature matching index corresponding to the video data, combined with the trajectory difference between the radar data and the video data, specifically includes:

[0035] For any time period, radar trajectory data is obtained through radar data, and video trajectory data is obtained through video data;

[0036] Calculating DTW data between radar trajectory data corresponding to the radar data and video trajectory data corresponding to the video data to obtain a first difference; calculating the absolute value of the difference between the feature matching index corresponding to the radar data and the feature matching index corresponding to the video data to obtain a second difference; performing negative correlation processing on the product of the first difference and the second difference to obtain a similarity measurement index between the radar data and the video data;

[0037] For the radar data corresponding to any target vehicle, when the similarity metric between the radar data and the video data is greater than a preset similarity threshold, the video data corresponding to the maximum value of the similarity metric between the radar data and all video data that meet the threshold is obtained as the matching result of the radar data and the video data.

[0038] In a second aspect, the present invention provides an intelligent highway supervision platform, which is used to implement the steps of a radar data fusion method for the intelligent highway supervision platform. The intelligent highway supervision platform includes:

[0039] The data acquisition module includes a millimeter-wave radar and a camera. The millimeter-wave radar is used to obtain radar data at each time period of the vehicle's driving on the highway, and the camera is used to obtain video data at each time period of the vehicle's driving on the highway.

[0040] A data preprocessing module, configured to obtain the driving data of the time period based on the driving data and obtain speed data and vehicle position data corresponding to each type of driving data;

[0041] a risk characteristic analysis module for respectively obtaining a lateral risk characteristic index and a longitudinal risk characteristic index corresponding to each type of driving data based on the speed data difference between the target vehicle and surrounding vehicles in the longitudinal direction, and the vehicle position data difference between the target vehicle and surrounding vehicles in the lateral direction;

[0042] a matching index determination module for obtaining a driving style characteristic index corresponding to each type of driving data based on a trend in displacement of the target vehicle under lane-changing driving behavior in historical data corresponding to each type of driving data; and determining a characteristic matching index for each type of driving data by combining the lateral risk characteristic index, the longitudinal risk characteristic index, and the driving style characteristic index;

[0043] The data fusion module is used to match radar data and video data based on the similarity between the feature matching indicators corresponding to the radar data and the feature matching indicators corresponding to the video data, combined with the trajectory differences between the radar data and the video data, and perform radar and video data fusion based on the matching results.

[0044] The embodiments of the present invention have at least the following beneficial effects:

[0045] The present invention represents the risk of a vehicle by evaluating the vehicle performance radar data and video data, and constructs risk indicators for the risk impact of lateral vehicles and longitudinal vehicles around the target vehicle for radar data and video data respectively. And when constructing the feature matching index of radar data and video data, the target vehicle driving style analysis results corresponding to the radar data are fully combined, and the risk feature index analysis results are combined to make the evaluation results of the additional data dimension more accurate. Finally, based on the risk characteristics and trajectory data, similarity metrics of radar data and video data are constructed to obtain the optimal matching relationship between radar data and video data. Based on the analysis of the trajectory differences of radar data, risk characteristics provide an additional information dimension for radar data matching. By comparing the vehicle risk coefficient of radar data and the vehicle risk coefficient of video data, the radar data of the same vehicle can be determined more accurately, the probability of mismatching can be reduced, the accuracy of radar data fusion can be improved, and a solid data foundation can be provided for the intelligent supervision platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 This is a flowchart of the steps of a radar data fusion method for a highway intelligent supervision platform provided by the present invention;

[0048] Figure 2 It is a flowchart of the steps of the method for obtaining the horizontal risk characteristic index and the vertical risk characteristic index provided by the present invention;

[0049] Figure 3 This is a flow chart of the sub-steps of the method for obtaining longitudinal risk characteristic indicators provided by the present invention;

[0050] Figure 4 This is a flow chart of the sub-steps of the method for obtaining the horizontal risk characteristic indicator provided by the present invention;

[0051] Figure 5 is a flowchart of the steps of the method for obtaining the driving style characteristic index provided by the present invention;

[0052] Figure 6 It is a structural diagram of an intelligent highway supervision platform provided by the present invention. DETAILED DESCRIPTION

[0053] To further illustrate the technical means and effectiveness of the present invention in achieving its intended objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a highway intelligent monitoring platform and radar-visual data fusion method proposed in accordance with the present invention. In the following description, references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0054] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0055] The specific scheme of the highway intelligent supervision platform and the radar data fusion method provided by the present invention is described in detail below with reference to the accompanying drawings.

[0056] See also Figure 1 , which shows a flowchart of a method for fusion of radar and visual data of a highway intelligent supervision platform provided by one embodiment of the present invention, the method comprising the following steps:

[0057] Step S100, obtaining driving data of the vehicle in each time period during driving on the highway, including radar data and video data; obtaining the driving data of the time period based on the driving data to obtain speed data and vehicle position data corresponding to each type of driving data.

[0058] Specifically, by real-time monitoring of the driving process of vehicles on the highway, the driving data of a target within a time period corresponds to a set of radar data and a set of video data. The purpose of this embodiment is to realize the matching process of radar data and video data by analyzing the characteristics of radar data and video data within the same time period respectively, so as to achieve the purpose of radar and video data fusion.

[0059] In this embodiment, millimeter-wave radar is used to collect radar data from a vehicle while it's in motion. By emitting electromagnetic waves and receiving reflected waves, the millimeter-wave radar accurately measures the straight-line distance between itself and a target vehicle based on the round-trip time of the signal, reflecting the target vehicle's position in space. The collected radar data can directly determine the target's distance, speed, and angle.

[0060] The average speed over a time period captured in the radar data of the target vehicle is used as the speed data corresponding to the radar data. The target vehicle's location is determined by the distance to the target vehicle in the radar data. The corresponding position coordinates of the target vehicle's radar data are obtained as the vehicle position data corresponding to the radar data. Based on the radar data, the temporal changes in the position of the same target vehicle constitute the radar trajectory data of the same target vehicle. The millimeter-wave radar acquisition frequency is 150Hz.

[0061] Furthermore, by using a high-definition camera to capture video data of a vehicle in motion, and employing target detection and tracking algorithms, the vehicle's position and unique identification information of the tracked target can be obtained. The vehicle's position coordinates within the video frame image of the target vehicle in the video data can be identified and obtained, thereby obtaining the vehicle position data corresponding to the target vehicle's video data. It should be understood that the vehicle position coordinates corresponding to the radar data and video data should be expressed in the same coordinate system.

[0062] Within a time period, the vehicle's position data at the initial and final moments of the video data can be used to determine the distance traveled by the target vehicle within that time period. The ratio of the distance traveled to the time period can be used to determine the target vehicle's speed. Based on the video data, the position of the same target vehicle in the video frame changes over time, forming the video trajectory data for the same target vehicle. The camera captures data at a frequency of 60 fps.

[0063] It should be noted that vehicle position data represents the position coordinates of the corresponding vehicle in the same coordinate system, where the vertical axis of the coordinate system is parallel to the vehicle's direction of travel. In this embodiment, the length of a time period is 10 seconds. Implementers can set this according to the specific implementation scenario. Adjacent time intervals are the same. This embodiment uses a single time period as an example for illustration. It is understandable that due to the different acquisition frequencies of the millimeter-wave radar and camera, the number of vehicles monitored in the same time period may vary.

[0064] Step S200, based on the speed data difference between the target vehicle and the surrounding vehicles in the longitudinal direction and the vehicle position data difference between the target vehicle and the surrounding vehicles in the lateral direction corresponding to each driving data, respectively obtain the lateral risk characteristic index and the longitudinal risk characteristic index corresponding to each driving data.

[0065] Considering that vehicles on highways are not isolated and that other vehicles around the target vehicle may affect its movement, the system can quantify the potential risk characteristics of the target vehicle by analyzing the characteristics of the surrounding vehicles. Risk characteristics are analyzed separately for radar data and video data, and the matching relationship between radar data and video data is derived by combining the similarities and differences in their trajectory distribution. Risk characteristics provide an additional dimension of information for radar and video data matching, allowing for more accurate identification of radar and video data for the same target.

[0066] Based on this, this embodiment analyzes the possible risk characteristics of radar data and video data from two aspects. First, considering that there may be other vehicles driving around the vehicle on the highway, which may have a risk impact on the target vehicle, the vehicles that may have a risk impact are divided into the characteristic performance of the horizontal dimension and the vertical dimension for analysis. In this embodiment, any one type of driving data is used as an example for explanation, that is, radar data or video data is used as an example for explanation. Figure 2 As shown, the method for obtaining the horizontal risk characteristic index and the vertical risk characteristic index can be implemented by step S201 and step S202.

[0067] Step S201 , obtaining a longitudinal risk characteristic index corresponding to the driving data based on a difference between speed data of a target vehicle corresponding to the driving data within a time period and speed data corresponding to adjacent vehicles in the same lane as the target vehicle.

[0068] Specifically, when vehicles are traveling in the same lane and are located adjacent to each other, with surrounding vehicles traveling in the same direction as the target vehicle, the speed differences between these vehicles can directly reflect the risk of rear-end collision. Specifically, when the leading vehicle's speed is lower than the target vehicle's, the greater the speed difference, the higher the target vehicle's rear-end collision risk. When the trailing vehicle's speed is higher than the target vehicle's, the greater the speed difference, the higher the target vehicle's rear-end collision risk. Based on this analysis, the target vehicle's longitudinal risk characteristics due to surrounding vehicles are quantified based on the speed differences between the target vehicle and the vehicles traveling in the same direction as the target vehicle, as identified by the driving data corresponding to a time period.

[0069] like Figure 3As shown, in this embodiment, the method for obtaining the longitudinal risk characteristic index can be implemented by steps S2011 to S2013.

[0070] Step S2011: The preceding vehicle in the same lane and the driving direction of the target vehicle corresponding to the driving data is used as the preceding reference vehicle, and the following vehicle in the same lane and the driving direction of the target vehicle corresponding to the driving data is used as the following reference vehicle.

[0071] It is understood that within a time period, a set of radar data or video data corresponds to a single target, namely, a target vehicle. Considering that closely spaced vehicles pose a mutual risk, this embodiment captures adjacent vehicles within the same video-monitorable area. Furthermore, in the direction of travel of the target vehicle, the vehicle immediately preceding the target vehicle is designated as the front reference vehicle, and the vehicle immediately following the target vehicle is designated as the rear reference vehicle. The front and rear reference vehicles represent vehicles that are located close to the target vehicle and may be at risk of rear-end collision or being rear-ended.

[0072] Step S2012: For any time period, a normalized value of the difference ratio between the speed data of the target vehicle and the previous reference vehicle in the same time period is calculated to obtain a forward risk characteristic index of the target vehicle's driving data corresponding to the current time period.

[0073] It will be appreciated that in this embodiment, a target vehicle corresponds to a speed representation value, or speed data, within a time period, representing the overall speed information of the target vehicle during that time period. The speed difference between the target vehicle and the preceding reference vehicle is then analyzed to quantify the risk of the target vehicle being affected by the preceding reference vehicle.

[0074] As a specific example, taking the target vehicle corresponding to the radar data in the t-th time period as an example, the forward risk characteristic index can be expressed as: ,in, represents the forward risk characteristic index of the target vehicle corresponding to the radar data in the t-th time period, Indicates the speed data of the target vehicle corresponding to the radar data in the t-th time period, represents the speed data of the preceding reference vehicle of the target vehicle corresponding to the radar data in the t-th time period, Indicates the minimum speed limit of the current highway section. is the normalization function.

[0075] Reflects the speed difference between the target vehicle and the previous reference vehicle in the same time period. It reflects the proportion of the speed difference between the two. The greater the speed difference between the target vehicle and the front reference vehicle, the faster the target vehicle is traveling and the slower the front reference vehicle is traveling. The greater the risk of rear-end collision for the target vehicle, the greater the degree of forward risk at this time, that is, the larger the value of the forward risk characteristic index.

[0076] Step S2013, calculate the normalized value of the difference ratio between the speed data of the reference vehicle and the target vehicle in the same time period to obtain the backward risk characteristic index of the driving data of the target vehicle corresponding to the current time period; the longitudinal risk characteristic index includes a forward risk characteristic index and a backward risk characteristic index.

[0077] Based on the same reasoning, the risk level of the target vehicle being affected by the following reference vehicle is quantified by analyzing the speed difference between the target vehicle and the following reference vehicle. As a specific example, taking the target vehicle corresponding to the radar data in the tth time period as an example, the backward risk characteristic index can be expressed as: ,in, It represents the backward risk characteristic index of the target vehicle corresponding to the radar data in the t-th time period, Indicates the speed data of the target vehicle corresponding to the radar data in the t-th time period, represents the speed data of the reference vehicle behind the target vehicle corresponding to the radar data in the t-th time period, Indicates the minimum speed limit of the current highway section. is the normalization function.

[0078] It reflects the speed difference between the target vehicle and the reference vehicle in the same time period. It reflects the proportion of the speed difference between the two. The greater the speed difference between the target vehicle and the rear reference vehicle, the faster the rear reference vehicle is and the slower the target vehicle is. The greater the risk of the target vehicle being rear-ended, the greater the degree of rearward risk at this time, that is, the larger the value of the rearward risk characteristic index.

[0079] Furthermore, in the longitudinal dimension, the risk impact of surrounding vehicles includes both the vehicles before and after the target vehicle. The comprehensive risk situation of various situations can more comprehensively analyze the degree of risk impact that the target vehicle may be subject to. It should be noted that if there is no front reference vehicle around the target vehicle, the analysis of the forward risk characteristic index will not be performed. Similarly, if there is no rear reference vehicle around the target vehicle, the analysis of the subsequent risk characteristic index will not be performed. It should be understood that if there is no front reference vehicle or rear reference vehicle around the target vehicle, it means that there are no vehicles with similar driving trajectories around the target vehicle, and thus there is no error. Therefore, this embodiment only performs feature analysis for the situation where there are other vehicles around the target vehicle.

[0080] Step S202: Analyze the distance change between the target vehicle and other vehicles based on the difference between the vehicle position data of the target vehicle at each moment in the time period corresponding to the driving data and the vehicle position data corresponding to the vehicle in the adjacent lane of the target vehicle, and obtain the lateral risk characteristic index corresponding to the driving data.

[0081] The target vehicle's lateral risk is impacted by lane changes by vehicles in adjacent lanes. If a vehicle in the lanes to the left or right of the target vehicle intends to change lanes and executes a lane change, the lane-changing vehicle may collide with the target vehicle if it fails to maintain a safe distance from the target vehicle or fails to accurately assess the speed difference between the target vehicle and the lane-changing vehicle. When the lateral distance between the target vehicle and vehicles in adjacent lanes changes, and the smaller the difference, the more likely the surrounding vehicle is changing lanes. The closer the speed of the surrounding vehicles to the target vehicle, the greater the risk to the target vehicle from the lane change.

[0082] Based on this analysis, by analyzing the lateral position difference between the target vehicle and the vehicles in the surrounding adjacent lanes, combined with the speed difference, the lateral risk characteristic index of the target vehicle corresponding to the driving data is quantified. Figure 4 As shown, the method for obtaining the horizontal risk characteristic indicator in this embodiment can be implemented by steps S2021 to S2023.

[0083] In step S2021, the vehicle closest to the target vehicle in the left lane adjacent to the target vehicle corresponding to the driving data is used as the left reference vehicle, and the vehicle closest to the target vehicle in the right lane adjacent to the target vehicle corresponding to the driving data is used as the right reference vehicle.

[0084] It's understandable that within a given time period, both radar data and video data correspond to a single target vehicle. The target vehicle's lane may have adjacent lanes to its left and right. Similarly, given that vehicles located close together pose a risk to each other, this embodiment uses the vehicles closest to the target vehicle in the lanes adjacent to its left and right within the same video surveillance area for reference analysis. In other words, the left and right reference vehicles represent vehicles located close to the target vehicle that may pose a risk of lane change.

[0085] Step S2022, based on the lateral difference between the corresponding vehicle position data of the target vehicle and the left reference vehicle in the same time period, combined with the corresponding speed data difference between the target vehicle and the left reference vehicle in the same time period, obtain the left-bound risk characteristic index of the target vehicle corresponding to the driving data.

[0086] In this embodiment, each vehicle corresponds to a piece of vehicle position data at each moment in a time period. Therefore, when analyzing the lateral position difference between two different vehicles, it is necessary to analyze the vehicle position data at the same moment separately. In this embodiment, the vehicle position data is the coordinate data of the vehicle's location. The difference between the horizontal coordinates corresponding to the vehicle position data of different vehicles can be used to quantify the change in the lateral distance between the two vehicles.

[0087] Specifically, for any time period, based on the difference between the horizontal coordinates of the vehicle position data of the target vehicle and the left reference vehicle at each moment in the time period, the lateral difference distance between the target vehicle and the left reference vehicle at each moment in the time period is determined; the lateral difference distances at all moments in the time period are accumulated and summed and negatively correlated to obtain a first difference coefficient; the difference between the speed data of the target vehicle and the left reference vehicle in the time period is negatively correlated to obtain a second difference coefficient; the normalized value of the product of the first difference coefficient and the second difference coefficient is used as the left-bound risk characteristic indicator.

[0088] As a specific example, taking the target vehicle corresponding to the radar data in the t-th time period as an example, the left-bound risk characteristic index can be expressed as:

[0089]

[0090] in, Indicates the left-bound risk characteristic index of the target vehicle corresponding to the radar data in the t-th time period, Indicates the horizontal coordinate of the vehicle position data of the target vehicle at the i-th moment corresponding to the radar data in the t-th time period, The horizontal coordinate of the vehicle position data of the left reference vehicle of the target vehicle corresponding to the radar data in the t-th time period at the i-th moment, N represents the total number of moments included in the t-th time period, Indicates the speed data of the target vehicle corresponding to the radar data in the t-th time period, Indicates the speed data of the left reference vehicle of the target vehicle corresponding to the radar data in the t-th time period, is the normalization function.

[0091] is the lateral difference distance between the target vehicle and the left reference vehicle at the i-th moment in the t-th time period, reflecting the lateral distance performance of the target vehicle and the left reference vehicle at the same time in the same time period. The smaller the comprehensive performance of the difference, the closer they are to each other, indicating that the vehicles have changed lanes. The first difference coefficient The larger the value of , the greater the risk to the corresponding target vehicle.

[0092] Indicates the difference between the speed data of the target vehicle and the speed data of the left reference vehicle. The greater the difference between the ratio of the speed data of the two and 1, the greater the speed difference between the two. This is the second difference coefficient. The denominator is added with 1 to prevent the denominator from being 0 and affecting the calculation.

[0093] When the lateral distance between the target vehicle and the left reference vehicle is smaller, the speed data difference between the target vehicle and the left reference vehicle is smaller, indicating that the degree of consistency between the target vehicle and the left reference vehicle is greater. At this time, there is a vehicle lane change behavior, and the risk of affecting the target vehicle is greater, and the corresponding left-bound risk characteristic index value is larger.

[0094] Step S2023, based on the lateral difference between the corresponding vehicle position data of the target vehicle and the right reference vehicle in the same time period, combined with the corresponding speed data difference between the target vehicle and the right reference vehicle in the same time period, obtain the right-hand risk characteristic index of the target vehicle corresponding to the driving data; the said lateral risk characteristic index includes a left-hand risk characteristic index and a right-hand risk characteristic index.

[0095] For the same reasoning, vehicles in the lane adjacent to the right of the target vehicle that may be changing lanes are analyzed. Using the same method as for the left reference vehicle, the difference between the horizontal coordinates of the target vehicle's and the right reference vehicle's vehicle position data at each moment in the same time period is calculated as the lateral difference distance between the target vehicle and the right reference vehicle at each moment in the same time period.

[0096] Specifically, the longitudinal difference distances at all moments in the time period are accumulated and summed up and negatively correlated to obtain the third difference coefficient; the difference between the speed data of the target vehicle and the right reference vehicle in the time period is negatively correlated to obtain the fourth difference coefficient; the normalized value of the product of the third difference coefficient and the fourth difference coefficient is used as the right-bound risk characteristic indicator.

[0097] As a specific example, taking the target vehicle corresponding to the radar data in the t-th time period as an example, the right-bound risk characteristic index can be expressed as:

[0098]

[0099] in, Indicates the rightward risk characteristic index of the target vehicle corresponding to the radar data in the t-th time period, Indicates the horizontal coordinate of the vehicle position data of the target vehicle at the i-th moment corresponding to the radar data in the t-th time period, The horizontal coordinate of the vehicle position data of the right reference vehicle of the target vehicle corresponding to the radar data in the t-th time period at the i-th moment, N represents the total number of moments included in the t-th time period, Indicates the speed data of the target vehicle corresponding to the radar data in the t-th time period, represents the speed data of the right reference vehicle of the target vehicle corresponding to the radar data in the t-th time period, is the normalization function.

[0100] The lateral difference distance between the target vehicle and the right reference vehicle at the i-th moment in the t-th time period reflects the lateral distance performance between the target vehicle and the right reference vehicle at the same moment in the same time period. The smaller the comprehensive performance of the difference, the closer they are to each other, indicating that the vehicles have changed lanes. The third difference coefficient The larger the value of , the greater the risk to the corresponding target vehicle.

[0101] Indicates the difference between the speed data of the target vehicle and the speed data of the right reference vehicle. The greater the difference between the ratio of the speed data of the two and 1, the greater the speed data difference between the two. This is the fourth difference coefficient. The denominator is added with 1 to prevent the denominator from being 0 and affecting the calculation.

[0102] When the lateral distance between the target vehicle and the right reference vehicle is smaller, the speed data difference between the target vehicle and the right reference vehicle is smaller, indicating that the degree of consistency between the target vehicle and the right reference vehicle is greater. At this time, there is a vehicle lane change behavior, and the risk of affecting the target vehicle is greater, and the corresponding right-bound risk characteristic index value is larger.

[0103] Furthermore, the risk of lane-changing vehicles potentially affecting both the left and right sides of the target vehicle is significant. To more accurately characterize the risk factor for the target vehicle, the lateral impact on the target vehicle primarily includes the left and right directions. Based on this, the lateral risk characteristic index includes a left-facing risk characteristic index and a right-facing risk characteristic index. It should be understood that, for similar reasons to the longitudinal risk characteristic index, if no reference vehicles exist on the left or right sides of the target vehicle, the corresponding risk characteristic analysis is not performed.

[0104] Thus, the longitudinal risk characteristic index represents the longitudinal risk impact of rear-end collisions and being rear-ended for a target vehicle corresponding to a type of driving data within a time period. The lateral risk characteristic index represents the lateral risk impact of lane-changing vehicles for a target vehicle corresponding to a type of driving data within a time period. Using the same method, risk quantification results can be obtained for both radar data and video data, providing a data foundation for subsequent comprehensive characteristic analysis.

[0105] Step S300: Obtain a driving style characteristic index corresponding to each type of driving data based on a trend in the displacement of the target vehicle corresponding to each type of driving data under lane-changing driving behavior in historical data; and determine a characteristic matching index for each type of driving data by combining the lateral risk characteristic index, the longitudinal risk characteristic index, and the driving style characteristic index.

[0106] Conduct a personalized assessment of the target vehicle's driver's driving behavior, evaluate the target vehicle's possible risk resistance limit by analyzing the target vehicle's driving habits, and combine this with the quantitative results of the target vehicle's risk level to more accurately reflect the actual risk level faced by the target vehicle, thereby improving the accuracy of the risk feature performance results.

[0107] The target vehicle's risk tolerance limit is closely related to its driving behavior habits. An aggressive driving style means that the driver is more frequent in driving operations and is more skilled in vehicle control. Since they are often in driving scenarios that require quick decision-making, they may respond more quickly to changes in the surrounding traffic environment. Therefore, for vehicles with an aggressive driving style, their risk tolerance limit is stronger. Conversely, for vehicles with a cautious driving style, their risk tolerance limit is weaker.

[0108] Considering that in a highway driving environment, target vehicles generally stay on the main road under normal circumstances and do not frequently change lanes unless there are special circumstances, vehicle drivers' lane changes occur in two sections of the highway. The first section is the normal section of the highway, where drivers may change lanes for behaviors such as overtaking. The second section is the on-ramp section of the highway, where vehicles inevitably change lanes due to the need to merge into and exit the main road. Therefore, this embodiment uses the lane change characteristics of vehicles on ramps as an example to evaluate vehicle driving habits and quantify the corresponding vehicle's risk tolerance limit.

[0109] Specifically, a vehicle's lane-changing driving behavior on a ramp includes entering the ramp and exiting the ramp, where exiting the ramp is when the vehicle merges into the main road. It should be noted that in order to more accurately characterize the driving behavior habits of vehicle drivers, this embodiment obtains the vehicle displacement data of the vehicle's lane-changing behavior within a year prior to the current time period. That is, the vehicle displacement data of the radar data and video data at each moment during the lane-changing driving behavior on the ramp within a year. One lane-changing driving behavior corresponds to a lane-changing driving time period, and includes an entering ramp sub-time period and an exiting ramp sub-time period. Each moment corresponds to the vehicle position data of a vehicle, reflecting the position change of the vehicle during the lane-changing process. Based on this, the driving style characteristic index corresponding to each driving data is obtained based on the difference in the vehicle displacement data of the target vehicle on the entering ramp and the difference in the vehicle displacement data of the target vehicle on the exiting ramp corresponding to each driving data.

[0110] In this embodiment, any type of driving data in any time period is used as an example for explanation. Figure 5 As shown, the method for obtaining the driving style characteristic index can be implemented by steps S301 to S303.

[0111] Step S301: Under any lane change driving behavior on the ramp, the difference between the horizontal coordinates of the vehicle displacement data at each moment when the target vehicle enters the ramp and the adjacent next moment corresponding to the driving data is recorded as the entry displacement data; the ratio of the cumulative sum of all the entry displacement data of the target vehicle entering the ramp to the total duration of the ramp entry is calculated to obtain the first displacement characteristic coefficient.

[0112] In this embodiment, taking radar data as an example, for any lane change driving behavior at a ramp, the horizontal coordinate changes of the vehicle displacement data at adjacent moments within the sub-time period when the target vehicle enters the ramp corresponding to the radar data are calculated, reflecting the changes in the vehicle's lateral displacement during the process of the target vehicle entering the ramp.

[0113] As a specific example, the lane change driving behavior of the target vehicle at the kth ramp corresponding to the radar data is described. The first displacement characteristic coefficient corresponding to the lane change driving behavior of the target vehicle at the kth ramp can be expressed as:

[0114] ,in, Indicates the first displacement characteristic coefficient corresponding to the lane-changing driving behavior of the target vehicle at the kth ramp corresponding to the radar data, represents the total duration of the ramp-entry sub-period of the lane-changing driving behavior of the target vehicle at the kth ramp corresponding to the radar data, The horizontal coordinate of the vehicle displacement data of the target vehicle at the kth ramp at the time of the ramp entry sub-time period corresponding to the radar data is r+1, The horizontal coordinate of the vehicle displacement data at the rth moment of the ramp-entering sub-time period of the lane-changing driving behavior of the target vehicle at the kth ramp corresponding to the radar data.

[0115] This reflects the change in lateral distance between the target vehicle and the ramp at two adjacent moments. The ratio of the cumulative displacement change to the ramp entry duration can be used to quantify the displacement change rate during ramp entry. A larger value for the first displacement characteristic coefficient indicates a faster ramp entry behavior during the current lane change.

[0116] In step S302, the difference between the horizontal coordinates of the vehicle displacement data at each moment when the target vehicle exits the ramp and the next adjacent moment is recorded as the exit displacement data, and the ratio of the cumulative sum of all the exit displacement data of the target vehicle exiting the ramp to the total duration of the exit ramp is calculated to obtain a second displacement characteristic coefficient.

[0117] In this embodiment, taking radar data as an example, for any lane change driving behavior at the ramp, the horizontal coordinate change of the vehicle displacement data at adjacent moments within the sub-time period when the target vehicle exits the ramp corresponding to the radar data is calculated, reflecting the change in the vehicle's lateral displacement during the process of the target vehicle exiting the ramp.

[0118] As a specific example, the lane change driving behavior of the target vehicle at the kth ramp corresponding to the radar data is described. The second displacement characteristic coefficient corresponding to the lane change driving behavior of the target vehicle at the kth ramp can be expressed as:

[0119] ,in, Indicates the second displacement characteristic coefficient corresponding to the lane-changing driving behavior of the target vehicle at the kth ramp corresponding to the radar data, represents the total duration of the off-ramp sub-period of the lane-changing driving behavior of the target vehicle at the k-th ramp corresponding to the radar data, The horizontal coordinate of the vehicle displacement data of the target vehicle at the kth ramp exit sub-time period corresponding to the radar data, The horizontal coordinate of the vehicle displacement data at the rth moment of the off-ramp sub-period of the lane-changing driving behavior of the target vehicle at the kth ramp corresponding to the radar data.

[0120] This reflects the change in lateral distance between the target vehicle and the vehicle at two adjacent moments during the off-ramp phase. The ratio of the cumulative displacement change to the off-ramp duration can be used to quantify the displacement change rate during the off-ramp phase. A larger value for the second displacement characteristic coefficient indicates a faster off-ramp behavior change during the current lane change.

[0121] In step S303, a normalized result of the cumulative sum of the first displacement characteristic coefficient and the second displacement characteristic coefficient is used as the driving characteristic factor of the current driving behavior; and the average value of the driving characteristic factors of the lane change driving behavior of the target vehicle on all secondary ramps is calculated to obtain a driving style characteristic index of the target vehicle corresponding to the driving data.

[0122] The first displacement characteristic coefficient and the second displacement characteristic coefficient respectively represent the speed of change of the on-ramp behavior characteristic and the speed of change of the off-ramp behavior characteristic of the target vehicle corresponding to the lane change behavior characteristics of the radar data or video data under a single lane change behavior characteristic. Combining these two aspects with the characteristic information of all lane change driving behaviors can more comprehensively reflect the driving behavior characteristics of the vehicle driver.

[0123] Specifically, the driving characteristic factor of the lane change driving behavior of the target vehicle at the kth ramp corresponding to the radar data is It can be expressed as ,in, Indicates the first displacement characteristic coefficient corresponding to the lane-changing driving behavior of the target vehicle at the kth ramp corresponding to the radar data, Indicates the second displacement characteristic coefficient corresponding to the lane-changing driving behavior of the target vehicle at the kth ramp corresponding to the radar data, is the normalization function.

[0124] Next, the mean driving characteristic factor for all lane-changing behaviors of the target vehicle corresponding to the radar data is calculated to obtain a driving style characteristic index. This index represents the rate of change in lane-changing behaviors in the historical driving data of the target vehicle corresponding to the radar data. In other words, the larger the value of the driving style characteristic index, the more responsive the driver of the corresponding target vehicle is to emergencies and the stronger the corresponding risk tolerance limit. Using the same calculation method, the driving style characteristic index of the target vehicle corresponding to the video data also represents the characteristic performance of the target vehicle in its historical lane-changing behaviors, reflecting the risk tolerance limit of the target vehicle corresponding to the video data. The risk tolerance limit corresponding to the target vehicle can be used to make further corrections when analyzing the risk characteristic level of the target vehicle in real-time over a period of time.

[0125] Furthermore, the characteristic matching index of each type of driving data is determined by combining the lateral risk characteristic index, the longitudinal risk characteristic index, and the driving style characteristic index. Specifically, for any type of driving data in any time period, the characteristic matching index is calculated as follows:

[0126]

[0127] in, Represents the feature matching index of the target vehicle corresponding to the driving data in the t-th time period, represents the forward risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, represents the backward risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, represents the left-bound risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, represents the right-bound risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, is the preset adjustment coefficient, , Indicates the driving style characteristic index of the target vehicle corresponding to the driving data.

[0128] It should be noted that the adjustment coefficient The value of is the same as the number of risk characteristic indicators, that is, in this embodiment, the risk impact of the target vehicle in four directions is considered, and the corresponding risk resistance limits are evaluated and corrected and adjusted respectively. When the risk resistance limit of the target vehicle is stronger, the risk impact on the corresponding target vehicle is smaller. The feature matching index finally obtained can more comprehensively and accurately characterize the risk characteristic performance corresponding to the radar data or video data.

[0129] It can be understood that this embodiment uses radar data as an example to introduce the specific method of obtaining the feature matching index of the target vehicle corresponding to this type of driving data. The feature matching index of the target vehicle corresponding to the video data can be obtained according to the same method, which will not be introduced in detail here.

[0130] In step S400, based on the similarity between the feature matching indicators corresponding to the radar data and the feature matching indicators corresponding to the video data, combined with the trajectory differences between the radar data and the video data, data matching is performed on the radar data and the video data, and radar-visual data fusion is performed based on the matching results.

[0131] In this embodiment, by calculating the similarity between the trajectory information corresponding to the radar data and the video data, combined with the degree of expression of additional feature dimensions, that is, the difference between the vehicle risk feature expression of the radar data and the vehicle risk feature expression of the video data, a similarity measurement result of the radar data is constructed, and finally the matching of the radar data and the video data is achieved.

[0132] First, it should be noted that for any time period, radar trajectory data can be obtained from radar data, and video trajectory data can be obtained from video data. The specific acquisition methods are well-known technologies and will not be described in detail here. The DTW data between the radar trajectory data corresponding to the radar data and the video trajectory data corresponding to the video data is calculated to obtain a first difference; the absolute value of the difference between the feature matching index corresponding to the radar data and the feature matching index corresponding to the video data is calculated to obtain a second difference; and a negative correlation is performed on the product of the first difference and the second difference to obtain a similarity metric for the radar data and the video data.

[0133] As a specific example, taking radar data m and video data n as an example in the t-th time period, the similarity measurement index between radar data m and video data n can be expressed as:

[0134]

[0135] in, represents the similarity metric between radar data m and video data n in the t-th time period, Represents the trajectory data of radar data m in the t-th time period, that is, the radar trajectory data; Represents the trajectory data of video data n in the t-th time period, that is, the video trajectory data; Indicates the DTW distance between the radar trajectory data and the video trajectory data, that is, the first difference; Represents the feature matching index of radar data m corresponding to the t-th time period, Represents the feature matching index of video data n corresponding to the t-th time period.

[0136] The first difference reflects the difference in trajectory information between radar data and video data in the same time period. The second difference reflects the difference in risk characteristics between radar data and video data within the same time period. The larger the first and second differences are, the less similar the trajectories between the radar data and video data are, and the more inconsistent the vehicle risk coefficients are. The smaller the possibility that the two represent the same target vehicle, the smaller the corresponding similarity is.

[0137] The radar data and video data are then matched using a similarity metric between them. Specifically, for any target vehicle's radar data, if the similarity metric between the radar data and the video data exceeds a preset similarity threshold, the video data corresponding to the maximum similarity between the radar data and all video data that meet the threshold is obtained as the matching result.

[0138] In this embodiment, the similarity threshold is set to 0.5, and implementers can set this threshold based on specific implementation scenarios. The larger the similarity metric value, the more likely the radar data and video data are representing the same target vehicle. Therefore, the video data corresponding to the maximum similarity metric value is considered the matching radar data within the same time period. During that time period, the radar data and video data represent the same target vehicle.

[0139] Finally, radar data and video data can be matched using similarity metrics, and the matching results can be used to fuse the radar data and video data corresponding to the same object. It can be understood that fusing radar data and video data here refers to preprocessing and extracting features from the radar data and video data, respectively, to achieve the goal of matching the radar data and video data. This allows for more accurate real-time monitoring of vehicle safety by taking into account multi-dimensional information during vehicle monitoring, avoiding errors caused by other vehicles with similar trajectories within a relatively close range of the target vehicle, and effectively improving the accuracy of the association between radar and video targets. It should be noted that matching radar data and video data is a well-known technique and will not be further elaborated upon here.

[0140] In summary, this embodiment evaluates the risk of a vehicle using radar data and video data, constructs a similarity matrix for the radar and video data based on the risk factor and trajectory data, and ultimately achieves the optimal matching relationship between radar and video data. The risk factor provides an additional dimension of information for radar and video data matching. By comparing the vehicle risk factor of radar data with the vehicle risk factor of video data, the radar and video data of the same vehicle can be more accurately determined, reducing the probability of mismatches and improving the accuracy of radar and video data fusion, providing a solid data foundation for the intelligent supervision platform.

[0141] like Figure 6 As shown, one embodiment of the present invention provides a highway intelligent supervision platform, which is used to implement the steps of a radar and visual data fusion method of the highway intelligent supervision platform. The highway intelligent supervision platform includes:

[0142] The data acquisition module includes a millimeter-wave radar and a camera. The millimeter-wave radar is used to obtain radar data at each time period of the vehicle's driving on the highway, and the camera is used to obtain video data at each time period of the vehicle's driving on the highway.

[0143] A data preprocessing module, configured to obtain the driving data of the time period based on the driving data and obtain speed data and vehicle position data corresponding to each type of driving data;

[0144] a risk characteristic analysis module for respectively obtaining a lateral risk characteristic index and a longitudinal risk characteristic index corresponding to each type of driving data based on the speed data difference between the target vehicle and surrounding vehicles in the longitudinal direction, and the vehicle position data difference between the target vehicle and surrounding vehicles in the lateral direction;

[0145] a matching index determination module for obtaining a driving style characteristic index corresponding to each type of driving data based on a trend in displacement of the target vehicle under lane-changing driving behavior in historical data corresponding to each type of driving data; and determining a characteristic matching index for each type of driving data by combining the lateral risk characteristic index, the longitudinal risk characteristic index, and the driving style characteristic index;

[0146] The data fusion module is used to match radar data and video data based on the similarity between the feature matching indicators corresponding to the radar data and the feature matching indicators corresponding to the video data, combined with the trajectory differences between the radar data and the video data, and perform radar and video data fusion based on the matching results.

[0147] Since an embodiment of a radar and visual data fusion method for an intelligent highway monitoring platform has been described in detail, it will not be further introduced here.

[0148] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A radar data fusion method for a highway intelligent supervision platform, characterized in that: The method comprises the following steps: Acquire driving data of the vehicle in each time period during the driving process on the highway, including radar data and video data; obtain speed data and vehicle position data corresponding to each type of driving data based on the driving data acquired in the time period; According to the speed data difference between the target vehicle and the surrounding vehicles in the longitudinal direction, and the vehicle position data difference between the target vehicle and the surrounding vehicles in the lateral direction corresponding to each driving data, a lateral risk characteristic index and a longitudinal risk characteristic index corresponding to each driving data are obtained respectively; Obtaining a driving style characteristic index corresponding to each type of driving data based on a change trend in vehicle displacement of the target vehicle under lane-changing driving behavior in historical data corresponding to each type of driving data; and determining a characteristic matching index for each type of driving data by combining the lateral risk characteristic index, the longitudinal risk characteristic index, and the driving style characteristic index. According to the similarity between the feature matching indicators corresponding to the radar data and the feature matching indicators corresponding to the video data, combined with the trajectory differences between the radar data and the video data, data matching is performed on the radar data and the video data is then fused based on the matching results.

2. The radar data fusion method for a highway intelligent supervision platform according to claim 1 is characterized in that: The lateral risk characteristic index and the longitudinal risk characteristic index corresponding to each driving data are obtained based on the speed data difference between the target vehicle and the surrounding vehicles in the longitudinal direction, and the vehicle position data difference between the target vehicle and the surrounding vehicles in the lateral direction. Specifically, the lateral risk characteristic index and the longitudinal risk characteristic index corresponding to each driving data are obtained. For any type of driving data, the longitudinal risk characteristic index corresponding to the driving data is obtained based on the difference between the speed data of the target vehicle within the time period corresponding to the driving data and the speed data of the adjacent vehicles in the same lane as the target vehicle; Based on the difference between the vehicle position data of the target vehicle at each moment in the time period corresponding to the driving data and the vehicle position data corresponding to the vehicles in the adjacent lanes of the target vehicle, the distance changes between the target vehicle and other vehicles are analyzed to obtain the lateral risk characteristic index corresponding to the driving data.

3. The radar data fusion method for a highway intelligent supervision platform according to claim 2 is characterized in that: The longitudinal risk characteristic index corresponding to the driving data is obtained based on the difference between the speed data of the target vehicle within the time period corresponding to the driving data and the speed data corresponding to adjacent vehicles in the same lane as the target vehicle, specifically including: The preceding vehicle in the same lane and in the driving direction of the target vehicle corresponding to the driving data is used as the preceding reference vehicle, and the following vehicle in the same lane and in the driving direction of the target vehicle corresponding to the driving data is used as the following reference vehicle; Each vehicle corresponds to a speed data in a time period. For any time period, the normalized value of the difference between the speed data of the target vehicle and the previous reference vehicle in the same time period is calculated to obtain the forward risk characteristic index of the target vehicle's driving data corresponding to the current time period. The normalized value of the difference between the speed data of the reference vehicle and the target vehicle in the same time period is calculated to obtain the backward risk characteristic index of the target vehicle's driving data corresponding to the current time period; The longitudinal risk characteristic indicators include forward risk characteristic indicators and backward risk characteristic indicators.

4. The radar data fusion method for a highway intelligent supervision platform according to claim 3 is characterized in that: The method of analyzing the distance change between the target vehicle and other vehicles based on the difference between the vehicle position data of the target vehicle at each moment in the time period corresponding to the driving data and the vehicle position data corresponding to the vehicle in the adjacent lane of the target vehicle to obtain the lateral risk characteristic index corresponding to the driving data specifically includes: The vehicle closest to the target vehicle in the left lane adjacent to the target vehicle corresponding to the driving data is used as the left reference vehicle, and the vehicle closest to the target vehicle in the right lane adjacent to the target vehicle corresponding to the driving data is used as the right reference vehicle; each vehicle corresponds to a vehicle position data at each moment in a time period; the lateral risk characteristic index includes a left-hand risk characteristic index and a right-hand risk characteristic index; The method for obtaining the leftward risk characteristic indicator includes: For any time period, based on the difference between the horizontal coordinates of the vehicle position data of the target vehicle and the left reference vehicle at each moment in the time period, the lateral difference distance between the target vehicle and the left reference vehicle at each moment in the time period is determined; the lateral difference distances at all moments in the time period are accumulated and summed and negatively correlated to obtain a first difference coefficient; the difference between the speed data of the target vehicle and the left reference vehicle in the time period is negatively correlated to obtain a second difference coefficient; the normalized value of the product of the first difference coefficient and the second difference coefficient is used as the left-bound risk characteristic indicator.

5. The radar data fusion method for a highway intelligent supervision platform according to claim 4 is characterized in that: The method for obtaining the rightward risk characteristic indicator includes: The longitudinal difference distances at all moments in the time period are accumulated and summed up and negatively correlated to obtain the third difference coefficient; the speed data between the target vehicle and the right reference vehicle in the time period are negatively correlated to obtain the fourth difference coefficient; the normalized value of the product of the third difference coefficient and the fourth difference coefficient is used as the right-bound risk characteristic indicator.

6. The radar data fusion method for a highway intelligent supervision platform according to claim 5 is characterized in that: The driving style characteristic index corresponding to each driving data is obtained based on the change trend of the vehicle displacement of the target vehicle under the lane change driving behavior in the historical data corresponding to each driving data, specifically including: The lane-changing driving behavior of a vehicle on a ramp includes entering the ramp and exiting the ramp. Based on the difference in vehicle displacement data of the target vehicle when entering the ramp and the difference in vehicle displacement data of the target vehicle when exiting the ramp corresponding to each type of driving data, the driving style characteristic index corresponding to each type of driving data is obtained.

7. The radar data fusion method for a highway intelligent supervision platform according to claim 6 is characterized in that: The driving style characteristic index corresponding to each driving data is obtained based on the difference in vehicle displacement data of the target vehicle at the on-ramp and the difference in vehicle displacement data of the target vehicle at the off-ramp corresponding to each driving data, specifically including: For any type of driving data within any time period and any lane change on a ramp, the difference between the horizontal coordinates of the vehicle displacement data at each moment the target vehicle enters the ramp and the next adjacent moment corresponding to the driving data is recorded as the entry displacement data. The first displacement characteristic coefficient is calculated by calculating the ratio of the cumulative sum of all the entry displacement data of the target vehicle entering the ramp and the total duration of the ramp entry. The difference between the horizontal coordinates of the vehicle displacement data at each moment when the target vehicle exits the ramp and the next adjacent moment is recorded as the exit displacement data, and the ratio of the cumulative sum of all the exit displacement data of the target vehicle exiting the ramp to the total duration of the exit ramp is calculated to obtain the second displacement characteristic coefficient; The normalized result of the cumulative sum of the first displacement characteristic coefficient and the second displacement characteristic coefficient is used as the driving characteristic factor of the current driving behavior; the average value of the driving characteristic factors of the lane change driving behavior of the target vehicle on all secondary ramps is calculated to obtain the driving style characteristic index of the target vehicle corresponding to the driving data.

8. The radar data fusion method for a highway intelligent supervision platform according to claim 5 is characterized in that: The step of combining the lateral risk characteristic index, the longitudinal risk characteristic index, and the driving style characteristic index to determine a characteristic matching index for each type of driving data specifically includes: For any type of driving data in any time period, the calculation formula of the feature matching index is: in, Represents the feature matching index of the target vehicle corresponding to the driving data in the t-th time period, represents the forward risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, represents the backward risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, represents the left-bound risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, represents the right-bound risk characteristic index of the target vehicle corresponding to the driving data in the t-th time period, is the preset adjustment coefficient, , Indicates the driving style characteristic index of the target vehicle corresponding to the driving data.

9. The radar data fusion method for a highway intelligent supervision platform according to claim 1 is characterized in that: The data matching of the radar data and the video data is performed based on the similarity between the feature matching index corresponding to the radar data and the feature matching index corresponding to the video data, in combination with the trajectory difference between the radar data and the video data, specifically including: For any time period, radar trajectory data is obtained through radar data, and video trajectory data is obtained through video data; Calculating DTW data between radar trajectory data corresponding to the radar data and video trajectory data corresponding to the video data to obtain a first difference; calculating the absolute value of the difference between the feature matching index corresponding to the radar data and the feature matching index corresponding to the video data to obtain a second difference; performing negative correlation processing on the product of the first difference and the second difference to obtain a similarity measurement index between the radar data and the video data; For the radar data corresponding to any target vehicle, when the similarity metric between the radar data and the video data is greater than a preset similarity threshold, the video data corresponding to the maximum value of the similarity metric between the radar data and all video data that meet the threshold is obtained as the matching result of the radar data and the video data.

10. A highway intelligent supervision platform, characterized in that: The supervision platform is used to implement the steps of the radar data fusion method of a highway intelligent supervision platform according to any one of claims 1 to 9, and the highway intelligent supervision platform includes: The data acquisition module includes a millimeter-wave radar and a camera. The millimeter-wave radar is used to obtain radar data at each time period of the vehicle's driving on the highway, and the camera is used to obtain video data at each time period of the vehicle's driving on the highway. A data preprocessing module, configured to obtain the driving data of the time period based on the driving data and obtain speed data and vehicle position data corresponding to each type of driving data; a risk characteristic analysis module for respectively obtaining a lateral risk characteristic index and a longitudinal risk characteristic index corresponding to each type of driving data based on the speed data difference between the target vehicle and surrounding vehicles in the longitudinal direction, and the vehicle position data difference between the target vehicle and surrounding vehicles in the lateral direction; a matching index determination module for obtaining a driving style characteristic index corresponding to each type of driving data based on a trend in displacement of the target vehicle under lane-changing driving behavior in historical data corresponding to each type of driving data; and determining a characteristic matching index for each type of driving data by combining the lateral risk characteristic index, the longitudinal risk characteristic index, and the driving style characteristic index; The data fusion module is used to match radar data and video data based on the similarity between the feature matching indicators corresponding to the radar data and the feature matching indicators corresponding to the video data, combined with the trajectory differences between the radar data and the video data, and perform radar and video data fusion based on the matching results.

Images