A radar and vision-integrated highway visibility detection method and system

Abstract

The present invention discloses a radar- and vision-fused highway visibility detection method and system. The method comprises the following steps: obtaining radar targets detected by radar and visual targets detected by camera; evaluating the visibility level within each distance interval based on the quantitative relationship between radar targets and visual targets; fusing the radar targets with the visual targets to obtain radar-visible targets; determining the maximum distance at which each radar-visible target can be detected by the camera based on the camera detection results, and then calculating the visibility distance value; and determining and outputting the visibility level based on the visibility level and the visibility distance value. The present invention has the advantages of low cost, high intelligence, and high prediction accuracy.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of highway visibility detection, and in particular to a highway visibility detection method and system integrating radar and vision. Background Art

[0002] Highway visibility is a key indicator of road traffic safety. Low visibility can obstruct the driver's vision, shortening their response time to emergencies and easily leading to traffic accidents. Therefore, accurate measurement of highway visibility is crucial in the transportation sector. Millimeter-wave radar and vision fusion technologies are increasingly being used in the transportation sector, often for traffic incident detection and traffic parameter measurement. Millimeter-wave radar can detect and accurately measure the distance to moving targets within a vehicle, while vision can detect targets of interest within the visibility range. Therefore, utilizing existing radar-vision fusion equipment on highways for highway visibility monitoring is a promising option.

[0003] At present, highway visibility detection methods can be divided into three categories: manual visual inspection, machine inspection and visual inspection, as follows:

[0004] 1. The manual visual inspection method estimates visibility by having staff observe the clarity of the target object. Staff can visually inspect the target object through video surveillance images at the monitoring center. This method can estimate visibility within a large range in a short period of time, but the efficiency is low. Alternatively, staff can observe on the highway. Although this method can continuously output the visibility in the scene, the coverage is very limited. Although the manual visual inspection method is simple to operate, it requires a lot of manpower and is greatly affected by subjective factors. The detection accuracy and consistency are low, and the coverage and work efficiency are difficult to effectively guarantee.

[0005] 2. The machine detection method monitors visibility through visibility detectors and other equipment installed near the road. It is generally based on laser or infrared technology and can directly measure visibility values. The detection results are very accurate. However, the visibility detector is expensive, complex to debug, has a single function, and has a low cost-effectiveness. It cannot be installed densely, and it is difficult to achieve all-round real-time monitoring of road sections.

[0006] 3. The visual inspection method uses surveillance cameras installed on highways to obtain on-site images and analyze the images to determine visibility. However, this method is only used in specific scenarios. Manual target setting and camera calibration are required for each scenario. It has poor universality and portability, and requires a lot of manpower in large-scale applications.

[0007] Existing vision-based visibility detection methods fall into two main categories. One is based on traditional image processing, which requires measuring two key parameters: the visibility value of the target in the image and the distance of the target relative to the detection camera. This method first selects a feature region within the scene and determines its distance from the detection camera through manual measurement or calibration of camera intrinsic and extrinsic parameters. Features of this feature region at different visibility levels are then modeled and stored in a feature library, and a correspondence between the feature library and visibility levels is established. In actual use, the feature region is obtained through automatic detection or by demarcating a fixed area. Features are extracted using a feature modeling method specific to the feature region, compared with the feature library, and the visibility level of the region is determined based on the correspondence between the feature library and visibility levels. Multiple feature regions are selected at different distances, enabling visibility detection at different distances. This method requires manual feature region selection and modeling for each scene, which is labor-intensive and difficult to widely apply.

[0008] Another type of method is based on deep learning. This method first constructs a dataset by collecting foggy highway images from real-world scenarios and annotating the visibility levels. The visibility levels can be determined by manual visual inspection or by obtaining visibility meter results from the same area and at the same time. A deep learning model is then constructed, and the dataset is divided into training, validation, and test sets according to a specific ratio. The deep learning model is trained using the dataset, resulting in a model weight file. Finally, the deep learning model and weight file are deployed on relevant hardware, and real-time video from highway surveillance cameras is captured. The video is decoded into frames, which are fed into the deep learning model to obtain visibility level classification results. This method requires a foggy dataset specifically for visibility detection, which takes a long time to collect, impacting project construction schedules. If the field of view of the cameras used in the real-world scenario differs significantly from that of the cameras used to construct the dataset, the detection results will be significantly inaccurate.

[0009] That is, among the two major categories of existing vision-based visibility detection methods, either they require manual selection of feature areas and feature modeling for each scene, which is a large workload and difficult to promote and apply on a large scale; or they require the construction of a dataset and neural network model specifically for visibility detection. If the field of view of the camera actually used is significantly different from that of the camera used to construct the dataset, the detection results will have large errors. This is because the imaging effects of different surveillance cameras vary greatly under different imaging time periods and lighting conditions. It is often difficult to obtain good prediction results by relying solely on visual features. Summary of the Invention

[0010] In response to the technical problems existing in the prior art, the present invention provides a highway visibility detection method and system that integrates radar and vision with high prediction accuracy.

[0011] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0012] A radar and vision fusion highway visibility detection method comprises the following steps:

[0013] Obtain radar targets detected by the radar and visual targets detected by the camera;

[0014] Evaluate the visibility level within each range interval based on the relationship between the number of radar targets and visual targets in that range interval;

[0015] The radar target and the visual target are integrated to obtain the radar-visible target, and the maximum distance at which each radar-visible target can be detected by the camera is determined according to the camera detection situation, and then the visibility distance value is calculated;

[0016] The visibility level is determined and output according to the visibility degree and the visibility distance value.

[0017] Preferably, based on the quantitative relationship between radar targets and visual targets in each distance interval, the specific process of evaluating the visibility level in the distance interval is as follows:

[0018] Visibility level classification: the visibility level is divided into multiple levels; each visibility level corresponds to a different distance interval;

[0019] Assessment of visibility levels in each distance interval: Obtain the number of radar target tracks and visual target tracks in each distance interval corresponding to the visibility level. Then, based on the quantitative relationship between radar target tracks and visual target tracks in each distance interval, derive the visibility level in that distance interval.

[0020] Preferably, when the radar and camera are time synchronized, the radar target trajectory is time-aligned with the visual target trajectory according to the camera's video frame rate and the radar trajectory update rate.

[0021] Preferably, in the process of obtaining the number of radar target tracks and visual target tracks in each distance interval corresponding to the visibility level, the visual target is projected into the radar coordinate system through the calibration parameters between the radar and the camera, and the distance of the visual target relative to the radar can be obtained, thereby obtaining the distance intervals in which the radar target and the visual target are located.

[0022] Preferably, the specific calculation formula for the visibility degree is: (number of visual targets / number of radar targets)*100%, and if the result is greater than 100%, it is set to 100%.

[0023] Preferably, the specific process of obtaining the visibility distance value is:

[0024] Radar and visual target fusion and correlation: The radar target track and the visual target track are temporally aligned and mapped to the same coordinate system. The radar target track and the visual target track, which have been temporally aligned and spatially unified to the same coordinate system, are then fused and correlated to obtain the radar-visual target track.

[0025] Obtain the maximum distance at which each radar target track can be visually detected;

[0026] The visibility distance value is calculated based on the maximum distance at which each radar target track can be visually detected.

[0027] Preferably, the specific process of obtaining the maximum distance at which each radar target trajectory can be visually detected is as follows: the radar target trajectory contains the distance information of the target, thereby obtaining the maximum distance at which the target can be visually detected; if the target moves away from the camera, the maximum distance at which the target can be visually detected is the distance at which the target was last visually detected; if the target moves close to the camera, the maximum distance at which the target can be visually detected is the distance at which the target was first visually detected.

[0028] Preferably, the specific process of calculating the visibility distance value according to the farthest distance at which each radar target track can be visually detected is:

[0029] The mean absolute deviation method is used to eliminate outliers. After eliminating outliers, the arithmetic average of the maximum distances at which multiple radar targets can be visually detected within the statistical time period is taken. If the average value is greater than the preset maximum value, it is directly judged that visibility is good. Otherwise, the average value is used as the visibility distance value at this time.

[0030] Preferably, the coordinate system is a camera coordinate system, a radar coordinate system or a world coordinate system.

[0031] The present invention also discloses a highway visibility detection system integrating radar and vision, comprising a radar processing module, a vision processing module, a radar and camera calibration module, a radar and camera fusion association module, a visibility degree detection module, a visibility distance calculation module and a visibility level fusion output module;

[0032] The radar processing module is used to acquire radar targets;

[0033] The visual processing module is used to obtain a visual target;

[0034] The radar and camera calibration module is used to make a one-to-one correspondence between the radar target and the visual target of the same target at the same time, forming a set of radar target and visual target point pairs, and then calculating the transformation relationship between the radar target and the visual target in each set of point pairs, thereby obtaining the calibration parameters between the coordinate system of the radar target and the coordinate system of the visual target;

[0035] The radar and camera fusion association module is used to align the radar target track with the visual target track in time, map the radar track and the visual track to the same coordinate system, and then fuse and associate the radar target track and the visual target track after being aligned in time and unified in space to the same coordinate system to obtain the radar target track;

[0036] The visibility detection module is used to evaluate the visibility level in each distance interval based on the quantitative relationship between radar targets and visual targets in the distance interval;

[0037] The visibility distance calculation module is used to detect the maximum distance at which the trajectory of the radar target can be visually detected, so as to calculate the visibility distance value;

[0038] The visibility level fusion output module is used to determine and output the visibility level according to the visibility degree and the visibility distance value.

[0039] Compared with the prior art, the advantages of the present invention are:

[0040] This method is highly intelligent and does not require human intervention. It can realize automatic real-time measurement of visibility, eliminating the influence of human subjectivity on the detection results. It has high prediction accuracy and strong universality. It does not require special settings and optimization for different scenarios, nor does it require the construction of a special fog data set for the detection scenario. It has a short development cycle and can be "plug and play". It is suitable for large-scale promotion and application, and can use the radar and vision fusion equipment installed on the highway for detection without any additional costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of an embodiment of the highway visibility detection method of the present invention.

[0042] Figure 2 This is a schematic diagram of the visibility level classification of the present invention.

[0043] Figure 3 Schematic diagram of visibility distance value calculation according to the present invention.

[0044] Figure 4 It is a block diagram of an embodiment of a highway visibility detection system of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0046] like Figure 1 As shown, the highway visibility detection method using radar and vision fusion according to an embodiment of the present invention includes the following steps:

[0047] Obtain radar targets detected by the radar and visual targets detected by the camera;

[0048] Evaluate the visibility level within each range interval based on the relationship between the number of radar targets and visual targets in that range interval;

[0049] The radar target and the visual target are integrated to obtain the radar-visual target. Based on the camera detection situation, the maximum distance at which each radar-visual target can be detected by the visual system (camera) is determined, and then the visibility distance value is calculated.

[0050] The output visibility level is determined based on the visibility degree and visibility distance value.

[0051] Specifically, the present invention first calibrates the millimeter-wave radar and camera to obtain a set of calibration parameters, and then uses the millimeter-wave radar and camera to respectively detect and track targets such as vehicles on the road in real time to obtain radar targets and visual targets;

[0052] On the basis of detection and tracking, on the one hand, the distance intervals are divided according to the visibility level definition, and the number of radar targets and visual targets in each distance interval are compared to evaluate the visibility degree in each distance interval;

[0053] On the other hand, radar targets and visual targets are fused to correlate the radar and camera detection results for the same target, thereby determining the maximum distance at which the target can be visually detected. The maximum detection distances of all targets are then averaged, and the average distance is used as the visibility distance value. Differences in target appearance lead to differences in the maximum detection distances of targets. For example, a gray-white vehicle is difficult to detect in fog, while an orange vehicle is more easily detected.

[0054] Finally, the visibility distance value is compared with the visibility degree in each interval, and if they match, the visibility level is output.

[0055] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0056] like Figure 1 As shown, the highway visibility detection method based on radar and vision fusion according to an embodiment of the present invention specifically includes the following steps:

[0057] S1, radar and camera calibration

[0058] First, the radar actively detects targets within its detection range and obtains detection results, which are radar targets. Each target is usually represented by a track. Radar targets usually include information such as the target's distance and speed relative to the radar.

[0059] The camera detects the target in the real-time image and obtains the detection result, which is the visual target. Usually each target is represented by a track. The visual target usually contains the coordinates, size, type and other information of the target in the image.

[0060] Then, manually or automatically, the radar detection results (i.e., radar targets) and camera detection results (i.e., visual targets) of the same target at the same time are matched one-to-one to form a set of radar target and visual target point pairs.

[0061] Finally, the transformation relationship between the radar target and the visual target in each set of point pairs is calculated by matrix solution or neural network, so as to obtain the calibration parameters between the coordinate system of the radar target and the coordinate system of the visual target.

[0062] S2, radar and visual target detection and tracking

[0063] The radar actively radiates electromagnetic waves and receives echo signals from the target. It processes and analyzes the echo signals to obtain radar target tracks. By correlating the radar target tracks of the same target at different times, a radar target track is obtained and assigned a unique tracking ID, thereby achieving radar target detection and tracking.

[0064] Acquire real-time images from the camera, use the target detection algorithm to detect the target of interest in the image, obtain the detection results, associate the detection results of the same target in adjacent frame images, obtain a visual target trajectory, and assign a unique tracking ID, thereby realizing visual target detection and tracking.

[0065] S3. Visibility level and visibility distance value

[0066] The specific process of obtaining the visibility degree is as follows:

[0067] S31. Classification of visibility levels

[0068] According to existing standards, visibility levels are divided into 5 levels, such as Figure 2As shown in the figure, visibility distances of 500 meters and above are considered good, 200 to 500 meters are considered good, 100 to 200 meters are considered fair, 50 to 100 meters are considered poor, and less than 50 meters are considered extremely poor. To detect visibility at a distance of 500 meters, radars and cameras must have a maximum stable detection range of greater than 500 meters. If the detection range of a single camera or radar does not meet the requirements, multiple radars or cameras can be used in a relay configuration.

[0069] S32. Assessment of visibility levels at various distance intervals

[0070] The number of radar targets and visual targets in the five range intervals above is compared. The number of radar targets is defined as the number of radar tracks with a unique ID number. A target may generate zero, one, or even multiple radar tracks from the time it enters the radar detection area to the time it leaves it. If it is not detected by the radar throughout the entire detection range, no radar track will be generated, but this probability is very small. If a target is detected after entering the detection area, but is not detected by the radar before leaving the detection area due to reasons such as obstruction, the track will stop updating. When it is detected again, a new track will be generated.

[0071] The definition of the number of visual targets is similar to that of radar targets, which is defined as the number of visual tracks with a unique ID number. From the time a target appears in the picture to the time it disappears, 0, 1, or even multiple visual tracks may be generated.

[0072] The radar target contains the distance information of the target relative to the radar. At the same time, through the calibration parameters between the radar and the camera, the visual target is projected into the radar coordinate system, and the distance of the visual target relative to the radar can be obtained, so that the distance between the radar target and the visual target can be obtained. Figure 2 Which distance interval in .

[0073] The number of tracks that appear within a certain range interval is considered the number of targets within that range interval. For example, if 80 radar tracks and 78 visual target tracks appear within a certain range interval, the number of radar targets within that range interval is 80 and the number of visual targets within that range interval is 78. If the number of radar targets and the number of visual targets within a certain range interval are basically the same, it means that the visibility within that range interval is very good; if the number of radar targets within a certain range interval is significantly greater than the number of visual targets within a certain range interval, it means that the visibility within that range interval is moderate; if there are no visual targets within a certain range interval or the number of visual targets and radar targets is significantly different, it means that the visibility within that range interval is very poor.

[0074] The visibility level for each range interval is calculated based on the relationship between the number of radar targets and visual targets. The specific calculation formula is: number of visual targets / number of radar targets * 100%. If the result is greater than 100%, it is set to 100%.

[0075] The process of obtaining the visibility distance value is:

[0076] S301, Radar and visual target fusion and correlation

[0077] When the radar and camera are time-synchronized, the radar target trajectory is temporally aligned with the visual target trajectory based on the camera's video frame rate and the radar trajectory update rate.

[0078] Based on the radar and camera calibration parameters obtained in the previous step, the radar trajectory and visual trajectory are mapped to the same coordinate system, which can be the camera coordinate system, the radar coordinate system, or the world coordinate system;

[0079] Finally, the radar target trajectory and the visual target trajectory, which are aligned in time and unified in space to the same coordinate system, are fused and associated to obtain the radar-visual target trajectory and assigned a new unique tracking ID.

[0080] The radar target contains all the information of the radar target and the visual target, namely the distance and speed of the target relative to the radar, as well as the coordinates, size, type and other information of the target in the image.

[0081] S302, visibility distance measurement

[0082] First, after the radar target and visual target are successfully fused and associated, the radar-visual target is obtained. Because each radar-visual target has a unique tracking ID number and contains the target's distance information, the maximum distance at which the target can be visually detected can be obtained. If the target is moving away from the camera, the maximum distance at which the target can be visually detected is the distance at which the target was last visually detected; if the target is moving toward the camera, the maximum distance at which the target can be visually detected is the distance at which the target was first visually detected.

[0083] Then, statistics are collected on the maximum distance at which the radar target can be visually detected. This maximum distance may contain outliers, which may be caused by the target being blocked. For example, when the target moves close to the radar or camera, it is blocked by other vehicles or objects when it just enters the radar and visual detection range, or the target is briefly blocked in the middle and then reappears. The radar and camera fail to match it with the previous trajectory, causing the target to form a new trajectory; when the target moves away from the radar or camera, it is blocked by other vehicles or objects when it is about to leave the radar and visual detection range. Figure 3 As shown in the figure, solid circles represent normal values, which are usually concentrated near a certain distance segment, while solid triangles represent outlier points, which are usually more scattered.

[0084] S303. Use the mean absolute deviation method to eliminate outliers. After eliminating outliers, take the arithmetic average of the maximum distances at which radar-visible targets can be visually detected. If this average is greater than 500 meters, visibility is directly determined to be good. Otherwise, this average is used as the visibility distance at that time. Alternatively, the 3σ method and the Z-Score method can be used to eliminate outliers.

[0085] S4, visibility level output

[0086] The visibility distance value is compared with the visibility classification level to obtain the visibility level, which is then compared with the visibility degree calculated previously. If the two results match, the visibility level is output; otherwise, the test result is considered invalid.

[0087] like Figure 4 As shown, an embodiment of the present invention further provides a highway visibility detection system with radar and visual fusion, comprising a radar processing module, a visual processing module, a radar and camera calibration module, a radar and camera fusion association module, a visibility degree detection module, a visibility distance calculation module, and a visibility level fusion output module;

[0088] Radar processing module: The radar actively radiates electromagnetic waves and receives echo signals from the target. It processes and analyzes the echo signals to obtain radar target tracks. By correlating radar target tracks of the same target at different times, a radar target track is obtained and assigned a unique tracking ID, thereby realizing radar target detection and tracking;

[0089] The visual processing module acquires the scene image in real time, uses the target detection algorithm to detect the target of interest in the image, obtains the detection result, associates the detection results of the same target in adjacent frames, obtains a visual target trajectory, and assigns a unique tracking ID, thereby realizing visual target detection and tracking;

[0090] The radar and camera calibration module obtains the results of the radar processing module and the visual processing module, and manually or automatically matches the radar detection results of the same target at the same time, that is, the radar target, with the camera detection results, that is, the visual target, to form a set of radar target and visual target point pairs. It then calculates the transformation relationship between the radar target and the visual target in each set of point pairs through matrix solving or neural network training, thereby obtaining the calibration parameters between the coordinate system of the radar target and the coordinate system of the visual target.

[0091] The radar and camera fusion and association module obtains the results of the radar processing module and the visual processing module. When the radar and camera are synchronized, the radar target track and the visual target track are temporally aligned according to the camera's video frame rate and the radar track update rate. Based on the radar and camera calibration parameters obtained in the previous step, the radar track and the visual track are mapped to the same coordinate system. The radar target track and the visual target track, which have been temporally aligned and spatially unified to the same coordinate system, are then fused and associated to obtain the radar-visual target track and assigned a new unique tracking ID.

[0092] The visibility detection module divides the distance intervals according to the requirements of the visibility level definition, obtains the results of the radar processing module and the visual processing module, counts the number of radar targets and visual targets in each distance interval, then compares the number of radar targets in each distance interval with the number of visual targets, and calculates the following formula: number of visual targets / number of radar targets * 100%. If the result is greater than 100%, it is set to 100%. Finally, the calculated result is used as the visibility level in each distance interval.

[0093] The visibility distance calculation module obtains the results of the radar and camera fusion association module, namely the radar-visible target, extracts the maximum distance at which the target can be visually detected from the radar-visible target, and calculates the arithmetic average of the maximum detection distances of all radar-visible targets in the statistical time period after removing outliers. This average is used as the visibility distance value for the statistical time period;

[0094] The visibility fusion output module obtains the results of the visibility detection module and the visibility distance module. If the two are consistent, the visibility level is output according to the visibility distance value and the visibility level definition.

[0095] The embodiment of the present invention also provides a road visibility detection device with radar and visual fusion, which specifically includes a millimeter-wave radar, a monitoring camera, and an edge intelligent computer;

[0096] The millimeter-wave radar and surveillance camera are installed in the same direction, with overlapping detection areas, allowing them to detect and track the same target. The edge intelligent computer captures the surveillance camera's video footage, detects and tracks the target of interest, and simultaneously obtains the radar's detection results, achieving radar and camera calibration and fusion of detection results.

[0097] Based on the calibration of millimeter-wave radar and surveillance camera, the radar detection results of the target are integrated with the visual detection results of the target to achieve one-to-one correspondence;

[0098] Radar can accurately measure the distance to a target and is unaffected by atmospheric visibility. Visual detection of targets is affected by atmospheric visibility and cannot detect targets outside the visible range. Furthermore, the worse the visibility, the lower the target detection rate. Therefore, by statistically analyzing the maximum distance at which targets can be visually detected, if the vast majority of targets can be visually detected within a certain distance range, it indicates that atmospheric visibility within that distance range is very good. If the vast majority of targets cannot be visually detected within a certain distance, and this distance is significantly less than the actual maximum detectable distance, it indicates that atmospheric visibility within that distance range is poor or even invisible. In short, the visibility within a distance range can be determined based on the radar's target detection distance and the camera's detection rate of all targets passing through that range within a certain statistical time period.

[0099] The above-mentioned visual method for detecting visibility mainly consists of two key parts: one is to determine the visibility of the target location by calculating the specific target in the image, and the other is to determine the distance between the target location and the camera. This method uses a visual target detection algorithm to determine whether the target location is visible. If the target can be detected within the effective visual detection range, it is considered visible; otherwise, it is not visible. This method uses millimeter-wave radar to automatically measure the distance between the target location and the camera, and the positional relationship between the radar and the camera is obtained through a pre-calibrated method. Therefore, the radar in this method can also be replaced by any other device capable of measuring distance, such as lidar.

[0100] The aforementioned radar can accurately measure the distance to a target and is unaffected by atmospheric visibility. However, visual detection of targets is affected by atmospheric visibility and cannot detect targets outside the visible range. Furthermore, the worse the visibility, the lower the target detection rate. Therefore, we can calculate the maximum distance at which a target can be visually detected. If the vast majority of targets can be visually detected within a certain range, this indicates that atmospheric visibility is excellent within that range. If the vast majority of targets cannot be visually detected at a certain distance, and this distance is significantly less than the maximum visually detectable distance, this indicates that atmospheric visibility within that range is poor.

[0101] This method is highly intelligent and does not require human intervention. It can realize automatic real-time measurement of visibility, eliminating the influence of human subjectivity on the detection results. At the same time, it has strong universality and does not require special settings and optimizations for different scenarios, nor does it require the construction of special fog data sets for the detection scenarios. It has a short development cycle and can be "plug and play". It is suitable for large-scale promotion and application, and can use the radar and vision fusion equipment installed on the highway for detection without any additional costs.

[0102] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A radar and vision fusion highway visibility detection method, characterized in that: Including steps: Obtain radar targets detected by the radar and visual targets detected by the camera; Evaluate the visibility level within each range interval based on the relationship between the number of radar targets and visual targets in that range interval; The radar target and the visual target are integrated to obtain the radar-visible target, and the maximum distance at which each radar-visible target can be detected by the camera is determined based on the camera detection situation, and then the visibility distance value is calculated; Determining and outputting a visibility level according to the visibility degree and the visibility distance value; Based on the relationship between the number of radar targets and visual targets in each distance interval, the specific process of evaluating the visibility level in this distance interval is as follows: Visibility level classification: the visibility level is divided into multiple levels; each visibility level corresponds to a different distance interval; Visibility assessment for each distance interval: Obtain the number of radar target tracks and visual target tracks in each distance interval corresponding to the visibility level. Then, based on the relationship between the number of radar target tracks and visual target tracks in each distance interval, derive the visibility level in that distance interval. The specific calculation formula for the visibility degree is: (number of visual targets / number of radar targets)*100%. If the result is greater than 100%, it is set to 100%; The specific process of obtaining the visibility distance value is: Radar and visual target fusion and correlation: The radar target track and the visual target track are temporally aligned and mapped to the same coordinate system. The radar target track and the visual target track, which have been temporally aligned and spatially unified to the same coordinate system, are then fused and correlated to obtain the radar-visual target track. Obtain the maximum distance at which each radar target track can be visually detected; Calculate the visibility distance value based on the maximum distance at which each radar target track can be visually detected; The specific process of calculating the visibility distance value based on the maximum distance at which each radar target track can be visually detected is as follows: The mean absolute deviation method is used to eliminate outliers. After eliminating outliers, the arithmetic average of the maximum distances at which multiple radar-visible targets can be visually detected within the statistical time period is taken. If the average value is greater than the preset maximum value, it is directly determined that visibility is good. Otherwise, the average value is used as the visibility distance value at that time. The visibility distance value is compared with the visibility classification level to obtain the visibility level, which is then compared with the visibility degree calculated previously. If the two results match, the visibility level is output; otherwise, the test result is considered invalid.

2. The radar and vision fusion highway visibility detection method according to claim 1, characterized in that: When the radar and camera are time-synchronized, the radar target trajectory is temporally aligned with the visual target trajectory based on the camera's video frame rate and the radar trajectory update rate.

3. The highway visibility detection method based on radar and vision fusion according to claim 1 or 2, characterized in that: In the process of obtaining the number of radar target tracks and visual target tracks in each distance interval corresponding to the visibility level, the visual target is projected into the radar coordinate system through the calibration parameters between the radar and the camera, and the distance of the visual target relative to the radar can be obtained, thereby obtaining the distance interval between the radar target and the visual target.

4. The highway visibility detection method based on radar and vision fusion according to claim 1, characterized in that: The specific process of obtaining the maximum distance at which each radar target trajectory can be visually detected is as follows: the radar target trajectory contains the target's distance information, thereby obtaining the maximum distance at which the target can be visually detected; if the target moves away from the camera, the maximum distance at which the target can be visually detected is the distance at which the target was last visually detected; If the target moves close to the camera, the farthest distance at which the target is visually detected is the distance at which the target is first visually detected.

5. The highway visibility detection method based on radar and vision fusion according to claim 1, characterized in that: The coordinate system is a camera coordinate system, a radar coordinate system or a world coordinate system.

6. A radar and vision fusion highway visibility detection system, configured to execute the steps of the radar and vision fusion highway visibility detection method according to any one of claims 1 to 5, characterized in that: It includes radar processing module, visual processing module, radar and camera calibration module, radar and camera fusion association module, visibility degree detection module, visibility distance calculation module and visibility level fusion output module; The radar processing module is used to acquire radar targets; The visual processing module is used to obtain a visual target; The radar and camera calibration module is used to make a one-to-one correspondence between the radar target and the visual target of the same target at the same time, forming a set of radar target and visual target point pairs, and then calculating the transformation relationship between the radar target and the visual target in each set of point pairs, thereby obtaining the calibration parameters between the coordinate system of the radar target and the coordinate system of the visual target; The radar and camera fusion association module is used to align the radar target track with the visual target track in time, map the radar track and the visual track to the same coordinate system, and then fuse and associate the radar target track and the visual target track after being aligned in time and unified in space to the same coordinate system to obtain the radar target track; The visibility detection module is used to evaluate the visibility level in each distance interval based on the quantitative relationship between radar targets and visual targets in the distance interval; The visibility distance calculation module is used to detect the maximum distance at which the trajectory of the radar target can be visually detected, so as to calculate the visibility distance value; The visibility level fusion output module is used to determine and output the visibility level according to the visibility degree and the visibility distance value.

Images