Radar and vision fused road visibility detection method and system

Abstract

The invention discloses a radar and vision fused road visibility detection method and system. The method comprises the following steps: acquiring a radar target detected by a radar and a vision target detected by a camera; according to the quantitative relationship between the radar target and the visual target in each distance interval, the visibility degree in the distance interval is evaluated; fusing the radar target and the visual target to obtain a thunder target, determining the farthest distance, which can be detected by the camera, of each thunder target according to the detection condition of the camera, and further calculating a visibility distance value; and judging and outputting the visibility grade according to the visibility degree and the visibility distance value. The method has the advantages of low cost, high intelligent degree, high prediction precision and the like.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of highway visibility detection, and in particular to a highway visibility detection method and system integrating radar and vision. Background Art

[0002] Highway visibility is an important indicator of road traffic safety. Low visibility will obstruct the driver's vision, shorten the response time to emergencies, and easily cause traffic accidents. Therefore, accurate measurement of highway visibility is of great significance in the field of transportation. Technology based on the fusion of millimeter-wave radar and vision is increasingly widely used in the field of transportation, and is often used for traffic event detection and traffic parameter measurement. Millimeter-wave radar can detect and accurately measure the distance of moving targets in vehicles, while vision can detect targets of interest within the visibility range. Therefore, it is a good choice to use the existing radar-vision fusion equipment on the highway to detect highway visibility.

[0003] At present, highway visibility detection methods can be divided into three categories: manual visual detection, machine detection and visual detection, as follows: 1. The manual visual method estimates visibility by having staff observe the clarity of the target object. Staff can visually observe through video surveillance images in the monitoring center. This method can estimate visibility in a large range in a short time, but the efficiency is low. They can also observe on the highway site. Although this method can continuously output the visibility in the scene, the coverage is very limited. Although the manual visual method is simple to operate, it requires more manpower and is greatly affected by subjective factors. The detection accuracy and consistency are low, and the coverage and work efficiency are difficult to effectively guarantee.

[0004] 2. The machine detection method monitors visibility through visibility detectors and other equipment installed near the road. It is generally based on laser or infrared technology and can directly measure visibility values. The detection results are very accurate, but the visibility detectors are expensive, complex to debug, have single functions, and have low cost-effectiveness. They cannot be densely installed and it is difficult to achieve all-round real-time monitoring of road sections.

[0005] 3. The visual detection method uses surveillance cameras installed on highways to obtain on-site images and analyze the images to determine visibility. However, this method is only used in specific scenarios, and manual operations such as target setting and camera calibration are required for each scenario. It has poor universality and transferability, and requires a lot of manpower in large-scale applications.

[0006] Existing vision-based visibility detection methods are mainly divided into two categories. One is based on traditional image processing methods. There are two key parameters that need to be measured in this method, one is the visibility value of the target in the image, and the other is the distance of the target relative to the detection camera. This method first selects a feature area in the scene, and determines the distance of the feature area from the detection camera by manually measuring or calibrating the camera's internal and external parameters. Then, the features of the feature area at different visibility levels are modeled and saved as a feature library, and the corresponding relationship between the feature library and the visibility level is established. In actual use, the feature area is obtained by automatic detection or demarcation of fixed areas, and features are extracted according to the feature modeling method of the feature area, and the extracted features are compared with the feature library. Then, the visibility level of the area at this time is obtained according to the corresponding relationship between the feature library and the visibility level. Select multiple feature areas at multiple different distances, that is, it can detect visibility at different distances. This method requires manual selection of feature areas and feature modeling for each scene, which is a large workload and difficult to promote and apply on a large scale.

[0007] The other is a method based on deep learning. The method first constructs a data set by collecting foggy images of highways in actual scenes and annotating the visibility levels of foggy images. The basis for the visibility level annotation can be determined by manual visual inspection or by obtaining the detection results of the visibility detector at the same time and in the same area, thus constructing a data set; then construct a deep learning model, and divide the data set into a training set, a validation set, and a test set according to a certain ratio, and use the data set to train the deep learning model. After the training, a model weight file is obtained; finally, the deep learning model and the weight file are deployed on the relevant hardware, and the video of the highway monitoring camera is obtained in real time. After decoding the video, it forms a frame of image, and the image is sent to the deep learning model to obtain the classification result of the visibility level. This method needs to build a foggy data set specifically for visibility detection. The data collection cycle is long, which affects the progress of project construction. If the field of view of the camera actually used is greatly different from that of the camera used to build the data set, the detection result will have a large error.

[0008] That is, among the two major categories of existing vision-based visibility detection methods, either they need to manually select feature areas and perform feature modeling for each scene, which is a large workload and difficult to promote and apply on a large scale; or they need to build a data set and a neural network model specifically for visibility detection. If the field of view of the camera actually used is significantly different from that of the camera used to build the data set, the detection result will have a large error, because the imaging effects of different surveillance cameras in different imaging periods and lighting conditions vary greatly, and it is often difficult to obtain good prediction results by relying solely on visual features. Summary of the invention

[0009] In view of the technical problems existing in the prior art, the present invention provides a highway visibility detection method and system that integrates radar and vision with high prediction accuracy.

[0010] In order to solve the above technical problems, the technical solution proposed by the present invention is: A radar and vision fusion highway visibility detection method comprises the following steps: Get radar targets detected by radar and visual targets detected by camera; According to the quantitative relationship between radar targets and visual targets in each distance interval, the visibility degree in the distance interval is evaluated; The radar target is merged with the visual target to obtain a radar-visible target, and the maximum distance at which each radar-visible target can be detected by the camera is determined according to the camera detection situation, and then the visibility distance value is calculated; The visibility level is determined and output according to the visibility degree and the visibility distance value.

[0011] Preferably, according to the quantitative relationship between radar targets and visual targets in each distance interval, the specific process of evaluating the visibility degree in the distance interval is: Visibility level classification: the visibility level is divided into multiple levels; each visibility level corresponds to a different distance interval; Assessment of visibility level at each distance interval: Obtain the number of radar target tracks and visual target tracks in each distance interval corresponding to the visibility level, and then derive the visibility level in the distance interval based on the quantitative relationship between radar target tracks and visual target tracks in each distance interval.

[0012] Preferably, in case of time synchronization between the radar and the camera, the radar target trajectory is time-aligned with the visual target trajectory according to the video frame rate of the camera and the update rate of the radar trajectory.

[0013] Preferably, in the process of obtaining the number of radar target tracks and visual target tracks in each distance interval corresponding to the visibility level, the visual target is projected into the radar coordinate system through the calibration parameters between the radar and the camera, and the distance of the visual target relative to the radar can be obtained, thereby obtaining the distance intervals in which the radar target and the visual target are located.

[0014] Preferably, the specific calculation formula for the visibility degree is: (number of visual targets / number of radar targets)*100%, and if the result is greater than 100%, it is set to 100%.

[0015] Preferably, the specific process of obtaining the visibility distance value is: Radar and visual target fusion and association: Align the radar target trajectory with the visual target trajectory in time, map the radar target trajectory and the visual target trajectory to the same coordinate system, and then fuse and associate the radar target trajectory and the visual target trajectory that are aligned in time and unified in space to the same coordinate system to obtain the radar visual target trajectory; Get the maximum distance at which each radar target track can be visually detected; The visibility distance value is calculated based on the maximum distance at which each radar target track can be visually detected.

[0016] Preferably, the specific process of obtaining the farthest distance at which each radar target trajectory can be visually detected is as follows: the radar target trajectory contains the distance information of the target, thereby obtaining the farthest distance at which the target can be visually detected; if the target moves away from the camera, the farthest distance at which the target is visually detected is the distance at which the target was last visually detected; if the target moves close to the camera, the farthest distance at which the target is visually detected is the distance at which the target is first visually detected.

[0017] Preferably, the specific process of calculating the visibility distance value according to the farthest distance at which each radar target track can be visually detected is: The mean absolute deviation method is used to eliminate outliers. After eliminating outliers, the arithmetic average of the maximum distances at which multiple radar targets can be visually detected within the statistical time period is taken. If the average value is greater than the preset maximum value, it is directly judged that visibility is good. Otherwise, the average value is used as the visibility distance value at this time.

[0018] Preferably, the coordinate system is a camera coordinate system, a radar coordinate system or a world coordinate system.

[0019] The present invention also discloses a highway visibility detection system integrating radar and vision, comprising a radar processing module, a vision processing module, a radar and camera calibration module, a radar and camera fusion association module, a visibility degree detection module, a visibility distance calculation module and a visibility level fusion output module; The radar processing module is used to acquire radar targets; The visual processing module is used to obtain a visual target; The radar and camera calibration module is used to make a one-to-one correspondence between the radar target and the visual target of the same target at the same time, forming a set of point pairs of the radar target and the visual target, and then calculating the conversion relationship between the radar target and the visual target in each set of point pairs, so as to obtain the calibration parameters between the coordinate system where the radar target is located and the coordinate system where the visual target is located; The radar and camera fusion association module is used to align the radar target trajectory with the visual target trajectory in time, map the radar trajectory and the visual trajectory to the same coordinate system, and then fuse and associate the radar target trajectory and the visual target trajectory that are aligned in time and unified in space to the same coordinate system to obtain the radar visual target trajectory; The visibility detection module is used to evaluate the visibility level in each distance interval according to the quantitative relationship between radar targets and visual targets in the distance interval; The visibility distance calculation module is used to detect the farthest distance at which the trajectory of the radar target can be visually detected, so as to calculate the visibility distance value; The visibility level fusion output module is used to determine and output the visibility level according to the visibility degree and the visibility distance value.

[0020] Compared with the prior art, the advantages of the present invention are: This method is highly intelligent and does not require human intervention. It can realize automatic real-time measurement of visibility, eliminating the influence of human subjectivity on the detection results. It has high prediction accuracy and strong universality. It does not require special settings and optimizations for different scenarios, nor does it require the construction of a special fog data set for the detection scenario. It has a short development cycle and can be "plug and play". It is suitable for large-scale promotion and application, and can use the radar and vision fusion equipment installed on the road for detection without any additional costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The figure is a flow chart of a highway visibility detection method in an embodiment of the present invention.

[0022] Figure 2 It is a schematic diagram of visibility level classification of the present invention.

[0023] Figure 3 It is a schematic diagram of calculating the visibility distance value of the present invention.

[0024] Figure 4 It is a block diagram of an embodiment of a highway visibility detection system of the present invention. DETAILED DESCRIPTION

[0025] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0026] like Figure 1 As shown, the highway visibility detection method of radar and vision fusion in an embodiment of the present invention comprises the steps of: Get radar targets detected by radar and visual targets detected by camera; According to the quantitative relationship between radar targets and visual targets in each distance interval, the visibility degree in the distance interval is evaluated; The radar target and the visual target are integrated to obtain the radar-visible target. According to the camera detection, the maximum distance at which each radar-visible target can be detected by the visual system (camera) is determined, and then the visibility distance value is calculated. The output visibility level is determined based on the visibility degree and visibility distance value.

[0027] Specifically, the present invention first calibrates the millimeter wave radar and the camera to obtain a set of calibration parameters, and then uses the millimeter wave radar and the camera to respectively detect and track targets such as vehicles on the road in real time to obtain radar targets and visual targets; On the basis of detection and tracking, on the one hand, the distance intervals are divided according to the visibility level definition, and the number of radar targets and visual targets in each distance interval are compared to evaluate the visibility degree in each distance interval; On the other hand, radar targets and visual targets are fused to achieve the association of radar and camera detection results for the same target, thereby determining the farthest distance at which the target can be visually detected. Next, the farthest detection distances of all targets are averaged, and the average distance is used as the visibility distance value. Differences in the appearance characteristics of targets lead to differences in the farthest detection distances of targets. For example, gray-white vehicles are not easy to be detected in fog, while orange ones are easier to be detected. Finally, the visibility distance value is compared with the visibility degree in each interval, and if they match, the visibility level is output.

[0028] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0029] like Figure 1 As shown, the highway visibility detection method of radar and vision fusion in an embodiment of the present invention specifically includes the following steps: S1, radar and camera calibration First, the radar actively detects the target within the detection range and obtains the detection result, which is the radar target. Usually, each target is represented by a track; the radar target usually contains information such as the distance and speed of the target relative to the radar; The camera detects the target in the real-time image and obtains the detection result, which is the visual target. Usually, each target is represented by a track. The visual target usually contains the coordinates, size, type and other information of the target in the image. Then, manually or automatically, the radar detection results (i.e., radar targets) and camera detection results (i.e., visual targets) of the same target at the same time are matched one by one to form a set of point pairs of radar targets and visual targets. Finally, the transformation relationship between the radar target and the visual target in each set of point pairs is calculated by matrix solution or neural network, so as to obtain the calibration parameters between the coordinate system of the radar target and the coordinate system of the visual target.

[0030] S2, radar and visual target detection and tracking The radar actively radiates electromagnetic waves and receives echo signals from the target. It processes and analyzes the echo signals to obtain radar target points. By correlating the radar target points of the same target at different times, a radar target track is obtained and a unique tracking ID is assigned, thereby realizing radar target detection and tracking. Acquire real-time images from the camera, use the target detection algorithm to detect the target of interest in the image, get the detection result, associate the detection results of the same target in adjacent frame images, get a visual target trajectory, and assign a unique tracking ID, thereby realizing visual target detection and tracking.

[0031] S3. Visibility level and visibility distance value The specific process of obtaining the visibility degree is as follows: S31. Classification of visibility levels According to the existing standards, visibility levels are divided into 5 levels, such as Figure 2 As shown in the figure, visibility distance of more than 500 meters is good, 200 to 500 meters is good, 100 to 200 meters is fair, 50 to 100 meters is poor, and less than 50 meters is extremely poor. In order to detect visibility at a distance of 500 meters, the maximum stable detection distance of radar and camera to the target is required to be greater than 500 meters. If the detection distance of a single existing camera or radar does not meet the requirements, multiple radars or cameras can be used in relay mode.

[0032] S32. Assessment of visibility levels at each distance interval The number of radar targets and visual targets in the above five distance intervals is compared. The number of radar targets is defined as the number of radar tracks with a unique ID number. A target may generate 0, 1 or even multiple radar tracks from entering the radar detection area to leaving it. If it is not detected by the radar throughout the whole process, no radar track will be generated, but the probability of this is very small. If the target is detected after entering the detection area, but it may not be detected by the radar before leaving the detection area due to reasons such as being blocked, the track will stop updating at this time. When it is detected again, a new track will be generated.

[0033] The definition of the number of visual targets is similar to that of radar targets, which is defined as the number of visual tracks with a unique ID number. From the time a target appears in the picture to the time it disappears, 0, 1, or even multiple visual tracks may be generated.

[0034] The radar target contains the distance information of the target relative to the radar. At the same time, through the calibration parameters between the radar and the camera, the visual target is projected into the radar coordinate system, and the distance of the visual target relative to the radar can be obtained, so that the distance between the radar target and the visual target can be obtained. Figure 2 Which distance interval is in ?

[0035] The number of tracks that appear in a certain distance interval is taken as the number of targets in that distance interval. For example, if there are 80 radar tracks and 78 visual target tracks in a certain distance interval, the number of radar targets in that distance interval is 80 and the number of visual targets is 78. If the number of radar targets and the number of visual targets in a certain distance interval are basically the same, it means that the visibility in that distance interval is very good; if the number of radar targets in a certain distance interval is significantly greater than the number of visual targets, it means that there is a certain degree of visibility in that distance interval; if there is no visual target in a certain distance interval or the number of visual targets and radar targets is very different, it means that the visibility in that distance interval is very poor.

[0036] According to the relationship between the number of radar targets and visual targets in each distance interval, the visibility degree in this distance interval is obtained. The specific calculation formula is: number of visual targets / number of radar targets*100%. If the result is greater than 100%, it is set to 100%.

[0037] The process of obtaining the visibility distance value is: S301, Radar and visual target fusion and association When the radar and camera are time synchronized, the radar target trajectory is time-aligned with the visual target trajectory according to the camera’s video frame rate and the radar trajectory update rate; Based on the radar and camera calibration parameters obtained in the previous steps, the radar trajectory and the visual trajectory are mapped to the same coordinate system, which can be the camera coordinate system, the radar coordinate system, or the world coordinate system; Finally, the radar target trajectory and the visual target trajectory, which are aligned in time and unified in space to the same coordinate system, are fused and associated to obtain the radar visual target trajectory and assigned a new unique tracking ID.

[0038] The radar target contains all the information of the radar target and the visual target, that is, the distance and speed of the target relative to the radar, as well as the coordinates, size, type and other information of the target in the image.

[0039] S302, visibility distance measurement First, after the radar target and the visual target are successfully fused and associated, the radar-visual target is obtained. Because each radar-visual target has a unique tracking ID number, the radar-visual target contains the distance information of the target, so the maximum distance at which the target can be visually detected can be obtained. If the target moves away from the camera, the maximum distance at which the target can be visually detected is the distance at which the target was last visually detected; if the target moves close to the camera, the maximum distance at which the target can be visually detected is the distance at which the target was first visually detected; Then, statistics are collected on the maximum distance at which the radar target can be detected visually. This maximum distance may contain outliers, which may be caused by the target being blocked. For example, when the target moves close to the radar or camera, it is blocked by other vehicles or objects when it just enters the detection range of the radar and vision, or the target is briefly blocked in the middle and then reappears. The radar and camera fail to match it with the previous trajectory, causing the target to form a new trajectory; when the target moves away from the radar or camera, it is blocked by other vehicles or objects when it is about to leave the detection range of the radar and vision. Figure 3 As shown, solid circles represent normal values, which are usually concentrated near a certain distance range, while solid triangles represent outlier points, which are usually more scattered.

[0040] S303, use the mean absolute deviation method to eliminate outliers. After eliminating outliers, take the arithmetic average of the maximum distance at which the radar target can be visually detected. If the average value is greater than 500 meters, it is directly determined that the visibility is good. Otherwise, the average value is used as the visibility distance value at this time. In addition, the 3σ method and Z-Score method can also be used to eliminate outliers.

[0041] S4, visibility level output The visibility distance value is compared with the visibility classification level to obtain the visibility level, and the visibility level is compared with the visibility degree calculated previously. If the two results match, the visibility level is output, otherwise the detection result is considered invalid.

[0042] like Figure 4 As shown, an embodiment of the present invention further provides a highway visibility detection system with radar-vision fusion, including a radar processing module, a visual processing module, a radar and camera calibration module, a radar and camera fusion association module, a visibility degree detection module, a visibility distance calculation module and a visibility level fusion output module; Radar processing module: the radar actively radiates electromagnetic waves and receives echo signals from the target. It processes and analyzes the echo signals to obtain radar target points. By correlating the radar target points of the same target at different times, a radar target track is obtained and a unique tracking ID is assigned, thereby realizing radar target detection and tracking. The visual processing module acquires the scene image in real time, uses the target detection algorithm to detect the target of interest in the image, obtains the detection result, associates the detection results of the same target in adjacent frame images, obtains a visual target trajectory, and assigns a unique tracking ID, thereby realizing visual target detection and tracking; The radar and camera calibration module obtains the results of the radar processing module and the visual processing module, and manually or automatically matches the radar detection results of the same target at the same time, that is, the radar target, with the camera detection results, that is, the visual target, to form a set of point pairs of radar targets and visual targets; then the conversion relationship between the radar target and the visual target in each set of point pairs is calculated by matrix solution or neural network training, so as to obtain the calibration parameters between the coordinate system of the radar target and the coordinate system of the visual target; The radar and camera fusion association module obtains the results of the radar processing module and the visual processing module. When the radar and camera are synchronized, the radar target trajectory and the visual target trajectory are aligned in time according to the camera's video frame rate and the radar trajectory update rate. According to the radar and camera calibration parameters obtained in the previous steps, the radar trajectory and the visual trajectory are mapped to the same coordinate system. The radar target trajectory and the visual target trajectory that are aligned in time and unified in space to the same coordinate system are then fused and associated to obtain the radar-visual target trajectory and assigned a new unique tracking ID. The visibility detection module divides the distance intervals according to the requirements of the visibility level definition, obtains the results of the radar processing module and the visual processing module, counts the number of radar targets and visual targets in each distance interval, then compares the number of radar targets with the number of visual targets in each distance interval, and calculates the following formula: number of visual targets / number of radar targets*100%. If the result is greater than 100%, it is set to 100%; finally, the calculation result is used as the visibility level in each distance interval.

[0043] The visibility distance calculation module obtains the result of the radar and camera fusion association module, that is, the radar-visible target, extracts the farthest distance at which the target can be visually detected from the radar-visible target, and takes the arithmetic average of the farthest detection distances of all radar-visible targets in the statistical time period after removing abnormal values, and uses the average value as the visibility distance value in the statistical time period; The visibility fusion output module obtains the results of the visibility detection module and the visibility distance module. If the two are consistent, the visibility level is output according to the visibility distance value and the visibility level definition.

[0044] The embodiment of the present invention also provides a highway visibility detection device with radar and vision fusion, which specifically includes a millimeter wave radar, a monitoring camera and an edge intelligent computer; The millimeter-wave radar and the surveillance camera are installed in the same direction, and their detection areas overlap, so they can detect and track the same target. The edge intelligent computer obtains the video image of the surveillance camera, detects and tracks the target of interest, and obtains the detection results of the radar at the same time, realizing the calibration of the radar and the camera and the fusion of the detection results. Based on the calibration of millimeter-wave radar and surveillance camera, the radar detection results of the target are integrated with the visual detection results of the target to achieve one-to-one correspondence; Among them, radar can accurately measure the distance of the target and is not affected by the atmospheric visibility; the visual detection result of the target is affected by the atmospheric visibility and cannot detect the target outside the visible range, and the worse the visibility, the lower the target detection rate. Therefore, by counting the farthest distance at which the target is visually detected, if most of the targets can be visually detected in a certain distance segment, it means that the atmospheric visibility in this distance segment is very good; if most of the targets cannot be visually detected at a certain distance, and the distance is significantly less than the actual farthest detectable distance of the vision, it means that the atmospheric visibility in this distance segment is poor or even invisible; in short, the visibility of this distance segment can be determined based on the detection distance of the radar to the target and the detection rate of the camera for all targets passing through the area in this distance segment and within a certain statistical time period.

[0045] The above-mentioned detection of visibility using visual methods mainly consists of two key parts. One is to determine the visibility of the target location by calculating the specific target in the image, and the other is to determine the distance between the target location and the camera. This method uses a visual target detection algorithm to determine whether the target location is visible. If the target can be detected within the effective visual detection range, it is considered visible; otherwise, it is not visible. This method uses millimeter-wave radar to automatically measure the distance between the target location and the camera, and the positional relationship between the radar and the camera is obtained by a pre-calibrated method. Therefore, the radar in this method can also be replaced by any other device that can measure distance, such as lidar.

[0046] The above radar can accurately measure the distance of the target and is not affected by the atmospheric visibility. The visual detection result of the target is affected by the atmospheric visibility and cannot detect the target outside the visible range. The worse the visibility, the lower the target detection rate. Therefore, the farthest distance at which the target is visually detected can be counted. If most of the targets can be visually detected in a certain distance segment, it means that the atmospheric visibility in this distance segment is very good; if most of the targets cannot be visually detected in a certain distance, and the distance is significantly less than the actual farthest detectable distance of the visual, it means that the atmospheric visibility in this distance segment is poor.

[0047] This method is highly intelligent and does not require human intervention. It can realize automatic real-time measurement of visibility, eliminating the influence of human subjectivity on the detection results. At the same time, it has strong universality and does not require special settings and optimizations for different scenarios, nor does it require the construction of a special fog data set for the detection scenario. It has a short development cycle and can be "plug and play". It is suitable for large-scale promotion and application, and can use the radar and vision fusion equipment installed on the road for detection without any additional costs.

[0048] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A highway visibility detection method based on radar and vision fusion, It is characterized in that Includes steps: Get radar targets detected by radar and visual targets detected by camera; According to the quantitative relationship between radar targets and visual targets in each distance interval, the visibility degree in the distance interval is evaluated; The radar target is merged with the visual target to obtain a radar-visible target, and the maximum distance at which each radar-visible target can be detected by the camera is determined according to the camera detection situation, and then the visibility distance value is calculated; The visibility level is determined and output according to the visibility degree and the visibility distance value.

2. The highway visibility detection method of radar and vision fusion according to claim 1, It is characterized in that According to the quantitative relationship between radar targets and visual targets in each distance interval, the specific process of evaluating the visibility degree in this distance interval is as follows: Visibility level classification: the visibility level is divided into multiple levels; each visibility level corresponds to a different distance interval; Assessment of visibility level at each distance interval: Obtain the number of radar target tracks and visual target tracks in each distance interval corresponding to the visibility level, and then derive the visibility level in the distance interval based on the quantitative relationship between radar target tracks and visual target tracks in each distance interval.

3. The highway visibility detection method of radar and vision fusion according to claim 2, It is characterized in that In the case of time synchronization between the radar and the camera, the radar target trajectory is temporally aligned with the visual target trajectory according to the camera's video frame rate and the radar trajectory update rate.

4. The highway visibility detection method of radar and vision fusion according to claim 2 or 3, It is characterized in that In the process of obtaining the number of radar target tracks and visual target tracks in each distance interval corresponding to the visibility level, the visual target is projected into the radar coordinate system through the calibration parameters between the radar and the camera, and the distance of the visual target relative to the radar can be obtained, thereby obtaining the distance interval of the radar target and the visual target.

5. The highway visibility detection method of radar and vision fusion according to claim 2 or 3, It is characterized in that The specific calculation formula of the visibility degree is: (number of visual targets / number of radar targets)*100%. If the result is greater than 100%, it is set to 100%.

6. The highway visibility detection method of radar and vision fusion according to any one of claims 1 to 3, It is characterized in that The specific process of obtaining the visibility distance value is: Radar and visual target fusion and association: Align the radar target trajectory with the visual target trajectory in time, map the radar target trajectory and the visual target trajectory to the same coordinate system, and then fuse and associate the radar target trajectory and the visual target trajectory that are aligned in time and unified in space to the same coordinate system to obtain the radar visual target trajectory; Get the maximum distance at which each radar target track can be visually detected; The visibility distance value is calculated based on the maximum distance at which each radar target track can be visually detected.

7. The highway visibility detection method of radar and vision fusion according to claim 6, It is characterized in that The specific process of obtaining the farthest distance at which each radar target track can be visually detected is as follows: the radar target track contains the distance information of the target, thereby obtaining the farthest distance at which the target can be visually detected; if the target moves away from the camera, the farthest distance at which the target can be visually detected is the distance at which the target was last visually detected; If the target is moving close to the camera, the farthest distance at which the target is visually detected is the distance at which the target is first visually detected.

8. The highway visibility detection method of radar and vision fusion according to claim 6, It is characterized in that The specific process of calculating the visibility distance value based on the maximum distance at which each radar target track can be visually detected is: The mean absolute deviation method is used to eliminate outliers. After eliminating outliers, the arithmetic average of the maximum distances at which multiple radar targets can be visually detected within the statistical time period is taken. If the average value is greater than the preset maximum value, it is directly judged that visibility is good. Otherwise, the average value is used as the visibility distance value at this time.

9. The highway visibility detection method of radar and vision fusion according to claim 6, It is characterized in that The coordinate system is a camera coordinate system, a radar coordinate system or a world coordinate system.

10. A highway visibility detection system integrating radar and vision. It is characterized in that It includes radar processing module, visual processing module, radar and camera calibration module, radar and camera fusion association module, visibility degree detection module, visibility distance calculation module and visibility level fusion output module; The radar processing module is used to acquire radar targets; The visual processing module is used to obtain a visual target; The radar and camera calibration module is used to make a one-to-one correspondence between the radar target and the visual target of the same target at the same time, forming a set of point pairs of the radar target and the visual target, and then calculating the conversion relationship between the radar target and the visual target in each set of point pairs, so as to obtain the calibration parameters between the coordinate system where the radar target is located and the coordinate system where the visual target is located; The radar and camera fusion association module is used to align the radar target trajectory with the visual target trajectory in time, map the radar trajectory and the visual trajectory to the same coordinate system, and then fuse and associate the radar target trajectory and the visual target trajectory that are aligned in time and unified in space to the same coordinate system to obtain the radar visual target trajectory; The visibility detection module is used to evaluate the visibility level in each distance interval according to the quantitative relationship between radar targets and visual targets in the distance interval; The visibility distance calculation module is used to detect the farthest distance at which the trajectory of the radar target can be visually detected, so as to calculate the visibility distance value; The visibility level fusion output module is used to determine and output the visibility level according to the visibility degree and the visibility distance value.

Images