A visibility measurement method based on millimeter-wave radar and multi-sensor information fusion
By combining millimeter wave radar with digital cameras and weather sensors, using multi-sensor information fusion to build a visibility measurement model, the problem of large measurement errors in severe weather is solved in the existing technology, and accurate visibility monitoring is achieved throughout the day and around the clock .
Patent Information
- Application Number
- CN202110554606.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-21
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-05-21
AI Technical Summary
The working performance of existing millimeter-wave radars in severe weather conditions such as rainfall is affected, and the measurement error is large, which cannot meet the needs of visibility monitoring throughout the day and around the clock.
Using a visibility measurement method based on the fusion of millimeter-wave radar and multi-sensor information, combining millimeter-wave radar, digital camera and weather sensor, through artificial neural network training data sets, multi-sensor information is fused to build a visibility measurement model to adapt to different weather and lighting conditions. , using radar reflectivity factor and image distortion rate to calculate visibility.
It achieves all-day and all-weather visibility measurement accuracy, has small equipment and low cost, and is adapted to a variety of environments, making up for the shortcomings of radar under harsh weather conditions and improving detection efficiency and accuracy.
Smart Images

Figure CN113762057B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of radar signal processing, and in particular to a visibility measurement method based on millimeter wave radar and multi-sensor information fusion. Background Art
[0002] Road traffic accidents, as one of the important factors endangering personal safety, have long been paid attention to by all sectors of society, especially the transportation and meteorological departments. The reduced visibility caused by meteorological factors such as haze, rain, snow, and dust seriously affects traffic safety.
[0003] At present, the main methods of monitoring visibility include manual visual inspection, optical measurement, digital camera, lidar, millimeter-wave radar, etc. In recent years, there has been observation and research on visibility using millimeter-wave radar, which mainly uses the scattering characteristics of electromagnetic waves of particles such as clouds, fog, and dust in the air to invert visibility. The measurement error is small and it can work all day and all weather. The integrated chip also reduces the size of the millimeter-wave radar equipment and reduces the cost. However, the detection horizontal range is small, and its working performance is seriously affected by rainfall. It is necessary to consider the serious impact of rain attenuation on radar echo energy. It is generally only suitable for non-precipitating clouds, drizzle, fog and sandstorm weather, and cannot meet people's requirements. Summary of the invention
[0004] 1. Technical issues to be solved
[0005] In view of the shortcomings of the prior art, the present invention provides a visibility measurement method based on millimeter-wave radar and multi-sensor information fusion, which can comprehensively utilize the respective advantages of multiple sensors, adapt to a variety of environments and weather conditions, and can measure accurately all day and all weather. The calibration and on-site debugging are simple, the equipment is small in size, low in cost, and easy to deploy on a large scale.
[0006] (II) Technical solution
[0007] To achieve the above object, the present invention provides the following technical solutions:
[0008] A visibility measurement method based on millimeter wave radar and multi-sensor information fusion, comprising the following steps:
[0009] S1: Select the millimeter-wave radar, camera and weather sensor models, then preset the reference objects and detection parameters, select the calibration test environment, then configure the radar parameters, obtain the radar reflectivity factor parameters, and then calculate the millimeter-wave radar visibility;
[0010] S2: Read weather sensor data to obtain data on fog droplet size and water content, air quality, metal particles, sand and dust content, light intensity, and precipitation;
[0011] S3: Determine whether the current illumination is lower than the threshold. If yes, use the night camera to detect the visibility model, then perform night image preprocessing, obtain the brightness of the target object and the light source, and then use the night digital camera method to calculate the visibility. If no, use the daytime camera to detect the visibility model, then perform daytime image preprocessing, calculate the average image distortion rate, and finally use the daytime digital camera method to calculate the visibility.
[0012] S4: Manual calibration and correction of abnormal data. The millimeter wave radar visibility calculation results in S1, the night digital camera visibility calculation results in S3, and the daytime digital camera visibility calculation results are all manually calibrated to correct abnormal data, and then a multi-sensor data set and a measured visibility data set are constructed;
[0013] S5: The artificial neural network training data set fuses the multi-sensor information and then obtains the multi-sensor visibility measurement model.
[0014] As a further solution of the present invention, the visibility in the air in S1 is a physical quantity that describes the degree of air transparency, which is defined as the visible distance of a special sign or luminous body, and according to Koschmider's law, satisfies the following formula:
[0015] ε=e -σV
[0016] In the formula, ε is the contrast threshold for the observer to identify the target object from the background, σ is the horizontal extinction coefficient, V is the horizontal visibility of the air, and the ε value recommended by ICAO is 0.05. Substituting it into the above formula, it is easy to get:
[0017]
[0018] Furthermore, the relationship between the millimeter-wave radar measurement data and visibility is calibrated in the calculation in S1. According to the derivation in S1, visibility is uniquely determined by the extinction coefficient σ, and its value is related to the total amount of particles in the air, that is, the size of each particle, and can be expressed as:
[0019]
[0020] In the formula, k i represents the scattering extinction coefficient, r i represents the particle radius, n i Represents the radius r i The number of particles, radar reflectivity factor Z and r i , n i The relationship can be expressed as:
[0021] Z=∑ i n i (2r i ) 6.
[0022] On the basis of the above scheme, the relationship between the image information obtained by the digital camera method and the visibility is first calibrated in S3. The intuitive feeling of the visibility is the change of clarity and blur of objects in the image field of view. The higher the visibility, the richer the detailed features of the image in the field of view and the smaller the color distortion.
[0023] Furthermore, the daytime visibility detection model in S3 is the same as that derived in S1. The visibility is uniquely determined by the extinction coefficient σ, which is similar to the way the human eye observes the clarity of an object. The visibility value is characterized by the color distortion condition. A reference object with a known color distribution is placed at a fixed distance L0 in front of the digital camera. Assuming that the reference object occupies a total of m rows and n columns of pixels in the image, the original grayscale value of each pixel is recorded as g0(i,j), 1≤i≤m, 1≤j≤n, and the grayscale value of the image captured by the digital camera after being transmitted through the air to be tested is recorded as g(i,j). The average distortion rate of the image can be expressed as:
[0024]
[0025] The relationship between the color distortion rate and visibility of digital images is calibrated by S4 artificial simulation test cabin or outdoor measurement. The night digital camera method uses the attenuation law of the light source after propagation through the air to be measured. Two groups of target objects and light sources are set in advance. The brightness of the light source itself is L0. The distances between the two groups of target objects and the light source and the camera are d1 and d2 respectively. The apparent brightness of the light source received by the camera is L a1 and L a2 , the target brightness is L b1 and L b2 , the calculation expression of visibility is:
[0026]
[0027] On the basis of the above scheme, in the selected calibration environment in S4, the millimeter-wave radar and the digital camera are placed in the same position, the corner reflector required for calibration in S1, the known color distribution reference object required for calibration in S3, and the light source and target object group required for calibration are placed in the same position, facing the millimeter-wave radar and the digital camera, and maintaining a distance. The drop spectrometer, the photosensor, the precipitation sensor, and the air quality meter are placed in the same measurement environment. When calibrating the relationship between the reflectivity factor and visibility of the radar in S1, the radar transmits a beam through a simulated air environment to illuminate the corner reflector, and the reflected echo is received by the radar through the simulated air environment, and the radar reflectivity factor data is recorded. When calibrating the relationship between the color distortion rate of the digital image and the visibility during the day in S3, the digital camera records the color distortion rate data of the known color distribution reference object. At night, the method in S3 is used to record visibility and light intensity data, and the drop spectrometer, the photosensor, the precipitation sensor, and the air quality meter are used to complete the calibration and measurement process.
[0028] A further solution of the present invention is to fully consider weather conditions, lighting conditions, temperature and humidity conditions during the artificial simulation test chamber or outdoor measurement, and establish a measured data set of multi-sensor data and visibility under various conditions.
[0029] (III) Beneficial effects
[0030] Compared with the prior art, the present invention provides a visibility measurement method based on millimeter wave radar and multi-sensor information fusion, which has the following beneficial effects:
[0031] 1. The present invention can comprehensively utilize the respective advantages of multiple sensors, can adapt to various environments and weather conditions, can measure accurately all day and all weather, is simple to calibrate and debug on site, has a small device size, low cost, and is easy to deploy on a large scale.
[0032] 2. The present invention is based on the different visibility detection models during the day and at night, and cooperates with a photosensitive sensor to sense the current ambient light intensity. When the brightness is higher than the threshold, the daytime visibility digital camera detection method is used, and when the brightness is lower than the threshold, the nighttime visibility digital camera detection method is used. This refines the detection method and improves the detection efficiency.
[0033] 3. The present invention selects two sensors, millimeter wave radar and digital camera, and cooperates with weather sensors such as drop spectrometer, photosensor, precipitation sensor and other multiple sensors to perform information fusion processing, making full use of the advantages of each sensor. The high measurement accuracy of millimeter wave radar can make up for the large measurement error of digital camera, and the high adaptability of digital camera to adverse meteorological conditions and complex sites can make up for the influence of rain attenuation on the working performance of millimeter wave radar.
[0034] 4. The weather sensors such as the drop spectrometer, photosensor, precipitation sensor, and air quality meter of the present invention are used to sense the current meteorological conditions. The drop spectrometer mainly provides the current air droplet spectrum (droplet radius-concentration information), which is used to determine the relationship curve between radar reflection factor and visibility. The photosensor mainly senses the current light intensity and is used to provide light intensity information and switch the digital camera day / night working model. The precipitation sensor provides the current precipitation information, and the air quality meter obtains the main components and content of the floating particles in the current air. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 A schematic diagram of the flow structure of a visibility measurement method based on millimeter-wave radar and multi-sensor information fusion proposed by the present invention;
[0036] Figure 2 A schematic diagram of visibility measurement by millimeter-wave radar based on a visibility measurement method of millimeter-wave radar and multi-sensor information fusion proposed by the present invention;
[0037] Figure 3 A schematic diagram of measuring daytime visibility using a digital camera method based on a visibility measurement method of millimeter-wave radar and multi-sensor information fusion proposed by the present invention;
[0038] Figure 4 A schematic diagram of night visibility measurement using a digital camera method based on a visibility measurement method of millimeter wave radar and multi-sensor information fusion proposed by the present invention;
[0039] Figure 5 A schematic diagram of the relationship between visibility and radar reflectivity factor under different droplet radii in a visibility measurement method based on millimeter wave radar and multi-sensor information fusion proposed by the present invention;
[0040] Figure 6 This is a schematic diagram of multi-sensor data fusion of a visibility measurement method based on millimeter-wave radar and multi-sensor information fusion proposed by the present invention. DETAILED DESCRIPTION
[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0042] Reference Figure 1-6 , a visibility measurement method based on millimeter wave radar and multi-sensor information fusion, comprising the following steps:
[0043] S1: Select the millimeter-wave radar, camera and weather sensor models, then preset the reference objects and detection parameters, select the calibration test environment, then configure the radar parameters, obtain the radar reflectivity factor parameters, and then calculate the millimeter-wave radar visibility;
[0044] S2: Read weather sensor data to obtain data on fog droplet size and water content, air quality, metal particles, sand and dust content, light intensity, and precipitation;
[0045] S3: Determine whether the current illumination is lower than the threshold. If yes, use the night camera to detect the visibility model, then perform night image preprocessing, obtain the brightness of the target object and the light source, and then use the night digital camera method to calculate the visibility. If no, use the daytime camera to detect the visibility model, then perform daytime image preprocessing, calculate the average image distortion rate, and finally use the daytime digital camera method to calculate the visibility.
[0046] S4: Manual calibration and correction of abnormal data. The millimeter wave radar visibility calculation results in S1, the night digital camera visibility calculation results in S3, and the daytime digital camera visibility calculation results are all manually calibrated to correct abnormal data, and then a multi-sensor data set and a measured visibility data set are constructed;
[0047] S5: The artificial neural network training data set fuses multi-sensor information, and then obtains a multi-sensor visibility measurement model, which can comprehensively utilize the respective advantages of multiple sensors, adapt to various environments and weather conditions, and measure accurately all day and all weather. The calibration and on-site debugging are simple, the equipment is small in size, low in cost, and easy to deploy on a large scale.
[0048] The visibility in the air in S1 of the present invention is a physical quantity that describes the degree of transparency of the air, and is defined as the visible distance of a special sign or luminous body, which satisfies the following formula according to Koschmider's law:
[0049] ε=e -σV
[0050] In the formula, ε is the contrast threshold for the observer to identify the target object from the background, σ is the horizontal extinction coefficient, V is the horizontal visibility of the air, and the ε value recommended by ICAO is 0.05. Substituting it into the above formula, it is easy to get:
[0051]
[0052] In S1, the relationship between the millimeter-wave radar measurement data and visibility is calibrated during calculation. According to the derivation in S1, visibility is uniquely determined by the extinction coefficient σ, and its value is related to the total amount of particles in the air, that is, the size of each particle, which can be expressed as:
[0053]
[0054] In the formula, k i represents the scattering extinction coefficient, r i represents the particle radius, n i Represents the radius r i The number of particles, radar reflectivity factor Z and r i , n i The relationship can be expressed as:
[0055] Z=∑ i n i (2r i ) 6
[0056] Therefore, the radar reflectivity factor can be used as the basis for characterizing visibility. In S3, the relationship between the image information obtained by the digital camera method and visibility is first calibrated. The intuitive feeling of visibility is the change in clarity and blur of objects in the image field of view. The higher the visibility, the richer the details of the image in the field of view, and the smaller the color distortion. For ordinary visible light digital camera methods, the daytime and nighttime visibility detection models are different. The photosensitive sensor is used to sense the current ambient light intensity. When it is higher than the brightness threshold, the daytime visibility digital camera detection method is used, and when it is lower than the brightness threshold, the nighttime visibility digital camera detection method is used. The detection method is refined and the detection efficiency is improved.
[0057] It should be noted that the daytime visibility detection model in S3 is the same as that derived in S1. The visibility is uniquely determined by the extinction coefficient σ. Similar to the way the human eye observes the clarity of an object, the visibility value is characterized by the color distortion condition. A reference object with a known color distribution is placed at a fixed distance L0 in front of the digital camera. Assuming that the reference object occupies a total of m rows and n columns of pixels in the image, the original gray value of each pixel is recorded as g0(i,j), 1≤i≤m, 1≤j≤n, and the gray value of the image captured by the digital camera after being transmitted through the air to be tested is recorded as g(i,j). The average distortion rate of the image can be expressed as:
[0058]
[0059] The relationship between the color distortion rate and visibility of digital images is calibrated by S4 artificial simulation test cabin or outdoor measurement. The night digital camera method uses the attenuation law of the light source after propagation through the air to be measured. Two groups of target objects and light sources are set in advance. The brightness of the light source itself is L0. The distances between the two groups of target objects and the light source are d1 and d2 respectively. The apparent brightness of the light source received by the camera is L a1 and L a2 , the target brightness is L b1 and L b2 , the calculation expression of visibility is:
[0060]
[0061] In particular, in the selected calibration environment in S4, the millimeter-wave radar and the digital camera are placed in the same position, the corner reflector required for the calibration in S1, the known color distribution reference required for the calibration in S3, and the light source and target group required for the calibration are placed in the same position, facing the millimeter-wave radar and the digital camera, keeping a distance, and the drop spectrometer, photosensor, precipitation sensor, and air quality meter are placed in the same measurement environment. When calibrating the relationship between the reflectivity factor and visibility of the radar in S1, the radar transmits a beam through the simulated air environment to illuminate the corner reflector, and the reflected echo is received by the radar through the simulated air environment, and the record is Record radar reflectivity factor data and calibrate S3. During the day, the digital camera records the color distortion rate data of the reference object with known color distribution. At night, the method in S3 is used to record visibility and light intensity data. At the same time, the calibration and measurement process is completed in conjunction with the drop spectrometer, photosensor, precipitation sensor, and air quality meter. Millimeter-wave radar and digital camera are selected, and weather sensors such as drop spectrometer, photosensor, precipitation sensor and other sensors are used for information fusion processing. The advantages of each sensor are fully utilized. The high measurement accuracy of millimeter-wave radar can be To make up for the large measurement error of the digital camera method, and the high adaptability of the digital camera method to adverse weather conditions and complex sites can make up for the impact of rain attenuation on the working performance of the millimeter-wave radar. Weather sensors such as drop spectrometer, photosensitive sensor, precipitation sensor, and air quality meter are used to sense the current weather conditions. The drop spectrometer mainly provides the current air droplet spectrum (droplet radius-concentration information) for determining the relationship curve between radar reflection factor and visibility. The photosensitive sensor mainly senses the current light intensity and is used to provide light intensity information and switch the digital camera method day / night working model. The precipitation sensor provides the current precipitation. Water volume information, the air quality meter obtains the main components and content of the floating particles in the current air. The method of using multiple sensors to detect and manually calibrate and correct abnormal data in S1-S6 can obtain the radar reflectivity factor, digital camera color distortion rate, fog water droplet size and water content, air quality such as metal particles and dust content, light intensity, and precipitation data relationship under various visibility conditions. In order to increase universality and environmental adaptability, in artificial simulation test cabins or outdoor measurements, full consideration is given to weather conditions, lighting conditions, temperature and humidity conditions, and a measured data set of multi-sensor data and visibility under various conditions is established.
[0062] In the description of this article, it should be noted that relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0063] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A visibility measurement method based on millimeter wave radar and multi-sensor information fusion, characterized in that: The following steps are involved: S1: Select the millimeter-wave radar, camera and weather sensor models, then preset the reference objects and detection parameters, select the calibration test environment, then configure the radar parameters, obtain the radar reflectivity factor parameters, and then calculate the millimeter-wave radar visibility; S2: Read weather sensor data to obtain data on fog droplet size and water content, air quality, metal particles, sand and dust content, light intensity, and precipitation; S3: Determine whether the current illumination is lower than the threshold. If yes, use the night camera to detect the visibility model, then perform night image preprocessing, obtain the brightness of the target object and the light source, and then use the night digital camera method to calculate the visibility. If no, use the daytime camera to detect the visibility model, then perform daytime image preprocessing, calculate the average image distortion rate, and finally use the daytime digital camera method to calculate the visibility. S4: Manual calibration and correction of abnormal data. The millimeter wave radar visibility calculation results in S1, the night digital camera visibility calculation results in S3, and the daytime digital camera visibility calculation results are all manually calibrated to correct abnormal data, and then a multi-sensor data set and a measured visibility data set are constructed; S5: The artificial neural network training data set fuses the multi-sensor information and then obtains the multi-sensor visibility measurement model.
2. The visibility measurement method based on millimeter wave radar and multi-sensor information fusion according to claim 1 is characterized in that: The visibility in the air in S1 is a physical quantity that describes the degree of air transparency. It is defined as the visible distance of a special sign or luminous body. According to Koschmider's law, it satisfies the following formula: e=e -σV In the formula, ε is the contrast threshold for the observer to identify the target object from the background, σ is the horizontal extinction coefficient, V is the horizontal visibility of the air, and the ε value recommended by ICAO is 0.
05. Substituting it into the above formula, it is easy to get:
3. The visibility measurement method based on millimeter wave radar and multi-sensor information fusion according to claim 2 is characterized in that: The relationship between the millimeter-wave radar measurement data and visibility is calibrated in the calculation in S1. According to the derivation in S1, visibility is uniquely determined by the extinction coefficient σ, and its value is related to the total amount of particles in the air, that is, the size of each particle, and can be expressed as: In the formula, k i represents the scattering extinction coefficient, r i represents the particle radius, n i Represents the radius r i The number of particles, radar reflectivity factor Z and r i , n i The relationship can be expressed as: Z=∑ i n i (2r i ) 6 。 4. The visibility measurement method based on millimeter wave radar and multi-sensor information fusion according to claim 1 is characterized in that: In the S3, the relationship between the image information obtained by the digital camera method and the visibility is first calibrated. The intuitive feeling of the visibility is the change in clarity and blur of objects in the image field of view. The higher the visibility, the richer the detailed features of the image in the field of view, and the smaller the color distortion.
5. The visibility measurement method based on millimeter wave radar and multi-sensor information fusion according to claim 4 is characterized in that: The daytime visibility detection model in S3 is the same as that derived in S1. The visibility is uniquely determined by the extinction coefficient σ. Similar to the way the human eye observes the clarity of an object, the visibility value is characterized by the color distortion condition. A reference object with a known color distribution is placed at a fixed distance L0 in front of the digital camera. Assuming that the reference object occupies a total of m rows and n columns of pixels in the image, the original gray value of each pixel is recorded as g0(i,j), 1≤i≤m, 1≤j≤n, and the gray value of the image captured by the digital camera after being transmitted through the air to be tested is recorded as g(i,j). The average distortion rate of the image can be expressed as: The relationship between the color distortion rate and visibility of digital images is calibrated by S4 artificial simulation test cabin or outdoor measurement. The night digital camera method uses the attenuation law of the light source after propagation through the air to be measured. Two groups of target objects and light sources are set in advance. The brightness of the light source itself is L0. The distances between the two groups of target objects and the light source and the camera are d1 and d2 respectively. The apparent brightness of the light source received by the camera is L a1 and L a2 , the target brightness is L b1 and L b2 , the calculation expression of visibility is:
6. The visibility measurement method based on millimeter wave radar and multi-sensor information fusion according to claim 5 is characterized in that: In the selected calibration environment in S4, the millimeter-wave radar and the digital camera are placed in the same position, the corner reflector required for the calibration in S1, the known color distribution reference object required for the calibration in S3, and the light source and target object group required for the calibration are placed in the same position, facing the millimeter-wave radar and the digital camera, and maintaining a distance. The drop spectrometer, the photosensor, the precipitation sensor, and the air quality meter are placed in the same measurement environment. When calibrating the relationship between the reflectivity factor and visibility of the radar in S1, the radar transmits a beam through a simulated air environment to illuminate the corner reflector, and the reflected echo is received by the radar through the simulated air environment, and the radar reflectivity factor data is recorded. When calibrating the relationship between the color distortion rate of the digital image and the visibility during the day in S3, the digital camera records the color distortion rate data of the known color distribution reference object. At night, the method in S3 is used to record visibility and light intensity data, and the drop spectrometer, the photosensor, the precipitation sensor, and the air quality meter are used to complete the calibration and measurement process.
7. The visibility measurement method based on millimeter wave radar and multi-sensor information fusion according to claim 6 is characterized in that: When conducting the artificial simulation test chamber or outdoor measurement, the weather conditions, lighting conditions, temperature and humidity conditions are fully considered to establish a measured data set of multi-sensor data and visibility under various conditions.
