Rain and fog weather multi-sensor fusion target detection system and method based on fuzzy control

Abstract

The invention discloses a rain and fog weather multi-sensor fusion target detection system and method based on fuzzy control, and belongs to the field of target detection. The system comprises a target distance and speed detection sensor group used for detecting the distance and the speed of a target and sending the distance and the speed to a data processing module; the environmental parameter acquisition sensor group is used for detecting current environmental parameters and sending the current environmental parameters to the data processing module, including rainfall, visibility and the like; and the data processing module is used for carrying out fuzzy processing on the environmental parameters, dynamically adjusting fusion weights of the sensors in the target distance and speed detection sensor group through a fuzzy control algorithm, and realizing target detection through weighted fusion. According to the invention, through the application of the fuzzy control algorithm, the accuracy and robustness of target detection in complex weather are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of target detection, and particularly relates to a rain and fog weather multi-sensor fusion target detection system and method based on fuzzy control. BACKGROUND

[0002] Currently, in the field of intelligent driving technology, there are mainly two common ways in sensor fusion processing.

[0003] First, the target filtering algorithm is determined according to the sensor type and obstacle information, so as to improve the fusion accuracy. However, this method does not fully consider the inherent defects of various sensors in rain and fog and other bad weather scenes. For example, millimeter wave radar may be disturbed by raindrop scattering, laser beam of laser radar may be scattered and absorbed by fog, and image quality of camera may be reduced due to raindrop attachment and fog blocking, which all result in the decrease of accuracy and reliability of sensor data.

[0004] Second, the sensor fusion is realized by weight distribution (mainly using weighted average method) according to weather state and network state. However, the weighted average method is based on fixed weight, which is too rough and cannot adapt to complex environmental changes, and the fusion result deviates from the actual situation, reducing the reliability and effectiveness of fusion.

[0005] Therefore, the present application proposes a rain and fog weather multi-sensor fusion target detection system and method based on fuzzy control. SUMMARY

[0006] The present application aims to overcome the shortcomings of the prior art and proposes a rain and fog weather multi-sensor fusion target detection system and method based on fuzzy control, so as to achieve the following purposes: improving the accuracy and robustness of target detection in complex weather.

[0007] In order to achieve the above purposes, the technical scheme adopted by the present application is as follows: a rain and fog weather multi-sensor fusion target detection system based on fuzzy control, which comprises a target distance and speed detection sensor group, an environmental parameter acquisition sensor group and a data processing module, wherein the data processing module is connected with the target distance and speed detection sensor group and the environmental parameter acquisition sensor group respectively.

[0008] The target distance and speed detection sensor group is used for detecting the distance and speed of the target and sending them to the data processing module.

[0009] The environmental parameter acquisition sensor group is used for detecting the current environmental parameters and sending them to the data processing module, including rainfall, visibility, etc.

[0010] The data processing module is used for dynamically adjusting the fusion weight of each sensor in the target distance and speed detection sensor group through a fuzzy control algorithm after the environment parameter is processed, and realizing target detection through weighted fusion.

[0011] Preferably, the target distance and speed detection sensor group comprises a millimeter wave radar, a camera, a laser radar, etc., and the millimeter wave radar, the camera and the laser radar are connected with the data processing module.

[0012] Preferably, the environment parameter acquisition sensor group comprises a visibility sensor and a rainfall sensor, and the visibility sensor and the rainfall sensor are connected with the data processing module.

[0013] Preferably, the data processing module adopts a microcontroller MCU.

[0014] The application further provides a fuzzy control-based rain and fog weather multi-sensor fusion target detection method.

[0015] Step S1: collecting distance and speed data of a target through a millimeter wave radar, a camera and a laser radar respectively and sending the data to the data processing module;

[0016] Step S2: collecting visibility and rainfall data of a current environment through a visibility sensor and a rainfall sensor respectively and sending the data to the data processing module;

[0017] Step S3: performing fuzzy processing on the visibility and rainfall data by the data processing module;

[0018] Step S4: according to the fuzzy processing result and the millimeter wave radar, camera and laser radar parameters, the data processing module correspondingly establishes a fuzzy rule base and calculates a fuzzy rule trigger strength;

[0019] Step S5: adjusting the weighted weight of the millimeter wave radar, camera and laser radar according to the fuzzy rule and the trigger strength thereof;

[0020] Step S6: obtaining target detection position and speed information through weighted fusion.

[0021] Preferably, the step S3 comprises:

[0022] The rainfall is processed in a fuzzy manner, three rainfall fuzzy sets, i.e., light rain, medium rain and heavy rain, are defined, and corresponding Gaussian membership functions are as follows:

[0023] Light rain:

[0024] Medium rain:

[0025] Heavy rain:

[0026] Wherein, R represents real-time rainfall;

[0027] The fog amount is fuzzed, and four fog amount fuzzy sets, i.e. no fog, light fog, medium fog, and heavy fog, are defined, and the corresponding Gaussian membership functions are as follows:

[0028] No fog:

[0029] Light fog:

[0030] Medium fog:

[0031] Heavy fog:

[0032] Wherein, F represents real-time visibility.

[0033] Preferably, the step S4 comprises:

[0034] Each fuzzy rule in the fuzzy rule base is in the following form:

[0035] If R = A i and F = B j , then

[0036] Wherein, A i represents the ith rainfall fuzzy set; B j represents the jth fog amount fuzzy set; respectively represent the preset weighting weights of the millimeter wave radar, laser radar, and camera under the current fuzzy rule, which are related to the corresponding sensor parameters, including the signal-to-noise ratio of the radar and the definition of the camera image;

[0037] The fuzzy rule triggering strength a ij is calculated by taking the minimum operation min:

[0038]

[0039] Wherein, represents the Gaussian membership degree function value of the ith rainfall fuzzy set represents the Gaussian membership degree function value of the jth fog amount fuzzy set.

[0040] Preferably, in the step S5, the weighting weights of the millimeter wave radar, camera, and laser radar are adjusted according to the fuzzy rule and the triggering strength thereof, and are expressed by the following formula:

[0041]

[0042] wherein, W radar ‘, W lidar ‘, W cam ‘ respectively represent the adjusted weighted weight of the millimeter wave radar, laser radar and camera.

[0043] Preferably, the step S6 comprises:

[0044] Distance fusion:

[0045] d fused =W radar ‘·d radar +W lidar ‘·d lidar +W cam ‘·d cam ;

[0046] wherein, d radar , d lidar , d cam respectively represent the target distance collected by the millimeter wave radar, laser radar and camera; d fused represents the target position information obtained after weighted fusion.

[0047] Preferably, the step S6 comprises:

[0048] Introducing variance reciprocal weighting for speed fusion:

[0049]

[0050] wherein, σ r , σ l , σ c are the speed measurement standard deviations of the millimeter wave radar, laser radar and camera; v radar , v lidar , v cam respectively represent the target speed collected by the millimeter wave radar, laser radar and camera; v fused represents the target speed information obtained after weighted fusion.

[0051] The technical effect of the present application is that the present application solves the three core problems of poor adaptability, low safety and high cost of traditional multi-sensor fusion scheme in bad weather through the innovative combination of fuzzy control and dynamic weight, and improves the robustness of target detection in complex weather through the dual action of environmental parameters and sensor performance. Further, the stability and accuracy of the intelligent driving system in complex environment are ensured. The present application adopts a fuzzy control algorithm, which reduces resource consumption compared to a deep learning scheme, and is suitable for vehicle embedded deployment. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 A structure block diagram of a rain and fog weather multi-sensor fusion target detection system based on fuzzy control is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0053] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings, and the description of the embodiments is to help the skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solutions of the present application, and to facilitate its implementation. It should be noted that the terms "first", "second" and the like in the present application are only for the convenience of describing the technical solutions and are used as a distinction between components, and the corresponding component configurations may be the same or different, and the present application is not limited to this. In order to make the technical solutions of the present application clearer, the present application is explained and described by the following embodiments.

[0054] The present embodiment provides a rain and fog weather multi-sensor fusion target detection system based on fuzzy control, as shown in Figure 1 The system includes a target distance and speed detection sensor group, an environment parameter acquisition sensor group, and a data processing module. The data processing module is connected to the target distance and speed detection sensor group and the environment parameter acquisition sensor group. The entire system has a simple structure and low layout cost. Among them:

[0055] The target distance and speed detection sensor group is used to detect the distance and speed of the target and send it to the data processing module;

[0056] The environment parameter acquisition sensor group is used to detect the current environment parameters and send them to the data processing module, including rainfall, visibility;

[0057] The data processing module is used to fuzz the environment parameters, dynamically adjust the fusion weights of each sensor in the target distance and speed detection sensor group through a fuzzy control algorithm, and realize target detection through weighted fusion.

[0058] Specifically, the target distance and speed detection sensor group of the present embodiment includes a millimeter wave radar, a camera, and a laser radar, all of which are connected to the data processing module. The millimeter wave radar installed at different positions of the vehicle can detect the distance, speed and other information of the target in real time, such as a 4D millimeter wave radar. The camera can obtain the distance, speed and other information of the target through images; the laser radar is used to construct a three-dimensional point cloud model of the target and obtain the distance, speed and other information of the target, allowing the use of low-cost sensors (such as 16-line laser radar instead of 64-line models) to maintain high performance in heavy rain scenarios.

[0059] The environmental parameter acquisition sensor group comprises an visibility sensor, a rainfall sensor, and the visibility sensor and the rainfall sensor are connected with the data processing module.

[0060] In addition, the data processing module of the embodiment adopts a microcontroller MCU, and other processors can also be selected according to actual conditions in specific implementation. The MCU has the advantages of small size and high integration, can reduce the system cost, and is compatible with data input of multiple sensors. The MCU supports the construction of an algorithm, and the real-time performance of the system is improved by virtue of the efficient data processing capability thereof.

[0061] The embodiment also provides a rain and fog weather multi-sensor fusion target detection method based on fuzzy control, using the rain and fog weather multi-sensor fusion target detection system based on fuzzy control.

[0062] Step S1, distance and speed data of a target are collected by a millimeter wave radar, a camera and a laser radar respectively and are sent to the data processing module;

[0063] Step S2, visibility and rainfall data of a current environment are collected by a visibility sensor and a rainfall sensor respectively and are sent to the data processing module;

[0064] Step S3, the data processing module performs fuzzy processing on the visibility and rainfall data;

[0065] Step S4, according to the fuzzy processing result and the millimeter wave radar, camera and laser radar parameters, the data processing module correspondingly establishes a fuzzy rule library and calculates a fuzzy rule trigger strength;

[0066] Step S5, the weighted weights of the millimeter wave radar, camera and laser radar are adjusted according to the fuzzy rule and the trigger strength thereof;

[0067] Step S6, weighted fusion is performed to obtain position and speed information of a target detection object.

[0068] Specifically, in steps S3 to S6 of the embodiment, the data processing module fuses rain and fog weather conditions and sensor detection data based on a fuzzy control algorithm, thereby avoiding the influence of adverse weather on the performance of the sensor, and enabling data of different sensors to be effectively fused under the premise of maintaining high reliability.

[0069] In step S3, a fuzzy set is first constructed, that is, rainfall and fog amount associated with rain and fog weather conditions are subjected to fuzzy processing by using a Gaussian membership function, including:

[0070] The rainfall amount is subjected to fuzzy processing, and three rainfall fuzzy sets, that is, light rain, moderate rain and heavy rain, are defined, and corresponding Gaussian membership functions are as follows:

[0071] Light rain (LR):

[0072] Moderate rain (MR):

[0073] Heavy rain (HR):

[0074] wherein R represents real-time rainfall; in μm LR (R) as an example, μ LR (R) represents the degree of membership of R to the fuzzy set (light rain) when the input value is R, i.e., the possibility of its classification as light rain when the input value is R.

[0075] The fog amount is fuzzified, and four fog amount fuzzy sets, i.e., no fog, light fog, moderate fog, and heavy fog, are defined, and the corresponding Gaussian membership functions are as follows:

[0076] No fog (NF):

[0077] Light fog (LF):

[0078] Moderate fog (MF):

[0079] Heavy fog (HF):

[0080] wherein F represents real-time visibility. in μm NF (F) as an example, μ NF (F) represents the degree of membership of F to the fuzzy set (no fog) when the input value is F, i.e., the possibility of its classification as no fog when the input value is F.

[0081] Further, in the preferred embodiment of the present application, by adjusting the membership function parameters, it can be extended to scenarios such as sand and snow, and the collection input of weather variables such as PM2.5 or temperature can be increased synchronously.

[0082] Then, in step S4, a fuzzy rule base is constructed, wherein the setting of each fuzzy rule needs to be associated with a set of weighted weights of millimeter wave radars, laser radars, and cameras. The size of the weight value is related to the corresponding sensor parameters, including the signal-to-noise ratio of the radar, the definition of the camera image, etc. In specific implementation, the system can reserve an API interface for online updating of the fuzzy rule base, so as to facilitate real-time adjustment of the fuzzy rules according to different situations. Specifically, the step S4 includes:

[0083] Each fuzzy rule in the fuzzy rule base is in the following form:

[0084] If R = A iand F = B j then

[0085] wherein A i represents the i-th rain amount fuzzy set; B j represents the j-th fog amount fuzzy set; respectively represent the preset weighting weights of the millimeter wave radar, laser radar and camera under the current fuzzy rule, which are related to corresponding sensor parameters, including the signal-to-noise ratio of the radar and the definition of the camera image. For example, if R belongs to the light rain fuzzy set and F belongs to the heavy fog fuzzy set, that is, the current weather is light rain and heavy fog, a set of weighting weights of the millimeter wave radar, laser radar and camera are set correspondingly. Such a fuzzy rule form realizes the setting of the sensor weighting weights under different weather conditions.

[0086] Then, the fuzzy rule triggering strength a ij is calculated by taking the minimum operation min.

[0087]

[0088] wherein, represents the Gaussian membership function value of the i-th rain amount fuzzy set represents the Gaussian membership function value of the j-th fog amount fuzzy set. a ij is used to adjust the weighting weights of the millimeter wave radar, camera and laser radar, so that the corresponding weighting weights are more suitable for the current weather environment, and the accuracy of the target detection result is improved.

[0089] Then, in step S5, the embodiment adjusts the weighting weights of the millimeter wave radar, camera and laser radar according to the fuzzy rule and its triggering strength, which is expressed by the following formula:

[0090]

[0091] W radar ‘+ W lidar ‘+ W cam ‘= 1

[0092] wherein W radar ‘, W lidar ‘, W camrespectively represent the adjusted weighting weights of millimeter wave radar, laser radar, camera. In rainy and foggy weather, the reliability of sensors may be different. The fuzzy control algorithm dynamically adjusts the sensor weights to ensure that the system can adapt flexibly according to different weather conditions, reduce false positives and omissions, and improve the robustness of target detection in complex weather; at the same time, it can avoid the influence of bad weather on the performance of sensors, so that the data of different sensors can be effectively fused under the premise of maintaining high reliability, and the accuracy of target detection is improved.

[0093] Finally, in step S6, the weighted fusion of the data collected by the millimeter wave radar, laser radar and camera is completed to realize target detection, specifically including:

[0094] Distance fusion: the target distance collected by the millimeter wave radar, laser radar and camera is weighted and fused to realize the detection of the target position, which is expressed by the following formula:

[0095] d fused +W radar ‘·d radar +W lidar ‘·d lidar +W cam ‘·d cam ;

[0096] Wherein, d radar , d lidar , d cam respectively represent the target distance collected by the millimeter wave radar, laser radar and camera; d fused represents the target position information obtained after weighted fusion.

[0097] In addition, considering the difference in sensor measurement accuracy, the reciprocal of variance weighting is introduced for speed fusion: the target speed collected by the millimeter wave radar, laser radar and camera is weighted and fused to realize the detection of the target speed, which is expressed by the following formula:

[0098]

[0099] Wherein, σ r , σ l , σ c are the speed measurement standard deviations of millimeter wave radar, laser radar and camera; v radar , v lidar , v cam respectively represent the target speed collected by the millimeter wave radar, laser radar and camera; v fused represents the target speed information obtained after weighted fusion. Among them, the reciprocal of variance weighting not only considers the influence of weather (fuzzy weight), but also retains the inherent accuracy of sensors, and the estimated error of actual speed is significantly reduced compared with single weighting method.

[0100] In summary, the present application solves the three core problems of poor adaptability, low safety and high cost of traditional multi-sensor fusion scheme in bad weather through the innovative combination of fuzzy control and dynamic weight, and improves the robustness of target detection in complex weather through the dual action of environmental parameters and sensor performance, thereby ensuring the stability and accuracy of the intelligent driving system in complex environments. The present application uses a fuzzy control algorithm, which reduces resource consumption compared to deep learning solutions and is suitable for vehicle embedded deployment.

[0101] The present application has been described above with reference to the drawings. Obviously, the specific implementation of the present application is not limited by the above manner. As long as various non-essential improvements are made using the method concept and technical solution of the present application, or the above concept and technical solution of the present application is directly applied to other occasions without improvement, they are all within the protection scope of the present application.

Claims

1. A multi-sensor fusion target detection system for rain and fog weather based on fuzzy control, characterized in that: The system includes a target distance and velocity detection sensor group, an environmental parameter acquisition sensor group, and a data processing module. The data processing module is connected to both the target distance and velocity detection sensor group and the environmental parameter acquisition sensor group. The target distance and velocity detection sensor group is used to detect the target's distance and velocity and send them to the data processing module; The environmental parameter acquisition sensor group is used to detect the current environmental parameters and send them to the data processing module, including rainfall and visibility; The data processing module is used to fuzzify the environmental parameters, dynamically adjust the fusion weights of each sensor in the target distance and speed detection sensor group through a fuzzy control algorithm, and achieve target detection through weighted fusion.

2. The multi-sensor fusion target detection system for rain and fog weather based on fuzzy control according to claim 1, characterized in that: The target distance and speed detection sensor group includes a millimeter-wave radar, a camera, and a lidar, all of which are connected to the data processing module.

3. The multi-sensor fusion target detection system for rain and fog weather based on fuzzy control according to claim 1, characterized in that: The environmental parameter acquisition sensor group includes a visibility sensor and a rainfall sensor, both of which are connected to the data processing module.

4. A multi-sensor fusion target detection system for rain and fog weather based on fuzzy control according to any one of claims 1-3, characterized in that: The data processing module uses a microcontroller (MCU).

5. A multi-sensor fusion target detection method for rain and fog weather based on fuzzy control, using a multi-sensor fusion target detection system for rain and fog weather based on fuzzy control according to any one of claims 1-4, characterized in that: The method includes the following steps: Step S1: Collect the target's distance and velocity data using millimeter-wave radar, camera, and lidar respectively, and send them to the data processing module; Step S2: Collect the visibility and rainfall data of the current environment through the visibility sensor and the rainfall sensor respectively, and send them to the data processing module; Step S3: The data processing module performs fuzzing processing on the visibility and rainfall data; Step S4: Based on the fuzzing processing result and the parameters of the millimeter-wave radar, camera, and lidar, the data processing module establishes a corresponding fuzzy rule library and calculates the fuzzy rule triggering strength. Step S5: Adjust the weighting of the millimeter-wave radar, camera, and lidar according to the fuzzy rules and their triggering strength; Step S6: Perform weighted fusion to obtain the position and velocity information of the target object.

6. The multi-sensor fusion target detection method for rain and fog weather based on fuzzy control according to claim 5, characterized in that: Step S3 includes: The rainfall data is fuzzified by defining three fuzzy sets: light rain, moderate rain, and heavy rain. The corresponding Gaussian membership functions are as follows: Light rain: Moderate rain: heavy rain: Where R represents real-time rainfall; The fog level is fuzzified, and four fuzzy fog sets are defined: no fog, light fog, medium fog, and heavy fog. The corresponding Gaussian membership functions are as follows: No fog: Light mist: Moderate fog: Dense fog: Where F represents real-time visibility.

7. The multi-sensor fusion target detection method for rain and fog weather based on fuzzy control according to claim 6, characterized in that: Step S4 includes: Each fuzzy rule in the fuzzy rule base takes the following form: If R = A i And F = B j ,but Among them, A i B represents the i-th fuzzy set of rainfall; j Represents the j-th fog fuzzy set; These represent the preset weights of millimeter-wave radar, lidar, and camera under the current fuzzy rules, respectively, which are related to the corresponding sensor parameters, including the signal-to-noise ratio of the radar and the clarity of the camera image; The triggering strength α of the fuzzy rule is calculated by taking the minimum value (min). ij : in, This represents the Gaussian membership function value of the i-th fuzzy rainfall set. This represents the Gaussian membership function value of the j-th fog fuzzy set.

8. The multi-sensor fusion target detection method for rain and fog weather based on fuzzy control according to claim 7, characterized in that: In step S5, the weights of the millimeter-wave radar, camera, and lidar are adjusted according to the fuzzy rules and their trigger strengths, as expressed by the following formula: IN radar '+W lidar '+W cam '=1 Among them, W radar '、W lidar '、W cam 'Represents the adjusted weights of millimeter-wave radar, lidar, and camera, respectively.

9. A multi-sensor fusion target detection method for rain and fog weather based on fuzzy control according to claim 8, characterized in that: Step S6 includes: Distance fusion: d fused =W radar ‘·d radar +W lidar ‘·d lidar +W cam ‘·d cam ; Where, d radar d lidar d cam These represent the target distances acquired by millimeter-wave radar, lidar, and camera, respectively; d fused This indicates that the target location information is obtained after weighted fusion.

10. A multi-sensor fusion target detection method for rain and fog weather based on fuzzy control, as described in claim 8 or 9, characterized in that: Step S6 includes: Introducing variance inverse weighting for speed fusion: Where, σ r ,σ l ,σ c Standard deviation for velocity measurements of millimeter-wave radar, lidar, and cameras; v radar v lidar v cam These represent the target velocities captured by millimeter-wave radar, lidar, and camera, respectively; v fused This indicates that the target speed information is obtained after weighted fusion.

Images