Rainfall sensor control system and method based on acousto-optic fusion

Abstract

The invention provides a rainfall sensor control system and method based on acousto-optic fusion, the control system comprises a sensor module and a signal processing system, the sensor module comprises a rainfall sensor and an acoustic sensor; wherein the rainfall sensor is used for detecting a rainfall signal in a current environment; the acoustic sensor is used for detecting an audio signal generated by knocking a front windshield of a vehicle by raindrops in the current environment; the signal processing system is used for processing the rainfall signal and the audio signal, generating weights corresponding to the rainfall signal and the audio signal respectively, and determining whether the current environment rains or not and the grade of rainfall according to the rainfall signal, the audio signal and the corresponding weights; the state of the vehicle is controlled according to whether it rains or not and the rainfall level. According to the invention, the defects of the rain sensor can be complemented through the acoustic sensor, and the monitoring accuracy of the automobile rain and light sensor is improved.

Description

Technical Field

[0001] The present invention relates to the field of sensor technology, and more specifically, to a rainfall sensor control system and a control method based on acoustic-optical fusion. Background Art

[0002] At present, more and more models on the market are equipped with rain and light sensors, which monitor the amount of rain on the car and identify the intensity of ambient light based on infrared reflection and photosensitive sensors. In particular, some new rain and light sensors integrate temperature and humidity sensors, which can identify the temperature and humidity in the car in real time, and control the wipers in conjunction with the domain controller and some actuators in the car, further improving the driver's and passengers' experience of using the car.

[0003] However, the infrared light-based rain sensor module does not sense rain by directly picking up rain signals, but rather achieves rain sensing through the principles of optical refraction and light intensity. The rain light sensor is generally installed on the windshield. In actual use, any change in refraction conditions (foreign objects, human touch) within the detection area of ​​the rain sensor will cause the sensor to misjudge. At the same time, the pickup area of ​​the rain sensor is relatively small. In the presence of wind interference, rain cannot fall evenly on the windshield, causing misjudgment and affecting the user experience. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a rain sensor control system and method based on sound and light fusion, so as to solve the problems of existing vehicle-mounted sensors, such as single basis for rainfall judgment, easy sensor misjudgment, and affected user experience.

[0005] The rain sensor control system based on sound and light fusion provided by the present invention comprises a sensor module and a signal processing system, wherein the sensor module comprises a rain sensor and an acoustic sensor; wherein the rain sensor is used to detect a rain signal in a current environment; the acoustic sensor is used to detect an audio signal generated by raindrops hitting a front windshield of a vehicle in the current environment; the signal processing system is used to process the rain signal and the audio signal, generate weights corresponding to the rain signal and the audio signal respectively, and determine whether it is raining in the current environment and the level of rain according to the rain signal, the audio signal and the corresponding weights, and control the state of the vehicle according to whether it is raining and the level of rain.

[0006] In addition, a preferred technical solution is that the rain sensor includes at least one group of infrared rain sensor chips, and the infrared rain sensor chip includes an infrared transmitting device and an infrared receiving device; wherein the infrared transmitting device is used to transmit infrared light to the windshield surface of the vehicle, and the infrared receiving device is used to receive infrared light reflected by the windshield.

[0007] In addition, a preferred technical solution is that the infrared rain sensor chip is attached to the front windshield of the vehicle, and the acoustic sensor is arranged on the contact surface between the infrared rain sensor chip and the front windshield; and, at least one acoustic sensor is arranged, and the number of the acoustic sensors arranged is the same as the number of the infrared rain sensor chips arranged.

[0008] In addition, a preferred technical solution is that the acoustic sensor is mounted on the windshield of the vehicle, and is used to pick up the vibration signal generated by the interaction of raindrop sounds, ambient sounds and wind noise signals generated by raindrops falling on the windshield and the windshield, and generate an audio signal corresponding to the vibration signal.

[0009] In addition, a preferred technical solution is that the signal processing system is also used to perform Fourier transform processing on the audio signal and extract the characteristic frequency points of the signal after Fourier transform processing; and, based on the characteristic frequency points, determine the matching degree between the current vibration signal and the preset signal; and determine whether it is raining in the current environment and the level of rainfall based on the matching degree.

[0010] In addition, a preferred technical solution is that the acoustic sensor includes a MEMS microphone or a VACC sensor.

[0011] In addition, a preferred technical solution is to further include an optical sensor; wherein the optical sensor is used to collect ambient light in the current environment of the vehicle, and convert the ambient light into an electrical signal and send it to the signal processing system; when the electrical signal is greater than a first preset threshold, the current environment is determined to be sunny; when the electrical signal is less than a second preset threshold, the current environment is determined to be cloudy and rainy; when the electrical signal is not greater than the first preset threshold and not less than the second preset threshold, the signal processing system determines that the current environment remains unchanged; the second preset threshold is less than the first preset threshold.

[0012] In addition, a preferred technical solution is that the signal processing system includes an AFE chip, an MCU and a LIN bus; wherein the AFE chip is used to convert the analog signals collected by the optical sensor and the rain sensor into digital signals, and transmit them to the MCU for processing through the serial port bus; the MCU is used to obtain the digital signal and the audio signal of the acoustic sensor for analysis to determine whether it is raining in the current environment and the level of rainfall.

[0013] In addition, a preferred technical solution is that the status of the vehicle includes wiper status, light status, air conditioning status, defog status, and HUD backlight brightness status.

[0014] On the other hand, the present invention also provides a rain sensor control method based on acoustic and optical fusion, including: collecting a rain signal from the external environment through a rain sensor; acquiring the weather conditions at the current location and the rain sensor's judgment on the rain signal through LIN bus communication, and generating calculation weights corresponding to the rain sensor and the acoustic sensor respectively; collecting audio information from the external environment through the acoustic sensor, and obtaining an audio signal after processing through a signal processing system; determining whether it is raining in the current environment and the level of rainfall according to the rainfall signal, the audio signal and the corresponding calculation weights, and controlling the state of the vehicle according to whether it is raining and the level of rainfall.

[0015] By utilizing the above-mentioned rain sensor control system and method based on acoustic-optical fusion, while collecting environmental rainfall signals through the rain sensor, the audio signal in the current environment is collected through the acoustic sensor, and the audio signal is used to assist in judging whether it is raining in the current environment. That is, the rainfall signal and the audio signal are comprehensively considered to avoid misjudgment of the rain sensor caused by foreign objects or human touch, and improve the accuracy of judging whether it is raining and the rainfall, thereby improving the user's driving experience.

[0016] In order to achieve the above and related purposes, one or more aspects of the present invention include the features that will be described in detail later. The following description and the accompanying drawings describe some exemplary aspects of the present invention in detail. However, these aspects indicate only some of the various ways in which the principles of the present invention can be used. In addition, the present invention is intended to include all these aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more apparent and easy to understand. In the accompanying drawings:

[0018] Figure 1 A principle block diagram of a rain sensor control system based on sound and light fusion according to an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of rainfall recognition of an acoustic sensor according to an embodiment of the present invention;

[0020] Figure 3 is a flow chart of a rain sensor control method based on sound and light fusion according to an embodiment of the present invention;

[0021] Figure 4Detailed flow chart of a rain sensor control method based on sound and light fusion according to an embodiment of the present invention.

[0022] The same reference numerals throughout the drawings indicate similar or corresponding features or functions. DETAILED DESCRIPTION

[0023] In the following description, for the purpose of illustration, in order to provide a comprehensive understanding of one or more embodiments, many specific details are set forth. However, it is apparent that these embodiments may also be implemented without these specific details. In other examples, for ease of describing one or more embodiments, known structures and devices are shown in the form of block diagrams.

[0024] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0025] In order to describe in detail the rain sensor control system and method based on sound and light fusion in the present invention, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Figure 1 The schematic principle of a rain sensor control system based on sound and light fusion according to an embodiment of the present invention is shown.

[0027] like Figure 1As shown, the rain sensor control system based on acoustic-optical fusion of the embodiment of the present invention includes a sensor module and a signal processing system, wherein the sensor module includes a rain sensor and an acoustic sensor; wherein the rain sensor is used to detect the rain signal in the current environment; the acoustic sensor is used to detect the audio signal generated by raindrops hitting the front windshield of the vehicle in the current environment; the signal processing system is used to process the rain signal and the audio signal, generate weights corresponding to the rain signal and the audio signal respectively, and determine whether it is raining in the current environment and the level of rain according to the rain signal, the audio signal and the corresponding weights, so as to control the state of the vehicle according to whether it is raining and the level of rain, and prevent the low accuracy of judgment only by the rain sensor, which is easy to cause misjudgment. Among them, the processing process of the signal processing system includes filtering the rain signal to filter out the influence of ambient light on the sensor, and filtering the audio signal to remove external noise, such as engine audio, wind noise and road noise, so as to improve the accuracy of subsequent signal processing.

[0028] Among them, the rain sensor further includes at least one group of infrared rain sensor chips, each infrared rain sensor chip includes an infrared transmitting device and an infrared receiving device; wherein the infrared transmitting device is used to transmit infrared light to the surface of the windshield of the vehicle, and the infrared receiving device is used to receive the infrared light reflected by the windshield. Based on the principle of light reflection, in a dry environment, the infrared light emitted by the infrared transmitting device is totally reflected when it reaches the surface of the front windshield, and the reflected light returns to the infrared receiving device through the lens. At this time, the light intensity signal received by the infrared receiving device is the strongest; and when there is rainfall, resulting in rainwater adhering to the glass surface, the light reflection medium changes, resulting in the condition of total reflection being destroyed, and a part of the infrared light is scattered. The infrared light emitted by the infrared transmitting device cannot be completely received by the infrared receiving device, resulting in the weakening of the light intensity signal received by the infrared receiving device, and then a preliminary judgment can be made on whether it is raining and the amount of rain through the change in the strength of the signal received by the infrared receiving device.

[0029] Specifically, the infrared rain sensor chip is attached to the front windshield of the vehicle, and the acoustic sensor is arranged at the center of the contact surface between the infrared rain sensor chip and the front windshield; and, the infrared rain sensor chip can be arranged in 2-4 groups, and the number of acoustic sensors arranged can be the same as the number of infrared rain sensor chips arranged. By attaching the acoustic sensor to the windshield of the vehicle, the vibration signal generated by the interaction of raindrop sound, ambient sound and wind noise signal generated by raindrops falling on the windshield and the windshield is picked up, and an audio signal corresponding to the vibration signal is generated, and transmitted to the signal processing system for processing through the PDM or I2S audio interface. Typical acoustic sensors may include MEMS microphones or VACC (voice accelerator) sensors.

[0030] Among them, the period of the pulse signal received by the infrared receiving device of the infrared rainfall sensor chip will change with the amount of rainfall, but its amplitude will not change. Therefore, by obtaining the relationship between the average voltage of the infrared receiving device of the infrared rainfall sensor chip and the amount of rainfall over a period of time, the current amount of rainfall can be calculated.

[0031] Figure 2 The schematic diagram of the principle of rainfall recognition based on the acoustic sensor signal according to the embodiment of the present invention is shown.

[0032] like Figure 2 As shown in the figure, the signal processing system processes the audio signal, including preprocessing, feature processing, rainfall classification and output results of the collected audio signal. Among them, the purpose of preprocessing the audio signal is to reduce noise, enhance signal quality, and improve signal recognizability and comprehensibility. The main steps of preprocessing include pre-emphasis, framing and windowing, and then feature processing is performed on the processed signal. Feature processing can use algorithms such as Log-Mel (Mel filter bank coefficients) and MFCC (Mel-Frequency Cepstral Coefficients) to process audio features and output the processed spectrum map. Then, the convolutional neural network CNN can be used for rainfall classification. Through the convolution operation of the convolutional layer of the convolutional neural network, the processed audio features are extracted and the convolution feature map is generated; then, the audio features are reduced in dimension through the pooling layer to reduce the size of the feature map and the complexity of the model. Finally, the features extracted by the convolutional layer and the pooling layer are classified and predicted through the fully connected layer to determine whether it is raining and the level of rainfall, and the final prediction result is output through the output layer.

[0033] In another specific embodiment of the present invention, in addition to using a convolutional neural network to process the audio signal, another processing method that can be used is: performing Fourier transform processing on the audio signal, and extracting the characteristic frequency points of the audio after the Fourier transform processing; then, judging the matching degree between the current vibration signal and the preset signal according to the characteristic frequency points; finally, determining whether it is raining in the current environment and the level of rainfall according to the matching degree.

[0034] It should be noted that in order to further improve the accuracy of rainfall detection, the sensor module may also include an optical sensor; wherein the optical sensor is used to collect ambient light in the current environment of the vehicle and convert the ambient light into an electrical signal and send it to the signal processing system; when the electrical signal is greater than the first preset threshold, the current environment is determined to be sunny; when the electrical signal is less than the second preset threshold, the current environment is determined to be cloudy and rainy; when the electrical signal is not greater than the first preset threshold and not less than the second preset threshold, the signal processing system determines that the current environment remains unchanged. The second preset threshold is less than the first preset threshold.

[0035] Specifically, the ambient light of the outside environment can be sensed by the built-in optical sensor of the sensor module, and converted into an electrical signal to be transmitted to the signal processing system for analysis. When the electrical signal is higher than the first preset threshold, the signal processing system determines that the outside world is more likely to be sunny; when the electrical signal is lower than the second preset threshold X2, the signal processing system will determine that the outside world is more likely to be cloudy and rainy. In order to prevent the signal processing system from misjudging due to the rapid change of the lighting environment of the outside environment, the hysteresis comparator principle of light detection can be adopted, that is, only when the signal detected by the light is lower than the lower limit threshold, it will be determined that the outside world is more likely to be cloudy and rainy; when the detected signal is higher than the upper limit threshold, it will be determined that the outside world is more likely to be sunny, and when it is within the upper and lower limit hysteresis range, the original judgment remains unchanged.

[0036] It should be noted that the signal processing system of the present invention includes an AFE chip, an MCU, a LIN bus, and a DC-DC (power supply module); wherein the AFE chip is used to convert the analog signals collected by the optical sensor and the rain sensor into digital signals, and transmit them to the MCU for processing through serial bus such as SPI and UART; the MCU is used to obtain the digital signal and the audio signal of the acoustic sensor for analysis to determine whether it is raining in the current environment and the level of rainfall. In the application process, the working state of the sensor module can be first detected according to the evaluation results of whether it is raining and the rainfall of the infrared rain sensor chip and the acoustic sensor, as well as the rainfall information of the weather forecast, to improve the reliability of the system. Secondly, when a sensor in the sensor module is in an abnormal state, other sensors are used to identify the rainfall separately, and the rainfall identification results of different sensors are no longer fused; finally, when the sensor state is normal, the evaluation results of the infrared rain sensor chip and the acoustic sensor are effectively fused to give play to their respective advantages under different light intensities.

[0037] Specifically, the fusion of multi-source sensor signals can be based on fuzzy control theory, with light intensity level and rainfall level as input, infrared reliability as output, and finally the result of the fuzzy controller obtained by the fuzzy controller. For example, rainfall level and light intensity jointly affect the reliability of infrared detection. When the light intensity is constant, the reliability of infrared detection gradually increases with the increase of rainfall level. When the rainfall intensity is high, as the light intensity decreases, the reliability of infrared detection increases; when the light intensity is high and the rainfall level is low, the reliability of infrared detection is almost 0, and the acoustic sensor is selected for detection; when the rainfall is heavy and the light intensity is low, the reliability of infrared detection is high, and infrared detection is more accurate. Therefore, the signal processing system can assign different calculation weights to the rain sensor, acoustic sensor and optical sensor according to the detection results, and then comprehensively judge whether it is raining and the rainfall level in the current environment through the calculation weights and the detection results of the rain sensor, acoustic sensor and optical sensor, and adjust the vehicle status according to the final whether it is raining and the rainfall level, thereby improving the detection accuracy and reducing the occurrence of misjudgment.

[0038] Among them, the vehicle status includes wiper status, light status, speaker status, microphone status, air conditioning status, defog status, HUD backlight brightness status, etc.

[0039] Corresponding to the above-mentioned rain sensor control system based on acoustic and optical fusion, the present invention also provides a rain sensor control method based on acoustic and optical fusion. Specifically, Figure 3 The schematic flow of a rain sensor control method based on sound and light fusion according to an embodiment of the present invention is shown.

[0040] like Figure 3 As shown, the rain sensor control method based on sound and light fusion of an embodiment of the present invention includes:

[0041] S100: collecting rainfall signals from the external environment through a rainfall sensor;

[0042] S200: Acquire the weather conditions at the current location and the judgment of the rainfall sensor on the rainfall signal through LIN bus communication, and generate calculation weights corresponding to the rainfall sensor and the acoustic sensor respectively;

[0043] S300: collecting audio information in the external environment through the acoustic sensor, and obtaining an audio signal after processing through the signal processing system;

[0044] S400: Determine whether it is raining in the current environment and the level of rainfall according to the rainfall signal, the audio signal and the corresponding calculation weights, and control the state of the vehicle according to whether it is raining and the level of rainfall.

[0045] As a specific example, Figure 4 The specific process of the rain sensor control method based on sound and light fusion according to an embodiment of the present invention is shown.

[0046] like Figure 4 As shown, in the rain sensor control method based on acoustic-optical fusion of an embodiment of the present invention, firstly, the rain sensor (or infrared rain sensor chip) is used to identify the rainfall of the environment and output the rainfall level, and the acoustic sensor is used to identify the rainfall of the environment and also output the corresponding rainfall level; then, the recognition results of the rain sensor and the acoustic sensor are combined with the weather forecast to perform a self-check of the sensor status of multiple information to improve the reliability of the system, and when the sensor status is normal, the multi-source information rainfall estimation is performed in combination with the light intensity detected by the optical sensor, and the final rainfall information (whether it is raining and the rainfall level) is output.

[0047] It can be seen that the present invention estimates rainfall respectively through a rain sensor and an acoustic sensor, and is responsible for collecting light intensity information through an optical sensor, and then combines the light intensity, infrared rainfall recognition results and acoustic rainfall recognition results. When the light intensity is high, the acoustic sensor is used to estimate rainfall mainly, and when the light intensity is weak, the rain sensor is used as the main rainfall estimation device, taking advantage of infrared in weak light intensity and the compensation advantage of the acoustic sensor to improve the advantage of rainfall recognition.

[0048] It should be noted that the embodiments of the above-mentioned rain sensor control method based on sound and light fusion can refer to the description in the above-mentioned rain sensor control system embodiment based on sound and light fusion, and will not be repeated here.

[0049] According to the rain sensor control system and method based on acoustic and optical fusion of the present invention, while collecting environmental rainfall signals through the rain sensor, audio signals in the current environment are collected through the acoustic sensor, and ambient light intensity is collected through the optical sensor. The audio signal and light intensity are then used to assist in determining whether it is raining in the current environment, thereby avoiding misjudgment of the rain sensor caused by foreign objects or human touch, and improving the accuracy of determining whether it is raining and the amount of rainfall, thereby improving the user's driving experience.

[0050] As described above with reference to the accompanying drawings, the rain sensor control system and method based on acoustic-optical fusion according to the present invention are described by way of example. However, those skilled in the art should understand that various improvements can be made to the rain sensor control system and method based on acoustic-optical fusion proposed by the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the attached claims.

Claims

1. A rain sensor control system based on acoustic and optical fusion, It is characterized in that It includes a sensor module and a signal processing system, wherein the sensor module includes a rain sensor and an acoustic sensor; wherein, The rain sensor is used to detect the rain signal in the current environment; The acoustic sensor is used to detect the audio signal generated by raindrops hitting the front windshield of the vehicle in the current environment; The signal processing system is used to process the rainfall signal and the audio signal, generate weights corresponding to the rainfall signal and the audio signal respectively, and determine whether it is raining in the current environment and the level of rainfall based on the rainfall signal, the audio signal and the corresponding weights, and control the state of the vehicle based on whether it is raining and the level of rainfall.

2. According to the rain sensor control system based on sound and light fusion according to claim 1, It is characterized in that The rain sensor includes at least one set of infrared rain sensor chips, and the infrared rain sensor chips include an infrared transmitting device and an infrared receiving device; wherein, The infrared emitting device is used for emitting infrared light to the surface of the windshield of the vehicle, and the infrared receiving device is used for receiving the infrared light reflected by the windshield.

3. The rain sensor control system based on sound and light fusion according to claim 2, It is characterized in that The infrared rain sensor chip is attached to the front windshield of the vehicle, and the acoustic sensor is arranged on the contact surface between the infrared rain sensor chip and the front windshield; and, At least one acoustic sensor is provided, and the number of the acoustic sensors provided is the same as the number of the infrared rainfall sensor chips provided.

4. The rain sensor control system based on sound and light fusion according to claim 1, It is characterized in that The acoustic sensor is attached to the windshield of the vehicle, and is used to pick up the vibration signal generated by the interaction of raindrop sound, ambient sound and wind noise signal generated by raindrops falling on the windshield and the windshield, and generate an audio signal corresponding to the vibration signal.

5. The rain sensor control system based on sound and light fusion according to claim 4, It is characterized in that The signal processing system is also used to perform Fourier transform processing on the audio signal and extract characteristic frequency points of the Fourier transformed signal; and, Determining the matching degree between the current vibration signal and the preset signal according to the characteristic frequency point; Whether it is raining in the current environment and the level of rainfall are determined according to the matching degree.

6. The rain sensor control system based on sound and light fusion according to claim 1, It is characterized in that The acoustic sensor includes a MEMS microphone or a VACC sensor.

7. The rain sensor control system based on sound and light fusion according to claim 1, It is characterized in that Also included is an optical sensor; wherein, The optical sensor is used to collect ambient light in the current environment of the vehicle, and convert the ambient light into an electrical signal and send it to the signal processing system; When the electrical signal is greater than a first preset threshold, it is determined that the current environment is sunny; When the electrical signal is less than a second preset threshold, it is determined that the current environment is cloudy and rainy; When the telecommunication is not greater than the first preset threshold and not less than the second preset threshold, the signal processing system determines that the current environment remains unchanged; The second preset threshold is smaller than the first preset threshold.

8. The rain sensor control system based on sound and light fusion according to claim 7, It is characterized in that The signal processing system includes an AFE chip, an MCU and a LIN bus; wherein, The AFE chip is used to convert the analog signals collected by the optical sensor and the rain sensor into digital signals, and transmit them to the MCU for processing through the serial port bus; The MCU is used to obtain the digital signal and the audio signal of the acoustic sensor for analysis to determine whether it is raining in the current environment and the level of rainfall.

9. The rain sensor control system based on sound and light fusion according to claim 1, It is characterized in that The vehicle status includes wiper status, light status, air conditioning status, defog status, and HUD backlight brightness status.

10. A rain sensor control method based on acoustic and optical fusion, It is characterized in that include: Collect rainfall signals from the external environment through a rainfall sensor; Acquire the weather conditions at the current location and the judgment of the rainfall sensor on the rainfall signal through LIN bus communication, and generate calculation weights corresponding to the rainfall sensor and the acoustic sensor respectively; The acoustic sensor collects audio information in the external environment, and obtains an audio signal after processing by the signal processing system; According to the rainfall signal, the audio signal and the corresponding calculation weights, it is determined whether it is raining in the current environment and the level of rainfall, and the state of the vehicle is controlled according to whether it is raining and the level of rainfall.

Images