An intelligent vehicle-mounted sunlight and rainfall sensing system and method based on multi-source heterogeneous data fusion, a storage medium and a computer program product

Abstract

The application provides an intelligent vehicle-mounted sunlight and rainfall sensing system and method based on multi-source heterogeneous data fusion, a storage medium and a computer program product. The system comprises a data acquisition module, a data preprocessing module, a data fusion module and a decision and output module. The application can significantly improve the accuracy and reliability of sunlight and rainfall detection. High-resolution visual data acquisition can capture environmental details in detail and effectively compensate for the sensing limitations of traditional sensors. The fusion algorithm can flexibly allocate weights according to the characteristics of different data sources to ensure accurate sensing results in complex and variable environmental conditions. Precise output information helps to optimize the control logic of vehicle automatic wiper, automatic headlight and air conditioning systems, and greatly improves the driving experience and driving safety.

Description

Technical Field

[0001] This invention belongs to the field of intelligent vehicle-mounted sunlight and rainfall sensing technology, specifically relating to an intelligent vehicle-mounted sunlight and rainfall sensing system, method, storage medium, and computer program product based on multi-source heterogeneous data fusion. Background Technology

[0002] Sunlight and rain sensors are core components of modern vehicle intelligent systems, directly impacting the control logic of automatic headlights, windshield wipers, and air conditioning systems. Traditional technologies primarily rely on sensors such as photoresistors and capacitive rain sensors. Traditional sunlight sensors, based on photosensitive elements, determine sunlight intensity by detecting changes in light intensity across specific wavelengths; rain sensors employ optical designs, inferring rainfall by observing the fluctuations in light refracted by raindrops. However, these technologies suffer from several problems: photosensitive elements are susceptible to obstructions or ambient light interference, leading to misjudgments of sunlight intensity; optical rain sensors exhibit weak signal changes when raindrops are sparse, making accurate identification difficult; and they operate independently, unable to integrate internet-based meteorological data, such as regional rainfall forecasts, for optimized decision-making. Some vehicles use capacitive rain sensors, which determine rainfall based on changes in windshield capacitance caused by raindrops. However, capacitance changes are easily affected by environmental parameters such as temperature and humidity, resulting in insufficient stability; and they cannot detect light intensity: requiring an additional sunlight sensor for rain detection alone, increasing system complexity. Summary of the Invention

[0003] To address the shortcomings of traditional solar and rainfall sensors in terms of detection accuracy, environmental adaptability, and data resource utilization, this invention proposes an intelligent vehicle-mounted solar and rainfall sensing system, method, storage medium, and computer program product based on multi-source heterogeneous data fusion.

[0004] One of the objectives of this invention is an intelligent vehicle-mounted sunlight and rainfall sensing system based on multi-source heterogeneous data fusion, comprising: The data fusion module is used to extract raindrop and light and shadow features from visual data in multi-source heterogeneous data to obtain raindrop density quantification values ​​and illumination distribution classification values; based on vehicle data, online meteorological data, and the aforementioned raindrop density quantification values ​​and illumination distribution classification values ​​from the multi-source heterogeneous data, an environmental state determination result is obtained through a probabilistic inference model; fusion weights are dynamically allocated according to the environmental state determination result, and weighted fusion is performed on rainfall-related data according to the fusion weights to obtain a rainfall fusion value, and weighted fusion is performed on illumination-related data according to the fusion weights to obtain a sunlight intensity fusion value; Decision and output module: Based on the rainfall fusion value and the sunlight intensity fusion value, it determines the corresponding rainfall level and sunlight intensity level through a preset threshold, and outputs them to the relevant electronic and electrical systems of the vehicle in a standard format adapted to the vehicle system.

[0005] Furthermore, it also includes a data acquisition module, used to collect visual data from cameras, vehicle data from sensors, and online meteorological data from the Internet obtained through the vehicle network, to obtain multi-source heterogeneous data.

[0006] Furthermore, it also includes a data preprocessing module for preprocessing the collected multi-source heterogeneous data; and inputs the preprocessed multi-source heterogeneous data into the data fusion module for fusion processing.

[0007] Furthermore, the data preprocessing includes processing the visual data, and the processing methods include: converting the RGB image into a grayscale image, performing median filtering and Gaussian filtering in sequence, calculating the gradient magnitude and gradient direction based on the Sobel operator, extracting edges through non-maximum suppression and double threshold detection, and finally scaling or cropping it into a grayscale image of a preset size.

[0008] Furthermore, it also includes a first verification module, which is used to perform consistency verification on the quantized values ​​of raindrop density and trigger the weight adjustment of the data fusion module based on the result of the consistency verification.

[0009] Furthermore, the method for consistency verification of raindrop density quantization values ​​includes: acquiring raindrop density quantization values ​​corresponding to multiple consecutive frames of visual data; and triggering weight adjustment when the rate of change of multiple consecutive raindrop density quantization values ​​is greater than a first set threshold. Specifically, this includes: acquiring three consecutive raindrop density quantization values ​​DV1, DV2, and DV3; calculating the change rates R1 and R2 of adjacent frames, where R1 is the change rate between the first and second frames R1=|DV2-DV1| / DV1×100%, and R2 is the change rate between the second and third frames R2=|DV3-DV2| / DV2×100%; comparing R1 and R2 with the first set threshold (30%) respectively to generate a first verification result; when both R1 and R2 exceed the first set threshold, the first verification result is data inconsistency, triggering the data fusion module to re-execute probabilistic inference and weight update.

[0010] Furthermore, it also includes a second verification module, which is used to perform consistency verification on vehicle-mounted rainfall data and internet rainfall probability data, and trigger the weight adjustment of the data fusion module based on the consistency verification result.

[0011] Furthermore, the method for consistency verification of vehicle-mounted rainfall data and internet rainfall probability data includes: mapping the internet rainfall probability data to an equivalent rainfall quantization value, calculating the normalized difference between the equivalent rainfall quantization value and the vehicle-mounted rainfall data, and triggering a weight adjustment when the normalized difference is greater than a second preset threshold. Specifically, this includes: acquiring vehicle-mounted rainfall data (CS) and internet rainfall probability data (Pi); mapping the internet rainfall probability data to an equivalent rainfall quantization value using the following method: Equivalent rainfall quantization value = Internet rainfall probability × Sensor range Smax S max The normalized quantization value for the capacitive rain gauge at the maximum rainfall level is set to 100. The normalized difference is calculated using the following formula: Normalized Difference = (|Vehicle Rainfall Data - Equivalent Rainfall Quantization Value| / S) max ) × 100%; compare the normalized difference with the second set threshold (50%) to generate a second verification result; when the normalized difference exceeds the second set threshold, the second verification result is a data conflict, triggering the data fusion module to re-execute probabilistic inference and weight update.

[0012] Furthermore, in the data fusion module, the probabilistic inference model is a Bayesian network model. It calculates the posterior probability of all possible values ​​of the environmental state variable using variable elimination, and selects the environmental state corresponding to the value with the highest posterior probability as the environmental state determination result. The nodes of the Bayesian network model include environmental state variables, visual data feature nodes (raindrop density quantification value, light distribution classification value), vehicle-mounted sensor data nodes (vehicle-mounted light intensity data, vehicle-mounted rainfall data), and internet meteorological data nodes (internet rainfall probability data, internet cloud coverage data). Directed edges represent the causal dependencies between variables. The environmental state determination result includes sunny, light rain, heavy rain, and snow / fog.

[0013] Furthermore, the method for dynamically allocating fusion weights based on the environmental state determination results includes: When the environmental condition is determined to be sunny, the fusion weights are allocated according to the first allocation method. When the environmental state is determined to be light rain, the fusion weights are allocated according to the second allocation method, and the environmental state determination must satisfy the rainfall probability parameter P. i ≥Set value 0.3; When the environmental state is determined to be heavy rain, the fusion weights are allocated according to the third allocation method; and the environmental state determination must satisfy the raindrop density D. V ≥ Set value 0.7 and visibility < Set distance 500m; When the environmental condition is determined to be snowy / foggy, the fusion weights are allocated according to the fourth allocation method.

[0014] Furthermore, the first allocation method is: α=0.6, β=0.3, γ=0.1.

[0015] The second allocation method is: α=0.4, β=0.2, γ=0.4.

[0016] The third allocation method is: α=0.2, β=0.3, γ=0.5.

[0017] The fourth allocation method is: α=0.3, β=0.4, γ=0.3.

[0018] Where α corresponds to the weight of the raindrop density quantization value and the illumination distribution classification value; β corresponds to the weight of the vehicle data collected by the sensors; γ corresponds to the weight of online meteorological data.

[0019] Furthermore, the formula for calculating the aggregated rainfall value includes: Rainfall agglomeration value = α·D V +β·C S +γ·P i ; The formulas for calculating the solar intensity blending value include: Sunlight intensity blending value = α·L V +β·S S +γ·C i S s and C s These represent the light intensity data and rainfall data after data preprocessing, respectively. P i and C i These represent the rainfall probability parameter and cloud cover rate parameter from meteorological data obtained from the Internet. D v and L v These represent the extracted raindrop density quantization value and the light distribution characteristic value, respectively.

[0020] Furthermore, the method for determining the corresponding rainfall level based on the rainfall fusion value and a preset threshold includes: When 0 ≤ rainfall fusion value < first set value, the rainfall level is 0, which means no rain, such as 24-hour rainfall < 1mm; When the first set value ≤ rainfall fusion value < the second set value, the rainfall level is level 1, which corresponds to light rain, such as 1-10 mm of rainfall in 24 hours; When the second set value ≤ rainfall fusion value < the third set value, the rainfall level is level 2, which corresponds to moderate rain, such as 10-25 mm of rainfall in 24 hours; When the third setting value ≤ rainfall fusion value < the fourth setting value, the rainfall level is level 3, which corresponds to heavy rain, such as 25-50 mm of rainfall in 24 hours; When the third set value ≤ rainfall fusion value ≤ 1, the rainfall level is level 4, which corresponds to heavy rain, such as 24-hour rainfall ≥ 50mm.

[0021] Furthermore, the first setting value, the second setting value, the third setting value, and the fourth setting value are 0.2, 0.4, 0.6, and 0.8 respectively.

[0022] Furthermore, the method for determining the corresponding sunlight intensity level based on the sunlight intensity blending value and a preset threshold includes: When 0 ≤ sunlight intensity fusion value < fifth setting value, the sunlight intensity level is 0, which corresponds to an extremely dark environment and is suitable for scenes without light, such as night or tunnels. When the fifth setting value ≤ sunlight intensity fusion value < the sixth setting value, the sunlight intensity level is 1. Level 1 corresponds to low light environment and is suitable for early morning, evening and cloudy scenes. When the sixth setting value ≤ sunlight intensity fusion value < the seventh setting value, the sunlight intensity level is level 2. Level 2 corresponds to low light environment and is suitable for cloudy and shaded scenes. When the seventh setting value ≤ sunlight intensity fusion value < the eighth setting value, the sunlight intensity level is level 3. Level 3 is suitable for normal environments in sunny days without direct sunlight.

[0023] Furthermore, the fifth, sixth, seventh, and eighth settings are 1, 2, 3, 4, and 5, respectively.

[0024] A second objective of this invention is an intelligent vehicle-mounted sunlight and rainfall sensing method based on multi-source heterogeneous data fusion, comprising: Raindrop and light / shadow features are extracted from visual data in multi-source heterogeneous data to obtain raindrop density quantification values ​​and illumination distribution classification values. Based on vehicle-mounted data, online meteorological data, and the aforementioned raindrop density quantification values ​​and illumination distribution classification values ​​from the multi-source heterogeneous data, an environmental state determination result is obtained through a probabilistic inference model. Fusion weights are dynamically allocated according to the environmental state determination result, and weighted fusion is performed on rainfall-related data according to the fusion weights to obtain a rainfall fusion value. Weighted fusion is also performed on illumination-related data according to the fusion weights to obtain a sunlight intensity fusion value. Based on the rainfall fusion value and the sunlight intensity fusion value, the corresponding rainfall level and sunlight intensity level are determined by a preset threshold and output to the relevant electronic and electrical systems of the vehicle in a standard format adapted to the vehicle system.

[0025] Furthermore, the method for acquiring multi-source heterogeneous data includes: acquiring visual data collected by cameras, vehicle-mounted data collected by sensors, and online meteorological data obtained through vehicle networks.

[0026] Furthermore, the method also includes preprocessing the collected multi-source heterogeneous data, and the preprocessed data is used to extract droplet features and light and shadow features, as well as to obtain environmental state determination results.

[0027] A non-transitory computer-readable storage medium for achieving the third objective of the present invention stores a computer program thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the intelligent vehicle-mounted sunlight and rainfall sensing method based on multi-source heterogeneous data fusion.

[0028] A computer program product for achieving the fourth objective of the present invention includes a computer program / instruction that, when executed by a processor, implements the steps of the intelligent vehicle-mounted sunlight and rainfall sensing method based on multi-source heterogeneous data fusion.

[0029] The beneficial effects of this invention include: By fusing multi-source data, the accuracy and reliability of sunlight and rainfall detection are significantly improved; high-resolution visual data acquisition can meticulously capture environmental details, effectively compensating for the perception limitations of traditional sensors; the fusion algorithm can flexibly allocate weights based on the characteristics of different data sources, ensuring accurate perception results even under complex and changing environmental conditions. Precise output information helps optimize the control logic of vehicle systems such as automatic wipers, automatic headlights, and air conditioning, greatly enhancing the driving experience and driving safety. Attached Figure Description

[0030] Figure 1 This is a structural schematic diagram of an embodiment of the system described in this invention. Detailed Implementation

[0031] The following detailed embodiments are provided to explain the technical solutions of the present invention, so that those skilled in the art can understand the present invention. The scope of protection of the present invention is not limited to the following specific embodiments. Any modifications or improvements made by those skilled in the art that incorporate the technical solutions of the present invention but differ from the following detailed embodiments are also within the scope of protection of the present invention.

[0032] Example 1 An intelligent vehicle-mounted solar and rainfall sensing system based on multi-source heterogeneous data fusion includes: The data fusion module is used to extract raindrop and light and shadow features from visual data in multi-source heterogeneous data to obtain raindrop density quantification values ​​and illumination distribution classification values; based on vehicle data, online meteorological data, and the aforementioned raindrop density quantification values ​​and illumination distribution classification values ​​from the multi-source heterogeneous data, an environmental state determination result is obtained through a probabilistic inference model; fusion weights are dynamically allocated according to the environmental state determination result, and weighted fusion is performed on rainfall-related data according to the fusion weights to obtain a rainfall fusion value, and weighted fusion is performed on illumination-related data according to the fusion weights to obtain a sunlight intensity fusion value; Decision and output module: It is used to determine the rainfall level and sunlight intensity level based on the rainfall fusion value and sunlight intensity fusion value through a preset threshold, and output them to the relevant electronic and electrical systems of the vehicle in a standard format adapted to the vehicle system.

[0033] In one embodiment, a multi-source data acquisition module is further included to integrate visual data acquired by a camera, vehicle-mounted data acquired by traditional sunlight and rain sensors, and online meteorological data obtained through a vehicle network to obtain a multi-source heterogeneous raw data set. The camera is a high-resolution, wide-angle lens with high dynamic range (HDR) functionality; for example, a wide-angle lens with a resolution ≥3840×2160 (4K), a horizontal field of view ≥140°, and a dynamic range (HDR) of ≥120dB is selected, supporting multi-frame exposure synthesis technology to eliminate motion blur. The sensors include a light intensity sensor and a capacitive rain sensor; the online meteorological data is acquired through a vehicle-mounted Tbox module, such as regional rainfall and cloud cover, and the online meteorological data adopts the national standard meteorological data interface format. In one embodiment, a data preprocessing module is further included, used for grayscale conversion, median filtering for noise reduction, and edge detection processing of the visual data; for amplification, filtering, and analog-to-digital conversion of the vehicle sensor data; and for parsing, filtering, and format unification of online meteorological data from the Internet; to solve the problems of noise interference, format incompatibility, and unclear features in multi-source heterogeneous data; specifically including: For visual data, the RGB image is first converted into a grayscale image to reduce computational complexity and adapt to the real-time processing requirements of in-vehicle scenarios, while retaining key brightness information. In this embodiment, the conversion formula is: gray=0.299R+0.587G+0.114B; gray represents the converted grayscale value, and R, G, and B represent the three color components of red, green, and blue pixels in the RGB color image, respectively. Images captured by vehicle-mounted cameras are susceptible to high-frequency detail noise, such as salt-and-pepper noise from splashing water and light reflections. This noise can easily be misinterpreted as raindrops or form false edges. Furthermore, the edges of raindrops overlap with glass reflections in the original image, blurring the raindrop outlines and light-shadow boundaries. Directly inputting these into a CNN network will lead to feature extraction errors. Therefore, this embodiment uses a 5×5 window for filtering; then, the Canny algorithm is executed to extract true edges. The first step of the Canny algorithm uses a Gaussian filter with a standard deviation σ=1.5 to smooth the image, suppressing high-frequency detail noise and accurately separating the raindrop outlines, light-shadow boundaries, and background, providing high-quality feature input for the CNN network. Specific steps include... Based on the smoothed image, the Sobel operator is used to calculate the lateral gradient G. x and longitudinal gradient G y ; Based on the lateral gradient G x and longitudinal gradient G y Calculate the gradient magnitude G and gradient direction θ, The calculation formula is as follows: , , θ The value range is [0°, 180°).

[0034] Non-maximum suppression is used to process the gradient magnitude image, which refines wide edges into single-pixel-width edges and eliminates redundant pixels in non-edge regions. After non-maximum suppression, a single-pixel-width edge candidate image is obtained, which contains only strong edge candidate points and weak edge candidate points. By setting two thresholds of different values, the above edge candidate points are divided into the following three categories: Strong edge points: Gradient magnitude > high threshold. These points are definite real edges and are directly retained.

[0035] Weak edge points: Low threshold < gradient magnitude ≤ high threshold. These points may be real edges or noise, and further verification is needed.

[0036] Non-edge points: If the gradient magnitude is less than or equal to the low threshold, set it to 0 directly and exclude it as noise or a false edge.

[0037] In this embodiment, the high threshold is 50 and the low threshold is 20.

[0038] Finally, the visual data is scaled or cropped to obtain a 224×224 pixel grayscale image to balance accuracy and computational efficiency, which is then used as input data for the CNN network to avoid feature extraction failure due to inconsistent size. For vehicle-mounted data, the data preprocessing includes: Since the original output signals of light intensity sensors and capacitive rain sensors are extremely weak (millivolt level), they are easily submerged by noise in the vehicle's electromagnetic environment, such as engines and radar. For the light intensity signal output by the vehicle sensor, an instrumentation amplifier with a gain of 100 is used to amplify the signal, and a low-pass filter circuit with a cutoff frequency of 10Hz is used to filter the amplified light intensity signal to eliminate high-frequency environmental interference such as electromagnetic radiation. For the capacitive signal output by the vehicle-mounted capacitive rain sensor, a bandpass filter circuit with a passband frequency of 1-100Hz is used for filtering to effectively eliminate the interference caused by temperature drift and electromagnetic radiation on the capacitive signal. After the amplification and filtering processes described above, the preprocessed light intensity signal and rainfall signal are converted from analog to digital using two independent 12-bit analog-to-digital converters (ADCs) to obtain the light intensity data S. s Rainfall data C s Furthermore, the sampling rate for the analog-to-digital conversion is set to 1 kHz. Among these, the light intensity data S... s Quantified into a numerical range of 0-5, where 0 corresponds to extremely dark and 5 corresponds to extremely heavy rainfall; Rainfall data CS The quantization is a numerical range of 0-1, corresponding to the original quantized value of raindrop density. The quantization formula is: Quantized value = (ADC conversion result / ADC full-scale value) × upper limit of the target quantization range.

[0039] For internet data, the data preprocessing includes: First, a JSON format parsing operation is performed to extract key weather-related fields such as rain_probability and cloud_cover. The rain_probability field represents the probability of rainfall in the next hour, and the cloud_cover field represents the current cloud cover. After extracting the key fields, the extracted data is standardized, and the rainfall probability parameter P is... i Mapping the cloud cover parameter C to a numerical range of 0-1 i The light intensity is converted into a 5-level quantification standard from 0 to 4, where 0 corresponds to a cloudless state and 4 corresponds to a state of complete cloud cover.

[0040] In one embodiment, the specific implementation steps of the data fusion module include: 1. Feature extraction using CNN networks A pre-trained CNN network is used to extract features from the preprocessed visual data. The input layer of this CNN network receives a 224×224 pixel grayscale image. Raindrop edge and illumination gradient features are extracted through three convolutional layers with a kernel size of 3×3, a stride of 1, and the ReLU activation function. The features are then reduced in dimensionality by a 2×2 window max pooling layer while retaining salient features. Finally, the raindrop density D, which is normalized to 0-1, is output through a fully connected layer. v And the light distribution value L of the 5-level classification v 0 corresponds to extreme darkness, and 4 corresponds to strong light; The training process of the CNN network includes: The training set uses visual data under various weather conditions labeled with raindrop information and light intensity. The training set contains over 100,000 frames covering eight typical weather scenarios (sunny, light rain, heavy rain, snow / fog), three time periods (morning, noon, and evening), and five road conditions (urban roads, highways, and rural roads). Each frame is labeled with raindrop density (0-1 continuous values) and light intensity level (0-4). The cross-entropy loss function is used to adapt the light distribution matrix L. V The classification task, mean squared error loss function adapted to raindrop density D v For the regression task, the model was trained using the Adam optimizer with a learning rate of 0.001. The number of training iterations was set to 50 rounds, and an early stopping strategy was adopted (training was stopped when the validation set loss did not decrease for 5 consecutive rounds) to ensure the model's generalization ability.

[0041] In one embodiment, the probabilistic inference model in the data fusion module is a Bayesian network model. It calculates the posterior probability of all possible values ​​of the environmental state variable using the variable elimination method, and selects the environmental state corresponding to the value with the highest posterior probability as the environmental state determination result. Specific implementation steps include: 2. Bayesian Network Probabilistic Modeling and Reliability Assessment 2.1 Constructing Bayesian Networks Construct a Bayesian network with environmental state variable E (including sunny, light rain, heavy rain, snow / fog) as the core; network nodes include environmental state variable E and visual data feature nodes (raindrop density D). V Light distribution L V ), vehicle-mounted sensor data nodes (light intensity data S) S Rainfall data C S Internet meteorological data nodes (rainfall probability parameter P) i And cloud cover parameter C i Directed edges between nodes represent causal dependencies between variables, E→D. V This indicates that environmental conditions directly affect raindrop density, E→P i This indicates that environmental conditions directly affect the probability of rainfall; 2.2 Establishing a joint probability distribution Based on historical measured data covering multiple weather scenarios across different regions and seasons, the conditional probabilities of each data origin node under different environmental conditions E are statistically analyzed. The historical measured data includes sensor and visual data from 20 typical climate regions nationwide (such as tropical monsoon climate and temperate continental climate), spanning four seasons and a three-year period, with a cumulative sample size of no less than 1 million sets to ensure the reliability of the conditional probability statistics, such as P(D). V |E)=Rainy day, P(L) V |E) = Sunny day, P(S) S |E) = snowy days, etc., and then establish the following complete joint probability distribution. Where P(E) is the prior probability of the environmental state variable, which is determined by historical environmental state statistics. 2.3 Calculate the posterior probability The preprocessed vehicle sensor data (light intensity data S) s Rainfall data C s Internet meteorological data (rainfall probability parameter P) i And cloud cover parameter C i ) and the raindrop density D output by the CNN network v Light distribution L vThe observed data is input into the Bayesian network probability model constructed above, and the posterior probabilities of all possible values ​​of the environmental state variable E are calculated using the variable elimination method. , For example: P(E=sunny day|observed data)=0.85, P(E=light rain|observed data)=0.12; compare the posterior probabilities of all possible states of the environmental state variable E, and select the environmental state variable E with the largest posterior probability value as the current environmental state determination result E. current ; 2.4 Based on the current environment state determination result E output in step 2.3 current The method of dynamically allocating fusion weights based on the environmental state determination results, specifically assigning weights α to visual data, β to vehicle sensor data, and γ to internet meteorological data, includes: When the environmental status determination result is E current When the weather is clear, the first allocation method is used for the fusion weights, specifically α=0.6, β=0.3, and γ=0.1; When the environmental status determination result is E current When it is light rain, the second allocation method is used for the fusion weights, specifically α=0.4, β=0.2, and γ=0.4. Furthermore, this environmental state determination must satisfy the rainfall probability parameter P. i ≥0.3; When the environmental status determination result is E current During heavy rain, the fusion weight allocation adopts the third allocation method, specifically α=0.2, β=0.3, γ=0.5, and this environmental state determination must satisfy the raindrop density D. V If the value is ≥0.7 and the visibility is <500m, the determination will be made with the assistance of visual data; When the environmental status determination result is E current For snowy / foggy days, the fourth allocation method is used for weighting, specifically α=0.3, β=0.4, and γ=0.3; 2.5 Weighted Fusion of Multi-Source Data Data fusion is performed according to a preset weighted fusion formula, where the rainfall fusion formula is: Rainfall agglomeration value = α·D V +β·C S +γ·P i ; The formula for combining sunlight intensity is: Sunlight intensity blending value = α·L V +β·S S +γ·C i In one embodiment, the implementation process of the decision-making and output module includes: The following methods are used to output rainfall levels of 0-4 and solar intensity levels of 0-4 based on rainfall blending values ​​and solar intensity blending values, respectively. Rainfall fusion value ∈ [0, 0.2) corresponds to level 0, that is, no rain, 24-hour rainfall < 1 mm; Rainfall fusion values ​​∈ [0.2, 0.4) correspond to level 1, i.e. light rain, with 24-hour rainfall of 1-10 mm; Rainfall fusion value ∈ [0.4, 0.6) corresponds to level 2, i.e. moderate rain, with 24-hour rainfall of 10-25 mm; Rainfall aggregation value ∈ [0.6, 0.8) corresponds to level 3, i.e. heavy rain, with 24-hour rainfall of 25-50 mm; Rainfall aggregation value ∈ [0.8, 1.0] corresponds to level 4, i.e., rainstorm, with 24-hour rainfall ≥ 50 mm; A sunlight intensity blending value ∈ [0,1) corresponds to level 0, which is extremely dark and has an ambient light intensity of <10lx. It is suitable for scenes without light, such as at night or in tunnels. The sunlight intensity fusion value ∈ [1,2) corresponds to level 1, which is low light, with an ambient light intensity of 10-250 lx, suitable for early morning, evening, and cloudy scenes; The sunlight intensity fusion value ∈ [2,3) corresponds to level 2, which is low light, with an ambient light intensity of 250-1000lx, suitable for cloudy and shaded scenes; The sunlight intensity fusion value ∈ [3,4) corresponds to level 3, which is normal, with an ambient light intensity of 1000-5000lx, suitable for sunny days without direct sunlight. The sunlight intensity fusion value ∈ [4,5) corresponds to level 4, which is strong light, with an ambient light intensity of 5000-45000lx, suitable for sunny midday direct sunlight and typical strong light scenes on plateaus; The determined sunlight intensity level and rainfall level are encapsulated in a standard format adapted to the vehicle system, including fields such as ECU address, signal routing identifier, and data frame index, and output to the relevant electronic and electrical systems of the vehicle, such as the automatic headlight controller, wiper ECU, and air conditioning control system.

[0042] In one embodiment, a first verification module is further included, used to perform consistency verification on the quantized raindrop density values. Based on the consistency verification result, the data fusion module is triggered to adjust the fusion weights. The adjustment method includes: obtaining the quantized raindrop density values ​​corresponding to multiple consecutive frames of visual data; when the rate of change of multiple consecutive quantized raindrop density values ​​is greater than a first set threshold, the data fusion module is triggered to adjust the fusion weights. In another embodiment, based on historical rainy day data statistics, if the raindrop density D of three consecutive frames of visual data... vIf the rate of change is greater than 30% (to avoid false triggering due to temporary water stain interference), the Bayesian network probability model will re-execute step 2.3 for posterior probability calculation and reliability assessment, thereby updating the weight coefficients; where the raindrop density D of three consecutive frames of visual data... v The method for calculating the rate of change includes: the rate of change R1 for frames 1-2 = |D V2 -D V1 | / D V1 ×100%; the rate of change for frames 2-3, R² = |D V3 -D V2 | / D V2 ×100%; dynamic weight adjustment is triggered if and only if R1 > 30% and R2 > 30%; D V1 D V2 D V3 These refer to the three raindrop density values ​​output after preprocessing (grayscale conversion, filtering, edge detection) and CNN feature extraction from three images continuously captured by the camera at a fixed frame rate.

[0043] In one embodiment, a second verification module is further included, which is used to perform consistency verification on vehicle-mounted rainfall data and internet rainfall probability data. Based on the consistency verification result, the data fusion module is triggered to adjust the fusion weight. The adjustment method includes: mapping the internet rainfall probability data to an equivalent rainfall quantization value, calculating the normalized difference between the equivalent rainfall quantization value and the vehicle-mounted rainfall data, and when the normalized difference is greater than a second set threshold, the data fusion module is triggered to adjust the fusion weight.

[0044] In another embodiment, the weights are dynamically adjusted according to the following method: The probability of rain on the internet, P i Mapped to the rainfall value C from the vehicle sensor s Equivalent rainfall quantification value in the same dimension: Internet equivalent rainfall quantification value = Internet rainfall probability P i ×Sensor range S max ; The absolute difference is calculated using the following formula: Absolute difference = |Rainfall value from vehicle sensor - Internet equivalent rainfall quantization value|; The normalized difference is calculated using the following formula to ensure that sensors with different measurement ranges are subject to the same quantization dimension threshold: Normalized Difference = (Absolute Difference / Sensor Range S) max ) × 100%; If the normalized difference is greater than 50%, it is determined to be an Internet data conflict, triggering the Bayesian network probability model to re-execute the posterior probability calculation and reliability assessment in step 2.3, and then update the weight coefficients.

[0045] In this invention, the sensor range S max The definition of is: Under the maximum rainfall level scenario defined in this invention, the capacitive rain sensor outputs a capacitive signal that, after being converted by a data preprocessing module and an analog-to-digital converter, yields a standardized rainfall quantization value with a quantization range of 0-100. This value is used to map the internet rainfall probability to the rainfall data C from the vehicle-mounted sensor. s The equivalent rainfall quantization value in the same dimension is calibrated to 100. That is, in a level 4 rainstorm scenario, the standardized rainfall quantization value output by the capacitive rain sensor after the above processing is 100.

[0046] Take, for example, a typical scenario where a sunny day is mistakenly reported as light rain: Assume sensor range S max =100, the vehicle sensor detected no rain (rain value = 5), the internet probability of rain = 10%, which corresponds to basically no rain; First, calculate the equivalent rainfall value on the Internet: 0.1 × 100 = 10; calculate the absolute difference: |5 - 10| = 5; calculate the normalized difference: (5 ÷ 100) × 100% = 5% ≤ 50%, which indicates that the data is consistent and no conflict is triggered.

[0047] Let's verify the conflict triggering scenario using a rainstorm underreporting scenario: Assume sensor range S max =100, the vehicle sensor detected heavy rain (rainfall value =90), the internet probability of rain =30%, corresponding to the probability of light rain; First, calculate the equivalent rainfall quantification value on the Internet: 0.3×100=30; calculate the absolute difference: |90-30|=60; calculate the normalized difference: (60÷100)×100%=60%>50%, which indicates a data conflict, and initiate a Bayesian network re-evaluation.

[0048] In one embodiment, regardless of whether the adjustment condition is triggered, the weight is automatically updated every set time interval (e.g., 5 seconds) to ensure adaptation to dynamic changes in the environment.

[0049] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0050] Example 2 This invention also provides an intelligent vehicle-mounted sunlight and rainfall sensing method based on multi-source heterogeneous data fusion, comprising: Visual data collected by cameras, vehicle data collected by sensors, and online meteorological data obtained through vehicle networks are collected to obtain a multi-source heterogeneous raw data set; The data in the multi-source heterogeneous original dataset are preprocessed to obtain a multi-source standardized dataset; Raindrop and light / shadow features are extracted from standardized visual data to obtain raindrop density quantification values ​​and illumination distribution classification values. Based on vehicle-mounted data and online meteorological data from a multi-source standardized dataset, along with the aforementioned raindrop density quantification values ​​and illumination distribution classification values, an environmental state determination result is obtained through a probabilistic inference model. Fusion weights are dynamically allocated according to the environmental state determination result, and weighted fusion is performed on rainfall-related data according to these weights to obtain a rainfall fusion value. Similarly, weighted fusion is performed on illumination-related data according to these weights to obtain a sunlight intensity fusion value. Based on the rainfall fusion value and the sunlight intensity fusion value, the corresponding rainfall level and sunlight intensity level are determined by a preset threshold and output to the relevant electronic and electrical systems of the vehicle in a standard format adapted to the vehicle system.

[0051] In one embodiment, the method further includes performing a consistency check on the quantized values ​​of raindrop density, and triggering the data fusion module to adjust the fusion weights based on the result of the consistency check.

[0052] In one embodiment, the method for consistency verification of raindrop density quantization values ​​includes: acquiring raindrop density quantization values ​​corresponding to multiple consecutive frames of visual data; and triggering the data fusion module to adjust the fusion weights when the rate of change of multiple consecutive raindrop density quantization values ​​is greater than a first set threshold.

[0053] In one embodiment, the method further includes performing a consistency check on the vehicle-mounted rainfall data and the internet rainfall probability data, and triggering the data fusion module to adjust the fusion weights based on the consistency check result.

[0054] In one embodiment, the method for consistency verification of vehicle-mounted rainfall data and internet rainfall probability data includes: mapping the internet rainfall probability data to an equivalent rainfall quantization value, calculating the normalized difference between the equivalent rainfall quantization value and the vehicle-mounted rainfall data, and triggering the data fusion module to adjust the fusion weight when the normalized difference is greater than a second preset threshold.

[0055] In one embodiment, the probabilistic reasoning model is a Bayesian network model, which calculates the posterior probability of all possible values ​​of the environmental state variable through variable elimination, and selects the environmental state corresponding to the value with the highest posterior probability as the environmental state determination result.

[0056] In one embodiment, the method for dynamically allocating fusion weights based on the environmental state determination result includes: When the environmental condition is determined to be sunny, the fusion weights are allocated according to the first allocation method. When the environmental state is determined to be light rain, the fusion weights are assigned according to the second allocation method; and the environmental state determination must satisfy the rainfall probability parameter P. i ≥ Set value; When the environmental state is determined to be heavy rain, the fusion weights are assigned according to the third allocation method; and the environmental state determination must satisfy the raindrop density D. V ≥ Set value and visibility < Set distance; When the environmental condition is determined to be snowy / foggy, the fusion weights are assigned according to the fourth allocation method.

[0057] The first allocation method is: α=0.6, β=0.3, γ=0.1; the second allocation method is: α=0.4, β=0.2, γ=0.4; the third allocation method is: α=0.2, β=0.3, γ=0.5; the fourth allocation method is: α=0.3, β=0.4, γ=0.3; where α corresponds to the weight of the raindrop density quantization value and the illumination distribution classification value; β corresponds to the weight of the vehicle data collected by the sensor; and γ corresponds to the weight of the online meteorological data.

[0058] In one embodiment, the formula for calculating the rainfall aggregation value includes: Rainfall agglomeration value = α·D V +β·C S +γ·P i ; The formulas for calculating the solar intensity blending value include: Sunlight intensity blending value = α·L V +β·S S +γ·C i S s and C s These represent the light intensity data and rainfall data after data preprocessing, respectively; P i and C i D represents the rainfall probability parameter and cloud cover parameter from meteorological data obtained from the Internet; v and L v These represent the extracted raindrop density quantization value and the light distribution characteristic value, respectively.

[0059] Example 3 This invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the various steps of the method described in this invention.

[0060] Example 4 This invention also provides a non-transitory computer-readable storage medium storing a computer program. The computer program includes program instructions that, when executed by a processor, implement the various steps of the method described in this invention, which will not be elaborated further here.

[0061] The computer-readable storage medium can be the data transmission apparatus or the internal storage unit of a computer device provided in any of the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device.

[0062] Furthermore, the computer-readable storage medium may include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that is to be output or has already been output.

[0063] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. An intelligent vehicle-mounted solar and rainfall sensing system based on multi-source heterogeneous data fusion, characterized in that, include: Data fusion module: used to extract raindrop and light and shadow features from visual data in multi-source heterogeneous data, and obtain raindrop density quantization value and illumination distribution classification value; Based on vehicle-mounted data, online meteorological data from the internet, and the quantified values ​​of raindrop density and the graded values ​​of light distribution from multi-source heterogeneous data, an environmental state determination result is obtained through a probabilistic inference model. Fusion weights are dynamically allocated according to the environmental state determination result, and a weighted fusion of rainfall-related data is performed according to the fusion weights to obtain a rainfall fusion value. A weighted fusion of light-related data is also performed according to the fusion weights to obtain a sunlight intensity fusion value. Decision and output module: It is used to determine the rainfall level and sunlight intensity level based on the rainfall fusion value and sunlight intensity fusion value through a preset threshold, and output them to the relevant electronic and electrical systems of the vehicle in a standard format adapted to the vehicle system.

2. The intelligent vehicle-mounted solar and rainfall sensing system based on multi-source heterogeneous data fusion as described in claim 1, characterized in that, It also includes a first verification module, which is used to perform consistency verification on the quantized values ​​of raindrop density, and triggers the data fusion module to adjust the fusion weights based on the result of the consistency verification.

3. The intelligent vehicle-mounted solar and rainfall sensing system based on multi-source heterogeneous data fusion as described in claim 2, characterized in that, The method for consistency verification of raindrop density quantization values ​​includes: acquiring raindrop density quantization values ​​corresponding to multiple consecutive frames of visual data; and triggering the data fusion module to adjust the fusion weights when the rate of change of multiple consecutive raindrop density quantization values ​​is greater than a first set threshold.

4. The intelligent vehicle-mounted solar and rainfall sensing system based on multi-source heterogeneous data fusion as described in claim 2 or 3, characterized in that, It also includes a second verification module, which is used to verify the consistency between vehicle-mounted rainfall data and internet rainfall probability data, and triggers the data fusion module to adjust the fusion weights based on the consistency verification results.

5. The intelligent vehicle-mounted solar and rainfall sensing system based on multi-source heterogeneous data fusion as described in claim 4, characterized in that, The method for consistency verification of vehicle-mounted rainfall data and internet rainfall probability data includes: mapping internet rainfall probability data to equivalent rainfall quantization values, calculating the normalized difference between the equivalent rainfall quantization values ​​and vehicle-mounted rainfall data, and triggering the data fusion module to adjust the fusion weights when the normalized difference is greater than a second set threshold.

6. The intelligent vehicle-mounted solar and rainfall sensing system based on multi-source heterogeneous data fusion as described in any one of claims 1 to 3 and 5, characterized in that, In the data fusion module, the probabilistic reasoning model is a Bayesian network model. The posterior probability of all possible values ​​of the environmental state variable is calculated by the variable elimination method, and the environmental state corresponding to the value with the largest posterior probability is selected as the environmental state determination result.

7. The intelligent vehicle-mounted solar and rainfall sensing system based on multi-source heterogeneous data fusion as described in claim 1, characterized in that, The method for dynamically allocating fusion weights based on the environmental state determination results includes: When the environmental condition is determined to be sunny, the fusion weights are allocated according to the first allocation method. When the environmental state is determined to be light rain, the fusion weights are assigned according to the second allocation method; and this environmental state determination must satisfy the rainfall probability parameter P. i ≥ Set value; When the environmental state is determined to be heavy rain, the fusion weights are assigned according to the third allocation method; and the environmental state determination must satisfy the raindrop density D. V ≥ Set value and visibility < Set distance; When the environmental condition is determined to be snowy / foggy, the fusion weights are assigned according to the fourth allocation method.

8. A smart vehicle-mounted solar and rainfall sensing method based on multi-source heterogeneous data fusion as described in claim 1, characterized in that, include: Raindrop and light / shadow features are extracted from standardized visual data to obtain raindrop density quantification values ​​and illumination distribution classification values. Based on vehicle-mounted data and online meteorological data from a multi-source standardized dataset, along with the aforementioned raindrop density quantification values ​​and illumination distribution classification values, an environmental state determination result is obtained through a probabilistic inference model. Fusion weights are dynamically allocated according to the environmental state determination result, and weighted fusion is performed on rainfall-related data according to these weights to obtain a rainfall fusion value. Similarly, weighted fusion is performed on illumination-related data according to these weights to obtain a sunlight intensity fusion value. Based on the rainfall fusion value and the sunlight intensity fusion value, the corresponding rainfall level and sunlight intensity level are determined by a preset threshold and output to the relevant electronic and electrical systems of the vehicle in a standard format adapted to the vehicle system.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the intelligent vehicle-mounted sunlight and rainfall sensing method based on multi-source heterogeneous data fusion as described in claim 8.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the intelligent vehicle-mounted sunlight and rainfall sensing method based on multi-source heterogeneous data fusion as described in claim 8.

Images