Intelligent driving control method, vehicle and storage medium

Abstract

The invention provides an intelligent driving control method, a vehicle and a storage medium, and relates to the technical field of intelligent driving. The method comprises the following steps: acquiring environment sensing data acquired by a sensor; the environment sensing data comprises rainfall detection data and target detection data; determining a real-time comprehensive rainfall level according to the rainfall detection data; dynamically adjusting the target detection weight of each sensor in data fusion according to the comprehensive rainfall level and the real-time target detection confidence of each sensor; based on the adjusted target detection weight, performing fusion processing on the target detection data collected by each sensor to obtain a target detection result; and adjusting an intelligent driving control strategy according to the comprehensive rainfall level and / or the target detection result. The environment sensing accuracy of the vehicle in a rainy environment is improved, the intelligent driving control strategy of the vehicle can adapt to the rainy environment change, and the driving safety and the driving experience are comprehensively improved from the sensing layer to the control layer.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to an intelligent driving control method, vehicle, and storage medium. Background Technology

[0002] With the popularization of new energy vehicles, intelligent driving has become an inevitable trend in the development of vehicle intelligence. Environmental perception is the foundation of intelligent driving, and the control strategy of intelligent driving is determined based on the results of environmental perception. Currently, the main configuration of vehicle environmental perception systems consists of sensor devices such as cameras and lidar installed around the vehicle.

[0003] In related technologies, when a user is driving a vehicle, sensors such as cameras and lidar collect information on target objects in the surrounding environment, such as pedestrians, obstacles, and other vehicles, and determine the position and speed of the target objects. Based on the target detection results, the user controls the vehicle's speed, lane changes, lane merging, turning, and other operations.

[0004] However, while sensors such as cameras and LiDAR each have their own unique advantages, in rainy conditions, the surface of cameras is easily covered by rainwater, and rain streaks obscure the image, resulting in unclear environmental images. LiDAR, on the other hand, has a short wavelength and weak penetration. In rainy conditions, especially with heavy rainfall, it is scattered by raindrops, generating a lot of noise. It is easy to misinterpret rainwater as obstacles, making it impossible to accurately identify the vehicle's surrounding environment. Failure of the perception system can easily lead to intelligent driving control errors, affecting driving safety and driving experience. Summary of the Invention

[0005] This application provides an intelligent driving control method, vehicle, and storage medium to address the problem that in the prior art, vehicle perception systems are prone to failure in rainy weather, leading to intelligent driving control errors that affect driving safety and driving experience.

[0006] In a first aspect, embodiments of this application provide an intelligent driving control method, including: Acquire environmental perception data collected by sensors; the environmental perception data includes rainfall detection data and target detection data. The real-time comprehensive rainfall level is determined based on the rainfall detection data. Based on the comprehensive rainfall level and the real-time target detection confidence of each sensor, the target detection weight of each sensor in the data fusion is dynamically adjusted. Based on the adjusted target detection weights, the target detection data collected by each sensor is fused to obtain the target detection results. The intelligent driving control strategy is adjusted based on the comprehensive rainfall level and / or the target detection results.

[0007] Based on the above technical content, the embodiments of this application first acquire environmental perception data collected by multiple source sensors; and the environmental perception data includes rainfall detection data and target detection data; then, the real-time comprehensive rainfall level is determined based on the rainfall detection data; the rainfall detection data from multiple sensors can accurately identify the rainy environment, providing an accurate environmental risk benchmark for the vehicle's environmental perception and control in intelligent driving mode, avoiding the problem of control strategy mismatch caused by misjudgment of rain conditions. Then, based on the comprehensive rainfall level and the real-time target detection confidence of each sensor, the target detection weight of each sensor in data fusion is dynamically adjusted. Based on the adjusted target detection weight, the target detection data collected by each sensor is fused to obtain the target detection result, ensuring that the data fusion result of multiple sensors preferentially relies on the more reliable sensor (such as prioritizing radar data in rainy weather), thereby improving the accuracy of target detection in rainy weather and enhancing the accuracy of the vehicle's environmental perception in rainy weather. Finally, based on the comprehensive rainfall level and / or target detection results, the intelligent driving control strategy is adjusted, forming a complete process of environmental perception, weight adjustment, data fusion and strategy optimization. This enables the vehicle's intelligent driving control strategy to adapt to changes in the rainy environment, comprehensively improving driving safety and driving experience from the perception layer to the control layer.

[0008] In one possible implementation, the sensor includes a vision sensor and a radar sensor; The step of dynamically adjusting the target detection weight of each sensor in data fusion based on the comprehensive rainfall level and the real-time target detection confidence of each sensor includes: If the comprehensive rainfall level is greater than or equal to the preset level, the target detection weight of the visual sensor is reduced and the target detection weight of the radar sensor is increased. In this system, the target detection weight of each sensor is positively correlated with its own real-time target detection confidence, and the sum of the target detection weights of all sensors is 1.

[0009] In this embodiment, to address the performance differences between visual sensors (susceptible to rain blurring) and radar (with superior rain resistance) in rainy conditions, the sensor weights are dynamically adjusted in a differentiated manner. By explicitly reducing the target detection weight of the visual sensor and increasing the target detection weight of the radar sensor when the overall rainfall level is greater than or equal to a preset level, the risk of target misses due to low clarity and unreliable data from visual sensors in rainy weather is specifically avoided. Furthermore, the target detection weight of each sensor is positively correlated with its own real-time target detection confidence level, and the sum of the target detection weights of all sensors after adjustment is 1. This ensures that high-confidence sensors contribute more during the target detection data fusion process, avoids redundancy or conflict in weight allocation, adapts to the hardware characteristics of multiple sensors (such as visual sensors excelling at target classification, lidar excelling at target distance measurement, and millimeter-wave radar excelling at measuring target movement speed), fully leverages the advantages of different sensors in different scenarios, maximizes strengths and minimizes weaknesses in rainy weather, improves the robustness of the fusion results, and makes the fusion results more scientific and reasonable, thereby improving the accuracy and reliability of the fused target detection results.

[0010] In one possible implementation, after reducing the target detection weight of the visual sensor and increasing the target detection weight of the radar sensor, the method further includes: Based on the images acquired by the vision sensor, determine whether there are severely blurred areas in the vision sensor; If the visual sensor has a severely blurred area, the spatial range of the severely blurred area relative to the vehicle is determined based on the location information of the severely blurred area. The target detection weight of the visual sensor within the spatial region is reduced, the target detection weight of the radar sensor within the spatial region is increased, and the main detection signal emitted by the radar sensor is adjusted to cover the spatial region or the detection signal emission density of the radar sensor within the spatial region is increased.

[0011] Here, for extreme scenarios where local areas of the visual sensor are severely blurred (such as parts of the lens being covered by rain), a refined weight adjustment and sensor performance optimization in local space is achieved. First, based on the spatial positioning of the severely blurred area, the weight of the visual sensor within that spatial area is reduced and the target detection weight of the radar sensor within that spatial area is increased. This avoids over-adjustment that negates global visual sensor data due to local blurring, balancing perception accuracy and efficiency. Furthermore, the main detection signal emitted by the radar sensor is adjusted to cover the blurred area, and the emission density of the radar sensor's detection signal within that blurred area is increased, strengthening the radar perception capability of that spatial area. This creates a redundant design where radar fills in areas where the visual sensor fails, avoiding perception blind spots. This achieves dynamic weight optimization in local spatial areas, which is more accurate than global weight adjustment and more suitable for complex scenarios where local visual sensors are obstructed in heavy rain, improving environmental perception accuracy in complex scenarios.

[0012] In one possible implementation, determining whether there is a severely blurred region in the visual sensor based on the image acquired by the visual sensor includes: The image acquired by the visual sensor is divided into multiple sub-regions; The sharpness of each sub-region is calculated to obtain a sharpness quantification value for each sub-region; Based on the pre-trained semantic segmentation model, the proportion of rain line occlusion area in each sub-region is determined; If the clarity quantification value corresponding to the sub-region is less than the first preset threshold, and the proportion of the rain line occlusion area in the sub-region is greater than the second preset threshold, then the sub-region is determined to be a severely blurred region.

[0013] In this embodiment, a quantitative judgment standard for severely blurred regions is provided to avoid subjectivity or misjudgment in the identification of blurred regions. By dividing the blurred regions into sub-regions, fine positioning of the blurred regions is achieved, avoiding the overall judgment of the entire image collected by the visual sensor. By combining the clarity quantification value and the proportion of rain line occlusion area for dual judgment, the positioning accuracy of severely blurred regions is greatly improved and the misjudgment rate is reduced.

[0014] In one possible implementation, the sensor includes a rain sensor, a visual sensor, and a radar sensor; The step of determining the real-time comprehensive rainfall level based on the rainfall detection data includes: The rainfall data collected by each sensor is quantified into a rainfall score; Rainfall detection weights are dynamically assigned to the corresponding sensor's rainfall score based on the rainfall detection confidence level of each sensor. Based on the rainfall detection weight corresponding to each sensor, the rainfall scores corresponding to all sensors are weighted and summed to obtain the comprehensive rainfall score. According to the preset mapping rules, the comprehensive rainfall score is mapped to the real-time comprehensive rainfall level.

[0015] In this embodiment, rainfall detection data collected by multiple sensors, such as rain gauges, visual sensors, and radar sensors, are fused. By utilizing data related to rainfall intensity collected by different sensors, the reliability and accuracy of rainfall assessment are improved. Furthermore, rainfall detection weights are dynamically allocated based on the confidence level of rainfall detection, making the comprehensive rainfall score more closely reflect the actual rainfall conditions. The quantification of the scoring and level mapping rules provides standardized input for subsequent weight adjustments and control strategy optimization, ensuring that all modules of the system have a consistent understanding of the rainfall environment. The fusion and judgment of rainfall detection data from multiple sensors improves the accuracy of real-time comprehensive rainfall levels and avoids misjudgments of rainfall conditions caused by the failure of a single sensor (such as a rain gauge sensor failure).

[0016] In one possible implementation, the radar sensor includes lidar and millimeter-wave radar; the quantification of rainfall detection data collected by each sensor into a rainfall score includes: The proportion of rain line obscuration area and image blur in the images acquired by the visual sensor are quantified into a rainfall score. The point cloud anomaly noise density of the lidar is quantified into a rainfall score; The echo scattering intensity of millimeter-wave radar is quantified into a rainfall score; The rainfall data collected by the rain gauge is quantified into a rainfall score; The higher the value of the rainfall detection data collected by each sensor, the higher the corresponding quantified rainfall score, and the higher the rainfall score, the higher the rainfall level.

[0017] In this embodiment, since different sensors collect different types of data related to rainfall, different quantization logics are designed for each sensor. For example, visual sensors are associated with rain lines and ambiguity, lidar is associated with noise density, and millimeter-wave radar is associated with echo scattering intensity, thus giving full play to the rainfall sensing advantages of each sensor. Furthermore, a unified quantization scale (converting rainfall detection data from different sensors into rainfall scores) enables direct fusion of multi-source data, avoiding fusion errors caused by differences in units or magnitudes. The quantization process is deeply bound to the physical characteristics of the sensors (e.g., the echo scattering intensity of millimeter-wave radar is positively correlated with raindrop density), improving the physical meaning and accuracy of rainfall scores.

[0018] In one possible implementation, the rainfall detection data collected by the rain sensor is rainfall data; the method further includes: Based on the rainfall data collected by the rain sensor, the comprehensive rainfall level, and the real-time external weather information, determine whether to control the driving mode to enter rain mode. When the rainfall data collected by the rain sensor is greater than or equal to the preset rainfall, or the comprehensive rainfall level is greater than or equal to the preset level, or the rainfall intensity indicated by the external weather information is greater than or equal to the preset rainfall intensity and the duration is greater than the preset duration, the driving mode is controlled to enter the rain mode. After the driving mode enters the rain mode, the data processing algorithm for each sensor is optimized.

[0019] In this embodiment, the rain mode is determined from multiple dimensions, including rain sensor data, comprehensive rainfall level, and external weather information, to avoid misjudgment based on a single condition (e.g., when external weather information indicates no rain but it is actually raining, multi-source sensor data can be combined for correction). After entering rain mode, sensor algorithms are optimized (e.g., LiDAR noise reduction, camera rain line segmentation, etc.) to improve perception quality and accuracy from the data source, rather than relying solely on subsequent fusion. This achieves accurate activation of rain mode and proactive optimization of sensor algorithms, shortening rain risk response time and improving the safety of intelligent driving in rainy environments.

[0020] In one possible implementation, adjusting the intelligent driving control strategy based on the comprehensive rainfall level includes: When the comprehensive rainfall level is greater than or equal to a preset level, the maximum cruising speed will be reduced to a range less than or equal to a preset speed threshold, and the following safety distance will be dynamically increased according to the comprehensive rainfall level; wherein, the comprehensive rainfall level is negatively correlated with the preset speed threshold and positively correlated with the increase in the following safety distance; When the comprehensive rainfall level is greater than or equal to the preset level, the lane change decision threshold is increased; the lane change decision threshold includes the longitudinal clearance threshold and the time margin threshold between vehicles in front and behind the target lane.

[0021] In this embodiment, rainfall conditions are directly linked to the control strategy to specifically reduce the risks of intelligent driving in rainy weather (such as increased braking distance and reduced lane change safety). As the overall rainfall level changes, the maximum cruising speed and following distance are dynamically adjusted (the higher the overall rainfall level, the lower the speed and the greater the following distance) to match the reduced road surface adhesion coefficient in rainy weather, reducing the risk of rear-end collisions. Increasing the lane change decision threshold (increasing longitudinal clearance and extending the time margin) can address the problem of reduced vehicle handling in rainy conditions, reducing the probability of collisions during lane changes. By dynamically adjusting the control strategy according to the overall rainfall level, driving safety in rainy conditions is further ensured.

[0022] Secondly, embodiments of this application provide an intelligent driving control device, including: The data acquisition module is used to acquire environmental perception data collected by multi-source sensors; the environmental perception data includes rainfall detection data and target detection data. A rainfall determination module is used to determine the real-time comprehensive rainfall level based on the rainfall detection data; The weight adjustment module is used to dynamically adjust the target detection weight of each sensor in the data fusion based on the comprehensive rainfall level and the real-time target detection confidence of each sensor. The target detection module is used to fuse the target detection data collected by each sensor based on the adjusted target detection weights to obtain the target detection results. The driving control module is used to adjust the intelligent driving control strategy based on the comprehensive rainfall level and / or the target detection result.

[0023] Thirdly, embodiments of this application provide a vehicle including a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor executes the computer program, it implements the intelligent driving control method as described in any of the first aspects.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the intelligent driving control method as described in any of the first aspects.

[0025] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application; Figure 2 This is a schematic flowchart of an embodiment of the intelligent driving control method provided in this application; Figure 3This is a flowchart illustrating an intelligent driving control method provided in another embodiment of this application; Figure 4 This is a schematic diagram of the software architecture of an intelligent driving control system provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an intelligent driving control device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a vehicle provided in one embodiment of this application. Detailed Implementation

[0029] The present application will be described more clearly below with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the function of the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0030] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0031] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0032] Most current intelligent driving systems use multiple sensors, such as cameras and radar, to perceive data on target objects in the surrounding environment, including pedestrians, vehicles, and obstacles. This data is then fused to obtain the final target detection result (including target position, distance, and speed). The system then uses this result to control the intelligent driving process (e.g., controlling speed, lane changes, and turns). However, intelligent driving systems face significant challenges in rainy conditions. While cameras and LiDAR sensors each have their advantages, in rainy environments, camera surfaces are easily covered by rainwater, obscuring the image and resulting in unclear environmental images and a sharp decline in the reliability of the output environmental data. LiDAR, with its short wavelength and weak penetration, is less affected by light rain, but in heavy rain, raindrop scattering generates significant noise, easily mistaking rainwater for obstacles and failing to accurately identify the vehicle's surroundings. This ultimately leads to low target recognition rates and high false / false detection rates. Therefore, in rainy conditions, if the intelligent driving system continues to perceive data in the surrounding environment in the same way as before, it is easy to experience problems such as perception failure or low perception accuracy, which in turn leads to intelligent driving control errors and affects driving safety and driving experience.

[0033] To address the aforementioned shortcomings, the embodiments of this application perform targeted optimization of both the perception and control layers in rainy conditions. Specifically, rain sensors, visual sensors, radar sensors, and external weather conditions are used to comprehensively identify rainy environments. Rainfall perception data from multiple sensors are fused to obtain a real-time comprehensive rainfall level, determining whether to enter rain mode and ensuring the accuracy and reliability of rain identification. After determining the rain mode, the target detection weight of each sensor in the target detection data fusion process is dynamically adjusted based on the real-time comprehensive rainfall level and the real-time target detection confidence of each sensor. Weighted fusion processing is performed on the target detection data from multiple sensors to obtain the final fusion result. Detection results with higher confidence levels have a greater weight in the final fusion result, exerting a greater influence on decision-making, ensuring perception accuracy and reliability, and avoiding perception failure or inaccuracy in rainy conditions. Then, in rainy conditions, the following distance is dynamically increased and the cruising speed is limited based on the comprehensive rainfall level to ensure following safety; the lane change decision threshold is increased to reduce lane change tendency, lowering the risk of lane changes in rainy conditions, thus achieving targeted optimization at the control level.

[0034] First refer to Figure 1 , Figure 1 The schematic diagram illustrates an application scenario provided according to an embodiment of this application. The devices involved in the application scenario include a host 101 on a vehicle, a rain sensor 102, a vision sensor 103, a lidar 104, and a millimeter-wave radar 105.

[0035] Among them, the rain sensor 102 can be set near the windshield of the vehicle, and the vision sensor 103, lidar 104 and millimeter-wave radar 105 are set around the vehicle body according to the vehicle design requirements, so that the detection range of the three sensors can cover the 360-degree area around the vehicle.

[0036] Rain sensor 102 collects rainfall data and sends it to host 101. Vision sensor 103 collects environmental data around the vehicle and sends it to host 101. LiDAR 104 emits a high-frequency pulsed laser beam around the vehicle. The scanning module covers the entire detection area with the laser beam. When the laser beam encounters a target (such as a vehicle, pedestrian, or motorcycle) while propagating in the air, some of the laser energy is reflected by the target surface to form an echo signal. By measuring the time and phase difference of the laser's round trip to the target, the precise distance and position of the target are calculated, and a 3D point cloud is generated to reconstruct the environmental contour. By reconstructing environmental details through high-precision point cloud, pixel-level environmental contour perception is achieved, and finally, the target's position, size, and motion direction are output. Based on the direction and target type (such as vehicles, pedestrians, etc.), the lidar 104 sends this perception data to the host 101; the millimeter-wave radar 105 emits millimeter waves into the environment around the vehicle. When the millimeter waves propagate in the air and encounter targets (such as vehicles, motorcycles, pedestrians, etc.), some of the energy is reflected. The echo signal reflected by the target is received, and by analyzing the time difference (range measurement), frequency difference (speed measurement), and phase difference (angle measurement) of the echo, the distance, speed, and angle of the target are detected. The target parameters (distance, speed, and angle of the target) of multiple consecutive frames are correlated by Kalman filtering to form a stable target trajectory, realizing target tracking and classification. The millimeter-wave radar 105 sends this perception data to the host 101. The host 101 adjusts the vehicle control strategy in real time based on the data collected by the above multi-source sensors to realize intelligent driving.

[0037] It should be noted that the visual sensor 103 can be a camera or other device that can capture images of the surrounding environment.

[0038] The following is combined Figure 1 Application scenarios, refer to Figures 2-3 This application describes an intelligent driving control method provided according to exemplary embodiments. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way. Rather, the embodiments of this application can be applied to any applicable scenario.

[0039] It should be noted that the embodiments of this application can be applied to vehicles, that is, the intelligent driving control method provided by the exemplary embodiments of this application can be executed on the host of the vehicle.

[0040] It should be noted that the intelligent driving control method provided according to the exemplary embodiments of this application can be executed on the same device or on different devices.

[0041] refer to Figure 2 , Figure 2 This is a schematic flowchart of an embodiment of the intelligent driving control method provided in this application. Figure 2 As shown, the method in the embodiments of this application may include: Step S201: Acquire environmental perception data collected by the sensor; the environmental perception data includes rainfall detection data and target detection data.

[0042] The sensors may include, but are not limited to, rain sensors, vision sensors (cameras) and radar sensors; the radar sensors may be lidar and millimeter-wave radar.

[0043] In this step, the rainfall detection data consists of data related to rainfall collected by various sensors. The rainfall sensor, acting as the direct sensing layer, directly detects the rainfall on the windshield, providing the most direct basic rainfall data. Visual sensors and radar sensors, acting as the environmental sensing layer, infer the rainfall amount based on the rainfall-related detection data. For example, a visual sensor could be a camera; the density of rain lines and image blur in the camera's captured image are related to rainfall intensity and can serve as an indirect indicator of rainfall amount. Similarly, radar sensors could be lidar or millimeter-wave radar; the density of anomalous noise in the effective point cloud of lidar's 3D point cloud is related to rainfall intensity and can serve as an indirect indicator of rainfall amount. Likewise, the intensity of the echo signal scattering from millimeter-wave radar is related to rainfall intensity and can serve as an indirect indicator of rainfall amount.

[0044] Target detection data may include, but is not limited to, the type of target in the environment surrounding the vehicle (such as vehicles, pedestrians, motorcycles, autonomous vehicles, signs, etc.), its location, size, speed of movement, and direction of movement.

[0045] Step S202: Determine the real-time comprehensive rainfall level based on rainfall detection data.

[0046] In this step, the types of rainfall detection data from different sensors differ. For example, rainfall data collected by rain gauges directly represents the amount of rainfall; the proportion of rain streaks obscuring the area and the blurriness of images captured by cameras indirectly indicate the amount of rainfall; and the anomalous noise density of the effective point cloud from lidar and the echo scattering intensity from millimeter-wave radar are also indirect indicators of rainfall amount. To determine the real-time comprehensive rainfall level, the rainfall detection weight of each sensor needs to be dynamically allocated based on the confidence level of the rainfall detection data. The rainfall detection data from each sensor are then weighted and fused to obtain the real-time comprehensive rainfall level.

[0047] It should be noted that the comprehensive rainfall level can be divided into 5 levels. The higher the level, the greater the rainfall and the greater the rainfall intensity. The levels are 0 (no rain), 1 (light rain), 2 (moderate rain), 3 (heavy rain), and 4 (torrential rain).

[0048] Step S203: Based on the comprehensive rainfall level and the real-time target detection confidence of each sensor, dynamically adjust the target detection weight of each sensor in the data fusion.

[0049] Among them, target detection confidence is used to measure the authenticity and accuracy of parameters of targets (such as vehicles, pedestrians, etc.) detected by the sensor, such as the probability that the target actually exists and the correctness of data such as the target's position, speed, and size. The higher the target detection confidence, the higher the reliability of the target detection data of the corresponding sensor.

[0050] In this step, the real-time target detection confidence level of each sensor is calculated based on the sensing characteristics of each sensor. Since the sensing characteristics of each sensor are different, their advantages vary under different environments. For example, cameras perceive the environment by acquiring images, and their sensing advantage is more pronounced in daylight without rain. LiDAR perceives the environment by emitting laser light and receiving echo energy, while millimeter-wave radar perceives the environment by emitting millimeter waves and receiving echo signals. Therefore, at night or in rainy conditions, LiDAR and millimeter-wave radar have more significant environmental sensing advantages. Consequently, the target detection confidence level of each sensor differs under different environments. To ensure that sensors with higher confidence levels contribute more, they need to be assigned higher weights. Furthermore, the higher the overall rainfall level (i.e., the greater the rainfall intensity), the greater the influence of cameras, and the greater the decrease in their target detection weight; similarly, the greater the increase in the target detection weight of millimeter-wave radar.

[0051] For example, in rainless daytime conditions, cameras have higher resolution and capture clearer environmental images. They can accurately identify target types, sizes, distances, and other information based on the images, resulting in higher target detection confidence. Therefore, cameras are assigned the highest weight. However, when the overall rainfall level increases, the vehicle is in a rainy environment, and the images captured by the cameras are covered by rain, increasing blurriness. In this case, the target detection confidence of the cameras decreases significantly. Millimeter-wave radar emits millimeter waves that can penetrate raindrops, and the echo signal is almost unaffected by rainwater, greatly improving target detection confidence. Therefore, millimeter-wave radar has a significant advantage in rainy conditions. To ensure more accurate target detection in rainy conditions, it is necessary to increase the target detection weight of millimeter-wave radar and decrease the target detection weight of cameras.

[0052] Step S204: Based on the adjusted target detection weights, the target detection data collected by each sensor is fused to obtain the target detection result.

[0053] In this step, the target detection data from all sensors are weighted and fused according to the adjusted target detection weights corresponding to each sensor to obtain the final target detection result. The target detection result includes information such as target type, position, size, outline, direction of movement, speed of movement, angle, and distance.

[0054] Step S205: Adjust the intelligent driving control strategy based on the comprehensive rainfall level and / or target detection results.

[0055] In one possible implementation, basic control parameters can be adjusted based on the real-time comprehensive rainfall level. For example, when the comprehensive rainfall level is greater than or equal to a preset level, the maximum cruising speed can be reduced and the following distance increased; and the lane change decision threshold can be increased.

[0056] It should be noted that during the weighted fusion of the target detection results output by each sensor, the real-time target detection confidence of each sensor also needs to be weighted and fused to obtain the fused confidence. This fused confidence is used to represent the credibility of the target detection result after multi-sensor fusion.

[0057] In another possible implementation, the dynamic control strategy is adjusted based on the target detection results: for example, for a target with a confidence score ≥ 0.8 after weight fusion, it indicates that the target detection result is relatively reliable and should be avoided first. Therefore, the target is included in the obstacle avoidance cost function first, with pedestrians and motorcycles having a higher obstacle avoidance priority than motor vehicles.

[0058] In another possible implementation, the control strategy can be adjusted based on the overall rainfall level and the target detection results. For example, when the overall rainfall level reaches the highest level (rainstorm level) and the confidence level of the target detection results is less than 0.6, a downgrade control is triggered to switch to the manual driving prompt mode to ensure driving safety.

[0059] In this embodiment, environmental perception data collected by multiple sensors is first acquired, including rainfall detection data and target detection data. Then, the real-time comprehensive rainfall level is determined based on the rainfall detection data. The rainfall detection data from multiple sensors accurately identifies the rainy environment, providing an accurate environmental risk benchmark for the vehicle's environmental perception and control in intelligent driving mode, avoiding control strategy mismatches caused by misjudgments of rain conditions. Next, based on the comprehensive rainfall level and the real-time target detection confidence of each sensor, the target detection weight of each sensor in data fusion is dynamically adjusted. Based on the adjusted target detection weights, the target detection data collected by each sensor is fused to obtain the target detection result. This ensures that the data fusion result from multiple sensors prioritizes the more reliable sensor (e.g., prioritizing radar data in rainy weather), thereby improving the accuracy of target detection in rainy conditions and enhancing the vehicle's environmental perception accuracy in rainy environments. Finally, the intelligent driving control strategy is adjusted based on the comprehensive rainfall level and / or target detection result, forming a complete process of environmental perception, weight adjustment, data fusion, and strategy optimization. This enables the vehicle's intelligent driving control strategy to adapt to changes in the rainy environment, comprehensively improving driving safety and driving experience from the perception layer to the control layer.

[0060] Figure 3 A flowchart illustrating another embodiment of the intelligent driving control method provided in this application is shown below. Figure 3 As shown, the method includes: Step S301: Acquire environmental perception data collected by the sensor; the environmental perception data includes rainfall detection data and target detection data.

[0061] In this step, the sensors include rain sensors, visual sensors (such as cameras), and radar sensors (such as lidar and millimeter-wave radar).

[0062] Step S302: Quantify the rainfall detection data collected by each sensor into a rainfall score.

[0063] In this step, since the types of rainfall detection data collected by different sensors are different, in order to facilitate the subsequent fusion of rainfall detection data from multiple sensors, it is necessary to first quantify the multi-source rainfall detection data into a rainfall score of a uniform scale.

[0064] In one possible implementation, the proportion of rain line occlusion area and image blur in images captured by visual sensors (such as cameras) are quantified into a rainfall score; the point cloud anomaly noise density of lidar is quantified into a rainfall score; the echo scattering intensity of millimeter-wave radar is quantified into a rainfall score; and the rainfall data collected by rain gauges is quantified into a rainfall score. The larger the value of the rainfall detection data collected by each sensor, the higher the corresponding quantified rainfall score, and the higher the rainfall score, the higher the rainfall level. For example, the greater the rainfall collected by the rain gauge, the higher the corresponding rainfall score.

[0065] In this embodiment, for the camera, the rainfall amount is indirectly verified by quantifying the proportion of rain line occlusion area and the image blur metric (denoted as S_cam). The proportion of rain line occlusion area can be calculated by using a pre-trained semantic segmentation model to identify and segment rain lines in the image captured by the camera, calculating the number of rain line pixels, and the ratio of the number of rain line pixels to the total number of image pixels. Image blur can be calculated using Laplacian variance. For the lidar, the rainfall metric (S_lidar) is quantified based on the point cloud anomalous noise density, which can be calculated using a spatial clustering algorithm based on density. For the millimeter-wave radar, the rainfall metric (S_radar) is quantified based on the mean of echo scattering intensity; higher scattering intensity results in a higher rainfall metric. For the rain sensor, the rainfall metric (S_rain) is quantified based on the rainfall per unit time (mm / h).

[0066] In one possible implementation, if the weather information of the vehicle's current location can be obtained through the vehicle weather service interface, the rainfall or rainfall intensity forecast can be quantified into a rainfall score (denoted as S_api) to facilitate subsequent comprehensive judgment of the rainy weather environment.

[0067] For example, the rainfall score range can be set from 0 to 4 points, corresponding to 5 levels of rainfall, to ensure consistent quantification scale. For rainfall data (unit: millimeters per hour) collected by rain sensors, the quantization logic is as follows: 0 points (no rain): <0.1 mm / h; 1 point (light rain): 0.1~2.5 mm / h; 2 points (moderate rain): 2.5~10 mm / h; 3 points (heavy rain): 10~50 mm / h; 4 points (torrential rain): >50 mm / h. The quantization logic for the proportion of rain line obstruction area (denoted as P) and Laplace variance (denoted as V) in images collected by cameras is: S_cam = round(4×(1 - V / 100) × P), where round() represents the rounding function; for example, if the proportion of rain line obstruction area P=60% and V=30, then S_cam=round(4×0.7×0.6)=2 points. For the point cloud anomalous noise density of lidar (denoted as D, unit: numbers / m²), 2 The quantization logic is as follows: 0 points: D < 5; 1 point: 5~20; 2 points: 20~50; 3 points: 50~100; 4 points: D > 100. The more noise, the greater the rainfall. For example, if the noise density D = 60 points / m², then S_lidar = 3 points. For the average echo scattering intensity of millimeter-wave radar (denoted as I, unit: dB), the quantization logic is as follows: Scattering intensity I (dB): 0 points: I < -80; 1 point: -80~-60; 2 points: -60~-40; 3 points: -40~-20; 4 points: > -20. The stronger the scattering, the more raindrops. For example, if the scattering intensity I = -30dB, then S_radar = 3 points. For weather information of the current location obtained from external information interfaces, the forecast rainfall level is directly mapped to a rainfall score: 0 points (no rain); 1 point (light rain); 2 points (moderate rain); 3 points (heavy rain); 4 points (heavy rain and above).

[0068] Step S303: Dynamically assign rainfall detection weights to the rainfall score of the corresponding sensor based on the rainfall detection confidence of each sensor.

[0069] In this step, the essence of dynamic weighting is to assign higher weights to reliable data. Therefore, it is necessary to calculate the real-time rainfall detection confidence level for each sensor's rainfall detection data. The reliability of the rainfall detection data provided by each sensor is evaluated through the real-time rainfall detection confidence level. The higher the confidence level, the greater the weight will be assigned in the subsequent process.

[0070] It should be noted that the confidence level of rainfall detection by a rain sensor can be calculated based on sensor fault codes and signal stability; the confidence level of rainfall detection by a camera can be calculated based on image clarity and rain line recognition reliability; the confidence level of rainfall detection by a lidar can be calculated based on effective point cloud rate and noise filtering rate; the confidence level of rainfall detection by a millimeter-wave radar can be calculated based on target tracking stability and echo signal-to-noise ratio; and the confidence level of external weather information obtained from the vehicle weather service interface can be determined based on signal timeliness and network quality.

[0071] In one possible implementation, the core of weight allocation is to convert the real-time confidence scores of each rainfall detection data source into weights that sum to 1, so as to ensure that the result obtained by subsequent weighted summation is within a reasonable range. The normalized weight method is adopted, that is, the rainfall detection confidence score of a certain sensor = the rainfall detection confidence score of that sensor / the sum of the rainfall detection confidence scores of all valid sensors.

[0072] Step S304: Based on the rainfall detection weight corresponding to each sensor, the rainfall scores corresponding to all sensors are weighted and summed to obtain a comprehensive rainfall score.

[0073] In this step, the quantitative rainfall score of each sensor is multiplied by the corresponding dynamic rainfall detection weight, and then summed to obtain the comprehensive rainfall score.

[0074] Step S305: Map the comprehensive rainfall score to the real-time comprehensive rainfall level according to the preset mapping rules.

[0075] In this step, the comprehensive rainfall score is mapped to a standard 5-level comprehensive rainfall level. The specific mapping rules are shown in Table 1 below.

[0076] Table 1

[0077] Step S306: Based on the rainfall data collected by the rain sensor, the comprehensive rainfall level, and the real-time external weather information, determine whether to control the driving mode to enter rain mode.

[0078] Step S307: When the rainfall data collected by the rain sensor is greater than or equal to the preset rainfall, or the comprehensive rainfall level is greater than or equal to the preset level, or the rainfall intensity indicated by the external weather information is greater than or equal to the preset rainfall intensity and the duration is greater than the preset duration, the driving mode is controlled to enter the rain mode.

[0079] In steps S306 and S307, the preset rainfall can be 10 mm / h (moderate rain), the preset level can be level 2 (moderate rain), and the rainfall intensity is moderate rain. The triggering conditions for activating the rain mode need to meet any one or more of the following and the duration must be greater than the preset duration (e.g., 2 minutes): 1. The comprehensive rainfall level is ≥ level 2; the external weather information forecasts moderate rain or above; the rainfall sensor detects a rainfall of ≥ 10 mm / h.

[0080] It should be noted that in this embodiment, the duration is set to prevent the rain mode from being falsely triggered due to brief sensor detection errors, thereby further improving the accuracy of rain environment recognition.

[0081] In this embodiment, compared to the traditional method of relying solely on rain sensors to detect rainfall intensity, multi-source rainfall detection data fusion is achieved from the direct sensing layer (rain sensors), the environmental sensing layer (cameras, lidar, and millimeter-wave radar), and the external information layer (weather information obtained from the vehicle weather service interface). This avoids single-point failures, and even if one type of rainfall detection data fails, the rainfall magnitude can still be detected through other data sources, making it more likely. Furthermore, by dynamically adjusting the rainfall detection weights based on the real-time rainfall detection confidence of each sensor, the system can adapt to changes in rainfall conditions at any time, accurately identify environmental rainfall conditions, ensure the accuracy of rain mode activation, and provide an accurate data foundation for subsequent adjustments to the control strategy.

[0082] Step S308: After the driving mode enters the rain mode, optimize the data processing algorithm for each sensor.

[0083] In this step, since different sensors are affected differently by rainy weather, it is necessary to optimize the sensor processing algorithms accordingly. After entering rain mode, for cameras, an attention-based de-raining algorithm can be activated to remove rain streaks or water droplets from the image, improving target recognition accuracy; for LiDAR, a targeted point cloud denoising algorithm can be activated to filter raindrop noise; for millimeter-wave radar, the beam focusing range and signal detection threshold can be adjusted to enhance target echo recognition capabilities.

[0084] In one possible implementation, a target detection and classification model is pre-trained and optimized for rainy scenes, and then deployed in the processor. When the rain mode is activated, the images captured by the camera are used to perform target detection using the specially trained target detection and classification model. This model is more robust to motion blur, low contrast, and partial occlusion (such as a motorcycle obscured by rain lines), which can improve the target detection accuracy of the camera in rainy environments.

[0085] In step S309, if the comprehensive rainfall level is greater than or equal to the preset level, the target detection weight of the visual sensor is reduced and the target detection weight of the radar sensor is increased; wherein, the target detection weight of each sensor is positively correlated with its own real-time target detection confidence, and the sum of the target detection weights of all sensors is 1.

[0086] Among them, the visual sensor can be a camera, and the radar sensor can include laser sensors and millimeter-wave sensors.

[0087] In one possible implementation, when the comprehensive rainfall level is greater than or equal to a preset level (e.g., level 2, corresponding to moderate rain), a rain mode is activated. In rain mode, the target detection weights of each sensor are dynamically adjusted by combining the real-time target detection confidence scores of each sensor with the comprehensive rainfall level. The real-time target detection confidence scores of each sensor include: the target detection confidence scores of the camera, the LiDAR, and the millimeter-wave radar.

[0088] In one possible implementation, before dynamically adjusting the target detection weights of each sensor, the method further includes: determining the real-time target detection confidence level of the corresponding sensor based on the sensing characteristics of each sensor. The target detection confidence level of the camera is calculated based on image sharpness and the proportion of rain-covered area; higher sharpness and less rain-covered area result in higher target detection confidence level for the camera. The target detection confidence level of the lidar is calculated based on effective point cloud rate, noise ratio, and point cloud clustering stability; the effective point cloud rate is the ratio of the number of target point clouds to the total number of point clouds; higher effective point cloud rate and less noise result in higher target detection confidence level for the lidar. The target detection confidence level of the millimeter-wave radar is calculated based on target tracking success rate, trajectory smoothness, and echo signal-to-noise ratio; the tracking success rate is the ratio of the number of consecutively successfully tracked frames to the total number of frames; higher target tracking stability and smoother trajectory result in higher target detection confidence level for the millimeter-wave radar.

[0089] In one possible implementation, when the overall rainfall level is greater than or equal to a preset level, the following conditions must be met when reducing the target detection weight of the camera and increasing the target detection weight of the lidar and millimeter-wave radar: 1. The adjusted target detection weight of each sensor is between 0 and 1; 2. The combined target detection weight of the three sensors is 1, dynamically responding to changes in the overall rainfall level and target detection confidence. The higher the overall rainfall level, the higher the target detection confidence of a certain sensor, and the more reliable the target detection data of that sensor, the higher the target detection weight assigned to that sensor.

[0090] In one possible implementation, the base weight of each sensor is first calculated. The base weight is determined by the sensor's own confidence level and reflects the sensor's inherent reliability without considering rainfall conditions: the base weight of a sensor = the sensor's own confidence level / the sum of the confidence levels of all sensors. For example, if the camera's inherent confidence level is 0.9, the lidar's is 0.85, and the millimeter-wave radar's is 0.8, then the camera's base weight = 0.9 / (0.9 + 0.85 + 0.8) = 0.35. Similarly, the lidar's base weight is calculated to be 0.33, and the millimeter-wave radar's base weight is 0.32 (the sum of the base weights of the three sensors is 1). Then, a rainfall correction factor is introduced to adjust the weights based on the comprehensive rainfall level: different sensors have different resistance to rainfall interference, so the basic weights need to be corrected based on the comprehensive rainfall level. Cameras have weak rain resistance: the higher the comprehensive rainfall level, the smaller the correction factor (suppressing weights); millimeter-wave radars have strong rain resistance: the higher the comprehensive rainfall level, the larger the correction factor (enhancing weights); lidar has moderate rain resistance: as the comprehensive rainfall level increases, the correction factor first stabilizes and then increases. After determining the rainfall correction factor for each sensor for different comprehensive rainfall levels, the basic weight of each sensor is multiplied by the corresponding rainfall correction factor, and then normalized to obtain the final target detection weight of each sensor under different comprehensive rainfall levels.

[0091] For example, the different sensor confidence levels corresponding to different comprehensive rainfall levels and the corresponding target detection weights assigned to different sensors are shown in Table 2 below.

[0092] Table 2

[0093] As shown in Table 2, C_cam represents the target detection confidence score of the camera, C_lidar represents the target detection confidence score of the lidar, and C_radar represents the target detection confidence score of the millimeter-wave radar; W_cam represents the target detection weight of the camera, W_lidar represents the target detection weight of the lidar, and W_radar represents the target detection weight of the millimeter-wave radar. F_cam represents the rain correction factor of the camera, F_lidar represents the rain correction factor of the lidar, and F_radar represents the rain correction factor of the millimeter-wave radar.

[0094] It should be noted that the data in Table 2 only provides illustrative examples of weight adjustments under some changes in rainfall conditions. In real-world scenarios, the relevant data can be adjusted according to the actual situation.

[0095] In this embodiment, the reliability of environmental perception is improved by dynamically adjusting the weights based on the combined rainfall conditions and the real-time target detection confidence of the sensors. The weights are rapidly adjusted according to the real-time status of the sensors to ensure that the final fusion result is always based on the most reliable data source, which directly guarantees the accuracy of subsequent target detection results and the security of the control strategy.

[0096] Step S310: Based on the image acquired by the vision sensor, determine whether there are severely blurred areas in the vision sensor.

[0097] In one possible implementation, determining whether a severely blurred region exists in an image captured by a visual sensor (such as a camera) includes: dividing the image captured by the visual sensor into multiple sub-regions; calculating the sharpness of each sub-region to obtain a sharpness metric value corresponding to each sub-region; determining the proportion of rain line occlusion area in each sub-region based on a pre-trained semantic segmentation model; and determining that the sub-region is a severely blurred region if the sharpness metric value corresponding to the sub-region is less than a first preset threshold and the proportion of rain line occlusion area in the sub-region is greater than a second preset threshold.

[0098] In this embodiment, before initiating special scene processing, it is necessary to accurately determine whether a certain local area of ​​the camera meets the severe blur standard to avoid misjudging slight rain occlusion or local shadows as severe blur, leading to excessive weight adjustment. Specifically, the image captured by the camera can be divided into multiple sub-regions, such as 16*16 or 32*32 pixel blocks, with each pixel block being a sub-region. Laplacian edge detection is performed on each sub-region, and the variance of the detection results is calculated. This variance is the quantification value of sharpness. The smaller the variance, the more blurred the edge of the region (rainwater causes loss of details), and the larger the variance, the sharper the region. Combined with a semantic segmentation model based on deep learning, the rain-occluded areas in the image are directly labeled. If the proportion of rain-occluded pixels in a certain sub-region is greater than or equal to a preset threshold (such as 60%), it indicates that the sub-region is blurred due to rain occlusion. The criteria for determining severe blur are predefined. The first preset threshold can be set to 50, and the second preset threshold can be set to 60%. When the Laplacian variance (i.e., the quantification of sharpness) is less than 50, and the area obscured by rain lines is greater than 60%, the corresponding sub-region is determined to be a severely blurred region. Furthermore, it is also possible to set the criteria to indicate the presence of a severely blurred region in the camera when a region in three or more consecutive frames of images meets the above criteria, in order to avoid misjudgment caused by noise in a single frame image.

[0099] Step S311: If there is a severely blurred area in the vision sensor, the spatial range of the severely blurred area relative to the vehicle is determined based on the location information of the severely blurred area.

[0100] For areas where the camera is severely blurry, the target detection capability in those areas needs to be optimized specifically, with a focus on adjusting the radar to monitor those areas where the camera is severely blurry.

[0101] It should be noted that the camera outputs two-dimensional image coordinates (such as pixel (x, y)), while the LiDAR and millimeter-wave radar output three-dimensional vehicle coordinates (such as (X, Y, Z), where X is the vehicle's forward direction, Y is the lateral direction, and Z is the height). In order for the millimeter-wave radar and LiDAR to know which area to focus on, the severely blurred sub-regions in the image must first be converted into three-dimensional spatial regions under a unified vehicle coordinate system to achieve spatial alignment.

[0102] In one possible implementation, the coordinate range of the severely blurred region in the image captured by the camera is first obtained. Based on the camera's internal and external calibration parameters, the image coordinate range is converted into three-dimensional coordinates in the camera coordinate system. Then, the three-dimensional coordinates in the camera coordinate system are converted into a three-dimensional spatial range in the vehicle coordinate system, thus obtaining the three-dimensional spatial region range of the severely blurred region in the vehicle coordinate system.

[0103] For example, assuming the resolution of the image captured by the camera is 1920×1080, and a severely blurred sub-region is the pixel block in the lower left corner of the image, the corresponding image coordinate range is: x∈[0, 480] (horizontal left half), y∈[540, 1080] (vertical lower half, corresponding to the near field area in front of the vehicle). After the coordinates are transformed by the above method, the three-dimensional spatial region range in the vehicle coordinate system is obtained, such as horizontal (Y-axis): Y∈[-3m, 0m] (region within 3m to the left of the vehicle); vertical (X-axis): X∈[5m, 20m] (near field area 5~20m in front of the vehicle); Z∈[0m, 1.5m] (ground to 1.5m height, covering targets such as motorcycles, pedestrians, and front bumpers).

[0104] Step S312: Reduce the target detection weight of the visual sensor within the spatial area, increase the target detection weight of the radar sensor within the spatial area, and adjust the coverage of the main detection signal emitted by the radar sensor within the spatial area or increase the detection signal emission density of the radar sensor within the spatial area.

[0105] Having already determined which 3D spatial region the camera failed in, this step requires controlling the radar sensor (such as lidar and millimeter-wave radar) to focus on that 3D spatial region.

[0106] In this step, the target detection weights of lidar and millimeter-wave radar in the three-dimensional spatial region are increased and the weight of camera in the three-dimensional spatial region is decreased. Essentially, this assigns a higher confidence weight to targets detected by radar in the region, ensuring that subsequent target detection fusion results prioritize radar data from the region rather than the blurry data from the camera.

[0107] It should be noted that when assigning weights to different sensors within the three-dimensional spatial region, the total weight of each sensor must still be 1 to ensure the reliability and accuracy of the final target detection data fusion.

[0108] It should be noted that the detection range of radar (millimeter-wave radar, lidar) is an omnidirectional or forward fan-shaped area. To ensure that it can accurately capture targets in areas where the camera is severely blurred, the radar's detection parameters and data processing range need to be optimized in a targeted manner to avoid wasting radar resources in areas where the camera is clear, while enhancing the detection sensitivity of the failed areas.

[0109] In one possible implementation, the core optimization of the millimeter-wave radar is to adjust the focusing range of the main detection beam, directionally covering the three-dimensional spatial range of the severely ambiguous area in the vehicle coordinate system. For example, the original radar beam covers ±45° laterally and 0~150m longitudinally; after optimization, for the area "Y∈[-3m,0m], X∈[5m,20m]", the lateral angle of the radar beam corresponding to this area is compressed to -30°~-10° (left side area), the longitudinal range is focused to 5-20m, and the signal transmission power of this beam is increased (e.g., from 10dBm to 15dBm) to enhance the target echo signal strength under raindrop interference.

[0110] In one possible implementation, the lidar emits lasers to form point cloud scan lines covering the detection area. The core optimization involves adjusting the point cloud scan line density and enhancing point cloud noise reduction. The lidar scan line density varies in different regions (typically denser in the near field and sparser in the far field). For the three-dimensional space region in the vehicle coordinate system corresponding to the severely blurred area of ​​the camera, the point cloud scan line density of the lidar within this three-dimensional space region is increased, for example, by doubling the number of scan lines corresponding to this three-dimensional space region. For example, the original lidar had 8 scan lines in the region "X∈[5m,20m], Y∈[-3m,0m]"; after optimization, this is increased to 16 lines, increasing the point cloud density in this region (from 50 points / m). 2 Increase to 100 points / m 2This ensures that small targets (such as motorcycle handlebars or pedestrian legs) are covered by a sufficient amount of point cloud data, preventing missed detections due to sparse point cloud coverage. Simultaneously, targeted point cloud denoising is implemented for severely blurred areas. Although the LiDAR is affected by raindrops, targets in severely blurred areas of the camera have higher priority; therefore, a "regional denoising algorithm" is used. For this area's point cloud, only isolated points (single points without other point clouds around them) are filtered out, while continuous point cloud clusters are retained (even if there are a few raindrop noises within the cluster, they will be verified using radar data during subsequent fusion), avoiding excessive denoising that might mistakenly delete real target point clouds.

[0111] Step S313: Based on the adjusted target detection weights, the target detection data collected by each sensor is fused to obtain the target detection result.

[0112] Step S314: Adjust the intelligent driving control strategy based on the comprehensive rainfall level and / or target detection results.

[0113] In one possible implementation, the intelligent driving control strategy is adjusted based on the comprehensive rainfall level, including: when the comprehensive rainfall level is greater than or equal to a preset level, reducing the maximum cruising speed to a range less than or equal to a preset speed threshold, and dynamically increasing the following safety distance based on the comprehensive rainfall level; wherein the comprehensive rainfall level is negatively correlated with the preset speed threshold and positively correlated with the increase in the following safety distance; when the comprehensive rainfall level is greater than or equal to a preset level, increasing the lane change decision threshold; the lane change decision threshold includes a longitudinal clearance threshold and a time margin threshold between vehicles in front and behind in the target lane.

[0114] For example, the preset level can be set to level 2 (moderate rain). When the comprehensive rainfall level reaches level 2, the rain mode is triggered. In rain mode, the maximum cruising speed is limited in segments according to the comprehensive rainfall level. The higher the level, the lower the maximum cruising speed is set to ensure that the vehicle speed matches the road surface adhesion. For example, if the comprehensive rainfall level = 3 (heavy rain), then the maximum cruising speed = min (user-set speed, 80 km / h), that is, the maximum cruising speed is limited to within 80 km / h; if the comprehensive rainfall level = 4 (torrential rain), then the maximum cruising speed = min (user-set speed, 60 km / h), that is, the maximum cruising speed is limited to within 60 km / h.

[0115] In this step, the safe following distance is the distance between the vehicle and the vehicle in front. In rain mode, the following distance dynamically increases according to the overall rainfall level. For example, as shown in Table 3 below, gap1 to gap4 are the following modes that users can choose at different vehicle speeds. gap1 is the most aggressive following mode (selecting the minimum following distance within the safe range), and gap4 is the most conservative following mode. Users can choose from these four following modes (gap1, gap2, gap3, and gap4) to control different following distances (unit: meters, m) according to their driving needs. The following distance before the "-" in the table is the safe following distance for different speeds and following modes in rainless conditions, and the following distance after the "-" is the safe following distance for different speeds and following modes in moderate rain. For example, if the vehicle speed is 36 km / h and the user selects gap1 mode, the following distance is 18 meters under normal rainless conditions. When the overall rainfall level reaches level 2 (moderate rain), the following distance at this speed and in gap1 mode automatically adjusts to 22 meters.

[0116] Table 3

[0117] It should be noted that the data in Table 3 are the changes in following distance corresponding to different vehicle speeds and modes under rainless and moderate rain conditions, as calibrated by real vehicle testing. As the comprehensive rainfall level increases, the increase in following distance will also increase. For example, at a vehicle speed of 36 km / h, if the user selects gap1 mode, the following distance is 18 meters under normal rainless conditions. When the comprehensive rainfall level reaches level 3 (heavy rain), the following distance at this speed and under gap1 mode will automatically adjust to 24 meters, etc. Specific data can be set according to the actual scenario.

[0118] In one possible implementation, when the overall rainfall level is greater than or equal to a preset level, the longitudinal clearance threshold and time margin threshold between vehicles in front and behind in the target lane are increased. The longitudinal clearance refers to the distance between vehicles in front and behind in the target lane. The longitudinal clearance threshold is used to determine whether there is sufficient space between vehicles in front and behind in the target lane to allow the vehicle to change lanes. When the overall rainfall level reaches the preset level (e.g., level 2), a rain mode is entered. Due to the slippery road surface, the longitudinal clearance threshold is increased. For example, the longitudinal clearance threshold is set to 35 meters in sunny weather, but can be set to 55 meters when the preset level is reached, to reduce the risk of lane changing. The time margin threshold can be a lane change execution time threshold, i.e., the maximum allowed time to complete a lane change. When entering rain mode, this time margin threshold needs to be extended to allow sufficient time for lane changing. For example, the time margin threshold can be set to 2 seconds in sunny weather, but can be set to 3.5 seconds in rain mode.

[0119] It should be noted that, in addition to adjusting the control strategy based on the comprehensive rainfall level, this embodiment also adjusts the control strategy based on the target detection results, so that the intelligent driving path changes in response to environmental targets.

[0120] In some embodiments, after entering rain mode using the above method, the lane change assist function and automatic emergency braking function can be linked with rain mode to maximize the protective effect; for example, after confirming that rain mode has been entered, the lane change assist function and automatic emergency braking function are automatically activated. This solves the problems of high lane change risk and long braking distance and untimely braking in rainy conditions.

[0121] For details on how to implement the steps not explained here, please refer to [link to relevant documentation]. Figure 2 The relevant descriptions in the embodiments will not be repeated here.

[0122] In this embodiment, a multi-sensor fusion identification mechanism for rainy weather is employed to avoid misjudging rainfall conditions by a single sensor, ensuring the accuracy of rain mode activation. By dynamically adjusting the fusion weights of multiple sensors and filling in fuzzy areas, the unreliability of sensor data in rainy weather is addressed, significantly improving target recognition rates in rainy environments and greatly enhancing the reliability and accuracy of environmental perception. Furthermore, based on comprehensive rainfall levels, end-to-end adaptation from mode switching to control strategy adjustment is achieved, enabling the system to cope with different rain conditions and enhancing environmental adaptability. The control logic for core scenarios such as following other vehicles and changing lanes is specifically optimized to reduce the risks associated with long braking distances and poor maneuverability in rainy weather, thereby improving driving safety.

[0123] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0124] Figure 4 This is a schematic diagram of the software architecture of an intelligent driving control system provided in an embodiment of this application.

[0125] like Figure 4As shown, the system's software architecture includes an input layer and software modules. The input layer includes ultrasonic sensors, LiDAR, millimeter-wave radar, cameras, a GPS system, and a rain sensor. The input layer primarily provides environmental and status data collected by multiple sensors to the software modules. The software modules include main software components and a mode selector. The mode selector dynamically adapts to the environment, switching the system between normal and rainy modes based on the rain detection data from the input layer to ensure robustness under different weather conditions. The main software components integrate the multi-sensor data from the input layer, construct an environmental model, and output control decisions (such as autonomous driving control signals). These components include several sub-modules: an end-to-end module that directly maps sensor inputs to decision outputs, simplifying the process and suitable for specific scenarios; a bird's-eye view detection module that converts sensor data into a bird's-eye view for better global environmental understanding; a vehicle localization and map modeling module that combines GPS and radar data to achieve vehicle localization and map construction; the bird's-eye view detection module and the vehicle localization and map modeling module interact with the world model; and the world model that integrates all perceived information to generate a unified environmental perception.

[0126] In this embodiment, the sensor data from the input layer first enters the mode selector, which switches modes based on this sensor data, and then transmits it to the main software components in the software module. After being written by multiple sub-modules, it is finally integrated by the world model into a global environmental cognition and outputs control decisions.

[0127] It should be noted that the software architecture of the intelligent driving control system provided in this embodiment is deployed in the vehicle's host computer to implement the above-mentioned intelligent driving control method.

[0128] Figure 5 This is a schematic diagram of the structure of an intelligent driving control device provided in one embodiment of this application. Figure 5 As shown, the intelligent driving control device provided in this embodiment may include: a data acquisition module 501, a rainfall determination module 502, a weight adjustment module 503, a target detection module 504, and a driving control module 505.

[0129] The data acquisition module 501 is used to acquire environmental perception data collected by the sensor; the environmental perception data includes rainfall detection data and target detection data.

[0130] The rainfall determination module 502 is used to determine the real-time comprehensive rainfall level based on the rainfall detection data.

[0131] The weight adjustment module 503 is used to dynamically adjust the target detection weight of each sensor in data fusion based on the comprehensive rainfall level and the real-time target detection confidence of each sensor. The target detection module 504 is used to fuse the target detection data collected by each sensor based on the adjusted target detection weights to obtain the target detection result. The driving control module 505 is used to adjust the intelligent driving control strategy based on the comprehensive rainfall level and / or the target detection result.

[0132] In one possible implementation, the sensor includes a visual sensor and a radar sensor; the weight adjustment module 503 is specifically used to: if the comprehensive rainfall level is greater than or equal to a preset level, reduce the target detection weight of the visual sensor and increase the target detection weight of the radar sensor; wherein, the target detection weight of each sensor is positively correlated with its own real-time target detection confidence, and the sum of the target detection weights of all sensors is 1.

[0133] In one possible implementation, the weight adjustment module 503 is further configured to: after reducing the target detection weight of the visual sensor and increasing the target detection weight of the radar sensor, determine whether there is a severely blurred region in the visual sensor based on the image acquired by the visual sensor; if there is a severely blurred region in the visual sensor, determine the spatial range of the severely blurred region relative to the vehicle based on the location information of the severely blurred region; reduce the target detection weight of the visual sensor within the spatial range, increase the target detection weight of the radar sensor within the spatial range, and adjust the coverage of the main detection signal emitted by the radar sensor within the spatial range or increase the detection signal emission density of the radar sensor within the spatial range.

[0134] In one possible implementation, the weight adjustment module 503 is specifically used to: divide the image acquired by the visual sensor into multiple sub-regions; calculate the sharpness of each sub-region to obtain a sharpness metric value corresponding to each sub-region; determine the proportion of rain line occlusion area in each sub-region based on a pre-trained semantic segmentation model; if the sharpness metric value corresponding to the sub-region is less than a first preset threshold and the proportion of rain line occlusion area in the sub-region is greater than a second preset threshold, then the sub-region is determined to be a severely blurred region.

[0135] In one possible implementation, the sensors include a rain gauge, a visual sensor, and a radar sensor; the rain determination module 502 is specifically used for: quantifying the rain detection data collected by each sensor into a rain score; dynamically assigning rain detection weights to the corresponding rain score of each sensor based on the rain detection confidence level of each sensor; performing a weighted summation of the rain scores corresponding to all sensors based on the rain detection weights corresponding to each sensor to obtain a comprehensive rain score; and mapping the comprehensive rain score to a real-time comprehensive rainfall level according to a preset mapping rule.

[0136] In one possible implementation, the radar sensor includes a lidar and a millimeter-wave radar; the rainfall determination module 502 is specifically used to: quantify the proportion of rain line occlusion area and image blur in the image acquired by the visual sensor into a rainfall score; quantify the point cloud anomaly noise density of the lidar into a rainfall score; quantify the echo scattering intensity of the millimeter-wave radar into a rainfall score; and quantify the rainfall data acquired by the rainfall sensor into a rainfall score; wherein, the larger the value of the rainfall detection data acquired by each sensor, the higher the corresponding quantized rainfall score, and the higher the rainfall score, the higher the rainfall level.

[0137] In one possible implementation, the rainfall determination module 502 is further configured to: determine whether to control the driving mode to enter rain mode based on the rainfall data collected by the rainfall sensor, the comprehensive rainfall level, and the real-time external weather information; control the driving mode to enter rain mode when the rainfall data collected by the rainfall sensor is greater than or equal to a preset rainfall amount, or the comprehensive rainfall level is greater than or equal to a preset level, or the rainfall intensity indicated by the external weather information is greater than or equal to a preset rainfall intensity and the duration is greater than a preset duration; and optimize the data processing algorithm for each sensor after the driving mode enters the rain mode.

[0138] In one possible implementation, the driving control module 505 is specifically configured to: reduce the maximum cruising speed to a range less than or equal to a preset speed threshold when the comprehensive rainfall level is greater than or equal to a preset level, and dynamically increase the following safety distance according to the comprehensive rainfall level; wherein the comprehensive rainfall level is negatively correlated with the preset speed threshold and positively correlated with the increase in the following safety distance; when the comprehensive rainfall level is greater than or equal to the preset level, increase the lane change decision threshold; the lane change decision threshold includes a longitudinal clearance threshold and a time margin threshold between vehicles in front and behind in the target lane.

[0139] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0140] Figure 6 This is a schematic diagram of the structure of a vehicle provided in one embodiment of this application. Figure 6As shown, the vehicle 600 in this embodiment includes a processor 610 and a memory 620, wherein the memory 620 stores a computer program 621 that can run on the processor 610. When the processor 610 executes the computer program 621, it implements the steps in any of the above method embodiments. Alternatively, when the processor 610 executes the computer program 621, it implements the functions of each module / unit in the above device embodiments.

[0141] For example, computer program 621 may be divided into one or more modules / units, one or more of which are stored in memory 620 and executed by processor 610 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 621 in vehicle 600.

[0142] Those skilled in the art will understand that Figure 6 This is merely an example of a vehicle and does not constitute a limitation on the vehicle. It may include more or fewer components than shown, or combinations of certain components, or different components, such as input / output devices, network access devices, buses, etc.

[0143] The processor 610 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0144] The memory 620 can be an internal storage unit of the vehicle, such as a hard drive or memory, or an external storage device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. The memory 620 can also include both internal and external storage devices. The memory 620 is used to store computer programs and other programs and data required by the vehicle. The memory 620 can also be used to temporarily store data that has been output or will be output.

[0145] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0146] An embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described intelligent driving control method.

[0147] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0148] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0149] In the embodiments provided in this application, it should be understood that the disclosed devices / vehicles and methods can be implemented in other ways. For example, the device / vehicle embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0150] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0152] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0153] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An intelligent driving control method, characterized in that, include: Acquire environmental perception data collected by sensors; The environmental sensing data includes rainfall detection data and target detection data; The real-time comprehensive rainfall level is determined based on the rainfall detection data. Based on the comprehensive rainfall level and the real-time target detection confidence of each sensor, the target detection weight of each sensor in the data fusion is dynamically adjusted. Based on the adjusted target detection weights, the target detection data collected by each sensor is fused to obtain the target detection results. The intelligent driving control strategy is adjusted based on the comprehensive rainfall level and / or the target detection results.

2. The intelligent driving control method according to claim 1, characterized in that, The sensors include visual sensors and radar sensors; The step of dynamically adjusting the target detection weight of each sensor in data fusion based on the comprehensive rainfall level and the real-time target detection confidence of each sensor includes: If the comprehensive rainfall level is greater than or equal to the preset level, the target detection weight of the visual sensor is reduced and the target detection weight of the radar sensor is increased. In this system, the target detection weight of each sensor is positively correlated with its own real-time target detection confidence, and the sum of the target detection weights of all sensors is 1.

3. The intelligent driving control method according to claim 2, characterized in that, After reducing the target detection weight of the visual sensor and increasing the target detection weight of the radar sensor, the method further includes: Based on the images acquired by the vision sensor, determine whether there are severely blurred areas in the vision sensor; If the visual sensor has a severely blurred area, the spatial range of the severely blurred area relative to the vehicle is determined based on the location information of the severely blurred area. The target detection weight of the visual sensor within the spatial region is reduced, the target detection weight of the radar sensor within the spatial region is increased, and the main detection signal emitted by the radar sensor is adjusted to cover the spatial region or the detection signal emission density of the radar sensor within the spatial region is increased.

4. The intelligent driving control method according to claim 3, characterized in that, The step of determining whether there are severely blurred areas in the visual sensor based on the image acquired by the visual sensor includes: The image acquired by the visual sensor is divided into multiple sub-regions; The sharpness of each sub-region is calculated to obtain a sharpness quantification value for each sub-region; Based on the pre-trained semantic segmentation model, the proportion of rain line occlusion area in each sub-region is determined; If the clarity quantification value corresponding to the sub-region is less than the first preset threshold, and the proportion of the rain line occlusion area in the sub-region is greater than the second preset threshold, then the sub-region is determined to be a severely blurred region.

5. The intelligent driving control method according to claim 1, characterized in that, The sensors include rain sensors, vision sensors, and radar sensors; The step of determining the real-time comprehensive rainfall level based on the rainfall detection data includes: The rainfall data collected by each sensor is quantified into a rainfall score; Rainfall detection weights are dynamically assigned to the corresponding sensor's rainfall score based on the rainfall detection confidence level of each sensor. Based on the rainfall detection weight corresponding to each sensor, the rainfall scores corresponding to all sensors are weighted and summed to obtain the comprehensive rainfall score. According to the preset mapping rules, the comprehensive rainfall score is mapped to the real-time comprehensive rainfall level.

6. The intelligent driving control method according to claim 5, characterized in that, The radar sensors include lidar and millimeter-wave radar; The process of quantifying the rainfall detection data collected by each sensor into a rainfall score includes: The proportion of rain line obscuration area and image blur in the images acquired by the visual sensor are quantified into a rainfall score. The point cloud anomaly noise density of the lidar is quantified into a rainfall score; The echo scattering intensity of millimeter-wave radar is quantified into a rainfall score; The rainfall data collected by the rain gauge is quantified into a rainfall score; The higher the value of the rainfall detection data collected by each sensor, the higher the corresponding quantified rainfall score, and the higher the rainfall score, the higher the rainfall level.

7. The intelligent driving control method according to claim 6, characterized in that, The rainfall data collected by the rain gauge sensor is rainfall data; the method further includes: Based on the rainfall data collected by the rain sensor, the comprehensive rainfall level, and the real-time external weather information, determine whether to control the driving mode to enter rain mode. When the rainfall data collected by the rain sensor is greater than or equal to the preset rainfall, or the comprehensive rainfall level is greater than or equal to the preset level, or the rainfall intensity indicated by the external weather information is greater than or equal to the preset rainfall intensity and the duration is greater than the preset duration, the driving mode is controlled to enter the rain mode. After the driving mode enters the rain mode, the data processing algorithm for each sensor is optimized.

8. The intelligent driving control method according to any one of claims 1 to 7, characterized in that, The step of adjusting the intelligent driving control strategy based on the comprehensive rainfall level includes: When the comprehensive rainfall level is greater than or equal to a preset level, the maximum cruising speed will be reduced to a range less than or equal to a preset speed threshold, and the following safety distance will be dynamically increased according to the comprehensive rainfall level; wherein, the comprehensive rainfall level is negatively correlated with the preset speed threshold and positively correlated with the increase in the following safety distance; When the comprehensive rainfall level is greater than or equal to the preset level, the lane change decision threshold is increased; the lane change decision threshold includes the longitudinal clearance threshold and the time margin threshold between vehicles in front and behind the target lane.

9. A vehicle comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the intelligent driving control method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent driving control method as described in any one of claims 1 to 8.

Images