Vehicle blind area risk early warning method, system, device and storage medium

Abstract

The application relates to a vehicle blind area risk early warning method, system, device and storage medium, and relates to the technical field of intelligent driving. The method comprises the following steps: acquiring first sensing data of a target object in a vehicle blind area in real time according to a plurality of sensors with a plurality of sensing angles; determining a corresponding image feature enhancement mode according to different image features in the first sensing data; performing image feature enhancement processing on corresponding sensing data in the first sensing data according to the image feature enhancement mode, so as to obtain second sensing data; performing space-time fusion processing on the second sensing data according to current environment information, so as to obtain a target sensing result of the target object; performing risk level evaluation on a vehicle blind area environment according to the target sensing result, so as to obtain a risk evaluation result; and determining at least one grading early warning mode according to the risk evaluation result.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to methods, systems, devices and storage media for vehicle blind spot risk warning. Background Technology

[0002] Research has shown that with the rapid development of intelligent driving technology, vehicle environmental perception systems have become a core component for ensuring driving safety. However, in low-visibility environments such as fog, haze, rain, snow, and nighttime, the performance of traditional perception systems deteriorates significantly, especially in critical areas like blind spots, where insufficient perception capabilities directly threaten driving safety.

[0003] Currently, sensor data quality deteriorates significantly in low-visibility environments. Traditional fusion algorithms often lack modeling of sensor characteristic degradation in low-visibility conditions, making it difficult to achieve effective feature-level and decision-level fusion, resulting in poor multi-sensor data fusion performance. Furthermore, most blind-spot perception systems employ fixed perception strategies and parameter settings, failing to dynamically adjust sensor operating modes and data processing methods according to changes in environmental visibility. This leads to low perception performance in complex and variable low-visibility environments and a lack of refined assessment of blind-spot environmental risks, making it difficult to provide accurate and timely early warning information in low-visibility environments. Summary of the Invention

[0004] This invention provides a method, system, device, and storage medium for vehicle blind spot risk warning, to at least solve the problems of low blind spot perception capability, low accuracy of blind spot environmental risk warning, and poor timeliness in complex and variable low visibility environments. The technical solution of this invention is as follows: According to a first aspect of the present invention, a vehicle blind spot risk warning method is provided. The method includes: acquiring first perception data of a target object in the vehicle blind spot in real time using multiple sensors from multiple perception angles; the first perception data represents multimodal data of the target object under the same high-precision time reference; determining corresponding image feature enhancement methods based on different image features in the first perception data; performing image feature enhancement processing on the corresponding perception data in the first perception data according to the image feature enhancement methods to obtain second perception data; performing spatiotemporal fusion processing on the second perception data based on current environmental information to obtain a target perception result of the target object; the target perception result represents the real-time operating state of the target object, including the type, location information, and speed of the target object; assessing the risk level of the vehicle blind spot environment based on the target perception result to obtain a risk assessment result; and determining to trigger at least one graded warning method based on the risk assessment result; the graded warning method includes visual warning, auditory warning, and tactile warning.

[0005] As one implementation method, the risk assessment results include high risk, medium risk, and low risk. If a high risk assessment result is detected, the assessment parameters are adjusted in real time based on the target object's perception results. The assessment parameters include the sampling frequency of the target sensor, the feature parameters of the target image feature enhancement method, and the confidence weight of the spatiotemporal fusion processing.

[0006] In this implementation, when a high risk is detected, real-time adjustments to key assessment parameters are triggered, directly improving the perception and response performance at the limits of high-risk scenarios. In medium- or low-risk situations, a mild alert is maintained to avoid continuous alarm triggering due to stationary, harmless objects in the environment, such as grass, guardrails, or distant safe targets, thereby significantly reducing false alarm rates and driver alarm fatigue, and improving the driving experience.

[0007] As one implementation method, multiple sensing data include visible light images, radar ranging and speed measurement data, infrared images, and radar point cloud images; multiple sensing angles include the vehicle's forward, lateral, and rearward directions.

[0008] Based on multiple sensors at multiple perception angles, the first perception data of a target object in the vehicle's blind spot is determined, including: acquiring multiple perception data from different perception angles in real time based on multiple sensors at multiple perception angles; filtering out multiple target perception data containing the target object from the multiple perception data; and performing time synchronization processing on the multiple target perception data to obtain the first perception data.

[0009] In this implementation, attention is focused on potential threat targets to achieve precision in the processing of objects. The perception data of the focused target objects is synchronized with high precision in time to achieve the unification of the data foundation and provide reliable data support for subsequent perception data fusion.

[0010] As one implementation method, multiple sensors include visible light sensors, millimeter-wave radar, lidar, ultrasonic sensors, and infrared sensors. Based on these sensors at multiple sensing angles, multiple sensing data from different sensing angles are acquired in real time, including: monitoring the blind spot in front of the vehicle using millimeter-wave radar, infrared sensors, and visible light sensors positioned at the front of the vehicle to obtain a first forward-facing radar point cloud image, a first forward-facing infrared image, and a first forward-facing visible light image; monitoring the lateral blind spot of the vehicle using lidar and ultrasonic sensors positioned on the left and right sides of the vehicle to obtain a first lateral radar point cloud image and radar ranging and speed measurement data; and monitoring the rearward blind spot of the vehicle using infrared sensors positioned at the rear of the vehicle to obtain a first rearward infrared image.

[0011] In this embodiment, by acquiring multimodal data from different types of sensors at multiple angles under a unified high-precision time reference, the limitations of a single sensor are effectively overcome, the adaptability of the vehicle's blind spot to low visibility environments is enhanced, and the integrity and consistency of target object perception within the vehicle's blind spot are ensured.

[0012] As one implementation method, image feature enhancement methods include visible light image enhancement algorithms, infrared image enhancement algorithms, and radar point cloud image enhancement algorithms. The image feature enhancement processing is performed on the corresponding perceptual data in the first perceptual data according to the image feature enhancement method to obtain the second perceptual data. This includes: removing haze and rain stripes from the first forward visible light image using the visible light image enhancement algorithm to obtain the second forward visible light image; extracting key heat source targets from the first forward infrared image or the first backward infrared image using the infrared image enhancement algorithm to determine the contour and structural features of the target object, thus obtaining the second forward infrared image or the second backward infrared image; and removing noisy point clouds from the first forward radar point cloud data or the first lateral radar point cloud image using the radar point cloud image enhancement algorithm, while retaining the real obstacle's point cloud structure and contour, thus obtaining the second forward radar point cloud data or the second lateral radar point cloud image.

[0013] In this embodiment, targeted image feature enhancement algorithms are used for different sensing images to enhance the sensing data, effectively improving the quality of sensing data of each modality. Subsequently, the enhanced effective information is fused, which significantly improves the data quality and reliability of blind spot sensing.

[0014] As one implementation method, based on the current environmental information, the second sensing data is subjected to spatiotemporal fusion processing to obtain the target perception result of the target object. This includes: constructing a unified coordinate system with the vehicle's center of mass as the origin; transforming the second sensing data to the unified coordinate system according to the calibration parameters of each sensor; extracting multiple modal features of the target object based on the second sensing data; the multiple modal features include visible light features, infrared features, and radar features; determining the confidence weight of each modal feature based on the current environmental information; and performing feature weighted fusion of multiple modal features based on the unified coordinate system and the confidence weight to obtain the target perception result.

[0015] In this implementation, a spatiotemporal fusion algorithm is used to achieve effective fusion of multi-source sensing data at the feature layer and decision layer, generating a more accurate and complete blind zone environmental representation, and providing a reliable basis for risk assessment.

[0016] As one implementation method, the risk level of the vehicle blind spot environment is assessed based on the target perception results to obtain the risk assessment results, including: constructing an initial risk assessment model; the risk assessment model represents the mapping relationship between the target perception results and the probability of accident occurrence; training the initial risk assessment model based on historical accident data to obtain the target risk assessment model; and inputting the currently determined target perception results into the target risk assessment model to obtain the risk assessment results.

[0017] According to a second aspect of the present invention, a vehicle blind spot risk warning system is provided, the system comprising: The multi-sensor data acquisition module is configured to acquire first perception data of target objects in the vehicle's blind spot in real time based on multiple sensors from multiple perception angles; the first perception data represents the multimodal data of the target object under the same high-precision time reference.

[0018] The data augmentation processing module is configured to determine the corresponding image feature enhancement method based on different image features in the first sensing data; and to perform image feature enhancement processing on the corresponding sensing data in the first sensing data according to the image feature enhancement method to obtain the second sensing data.

[0019] The spatiotemporal fusion processing module is configured to perform spatiotemporal fusion processing on the second sensing data based on the current environmental information to obtain the target perception result of the target object; the target perception result represents the real-time operating status of the target object, including the type, location information, and speed of the target object.

[0020] The risk assessment module is configured to assess the risk level of the vehicle's blind spot environment based on the target perception results, and obtain the risk assessment results; based on the risk assessment results, determine to trigger at least one graded warning method; the graded warning methods include visual warning, auditory warning and tactile warning.

[0021] The vehicle blind spot risk warning system is configured to perform a vehicle blind spot risk warning method as described in the first aspect and any of its possible implementations.

[0022] According to a third aspect of the present invention, a vehicle blind spot risk warning device is provided, the device being configured to perform a vehicle blind spot risk warning method as described in the first aspect and any possible implementation thereof.

[0023] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a processor of a vehicle blind spot risk warning device, the vehicle blind spot risk warning device is able to perform a vehicle blind spot risk warning method as described in the first aspect and any possible implementation thereof.

[0024] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including computer instructions, which, when executed on a vehicle blind spot risk warning device, cause the vehicle blind spot risk warning device to execute the vehicle blind spot risk warning method of the first aspect and any possible implementation thereof.

[0025] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects: By acquiring multimodal data from multiple angles and different types of sensors under a unified high-precision time reference, the limitations of a single sensor are effectively overcome, enhancing the adaptability of vehicle blind spots in low-visibility environments and ensuring the integrity and consistency of target object perception within the vehicle blind spot; dynamically selecting enhancement methods based on different image features can specifically improve the recognition of key features, improve perception quality in complex low-visibility environments, and enhance the accuracy and real-time performance of target object operation status perception by combining current environmental information with spatiotemporal fusion of multi-source perception data, achieving continuous and stable tracking of target dynamics within the blind spot. Risk level assessment is performed based on the target perception results, triggering at least one graded warning method among visual, auditory, and tactile senses, matching the warning with the risk level, realizing multi-level risk assessment and warning, enhancing driving safety, and reducing the risk of blind spot accidents.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0028] Figure 1 This is a schematic diagram illustrating a vehicle blind spot risk warning system according to an exemplary embodiment; Figure 2 This is a flowchart illustrating a vehicle blind spot risk warning method according to an exemplary embodiment; Figure 3 This is a schematic diagram illustrating a vehicle blind spot risk warning device according to an exemplary embodiment. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0030] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0031] Before providing a detailed introduction to the vehicle blind spot risk warning method provided in this application embodiment, let's briefly introduce the application scenarios and implementation environment involved in this application embodiment.

[0032] With the rapid development of intelligent driving technology, vehicle environmental perception systems have become a core component for ensuring driving safety. However, in low-visibility environments such as fog, haze, rain, snow, and nighttime, the performance of traditional perception systems deteriorates significantly, especially in critical areas like blind spots, where insufficient perception capabilities directly threaten driving safety. Currently, vehicle blind spot perception mainly relies on sensors such as ultrasonic radar, millimeter-wave radar, and visual cameras. Ultrasonic radar is relatively inexpensive, but has a short detection range, limited accuracy, and is susceptible to weather conditions; millimeter-wave radar has good ranging and speed measurement capabilities, but is weak in target recognition and classification; visual cameras can provide rich texture information, but image quality degrades severely in low-visibility environments, making feature extraction difficult.

[0033] Currently, single-sensor blind zone perception in low-visibility environments has significant limitations. While multi-sensor fusion technology can improve perception reliability to some extent, it still faces the following technical challenges under low-visibility conditions: First, sensor data quality deteriorates severely in low-visibility environments. Haze, rain, and snow reduce the contrast and detail of visible light images; radar signals are affected by atmospheric attenuation and clutter interference; and while infrared images have some penetration capability, their resolution is limited and they are easily affected by ambient temperature. Second, multi-sensor data fusion is ineffective. Different sensors experience varying degrees of performance degradation in low-visibility environments, weakening the complementarity between data. Traditional fusion algorithms often lack modeling of sensor characteristic degradation in low-visibility environments, making it difficult to achieve effective feature-level and decision-level fusion. Third, existing systems lack adaptability to low-visibility environments. Most blind zone perception systems employ fixed perception strategies and parameter settings, failing to dynamically adjust sensor operating modes and data processing methods according to changes in environmental visibility, resulting in unstable perception performance in complex and variable low-visibility environments. Furthermore, current systems have relatively simple risk assessment and early warning mechanisms. Fixed distance thresholds or simple logical judgments are typically used, lacking a refined assessment of the risks in blind zone environments, making it difficult to provide accurate and timely early warning information in low visibility environments.

[0034] To address the aforementioned issues, this application proposes a vehicle blind spot risk warning method. By acquiring multimodal data from multiple angles and different types of sensors under a unified high-precision time reference, it effectively overcomes the limitations of a single sensor, enhances the adaptability of vehicle blind spots in low-visibility environments, and ensures the integrity and consistency of target object perception within the vehicle blind spot. Dynamically selecting enhancement methods based on different image features can specifically improve the recognition of key features and enhance perception quality in complex low-visibility environments. By combining current environmental information with spatiotemporal fusion of multi-source perception data, the accuracy and real-time performance of target object motion status perception are enhanced, achieving continuous and stable dynamic tracking of targets within the blind spot. Based on the target perception results, a risk level assessment is performed, triggering at least one graded warning method from visual, auditory, and tactile senses, ensuring that the warning matches the risk level. This achieves multi-level risk assessment and warning, enhancing driving safety and reducing the risk of blind spot accidents.

[0035] Secondly, the implementation architecture involved in this application will be briefly introduced below.

[0036] Figure 1 This is a schematic diagram of a vehicle blind spot risk warning system provided in this application. Figure 1 As shown, the vehicle blind spot risk warning system includes a multi-sensor data acquisition module 11, a data enhancement processing module 12, a spatiotemporal fusion processing module 13, and a risk assessment module 14.

[0037] The multi-sensor data acquisition module 11, data enhancement processing module 12, spatiotemporal fusion processing module 13, and risk assessment module 14 are connected via communication.

[0038] The multi-sensor data acquisition module 11 is configured to acquire first perception data of a target object in the vehicle's blind spot in real time based on multiple sensors from multiple perception angles; the first perception data represents multimodal data of the target object under the same high-precision time reference.

[0039] The data augmentation processing module 12 is configured to determine the corresponding image feature enhancement method based on different image features in the first perception data; and to perform image feature enhancement processing on the corresponding perception data in the first perception data according to the image feature enhancement method to obtain the second perception data.

[0040] The spatiotemporal fusion processing module 13 is configured to perform spatiotemporal fusion processing on the second sensing data based on the current environmental information to obtain the target perception result of the target object; the target perception result represents the real-time operating state of the target object, including the type, location information, and speed of the target object.

[0041] The risk assessment module 14 is configured to assess the risk level of the vehicle blind spot environment based on the target perception result, and obtain a risk assessment result; based on the risk assessment result, determine to trigger at least one graded warning method; the graded warning method includes visual warning, auditory warning and tactile warning.

[0042] As one approach, risk assessment results include high risk, medium risk, and low risk.

[0043] The vehicle blind spot risk warning system is specifically configured to, when a risk assessment result is detected as high risk, adjust the assessment parameters in real time based on the target perception result of the target object; the assessment parameters include the sampling frequency of the target sensor, the feature parameters of the target image feature enhancement method, and the confidence weight of the spatiotemporal fusion processing.

[0044] As one implementation method, multiple sensing data include visible light images, radar ranging and speed measurement data, infrared images, and radar point cloud images; multiple sensing angles include the vehicle's forward, lateral, and rearward directions.

[0045] The vehicle blind spot risk warning system is specifically configured to determine the first perception data of a target object in the vehicle's blind spot based on multiple sensors at multiple perception angles, including: acquiring multiple perception data from different perception angles in real time based on multiple sensors at multiple perception angles; filtering out multiple target perception data containing the target object from the multiple perception data; and performing time synchronization processing on the multiple target perception data to obtain the first perception data.

[0046] As one implementation method, multiple sensors include visible light sensors, millimeter-wave radar, lidar, ultrasonic sensors, and infrared light sensors.

[0047] The vehicle blind spot risk warning system is specifically configured to acquire multiple perception data from different perception angles in real time based on multiple sensors at multiple perception angles, including: monitoring the blind spot in front of the vehicle using millimeter-wave radar, infrared light sensor, and visible light sensor arranged in front of the vehicle to obtain a first forward radar point cloud image, a first forward infrared image, and a first forward visible light image; monitoring the lateral blind spot of the vehicle using lidar and ultrasonic sensors arranged on the left and right sides of the vehicle to obtain a first lateral radar point cloud image and radar ranging and speed measurement data; and monitoring the rear blind spot of the vehicle using infrared light sensor arranged behind the vehicle to obtain a first rear infrared image.

[0048] As one implementation method, image feature enhancement methods include visible light image enhancement algorithms, infrared image enhancement algorithms, and radar point cloud image enhancement algorithms.

[0049] The vehicle blind spot risk warning system is specifically configured to perform image feature enhancement processing on the corresponding perception data in the first perception data according to the image feature enhancement method to obtain the second perception data, including: removing haze and rain stripes from the first forward visible light image according to the visible light image enhancement algorithm to obtain the second forward visible light image; extracting key heat source targets from the first forward infrared image or the first rear infrared image according to the infrared image enhancement algorithm, determining the outline and structural features of the target object, to obtain the second forward infrared image or the second rear infrared image; and removing noisy point clouds from the first forward radar point cloud data or the first lateral radar point cloud image according to the radar point cloud image enhancement algorithm, retaining the point cloud structure and outline of the real obstacle, to obtain the second forward radar point cloud data or the second lateral radar point cloud image.

[0050] As one implementation method, the vehicle blind spot risk warning system is specifically configured to perform spatiotemporal fusion processing on the second perception data based on the current environmental information to obtain the target perception result of the target object. This includes: constructing a unified coordinate system with the vehicle's center of mass as the origin; transforming the second perception data to the unified coordinate system according to the calibration parameters of each sensor; extracting multiple modal features of the target object based on the second perception data; the multiple modal features include visible light features, infrared features, and radar features; determining the confidence weight of each modal feature based on the current environmental information; and performing feature weighted fusion of the multiple modal features based on the unified coordinate system and the confidence weight to obtain the target perception result.

[0051] As one implementation method, the vehicle blind spot risk warning system is specifically configured to assess the risk level of the vehicle blind spot environment based on the target perception results, and obtain the risk assessment results, including: constructing an initial risk assessment model; the risk assessment model characterizing the mapping relationship between the target perception results and the probability of accident occurrence; training the initial risk assessment model based on historical accident data to obtain a target risk assessment model; and inputting the currently determined target perception results into the target risk assessment model to obtain the risk assessment results.

[0052] For ease of understanding, the vehicle blind spot risk warning method provided in this application will be described in detail below with reference to the accompanying drawings.

[0053] Figure 2 This is a flowchart illustrating a vehicle blind spot risk warning method according to an exemplary embodiment, such as... Figure 2 As shown, the vehicle blind spot risk warning method includes the following steps.

[0054] S21, based on multiple sensors at multiple perception angles, acquires first perception data of target objects in the vehicle's blind spot in real time.

[0055] Multiple sensing data sources include visible light images, radar ranging and velocity measurement data, infrared images, and radar point cloud images.

[0056] The multiple sensing angles include the vehicle's forward, lateral, and rearward directions.

[0057] Multiple sensors include visible light sensors, millimeter-wave radar, lidar, ultrasonic sensors, and infrared sensors.

[0058] The first perception data representation is the multimodal data of the target object under the same high-precision time reference.

[0059] Target objects include: other vehicles, pedestrians, obstacles, etc.

[0060] To comprehensively acquire multi-source perception data of the vehicle's blind spot environment, a heterogeneously arranged multi-source sensor array is used to acquire real-time perceived images of target objects in the vehicle's blind spots from various angles, as well as speed and distance measurement data. This involves the following two steps.

[0061] Firstly, multiple sensors at multiple sensing angles acquire multiple sensing data from different sensing angles in real time.

[0062] In one implementation, a combination of millimeter-wave radar and infrared cameras is arranged on both sides of the vehicle's front bumper to detect targets in the blind spot ahead.

[0063] A combination of lidar and ultrasonic sensors is placed below the side mirrors of the vehicle to monitor the side blind spots.

[0064] A wide-angle infrared camera is placed in the center of the vehicle's rear bumper to cover the blind spot behind.

[0065] Specifically, using millimeter-wave radar, infrared light sensors, and visible light sensors positioned at the front of the vehicle, the blind spots in front of the vehicle are monitored, resulting in a first forward-facing radar point cloud image, a first forward-facing infrared image, and a first forward-facing visible light image. Using lidar and ultrasonic sensors positioned on the left and right sides of the vehicle, the lateral blind spots are monitored, resulting in a first lateral radar point cloud image and radar ranging and speed measurement data. Using infrared light sensors positioned at the rear of the vehicle, the rearward blind spots are monitored, resulting in a first rearward infrared image.

[0066] Secondly, from multiple sensing data sets, multiple target sensing data sets containing the target object are selected. These multiple target sensing data sets are then processed in time synchronization to obtain the first sensing data set.

[0067] In one implementation, multiple sensors acquire omnidirectional perception data of the vehicle in real time, collecting multiple perception data sets. A data acquisition mode combining polling and event triggering is employed. Hardware timestamps and software synchronization algorithms are used to perform data time synchronization processing, ensuring the temporal consistency of multi-source perception data. This guarantees that the timestamps of sensor data from the same environment and the same event are strictly aligned, avoiding ghosting or positioning deviations caused by time asynchrony during subsequent perception data fusion.

[0068] In this implementation, raw data acquired asynchronously or with imprecise synchronization are unified onto the same timeline, ensuring that the physical events described by all sensors occur at the same moment. Under strict time synchronization, the same target detected by different sensors can be accurately and correctly associated, improving the accuracy of target association and state estimation, and enhancing the subsequent feature matching and fusion effects.

[0069] S22, determine the corresponding image feature enhancement method based on the different image features in the first sensing data. Perform image feature enhancement processing on the corresponding sensing data in the first sensing data according to the image feature enhancement method to obtain the second sensing data.

[0070] Image features include visible light image features, infrared image features, and radar image features.

[0071] Image feature enhancement methods include visible light image enhancement algorithms, infrared image enhancement algorithms, and radar point cloud image enhancement algorithms.

[0072] There is a one-to-one correspondence between image features and image feature enhancement methods. Different image features of the first perceptual data are detected, and the corresponding image feature enhancement methods are determined.

[0073] Specifically, based on the visible light image enhancement algorithm, the occlusion of haze and rain streaks in the first forward visible light image is removed to obtain the second forward visible light image. Based on the infrared image enhancement algorithm, key heat source targets are extracted from the first forward infrared image or the first backward infrared image to determine the contour and structural features of the target objects, resulting in the second forward infrared image or the second backward infrared image. Based on the radar point cloud image enhancement algorithm, noisy point clouds are removed from the first forward radar point cloud data or the first lateral radar point cloud image, while preserving the true point cloud structure and contour of the obstacles, resulting in the second forward radar point cloud data or the second lateral radar point cloud image.

[0074] In one implementation, the visible light image enhancement algorithm is a dehazing and deraining algorithm based on a conditional generative adversarial network (CGAN). The generator network of this algorithm embeds an atmospheric scattering physics model and achieves image sharpening in hazy scenes through end-to-end training, thereby enhancing the features of visible light images.

[0075] The infrared image enhancement algorithm is a deep learning network based on an attention mechanism. It adaptively focuses on key heat source targets in infrared images, enhancing the salience of target features.

[0076] The radar point cloud image enhancement algorithm is an adaptive filtering algorithm based on voxel grids. It performs noise suppression and compensation on radar point cloud data, effectively suppressing rain and snow noise while preserving the point cloud structure of real obstacles.

[0077] In this implementation, feature enhancement technology effectively overcomes the perception limitations of a single sensor in low-visibility environments, significantly improving the data quality and reliability of blind zone perception.

[0078] S23. Based on the current environmental information, perform spatiotemporal fusion processing on the second perception data to obtain the target perception result of the target object.

[0079] The target perception results characterize the real-time operating status of the target object, including the target object's type, location information, and speed.

[0080] In one implementation, the feature-enhanced second sensory data undergoes spatiotemporal fusion processing. This specifically includes the following four steps.

[0081] First, a unified coordinate system with the vehicle's center of gravity as the origin is constructed. Based on the calibration parameters of each sensor, the second sensing data is transformed into the unified coordinate system.

[0082] Establish a unified world coordinate system to align multi-source sensing data in the spatiotemporal dimensions.

[0083] Secondly, based on the second perception data, multiple modal features of the target object are extracted.

[0084] Multiple modal features include visible light features, infrared features, and radar features.

[0085] A feature fusion network employing a cross-modal attention mechanism is used to calculate the correlation weights between features of different modalities, determine the shared features of the target object in the second perception data, and thus extract the multimodal features that commonly describe the same target object from the perception data collected by different sensors.

[0086] Third, based on the current environmental information, determine the confidence weights of each modal feature.

[0087] The performance of different sensors in the current environment, such as the more reliable infrared features in foggy weather and the less affected radar in rainy weather, is dynamically assigned different confidence weights to each modal feature by combining historical data and real-time environmental information.

[0088] First, environmental perception is performed to determine whether it is fog, rain, snow, or nighttime. Second, the data quality of each sensor in the current environment is evaluated. Finally, a feature fusion network with a cross-modal attention mechanism is used to assign weights to different features and determine the confidence weight of each modality feature.

[0089] Fourth, based on a unified coordinate system, multiple modal features are weighted and fused according to confidence weights to obtain the target perception result.

[0090] Confidence fusion is performed based on a Bayesian filtering framework. The weighted features are fused in a unified coordinate system to obtain the target perception result of the target object.

[0091] In this implementation, a spatiotemporal fusion algorithm is used to achieve effective fusion of multi-source sensing data at the feature layer and decision layer, generating a more accurate and complete blind zone environmental representation, and providing a reliable basis for risk assessment.

[0092] S24. Based on the target perception results, assess the risk level of the vehicle's blind spot environment to obtain a risk assessment result. Based on the risk assessment result, determine the triggering of at least one graded warning method.

[0093] The tiered early warning system includes visual, auditory, and tactile warnings.

[0094] The risk assessment results include high risk, medium risk, and low risk.

[0095] Specifically, an initial risk assessment model is constructed; the risk assessment model represents the mapping relationship between the target perception result and the probability of accident occurrence; the initial risk assessment model is trained based on historical accident data to obtain the target risk assessment model; the currently determined target perception result is input into the target risk assessment model to obtain the risk assessment result.

[0096] The risk assessment model is based on a deep learning network. It performs dual functions based on the risk assessment results.

[0097] First, a graded early warning mechanism is triggered based on the risk level, including dashboard visual warnings, multi-band auditory alarms, and tactile vibration warnings.

[0098] Secondly, if a high-risk risk assessment result is detected, the assessment parameters are adjusted in real time based on the target object's perception results.

[0099] The evaluation parameters include the sampling frequency of the target sensor, the feature parameters of the target image feature enhancement method, and the confidence weight of the spatiotemporal fusion processing.

[0100] In some specific embodiments, different environmental information and different risk assessment results correspond to different graded early warning mechanisms.

[0101] Firstly, in the scenario of highways during dense fog.

[0102] In a highway driving scenario with dense fog, millimeter-wave radar detected a rapidly approaching target on the right rear. However, due to fog interference, the radar signature confidence level was low. Simultaneously, an infrared camera captured a corresponding heat source signal. Feature enhancement was applied to the infrared image, clearly revealing the thermal outline of the vehicle's engine. Cross-modal fusion of the radar's ranging and speed measurement data with the target outline information from the infrared image accurately identified it as a truck accelerating to overtake. Based on the fusion result, a risk assessment was performed, determining the risk level to be high, and triggering a tiered warning: the right blind spot indicator on the dashboard flashed violently, emitted a continuous beeping sound, and the right side of the driver's seat began to vibrate at a high frequency. Simultaneously, feedback control was triggered, increasing the sampling frequency of the right-side sensor and optimizing the data fusion weights to achieve adaptive performance improvement for the system.

[0103] Secondly, in the scenario of urban roads at night and in rain.

[0104] It exhibits good environmental adaptability in urban road environments with nighttime rainfall. However, under low light and raindrop interference conditions, the image quality of the visible light camera is severely degraded. By employing raindrop removal and infrared image enhancement algorithms, feature enhancement is applied to both visible light and infrared images, clearly identifying pedestrian targets in blind spots. The spatiotemporal fusion processing module integrates infrared features and millimeter-wave radar data to accurately determine the pedestrian's location and movement status. Based on the pedestrian's distance and trajectory, a high-risk level is determined, immediately triggering the highest-level warning and feedback control, prioritizing system resources for forward perception tasks to ensure driving safety.

[0105] Thirdly, in low-speed scenarios in parking lots during snowy weather.

[0106] In low-speed scenarios within snow-covered parking lots, precise close-range perception is crucial. A rear-facing wide-angle infrared camera detects low-lying stone blocks partially covered by snow, while ultrasonic sensors provide close-range obstacle information. The thermal outline features of the stone blocks in the infrared image are enhanced. Through spatiotemporal fusion processing, the infrared outline and ultrasonic distance information are combined to accurately mark the stone block's location in the reversing camera image. Based on vehicle speed and obstacle distance, the obstacle is classified as medium-risk, and appropriate driving assistance is provided through a reversing camera warning frame and intermittent audible alerts.

[0107] Therefore, when high risk is detected, real-time adjustments to key assessment parameters are triggered, directly improving perception and response performance at the limits of high-risk scenarios. At critical moments, the system can closely monitor threat targets with a higher data refresh rate, acquiring denser and more continuous trajectory points, thereby significantly improving the ability to capture sudden maneuvers and tracking accuracy. This provides valuable milliseconds of time for emergency decision-making, ensuring the safe operation of vehicles in low-visibility environments. In medium- or low-risk situations, a mild alert is maintained to avoid continuous alarm triggering due to stationary, harmless objects in the environment, such as grass, guardrails, or distant safe targets, thus significantly reducing false alarm rates and driver fatigue, and improving the driving experience.

[0108] Figure 3 This is a schematic diagram of a vehicle blind spot risk warning device provided in this application. Figure 3 The monitoring device 50 includes: a first processor 501, a communication bus 502, a memory 503, a communication interface 504, an output device 505, an input device 506, and a second processor 507.

[0109] The vehicle blind spot risk warning device 50 may include at least one first processor 501 and a memory 503 for storing processor-executable instructions. The first processor 501 is configured to execute the instructions in the memory 503 to implement the vehicle blind spot risk warning method in the following embodiments.

[0110] In addition, the vehicle blind spot risk warning device 50 may also include a communication bus 502, at least one communication interface 504, an input device 506, and an output device 505.

[0111] The first processor 501 may be a processor (central processing unit, CPU), a microprocessor unit, an ASIC, or one or more integrated circuits for controlling the execution of programs according to the present application.

[0112] The communication bus 502 may include a path for transmitting information between the aforementioned components.

[0113] Communication interface 504 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0114] Input device 506 is used to receive input signals and output device 505 is used to output signals.

[0115] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processing unit via a bus. Memory may also be integrated with the processing unit.

[0116] The memory 503 stores instructions for executing the scheme of this application, and the execution is controlled by the first processor 501. The first processor 501 executes the instructions stored in the memory 503 to realize the functions of the method of this application.

[0117] In a specific implementation, as one example, the first processor 501 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 in the CPU.

[0118] In a specific implementation, as one example, the vehicle blind spot risk warning device 50 may include multiple processors, such as... Figure 3 The first processor 501 and the second processor 507 are described. Each of these processors can be a single-core processor or a multi-core processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0119] The vehicle's blind spot risk warning device, such as Figure 3 The diagram shows a first processor 501 and a memory 503 for storing executable instructions of the first processor 501. The first processor 501 is configured to execute the executable instructions to implement a vehicle blind spot risk warning method as described in any of the possible embodiments above. Since the same technical effects can be achieved, further details are omitted here to avoid repetition.

[0120] This application also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by the processor of a vehicle blind spot risk warning device, the vehicle blind spot risk warning device can perform a vehicle blind spot risk warning method as described in any of the possible implementations above. And it can achieve the same technical effect; to avoid repetition, it will not be described again here.

[0121] This application also provides a computer program product, including a computer program or instructions, which are executed by a processor as a vehicle blind spot risk warning method according to any of the possible implementations described above. It achieves the same technical effect, and to avoid repetition, will not be described again here.

[0122] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0123] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for vehicle blind spot risk warning, characterized in that, The method includes: Based on multiple sensors from multiple sensing angles, the first sensing data of the target object in the vehicle's blind spot is acquired in real time; the first sensing data represents the multimodal data of the target object under the same high-precision time reference. Based on the different image features in the first perception data, a corresponding image feature enhancement method is determined; the image feature enhancement processing is performed on the corresponding perception data in the first perception data according to the image feature enhancement method to obtain the second perception data; Based on the current environmental information, the second sensing data is subjected to spatiotemporal fusion processing to obtain the target perception result of the target object; the target perception result represents the real-time operating state of the target object, including the type, location information, and speed of the target object; Based on the target perception results, a risk level assessment is performed on the vehicle blind spot environment to obtain a risk assessment result; based on the risk assessment result, at least one graded warning method is determined to be triggered; the graded warning method includes visual warning, auditory warning and tactile warning.

2. The vehicle blind spot risk warning method according to claim 1, characterized in that, The risk assessment results include high risk, medium risk, and low risk; The method further includes: If the risk assessment result is detected as high risk, the assessment parameters are adjusted in real time based on the target perception result of the target object. The assessment parameters include the sampling frequency of the target sensor, the feature parameters of the target image feature enhancement method, and the confidence weight of the spatiotemporal fusion processing.

3. The vehicle blind spot risk warning method according to claim 1, characterized in that, The multi-sensing angles include the vehicle's forward, lateral, and rearward directions; The first perception data for determining a target object in the vehicle's blind spot based on multiple sensors from multiple perception angles includes: Based on multiple sensors at multiple sensing angles, multiple sensing data from different sensing angles are acquired in real time; the multiple sensing data include visible light images, radar ranging and velocity measurement data, infrared images, and radar point cloud images. From the plurality of sensory data, select the plurality of target sensory data that contain the target object; The multiple target perception data are processed in time synchronization to obtain the first perception data.

4. The vehicle blind spot risk warning method according to claim 3, characterized in that, The multiple sensors include visible light sensors, millimeter-wave radar, lidar, ultrasonic sensors, and infrared light sensors; The method of acquiring multiple sensing data from different sensing angles in real time using multiple sensors at multiple sensing angles includes: Based on the millimeter-wave radar, the infrared light sensor, and the visible light sensor arranged in front of the vehicle, the blind spot in front of the vehicle is monitored to obtain a first forward radar point cloud image, a first forward infrared image, and a first forward visible light image. Based on the lidar and ultrasonic sensors arranged on the left and right sides of the vehicle, the lateral blind spots of the vehicle are monitored to obtain the first lateral radar point cloud image and radar ranging and speed measurement data. The infrared light sensor arranged at the rear of the vehicle monitors the rear blind spot of the vehicle to obtain a first rearward infrared image.

5. The vehicle blind spot risk warning method according to claim 4, characterized in that, The image feature enhancement methods include visible light image enhancement algorithms, infrared image enhancement algorithms, and radar point cloud image enhancement algorithms. The step of performing image feature enhancement processing on the corresponding perceptual data in the first perceptual data according to the image feature enhancement method to obtain the second perceptual data includes: According to the visible light image enhancement algorithm, the haze and rain stripes in the first forward visible light image are removed to obtain the second forward visible light image; According to the infrared image enhancement algorithm, the key heat source target of the first forward infrared image or the first backward infrared image is extracted, the outline and structural features of the target object are determined, and a second forward infrared image or a second backward infrared image is obtained. According to the radar point cloud image enhancement algorithm, noisy point clouds are removed from the first forward radar point cloud data or the first lateral radar point cloud image, while retaining the point cloud structure and contour of the real obstacle, to obtain the second forward radar point cloud data or the second lateral radar point cloud image.

6. The vehicle blind spot risk warning method according to claim 5, characterized in that, The step of performing spatiotemporal fusion processing on the second sensing data based on current environmental information to obtain the target perception result of the target object includes: Construct a unified coordinate system with the vehicle's center of gravity as the origin; transform the second sensing data into the unified coordinate system according to the calibration parameters of each sensor; Based on the second sensing data, multiple modal features of the target object are extracted; the multiple modal features include visible light features, infrared features, and radar features; Based on the current environmental information, determine the confidence weight of each modal feature; Based on the unified coordinate system, the multiple modal features are weighted and fused according to the confidence weights to obtain the target perception result.

7. The vehicle blind spot risk warning method according to claim 6, characterized in that, The step of assessing the risk level of the vehicle blind spot environment based on the target perception result, and obtaining the risk assessment result, includes: Construct an initial risk assessment model; the risk assessment model represents the mapping relationship between the target perception results and the probability of accident occurrence; Based on historical accident data, the initial risk assessment model is trained to obtain the target risk assessment model; The current target perception result is input into the target risk assessment model to obtain the risk assessment result.

8. A vehicle blind spot risk warning system, characterized in that, The system includes, The multi-sensor data acquisition module is configured to acquire first perception data of target objects in the vehicle's blind spot in real time based on multiple sensors from multiple perception angles. The first sensing data represents the multimodal data of the target object under the same high-precision time reference; The data augmentation processing module is configured to determine the corresponding image feature augmentation method based on different image features in the first perceived data; The image feature enhancement process is performed on the corresponding perceptual data in the first perceptual data according to the image feature enhancement method to obtain the second perceptual data. The spatiotemporal fusion processing module is configured to perform spatiotemporal fusion processing on the second sensing data based on the current environmental information to obtain the target sensing result of the target object; The target perception result characterizes the real-time operating status of the target object, including the type, location information, and speed of the target object; The risk assessment module is configured to assess the risk level of the vehicle blind spot environment based on the target perception results, and obtain the risk assessment results. Based on the risk assessment results, at least one tiered early warning method is determined to be triggered; the tiered early warning method includes visual early warning, auditory early warning, and tactile early warning. It is configured to perform the vehicle blind spot risk warning method as described in any one of claims 1-7.

9. A vehicle blind spot risk warning device, characterized in that, It is configured to perform the vehicle blind spot risk warning method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the vehicle blind spot risk warning method as described in any one of claims 1-7.

Images