Driving risk detection method, device, equipment and storage medium

Abstract

The application discloses a driving risk detection method, device and equipment and a storage medium. Multimodal data in the current environment of a vehicle is acquired, dimension reduction is performed and feature fusion is performed to generate a unified multimodal fusion feature vector. The complexity level of the current road condition is detected, and the K value range used for scene retrieval is dynamically determined according to the level, and then a target K value is selected, wherein a larger K value is used for a complex road condition to ensure the comprehensiveness of retrieval, and a smaller K value is used for a normal road condition to improve efficiency. From a pre-stored historical risk scene library, a target K historical scene most similar to the current fusion feature vector is retrieved. Based on the obstacle avoidance strategy associated with the K historical scenes, the risk detection result of the current vehicle is comprehensively determined. The application can improve the efficiency of risk detection, and is beneficial to improving the response speed and decision reliability of the vehicle in a sudden scene. The technical scheme of the application can be widely applied to the technical field of vehicles.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a driving risk detection method, device, equipment, and storage medium. Background Technology

[0002] With the deepening development of autonomous driving technology, the decision-making system of intelligent vehicles is shifting from a single-model driven approach to a composite decision-making paradigm that combines multimodal perception with historical experience. To cope with complex and ever-changing road environments, the ability to quickly and accurately assess driving risks and make pre-decision decisions using multi-dimensional sensor data has become crucial for improving the safety and reliability of autonomous driving.

[0003] In related technologies, mainstream driving risk detection solutions typically rely on end-to-end deep learning models for real-time perception and decision-making. These methods heavily depend on massive amounts of labeled data during model training, resulting in limited generalization ability for extreme or rare long-tail scenarios. Furthermore, computational latency during model inference makes it difficult to generate effective obstacle avoidance decisions in a timely manner when facing sudden high-risk scenarios, posing a certain risk of response lag. Summary of the Invention

[0004] This application provides a driving risk detection method, apparatus, device, and storage medium, which can improve the efficiency of risk detection.

[0005] One aspect of this application provides a driving risk detection method, the method comprising: Acquire multimodal data of the vehicle in the current environment; wherein the multimodal data includes at least one of lidar point cloud, camera image, and millimeter-wave radar signal; The multimodal data is subjected to dimensionality reduction and feature fusion to obtain a multimodal fused feature vector; The complexity level of the current road conditions of the vehicle is detected, the corresponding range of K values ​​is determined according to the complexity level, and a target K value is selected from the range of K values; wherein, the K value corresponding to normal road conditions is less than the K value corresponding to complex road conditions; Retrieve the K most similar historical scenarios from the pre-stored historical risk scenario library that are most similar to the multimodal fusion feature vector; Based on the target K value and the historical scenarios, determine the current risk detection result of the vehicle.

[0006] For example, in some embodiments, the dimensionality reduction and feature fusion of the multimodal data to obtain a multimodal fused feature vector includes: Extract environmental features, target features, and trajectory features from the multimodal data; The attention weights between different modal features are calculated using a cross-attention mechanism, and the target features of a specified category are weighted and enhanced; wherein, the specified category includes pedestrians and / or construction signs. Based on the attention weights, the environmental features, the target features, and the trajectory features are weighted to obtain multimodal features; Principal component analysis is performed on the multimodal features to reduce their dimensionality, generating the multimodal fusion feature vector of a specified dimension.

[0007] For example, in some embodiments, the weight enhancement processing of the target features of the specified category includes: Identify the target category corresponding to the target features; If the target category belongs to pedestrians or construction signs, the attention weight of the target feature corresponding to the target category is increased according to the preset hazard category weight coefficient.

[0008] For example, in some embodiments, detecting the complexity level of the vehicle's current road conditions includes: Based on the multimodal fusion feature vector, the density of target objects in the current road conditions and the real-time driving speed of the vehicle are determined. The target density and the real-time driving speed are input into a preset complexity evaluation model to obtain an initial complexity score; If at least one preset high-risk event feature is identified from the multimodal fusion feature vector, the initial complexity score is weighted and adjusted upwards to obtain the target complexity score; The target complexity score is compared with a preset complexity threshold to determine the complexity level of the current road condition.

[0009] For example, in some embodiments, the historical risk scenario library uses a dynamic sparse voxel structure to store data.

[0010] For example, in some embodiments, retrieving the K most similar target historical scenarios to the multimodal fusion feature vector from a pre-stored historical risk scenario library includes: The dynamic sparse voxel structure is mapped to memory, and multiple candidate voxels adjacent to the multimodal fusion feature vector space are determined based on the spatial distribution of the current multimodal fusion feature vector. In the feature vectors of multiple historical scenes contained in the candidate voxel, the Manhattan distance between the multimodal fusion feature vector and the feature vector of each historical scene is calculated in parallel. From the calculated Manhattan distances, select the historical scene with the smallest target K value.

[0011] Exemplarily, in some embodiments, the method further includes: From the obstacle avoidance strategies associated with the target K value and historical scenarios, select the target strategy with the highest frequency of occurrence, or determine the target strategy through a weighted voting mechanism; When a risk is detected in the vehicle, control commands such as deceleration, lane change, or emergency braking are output according to the target strategy.

[0012] On the other hand, embodiments of this application provide a driving risk detection device, the device comprising: An acquisition unit is used to acquire multimodal data of the vehicle in the current environment; wherein the multimodal data includes at least one of lidar point cloud, camera image, and millimeter-wave radar signal; The fusion unit is used to perform dimensionality reduction and feature fusion on the multimodal data to obtain a multimodal fusion feature vector; The detection unit is used to detect the complexity level of the current road conditions of the vehicle, determine the corresponding range of K values ​​based on the complexity level, and select a target K value from the range of K values; wherein, the K value corresponding to normal road conditions is less than the K value corresponding to complex road conditions; The retrieval unit is used to retrieve the K most similar historical scenarios to the multimodal fusion feature vector from the pre-stored historical risk scenario library; The processing unit is used to determine the current risk detection result of the vehicle based on the target K value and the historical scenario.

[0013] On the other hand, embodiments of this application provide an electronic device, including a processor and a memory; The memory is used to store computer programs; The processor executes the computer program to implement the aforementioned driving risk detection method.

[0014] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program that is executed by a processor to implement the aforementioned driving risk detection method.

[0015] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the aforementioned driving risk detection method.

[0016] The embodiments of this application include at least the following beneficial effects: This application provides a driving risk detection method, device, equipment, and storage medium, aiming to improve the problems of insufficient generalization ability and slow response of related solutions. This application first acquires multimodal data of the vehicle in the current environment, and performs dimensionality reduction and feature fusion to generate a unified multimodal fusion feature vector. The complexity level of the current road condition is detected, and the range of K values ​​used for scene retrieval is dynamically determined based on this level, thereby selecting a target K value. Complex road conditions correspond to larger K values ​​to ensure the comprehensiveness of the retrieval, while normal road conditions use smaller K values ​​to improve efficiency. The K most similar target historical scenes to the current fusion feature vector are retrieved from a pre-stored historical risk scene library. Based on the obstacle avoidance strategies associated with these K historical scenes, the risk detection result of the current vehicle is comprehensively determined. This application, through dynamic K value retrieval and historical scene matching, can utilize verified historical experience for rapid risk prediction, reducing the dependence on massive labeled data and single deep learning models, and is expected to improve the system's response speed and decision reliability in long-tail and sudden scenarios. Attached Figure Description

[0017] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0018] Figure 1 This is a system architecture diagram of a driving risk detection method provided in the embodiments of this application; Figure 2 This is a flowchart illustrating a driving risk detection method provided in an embodiment of this application; Figure 3 This is a structural block diagram of a driving risk detection device provided in the embodiments of this application; Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] It is understood that the terms “first,” “second,” etc., used in this application may be used to describe various concepts herein, but unless otherwise stated, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another.

[0021] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0022] With the deepening development of autonomous driving technology, the decision-making system of intelligent vehicles is shifting from a single-model driven approach to a composite decision-making paradigm that combines multimodal perception with historical experience. To cope with complex and ever-changing road environments, the ability to quickly and accurately assess driving risks and make pre-decision decisions using multi-dimensional sensor data has become crucial for improving the safety and reliability of autonomous driving.

[0023] In related technologies, mainstream driving risk detection solutions typically rely on end-to-end deep learning models for real-time perception and decision-making. These methods heavily depend on massive amounts of labeled data during model training, resulting in limited generalization ability for extreme or rare long-tail scenarios. Furthermore, computational latency during model inference makes it difficult to generate effective obstacle avoidance decisions in a timely manner when facing sudden high-risk scenarios, posing a certain risk of response lag.

[0024] In view of this, this application provides a driving risk detection method, device, equipment, and storage medium, aiming to improve the problems of insufficient generalization ability and slow response of related solutions. This application first acquires multimodal data of the vehicle in the current environment and performs dimensionality reduction and feature fusion to generate a unified multimodal fusion feature vector. The complexity level of the current road condition is detected, and the range of K values ​​used for scene retrieval is dynamically determined based on this level. A target K value is then selected, with a larger K value corresponding to complex road conditions to ensure comprehensive retrieval, and a smaller K value used for normal road conditions to improve efficiency. The K most similar target historical scenes to the current fusion feature vector are retrieved from a pre-stored historical risk scene library. Based on the obstacle avoidance strategies associated with these K historical scenes, the risk detection result of the current vehicle is comprehensively determined. This application, through dynamic K-value retrieval and historical scene matching, can utilize verified historical experience for rapid risk prediction, reducing the dependence on massive labeled data and a single deep learning model, and is expected to improve the system's response speed and decision reliability in long-tail and sudden scenarios.

[0025] System architecture and scenario description used in the embodiments of this application Please refer to Figure 1 , Figure 1 The diagram shows a system architecture diagram of a driving risk detection method provided in this application embodiment, which includes a terminal device 140, an Internet 130, a gateway 120, a back-end server 110, etc.

[0026] In this embodiment, the terminal device 140 may be an in-vehicle device, specifically a control unit integrated in a vehicle, which includes a multimodal sensor for collecting data such as lidar point clouds, camera images, and millimeter-wave radar signals, as well as a processor and memory for executing the driving risk detection method described in this application.

[0027] Backend server 110 refers to a computer system that can provide certain services to terminal device 140. Compared with ordinary terminal device 140, backend server 110 has higher requirements in terms of stability, security, and performance. Backend server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines).

[0028] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator, enabling network interconnection at the transport layer. It bridges the gap between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from terminal device 140 to backend server 110 are forwarded to the corresponding backend server 110 via gateway 120. Messages sent from backend server 110 to terminal device 140 are also forwarded to the corresponding terminal device 140 via gateway 120.

[0029] The backend server 110 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0030] The driving risk detection method provided in this application embodiment can be executed independently on the terminal device 140, or based on data interaction between the terminal device 140 and the backend server 110.

[0031] Of course, it is understood that the implementation environment corresponding to the method in the embodiments of this application is not limited to that of the implementation environment. Figure 1 As shown, those skilled in the art can flexibly select the specific implementation environment according to actual needs, and this application does not impose any restrictions on this.

[0032] General Description of Embodiments in this Application Please refer to Figure 2 , Figure 2A flowchart illustrating a driving risk detection method provided in an embodiment of this application is shown. Figure 2 As shown, a driving risk detection method according to an embodiment of this application includes, but is not limited to, the following steps: Step 210: Acquire multimodal data of the vehicle in the current environment; wherein the multimodal data includes at least one of lidar point cloud, camera image, and millimeter-wave radar signal; Step 220: Perform dimensionality reduction and feature fusion on the multimodal data to obtain a multimodal fused feature vector; Step 230: Detect the complexity level of the current road conditions of the vehicle, determine the corresponding range of K values ​​based on the complexity level, and select a target K value from the range of K values; wherein, the K value corresponding to normal road conditions is less than the K value corresponding to complex road conditions; Step 240: Retrieve the K most similar historical scenarios to the multimodal fusion feature vector from the pre-stored historical risk scenario library; Step 250: Determine the current risk detection result of the vehicle based on the target K value and the historical scenario.

[0033] This application provides a driving risk detection method, which aims to improve the problems of limited generalization ability and delayed response to sudden risk scenarios in related technologies that rely on a single deep learning model. By fusing multimodal perception data and introducing historical scene similarity matching and dynamic K-value adjustment mechanisms, it achieves efficient and reliable forward-looking detection of driving risks.

[0034] Specifically, this method first performs feature fusion and dimensionality reduction on the multimodal data perceived by the vehicle to form a unified scene feature vector. Based on this, the retrieval range (K value) is dynamically determined according to the real-time road condition complexity to balance matching accuracy and decision-making efficiency. Next, the K most similar historical cases to the current scenario are quickly retrieved from a pre-built historical risk scenario library. Finally, based on the validated obstacle avoidance strategies associated with these historical cases, a comprehensive decision is made regarding the current risk level and corresponding countermeasures. This method combines rule-based historical experience matching with data-driven real-time perception, effectively reducing reliance on massive amounts of labeled data and single models. It can leverage historical experience to make advance predictions in unexpected or rare road conditions, thereby improving the safety, real-time performance, and decision-making reliability of autonomous driving systems in complex environments.

[0035] Below, in conjunction with Figure 2 This paper introduces and explains each step of the driving risk detection method in the embodiments of this application.

[0036] In step 210, the vehicle's multimodal data in the current environment is first acquired. Here, multimodal data is the raw signal set obtained by the vehicle through various onboard sensors to perceive the environment, typically including but not limited to: 3D point cloud data generated by lidar sensors (used to accurately obtain the three-dimensional contours and positions of targets), 2D image sequences captured by cameras (used to identify the category, texture, and semantic information of targets), and 4D point clouds containing velocity information provided by millimeter-wave radar (used to stably detect targets and obtain their radial velocity in adverse weather conditions). Through the complementarity of multi-source heterogeneous data, a redundant and robust perception system can be constructed.

[0037] In step 220, the multimodal data obtained in step 210 is subjected to feature extraction, fusion, and dimensionality reduction to generate a unified and compact multimodal fusion feature vector.

[0038] Specifically, firstly, pre-trained feature extraction networks (such as convolutional neural networks, PointNet, etc.) can be used to extract environmental features (such as lane lines, curbs), target features (such as the category, size, and location of vehicles, pedestrians, and cones), and trajectory features (such as the speed and direction of the target) from images, point clouds, and radar signals, respectively. Then, a cross-attention mechanism is introduced to calculate the correlation weights between different modal features. For example, in rainy or foggy weather, the weights of millimeter-wave radar features can be automatically enhanced to compensate for the attenuation of visual modalities. This mechanism can also enhance the feature weights of specific target categories (such as pedestrians and construction signs) to highlight the decision-making impact of high-risk targets. Finally, dimensionality reduction processing, such as principal component analysis, is performed on the weighted fused high-dimensional features to generate a low-dimensional multimodal fusion feature vector that comprehensively represents the current driving scenario, creating conditions for subsequent rapid matching calculations.

[0039] In step 230, the complexity level of the current road conditions is detected, and the range of K values ​​used when retrieving historical scenes is dynamically determined accordingly, thereby selecting a target K value. Specifically, based on the multimodal fusion feature vector generated in step 220, or directly combined with the original perception data, several key indicators of the current scene can be analyzed, such as: lane line clarity, density of surrounding dynamic and static objects, and whether there are construction signs or pedestrian crossing intentions.

[0040] Based on these indicators, the current road condition can be classified into different levels, such as "normal" or "complex." This allows for determining the range of K-values ​​used when retrieving historical scenarios based on the complexity level. For example, in simple, open, normal road conditions, to maximize decision-making efficiency, a target K-value is typically selected from a smaller range (e.g., 3-5) to achieve rapid retrieval and decision-making. Conversely, in complex road conditions with dense targets and fluctuating situations, to ensure comprehensiveness and accuracy in decision-making, a target K-value is selected from a larger range (e.g., 7-12) to match more historical scenarios for comprehensive analysis. This allows for scenario-adaptive retrieval, balancing the efficiency and accuracy of risk detection.

[0041] In step 240, based on the target K value determined in step 230, the K most similar historical scenarios to the current "multimodal fusion feature vector" are retrieved from the pre-stored historical risk scenario library. In this embodiment, the historical risk scenario library is a structured database that stores a large number of de-identified historical driving scenarios and their corresponding feature vectors, risk labels, and verified effective obstacle avoidance strategies. The retrieval process can employ an optimized nearest neighbor search algorithm to calculate the distance metric (such as Manhattan distance or cosine similarity) between the current feature vector and all historical feature vectors in the library, and quickly filter out the K historical scenarios with the smallest distance. In this way, the new scenario currently being faced can be quickly associated with similar scenarios accumulated in the past and with which there is processing experience.

[0042] In step 250, based on the K historical scenarios retrieved in step 240, the risk detection result for the current vehicle is finally determined. Specifically, the pre-labeled risk tags associated with each of these K historical scenarios can be analyzed. These tags indicate whether a risk event actually occurred in the historical scenario, such as "risky" (corresponding to a collision, emergency avoidance, etc.) or "no risk" (corresponding to safe passage). Next, a corresponding confidence weight is calculated for each historical scenario based on its similarity to the current scenario. Historical scenarios with higher similarity have higher weights in the final decision. Subsequently, the risk tags of all K scenarios are weighted statistically: if the total weight of the "risky" tag exceeds a preset judgment threshold, the detection result of the current scenario is determined to be "risky"; otherwise, it is determined to be "no risk". In some implementations, the system can also calculate a specific risk probability value as a quantitative output. Finally, the system generates a risk detection result containing a binary judgment of "risky / no risk" or a risk probability value. This result can be used for subsequent warning prompts or as input for higher-level decision-making modules.

[0043] It is understood that, in the embodiments of this application, a decision-making framework that does not solely rely on end-to-end computation of real-time data is constructed through the aforementioned progressive process. This application combines the advantages of deep learning in feature extraction with the advantages of instance-based matching learning in interpretability and few-shot learning. This system can not only handle common scenarios but also, by retrieving similar historical cases, provide reliable and rapid decision-making references even when facing sudden "long-tail" risk scenarios with insufficient coverage in the training data. This effectively improves the safety, robustness, and responsiveness of autonomous driving systems in complex real-world environments.

[0044] Specifically, in some embodiments, the dimensionality reduction and feature fusion of the multimodal data to obtain a multimodal fused feature vector includes: Extract environmental features, target features, and trajectory features from the multimodal data; The attention weights between different modal features are calculated using a cross-attention mechanism, and the target features of a specified category are weighted and enhanced; wherein, the specified category includes pedestrians and / or construction signs. Based on the attention weights, the environmental features, the target features, and the trajectory features are weighted to obtain multimodal features; Principal component analysis is performed on the multimodal features to reduce their dimensionality, generating the multimodal fusion feature vector of a specified dimension.

[0045] Specifically, in some embodiments, the weight enhancement processing of the target features of the specified category includes: Identify the target category corresponding to the target features; If the target category belongs to pedestrians or construction signs, the attention weight of the target feature corresponding to the target category is increased according to the preset hazard category weight coefficient.

[0046] In this embodiment of the application, when performing weight enhancement processing on target features of a specified category, it can be achieved in the following way: After extracting target features from the multimodal data, the built-in target classification module is invoked to identify the specific target category corresponding to the feature, such as vehicles, pedestrians, cones, and traffic signs. Based on the category identification, it is compared with a pre-defined "hazard category list." This list pre-defines high-risk target types that require special attention in driving risk assessment, typically including "pedestrians" and "construction markers" (such as cones and construction barriers).

[0047] When the system determines that the category of the current target feature matches the list, i.e., the target category belongs to pedestrians or construction markers, it amplifies the attention weight of that target feature in the subsequent feature fusion process based on a pre-configured "hazard category weight coefficient" greater than 1.0. This mechanism ensures that information representing high-risk targets such as pedestrians, construction vehicles, and cones is more prominently represented when generating the final multimodal fusion feature vector. This allows the system to maintain higher sensitivity and vigilance towards scenarios containing such targets in subsequent historical scene matching and risk assessment, thereby improving its ability to detect potential collision risks.

[0048] Specifically, in some embodiments, detecting the complexity level of the current road conditions of the vehicle includes: Based on the multimodal fusion feature vector, the density of target objects in the current road conditions and the real-time driving speed of the vehicle are determined. The target density and the real-time driving speed are input into a preset complexity evaluation model to obtain an initial complexity score; If at least one preset high-risk event feature is identified from the multimodal fusion feature vector, the initial complexity score is weighted and adjusted upwards to obtain the target complexity score; The target complexity score is compared with a preset complexity threshold to determine the complexity level of the current road condition.

[0049] In this embodiment of the application, the complexity level of detecting the current road conditions of the vehicle can be achieved through a comprehensive evaluation method that combines quantitative indicators and event judgment.

[0050] Specifically, firstly, based on the multimodal fusion feature vector generated in the previous steps, two core dynamic indicators for quantitative evaluation are parsed out: one is the density of objects in the current road conditions, which reflects the density of surrounding traffic participants; the other is the real-time driving speed of the vehicle itself, which reflects the current motion state of the vehicle.

[0051] Subsequently, these two metrics are input into a pre-defined complexity assessment model. This model, trained by analyzing a large amount of historical driving data, can map the basic environmental complexity under different combinations of object density and speed, and output an initial complexity score. To further enhance sensitivity to sudden and high-risk scenarios, this embodiment also detects in parallel whether the same multimodal fusion feature vector contains certain predefined high-risk event features that characterize immediate risks, such as detecting a pedestrian's sudden lateral movement intention or an abnormal cutting trend of vehicles in adjacent lanes.

[0052] If any of these high-risk event characteristics are identified, the system will trigger a weighted adjustment mechanism to significantly increase the initial complexity score obtained above, thus arriving at a final target complexity score. The system then compares this target complexity score with a preset complexity threshold to clearly categorize the current road condition into different complexity levels, such as normal, complex, or other levels; this application does not impose any restrictions on this.

[0053] It is understood that the embodiments of this application not only consider the static density of the environment and the dynamics of vehicles, but also enhance the response to key risk events, making the determination of road condition complexity more comprehensive and accurate, and providing a more reliable basis for subsequent dynamic adjustment of the K value.

[0054] Specifically, in some embodiments, the historical risk scenario library uses a dynamic sparse voxel structure to store data.

[0055] Specifically, in some embodiments, retrieving the K target historical scenarios most similar to the multimodal fusion feature vector from a pre-stored historical risk scenario database includes: The dynamic sparse voxel structure is mapped to memory, and multiple candidate voxels adjacent to the multimodal fusion feature vector space are determined based on the spatial distribution of the current multimodal fusion feature vector. In the feature vectors of multiple historical scenes contained in the candidate voxel, the Manhattan distance between the multimodal fusion feature vector and the feature vector of each historical scene is calculated in parallel. From the calculated Manhattan distances, select the historical scene with the smallest target K value.

[0056] In this embodiment of the application, to achieve efficient retrieval of massive historical risk scenarios, the historical risk scenario database can adopt a data storage structure based on Dynamic Sparse Voxel Structure. This structure discretizes the high-dimensional feature vector space into a series of voxels (grid units in three-dimensional space) and allocates storage resources only in voxels where actual data points (i.e., feature vectors of historical scenarios) fall, thereby greatly saving storage space and optimizing the logical structure of data organization.

[0057] When scene retrieval is required, the entire dynamic sparse voxel structure can be mapped into memory for fast access. Then, based on the coordinates of the currently generated multimodal fusion feature vector in the feature space, the region where it is located is quickly located, and multiple candidate voxels that are spatially adjacent to it are identified. These voxels contain potentially similar historical scenes that are "neighboring" to the current scene in the feature space.

[0058] Subsequently, the system initiates parallel computation, simultaneously calculating the Manhattan distance between the current feature vector and each historical feature vector within the feature vectors of all historical scenes contained in these candidate voxels. This is a commonly used and computationally efficient similarity metric in high-dimensional spaces. From all the Manhattan distances obtained through this batch of parallel computation, the K smallest distances (i.e., the K target distances) are selected, and the corresponding K historical scenes are determined as the most similar matching results to the current scene.

[0059] It is understood that the retrieval method based on dynamic sparse voxels provided in this application embodiment significantly improves the speed of finding nearest neighbors in a large-scale scene library by restricting the global search to local adjacent voxels and combining it with parallel distance calculation, thus meeting the stringent real-time requirements of autonomous driving systems.

[0060] Specifically, in some embodiments, the method further includes: From the obstacle avoidance strategies associated with the target K value and historical scenarios, select the target strategy with the highest frequency of occurrence, or determine the target strategy through a weighted voting mechanism; When a risk is detected in the vehicle, control commands such as deceleration, lane change, or emergency braking are output according to the target strategy.

[0061] In this embodiment of the application, after completing the historical scene retrieval and risk assessment, a targeted decision output step can also be provided.

[0062] Specifically, when the system determines that the current vehicle is at risk (i.e., "risky") based on the risk labels of the K most similar historical scenarios, it can generate specific control commands to avoid the risk. To this end, the system focuses on the practically validated obstacle avoidance strategies associated with each of these K historical scenarios, such as "decelerate by 20%", "change lanes to the left", and "emergency braking". These strategies are integrated and analyzed, and the final "target strategy" is determined through two main mechanisms: one is the simple and efficient "frequency priority" principle, which directly selects the obstacle avoidance strategy that appears most frequently in these K scenarios; the other is a more refined "weighted voting" mechanism, which assigns voting weights to each historical scenario based on its similarity to the current scenario, with higher similarity resulting in higher weights. All strategy options are then weighted and statistically analyzed, and the strategy with the highest total number of votes (based on weights) wins.

[0063] Once the target strategy is determined, it is translated into specific, executable vehicle control commands. For example, if the target strategy is "deceleration," a braking command containing the specific deceleration value is output; if it is "lane change," a steering command containing the target lane and steering angle is output; and if it is "emergency braking," the highest priority full braking command is triggered. This process achieves a closed loop from risk detection based on historical similarity to specific, operable avoidance actions, enabling the system not only to warn of risks but also to directly provide effective response solutions validated by historical cases.

[0064] Reference Figure 3 In this embodiment of the application, a driving risk detection device is also provided, which includes: The acquisition unit 310 is used to acquire multimodal data of the vehicle in the current environment; wherein the multimodal data includes at least one of lidar point cloud, camera image, and millimeter-wave radar signal; The fusion unit 320 is used to perform dimensionality reduction and feature fusion on the multimodal data to obtain a multimodal fusion feature vector; The detection unit 330 is used to detect the complexity level of the current road conditions of the vehicle, determine the corresponding range of K values ​​based on the complexity level, and select a target K value from the range of K values; wherein, the K value corresponding to normal road conditions is less than the K value corresponding to complex road conditions; The retrieval unit 340 is used to retrieve the K most similar historical scenarios to the multimodal fusion feature vector from the pre-stored historical risk scenario library; The processing unit 350 is used to determine the current risk detection result of the vehicle based on the target K value and the historical scenario.

[0065] It is understandable that, such as Figure 2 The content of the driving risk detection method embodiments shown is applicable to the driving risk detection device embodiments. The specific functions implemented by the driving risk detection device embodiments are the same as those shown in the examples. Figure 2 The driving risk detection method shown in the embodiment is the same, and the beneficial effects achieved are the same as those described above. Figure 2 The beneficial effects achieved by the driving risk detection method embodiment shown are also the same.

[0066] Reference Figure 4 This application also discloses an electronic device, including: At least one processor 410; At least one memory 420 is used to store at least one program; When at least one program is executed by at least one processor 410, such that at least one processor 410 performs as follows: Figure 2 The illustrated embodiment of the driving risk detection method.

[0067] The electronic device in the embodiments of this application may be a terminal device, a computer device, or a server device.

[0068] Understandable Figure 2 The content of the driving risk detection method embodiments shown is applicable to the embodiments of this electronic device, and the specific functions implemented by the embodiments of this electronic device are the same as those shown. Figure 2 The driving risk detection method shown in the embodiment is the same, and the beneficial effects achieved are the same. Figure 2 The beneficial effects achieved by the driving risk detection method embodiment shown are also the same.

[0069] This application also discloses a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement, for example... Figure 2 The illustrated embodiment of the driving risk detection method.

[0070] Understandable Figure 2 The content of the driving risk detection method embodiments shown is applicable to the embodiments of this computer-readable storage medium. The specific functions implemented by the embodiments of this computer-readable storage medium are the same as those shown in the embodiments. Figure 2 The driving risk detection method shown in the embodiment is the same, and the beneficial effects achieved are the same. Figure 2 The beneficial effects achieved by the driving risk detection method embodiment shown are also the same.

[0071] This application also discloses a computer program product or computer program, which includes computer instructions stored in the aforementioned computer-readable storage medium. Figure 4 The processor of the illustrated electronic device can read the computer instructions from the aforementioned computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform... Figure 2 The illustrated embodiment of the driving risk detection method.

[0072] Understandable Figure 2 The content of the driving risk detection method embodiments shown is applicable to this computer program product or computer program embodiment, and the specific functions implemented by this computer program product or computer program embodiment are the same as those shown. Figure 2 The driving risk detection method shown in the embodiment is the same, and the beneficial effects achieved are the same. Figure 2 The beneficial effects achieved by the driving risk detection method embodiment shown are also the same.

[0073] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0074] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional technology for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0075] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0076] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0077] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable storage medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0078] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0079] In the foregoing description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0080] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0081] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for detecting driving risks, characterized in that, The method includes: Acquire multimodal data of the vehicle in the current environment; wherein the multimodal data includes at least one of lidar point cloud, camera image, and millimeter-wave radar signal; The multimodal data is subjected to dimensionality reduction and feature fusion to obtain a multimodal fused feature vector; The complexity level of the current road conditions of the vehicle is detected, the corresponding range of K values ​​is determined according to the complexity level, and a target K value is selected from the range of K values; wherein, the K value corresponding to normal road conditions is less than the K value corresponding to complex road conditions; Retrieve the K most similar historical scenarios from the pre-stored historical risk scenario library that are most similar to the multimodal fusion feature vector; Based on the target K value and the historical scenarios, determine the current risk detection result of the vehicle.

2. The driving risk detection method according to claim 1, characterized in that, The step of performing dimensionality reduction and feature fusion on the multimodal data to obtain a multimodal fused feature vector includes: Extract environmental features, target features, and trajectory features from the multimodal data; The attention weights between different modal features are calculated using a cross-attention mechanism, and the target features of a specified category are weighted and enhanced; wherein, the specified category includes pedestrians and / or construction signs. Based on the attention weights, the environmental features, the target features, and the trajectory features are weighted to obtain multimodal features; Principal component analysis is performed on the multimodal features to reduce their dimensionality, generating the multimodal fusion feature vector of a specified dimension.

3. The driving risk detection method according to claim 2, characterized in that, The weight enhancement processing of the target features of the specified category includes: Identify the target category corresponding to the target features; If the target category belongs to pedestrians or construction signs, the attention weight of the target feature corresponding to the target category is increased according to the preset hazard category weight coefficient.

4. The driving risk detection method according to claim 1, characterized in that, The complexity level of detecting the current road conditions of the vehicle includes: Based on the multimodal fusion feature vector, the density of target objects in the current road conditions and the real-time driving speed of the vehicle are determined. The target density and the real-time driving speed are input into a preset complexity evaluation model to obtain an initial complexity score; If at least one preset high-risk event feature is identified from the multimodal fusion feature vector, the initial complexity score is weighted and adjusted upwards to obtain the target complexity score; The target complexity score is compared with a preset complexity threshold to determine the complexity level of the current road condition.

5. The driving risk detection method according to claim 1, characterized in that, The historical risk scenario library uses a dynamic sparse voxel structure to store data.

6. The driving risk detection method according to claim 5, characterized in that, The step of retrieving the K most similar historical scenarios from the pre-stored historical risk scenario database to the multimodal fusion feature vector includes: The dynamic sparse voxel structure is mapped to memory, and multiple candidate voxels adjacent to the multimodal fusion feature vector space are determined based on the spatial distribution of the current multimodal fusion feature vector. In the feature vectors of multiple historical scenes contained in the candidate voxel, the Manhattan distance between the multimodal fusion feature vector and the feature vector of each historical scene is calculated in parallel. From the calculated Manhattan distances, select the historical scene with the smallest target K value.

7. The driving risk detection method according to any one of claims 1 to 6, characterized in that, The method further includes: From the obstacle avoidance strategies associated with the target K value and historical scenarios, select the target strategy with the highest frequency of occurrence, or determine the target strategy through a weighted voting mechanism; When a risk is detected in the vehicle, control commands such as deceleration, lane change, or emergency braking are output according to the target strategy.

8. A driving risk detection device, characterized in that, The device includes: An acquisition unit is used to acquire multimodal data of the vehicle in the current environment; wherein the multimodal data includes at least one of lidar point cloud, camera image, and millimeter-wave radar signal; The fusion unit is used to perform dimensionality reduction and feature fusion on the multimodal data to obtain a multimodal fusion feature vector; The detection unit is used to detect the complexity level of the current road conditions of the vehicle, determine the corresponding range of K values ​​based on the complexity level, and select a target K value from the range of K values; wherein, the K value corresponding to normal road conditions is less than the K value corresponding to complex road conditions; The retrieval unit is used to retrieve the K most similar historical scenarios to the multimodal fusion feature vector from the pre-stored historical risk scenario library; The processing unit is used to determine the current risk detection result of the vehicle based on the target K value and the historical scenario.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the driving risk detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the driving risk detection method according to any one of claims 1 to 7.

Images