Emergency braking decision model generation method, emergency braking decision method and emergency braking decision device

By integrating the decision-making component into the model architecture of the automatic emergency braking system and using point cloud and image data to construct feature sequences for interactive modeling, the problems of coupling and poor decision-making effect caused by manual module division in existing technologies are solved, and more efficient emergency braking decision-making is achieved.

CN121133725APending Publication Date: 2025-12-16JINGDONG KUNPENG (JIANGSU) TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202511547129.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-10-28
Publication Date
2025-12-16

AI Technical Summary

Technical Problem

The existing automatic emergency braking system adopts a two-stage separate design, which relies on manual module division and cannot get rid of the limitations of human experience, resulting in poor coupling and decision-making effectiveness.

Method used

The decision-making component is efficiently integrated into the model architecture. By constructing the feature sequences of the target and the main vehicle through point cloud data and image data, interactive modeling processing is performed to generate an emergency braking decision model. Features are directly extracted from the acquired relevant information without the need for manual splitting.

Benefits of technology

It improves coupling and decision-making effectiveness, enables direct interaction and decision-making between master vehicle information and target information, and enhances the safety and real-time performance of the autonomous driving system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121133725A_ABST
    Figure CN121133725A_ABST
Patent Text Reader

Abstract

The invention discloses an emergency braking decision model generation method, an emergency braking decision method and an emergency braking decision device, and relates to the technical field of computers. A specific embodiment of the training method of the information recommendation prediction model comprises the steps of determining motion state information of a target based on point cloud data and image data, and constructing a target feature sequence according to the motion state information; constructing a main vehicle feature sequence based on the key point information of the main vehicle; and performing interactive modeling on the target feature sequence and the main vehicle feature sequence, and generating an emergency braking decision model. According to the embodiment of the invention, the interaction and decision-making of the main vehicle information and the target information can be realized without manual module splitting, and the coupling and decision-making effect are improved. A specific embodiment of the emergency braking decision-making method comprises the steps that to-be-decided point cloud data and to-be-decided image data of a user are input into an emergency braking decision-making model, and the emergency braking decision-making model outputs an emergency braking decision-making result and feeds back the emergency braking decision-making result to the user. According to the embodiment, the feature mining capability is higher, and the decision-making effect is better.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method for generating an emergency braking decision model, an emergency braking decision method, and an apparatus. Background Technology

[0002] Automatic emergency braking (AEB) refers to the real-time monitoring of the vehicle's surrounding environment and the automatic activation of the brakes to slow the vehicle down in the event of a potential collision, thereby avoiding or mitigating the impact. Existing AEB systems primarily employ a two-stage, separate design, consisting of a perception module and a decision module. The perception module receives data from sensors as input and senses the position and motion of the target. It then outputs its results to the downstream decision module, which, based on the perception results and the vehicle's motion information, determines whether the vehicle is in danger and, if necessary, issues an emergency braking command to the chassis. This two-stage, separate design relies on manual module division, which cannot escape the limitations of human experience and the inherent irrationality of module functional division, resulting in poor coupling and decision-making effectiveness. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method for generating an emergency braking decision model, an emergency braking decision method, and an apparatus, which efficiently integrate the decision-making part into the model architecture, directly extract features from the acquired relevant information, and realize the interaction and decision-making between the main vehicle information and the target information without manual decomposition, thereby improving coupling and decision-making effectiveness.

[0004] To achieve the above objectives, according to one aspect of the present invention, a method for generating an emergency braking decision model is provided, comprising:

[0005] Based on the acquired point cloud data and image data, the motion state information of the target is determined, and a target feature sequence is constructed based on the motion state information of the target.

[0006] Construct a main vehicle feature sequence based on the acquired key point information of the main vehicle;

[0007] The target feature sequence and the main vehicle feature sequence are interactively modeled and processed, and an emergency braking decision model is generated based on the modeling results.

[0008] Optionally, the motion state information of the target is determined based on the acquired point cloud data and image data, including:

[0009] A first bird's-eye view feature map is generated based on point cloud data, and a second bird's-eye view feature map is generated based on image data.

[0010] The first bird's-eye view feature map and the second bird's-eye view feature map are stitched together to obtain a multimodal bird's-eye view feature map.

[0011] The multimodal bird's-eye view feature map is concatenated with the pre-acquired historical frame feature map to obtain the target feature map sequence;

[0012] The motion state information of the target is predicted based on the target feature map sequence and neural network.

[0013] Optionally, a first bird's-eye view feature map is generated based on the point cloud data, including:

[0014] Voxelization is performed on the point cloud data to obtain point cloud voxels;

[0015] Point cloud features are extracted from point cloud voxels based on neural networks;

[0016] The point cloud features are flattened along the height dimension to generate a feature map from the first bird's-eye view perspective.

[0017] Optionally, a second bird's-eye view feature map is generated based on the image data, including:

[0018] Image features are extracted from image data based on neural networks, and dense depth maps are predicted based on depth estimation networks and image features.

[0019] Image features are back-projected into three-dimensional space based on dense depth maps to generate a second bird's-eye view feature map.

[0020] Optionally, a target feature sequence is constructed based on the target's motion state information, including:

[0021] The motion state information of the target is decoded to obtain a set of target points;

[0022] The target point set is input into a neural network for feature extraction to obtain the target feature sequence.

[0023] Optionally, interactive modeling processing is performed on the target feature sequence and the main vehicle feature sequence, and an emergency braking decision model is generated based on the modeling results, including:

[0024] The target feature sequence is used as key information and the main vehicle feature sequence is used as query information. These are input into the neural network to perform feature interaction and obtain the target interaction features.

[0025] The target interaction features are flattened and then input into a fully connected network to obtain emergency braking decision information.

[0026] An emergency braking decision model is generated based on emergency braking decision information.

[0027] According to a second aspect of the present invention, an emergency braking decision-making method is provided, comprising:

[0028] Acquire the user's point cloud data and image data to be decided;

[0029] The point cloud data and image data to be decided are input into the emergency braking decision model, and the emergency braking decision model outputs the emergency braking decision result; wherein, the emergency braking decision model is obtained by any of the methods in the first aspect of the present invention;

[0030] The emergency braking decision results are fed back to the user.

[0031] According to a third aspect of the present invention, an apparatus for generating an emergency braking decision model is provided, comprising:

[0032] The determination module is used to determine the motion state information of the target based on the acquired point cloud data and image data, and to construct a target feature sequence based on the motion state information of the target;

[0033] The module is used to construct the main vehicle feature sequence based on the acquired key point information of the main vehicle;

[0034] The modeling module is used to perform interactive modeling processing on the target feature sequence and the main vehicle feature sequence, and generate an emergency braking decision model based on the modeling results.

[0035] According to a fourth aspect of the present invention, an emergency braking decision device is provided, comprising:

[0036] The acquisition module is used to acquire the user's point cloud data and image data to be decided;

[0037] The decision module is used to input the point cloud data and image data to be decided into the emergency braking decision model, and the emergency braking decision model outputs the emergency braking decision result; wherein, the emergency braking decision model is obtained by any of the methods in the first aspect of the present invention;

[0038] The feedback module is used to provide the user with feedback on the emergency braking decision.

[0039] According to a fifth aspect of the present invention, an electronic device is provided, comprising:

[0040] One or more processors;

[0041] Storage device for storing one or more programs.

[0042] When one or more programs are executed by one or more processors, the one or more processors implement the methods of any of the above embodiments.

[0043] According to a sixth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method of any of the above embodiments.

[0044] According to a seventh aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method of any of the above embodiments.

[0045] An embodiment of the above invention has the following advantages or beneficial effects: It determines the target's motion state information based on acquired point cloud data and image data, and constructs a target feature sequence based on the target's motion state information; it constructs a main vehicle feature sequence based on acquired key point information of the main vehicle; it performs interactive modeling processing on the target feature sequence and the main vehicle feature sequence, and generates an emergency braking decision model based on the modeling results; this embodiment efficiently integrates the decision-making part into the model architecture, directly mining features from the acquired relevant information, without manual splitting, thus realizing the interaction and decision-making between main vehicle information and target information, improving coupling and decision-making effectiveness. It acquires the user's point cloud data and image data to be decided; inputs the point cloud data and image data to be decided into the emergency braking decision model, and the emergency braking decision model outputs the emergency braking decision result; wherein, the emergency braking decision model is obtained through any method in the first aspect of the present invention; the emergency braking decision result is fed back to the user; it can directly output the final emergency braking decision result based on the user's point cloud data and image data, with stronger feature mining capabilities and better decision-making effects.

[0046] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0047] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0048] Figure 1 This is a schematic diagram of the main flow of the method for generating an emergency braking decision model according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the main flow of a method for generating an emergency braking decision model according to an optional embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the main flow of the emergency braking decision method according to an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the main modules of the emergency braking decision model generation device according to an embodiment of the present invention;

[0052] Figure 5 This is a schematic diagram of the main modules of the emergency braking decision device according to an embodiment of the present invention;

[0053] Figure 6 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0054] Figure 7 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0055] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0056] It should be noted that the acquisition, storage, and application of personal information involved in the embodiments of the present invention comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0057] Automatic emergency braking (AEB) refers to the real-time monitoring of the vehicle's surrounding environment and the automatic activation of the brakes to slow the vehicle down in the event of a potential collision, thereby avoiding or mitigating the impact. Existing AEB systems primarily employ a two-stage, separate design, consisting of a perception module and a decision module. The perception module receives data from sensors as input and senses the position and motion of the target. It then outputs its results to the downstream decision module, which, based on the perception results and the vehicle's motion information, determines whether the vehicle is in danger and, if necessary, issues an emergency braking command to the chassis. This two-stage, separate design relies on manual module division, which cannot escape the limitations of human experience and the inherent irrationality of module functional division, resulting in poor coupling and decision-making effectiveness.

[0058] In view of this, according to one aspect of the present invention, a method for generating an emergency braking decision model is provided.

[0059] Figure 1 This is a schematic diagram of the main flow of the emergency braking decision model generation method according to an embodiment of the present invention. Figure 1 As shown, the method for generating an emergency braking decision model according to an embodiment of the present invention mainly includes the following steps S101 to S103.

[0060] Step S101: Determine the motion state information of the target based on the acquired point cloud data and image data, and construct the target feature sequence based on the motion state information of the target.

[0061] Point cloud data is a collection of numerous discrete points in three-dimensional space collected by sensors such as LiDAR. It provides spatial structural information about objects in the environment, including features such as position, shape, and height. Image data is two-dimensional visual information collected by vehicle-mounted cameras, including texture, color, and semantic details, which can be used to identify objects and their categories in the environment. When determining the motion state information of a target based on the above data, the point cloud data can be voxelized to reduce data density, and then input into a neural network for feature extraction. Compressed projection is performed in the height dimension to generate a point cloud feature map from a bird's-eye view. Image data is input into a neural network to extract two-dimensional image features. Simultaneously, a depth estimation network is used to predict the depth of the scene, and then the two-dimensional image features are back-projected into three-dimensional space to construct a visual modality feature map from a bird's-eye view. After the feature maps from these two bird's-eye view perspectives are stitched and fused in the channel dimension, a spatial feature representation in the fused modality is obtained. This fused representation is then input into a neural network to estimate the motion state information of multiple targets, including the position, velocity, acceleration, and category information of each target.

[0062] The aforementioned targets refer to traffic participants in the environment who have potential interactions with the main vehicle, such as pedestrians, bicycles, motorcycles, and other motor vehicles. After determining the motion state information of the targets, a target feature sequence can be further constructed based on this motion state information. Specifically, the motion state information of each target can be decoded to form a set of target points containing features such as position, velocity, acceleration, and category. This set of target points is input into a neural network, and a feature vector of uniform dimension is generated through feature transformation and embedding operations, thereby constructing the target feature sequence. Through the above processing, the spatiotemporal dynamic behavior of each target in the scene can be fully expressed, improving the model's environmental understanding and decision-making accuracy in emergency braking situations.

[0063] Step S102: Construct the main vehicle feature sequence based on the acquired key point information of the main vehicle.

[0064] The key point information of the main vehicle refers to representative location points selected from the current geometric frame of the main vehicle and their corresponding motion attributes. Specifically, these key points can be the four corner points and the center point of the main vehicle's rectangle, totaling five points, or other points, etc. Each key point has different position coordinate information, reflecting the specific distribution pattern of the main vehicle in space. However, the associated velocity, acceleration, and category information remain consistent across the five key points, reflecting the overall motion state of the main vehicle. When constructing the main vehicle feature sequence, the position coordinate information of the five key points of the main vehicle can be combined with the shared velocity, acceleration, and category information to form five structured original feature sets. These original features are input into a neural network, and the expressive power of the neural network is used to encode and embed the features of each key point, thereby obtaining a feature representation sequence with a unified dimension. The final generated main vehicle feature sequence can be used as query information, participating together with the target feature sequence in the attention modeling and inference calculation of emergency braking decision-making, improving the model's accurate understanding and response capability to the relationship between the main vehicle's state and the surrounding environment.

[0065] Step S103: Perform interactive modeling processing on the target feature sequence and the main vehicle feature sequence, and generate an emergency braking decision model based on the modeling results.

[0066] When performing interactive modeling of the target feature sequence and the vehicle feature sequence, the vehicle feature sequence can be used as the query information input, and the target feature sequence as the key-value information input. A multi-head attention mechanism is used to calculate the dynamic attention level of the vehicle's key points to each target, thereby uncovering the potential collision risks faced by the vehicle in the current state. In specific implementations, a standard multi-head attention module can be used to map the vehicle and target features, obtaining the interactive representation from the vehicle's perspective. Alternatively, a fused, layered decoder structure can be used, with a feedforward neural network connected after the attention layer to further enhance the nonlinear modeling capability between features, thereby improving the model's ability to capture potential risk relationships in complex traffic scenarios. Through the above interactive modeling process, the modeling result is obtained, which is the interactive feature representation comprehensively perceived and inferred by the vehicle in the current scenario. This interactive feature not only includes the dynamic connections between the vehicle and various surrounding targets but also contains predictive clues about potential future hazards. This interactive feature can be flattened into a one-dimensional vector representation for subsequent network processing. Based on this modeling result, an emergency braking decision model can be generated. The flattened interaction features can be input into a fully connected neural network, and through a series of nonlinear transformation layers, the decision result of whether emergency braking should be performed can be output. Alternatively, the interaction features and the current global state information of the vehicle can be combined and input into a discriminative network for joint modeling, thereby improving decision accuracy and response robustness. The final emergency braking decision model can directly infer whether emergency braking needs to be triggered from multimodal inputs end-to-end, effectively avoiding the limitations of manually set rules and improving the safety and real-time performance of the autonomous driving system in complex scenarios.

[0067] This embodiment determines the target's motion state information based on acquired point cloud data and image data, and constructs a target feature sequence based on the target's motion state information; it also constructs a main vehicle feature sequence based on acquired key point information of the main vehicle; it performs interactive modeling processing on the target feature sequence and the main vehicle feature sequence, and generates an emergency braking decision model based on the modeling results; this embodiment efficiently integrates the decision-making part into the model architecture, directly mining features from the acquired relevant information, without the need for manual splitting, and can realize the interaction and decision-making between main vehicle information and target information, improving coupling and decision-making effect.

[0068] Figure 2 This is a schematic diagram of a method for generating an emergency braking decision model according to an optional embodiment of the present invention. Figure 2 As shown, the motion state information of the target is determined based on the acquired point cloud data and image data, including the following steps S201 to S204.

[0069] Step S201: Generate a first bird's-eye view feature map based on point cloud data, and generate a second bird's-eye view feature map based on image data.

[0070] Step S202: The first bird's-eye view feature map and the second bird's-eye view feature map are stitched together to obtain a multimodal bird's-eye view feature map.

[0071] Step S203: The multimodal bird's-eye view feature map is stitched together with the pre-acquired historical frame feature map to obtain the target feature map sequence.

[0072] Step S204: Predict the motion state information of the target based on the target feature map sequence and neural network.

[0073] In this embodiment, a first bird's-eye view feature map is first generated based on the acquired point cloud data. By projecting the three-dimensional point cloud onto a two-dimensional plane, the spatial location information and geometric structural features of the target can be effectively preserved. A second bird's-eye view feature map is then generated based on the acquired image data, leveraging the image's expressive power in details such as texture, edge, and color to enhance the overall environmental perception accuracy. The first and second bird's-eye view feature maps are then stitched and fused along the channel dimension to construct a multimodal bird's-eye view feature map, thereby integrating complementary information on structure and appearance. To consider the continuity and dynamic changes of the target in the time dimension, the current multimodal bird's-eye view feature map is further stitched with pre-acquired historical frame feature maps to construct a target feature map sequence. This target feature map sequence can be used as a time series input to a neural network for end-to-end temporal modeling and state estimation, ultimately predicting the target's motion state information at the current moment, including key parameters such as position, velocity, and acceleration. In addition to the aforementioned method of fusing bird's-eye view images, spatial features can also be extracted directly using point cloud-based voxel grid encoding and image convolutional encoding, respectively. These features are then aligned and fused using a cross-modal attention mechanism or graph neural network, thereby achieving high-precision prediction of the target's motion state. This embodiment not only achieves effective complementarity between point cloud and image information but also introduces temporal contextual information, thereby improving the accuracy and stability of target motion state estimation and providing more comprehensive and accurate perceptual input for subsequent emergency braking decisions.

[0074] Optionally, generating a first bird's-eye view feature map based on point cloud data includes: performing voxelization on the point cloud data to obtain point cloud voxels; extracting point cloud features from the point cloud voxels based on a neural network; and flattening the point cloud features along the height dimension to generate a first bird's-eye view feature map.

[0075] In this embodiment, the acquired raw point cloud data is first voxelized, that is, the irregularly distributed 3D point cloud is divided into regular voxel grid units, giving the point cloud data stronger spatial structure and processability. Based on the point cloud voxels, point cloud features containing spatial geometry and semantic information are extracted by a deep neural network. The neural network can encode the internal point distribution of each point cloud voxel and its spatial relationship with surrounding point cloud voxels. The extracted point cloud features are flattened in the height dimension, that is, the 3D voxel features are compressed into a 2D plane, thereby generating a first bird's-eye view feature map. This feature map shows the structural distribution of the point cloud in horizontal space from a top-down perspective and retains key information for subsequent multimodal fusion and temporal modeling. In addition to using voxelization and neural network extraction, features can also be extracted by directly performing sparse spatial convolution operations on the point cloud data using a sparse convolutional network based on the raw point cloud, without the need to construct a regular voxel grid. The above processing method can effectively enhance the expressive power of point cloud data from a bird's-eye view perspective, enabling the model to retain more environmental information during spatial feature extraction, improving the accuracy of judging target contours, positions and densities, and providing a more stable and structured input foundation for subsequent motion state estimation and model training.

[0076] Optionally, generating a second bird's-eye view feature map based on image data includes: extracting image features from image data based on a neural network, and predicting a dense depth map based on a depth estimation network and image features; and back-projecting the image features onto a three-dimensional space based on the dense depth map to generate the second bird's-eye view feature map.

[0077] In this embodiment, a neural network is used to extract features from the acquired image data, extracting image features containing information such as texture, edges, and semantics. A depth estimation network is used to predict the image features, generating a dense depth map. This dense depth map provides corresponding spatial depth information for each pixel in the image. Based on this dense depth map, the image features are back-projected from the two-dimensional image plane to three-dimensional space, giving the image features spatial distribution attributes. This is further projected onto a bird's-eye view, generating a second bird's-eye view feature map. This feature map integrates the image's advantages in texture and semantics, effectively converting it into an expression suitable for spatial modeling. Alternatively, a multi-view image joint reconstruction method can be used to generate a higher-precision three-dimensional structure by combining information from multiple frames, and then the second bird's-eye view feature map can be extracted. This embodiment realizes the transformation and expression of image features in a spatial perspective, not only enhancing the model's understanding of scene details and semantic elements but also providing a structurally consistent input format for subsequent multimodal fusion with point cloud features, thereby improving overall perception accuracy and model generation performance.

[0078] Optionally, constructing a target feature sequence based on the target's motion state information includes: decoding the target's motion state information to obtain a set of target points; and inputting the set of target points into a neural network for feature extraction to obtain a target feature sequence.

[0079] The motion state information of targets predicted from multimodal data is decoded to extract the spatial and motion attributes of each target, thus obtaining a set of target points. This set of target points contains key state information of multiple traffic participants at a specific time point, such as position, speed, acceleration, and category attributes. The set of target points is input into a neural network model, and the state of each target is abstracted and expressed in high dimension through a multilayer perceptual structure, extracting deep features with spatial, temporal, and category semantics to form a target feature sequence in a unified format. This target feature sequence can serve as an important input to characterize target behavior patterns in the subsequent interaction modeling stage. In addition to the above method, graph neural networks can also be used to model the relative motion relationships between targets, thereby constructing a target feature sequence that integrates interaction information. This embodiment can effectively capture the spatiotemporal behavioral characteristics of various traffic participants in different scenarios and uniformly encode them into a high-dimensional target feature sequence, thereby providing a structured and highly discriminative input for the emergency braking decision model, improving the model's responsiveness and decision accuracy in complex traffic scenarios.

[0080] Optionally, interactive modeling processing is performed on the target feature sequence and the main vehicle feature sequence, and an emergency braking decision model is generated based on the modeling results. This includes: inputting the target feature sequence as key-value information and the main vehicle feature sequence as query information into a neural network for feature interaction to obtain target interactive features; flattening the target interactive features and inputting the flattened target interactive features into a fully connected network to obtain emergency braking decision information; and generating an emergency braking decision model based on the emergency braking decision information.

[0081] In this embodiment, the target feature sequence is used as key-value information, and the main vehicle feature sequence is used as query information. Both are input into a neural network for feature interaction processing. This neural network can adopt an attention-based structure, such as a Transformer or a self-attention module, to achieve weighted modeling of key targets and their behavioral features, thereby obtaining target interaction features containing the semantics of interaction between the main vehicle and surrounding targets. The target interaction features are then flattened in the spatial dimension to fit into the subsequent fully connected network structure. The fully connected network further integrates and transforms multi-dimensional features, outputting decision information reflecting whether emergency braking is necessary in the current traffic scenario. Supervised training is then performed based on this emergency braking decision information to generate an emergency braking decision model with discriminative capabilities. Besides the above interaction method, a graph neural network can also be used to construct a graph structure relationship between the main vehicle and surrounding targets. More refined relationship modeling can be achieved through edge weights and node information. Alternatively, a structure incorporating temporal dynamics, such as a temporal attention network, can be introduced to further enhance the model's ability to perceive the evolution trend of interactions. This embodiment can efficiently model the complex interaction between the main vehicle and surrounding targets, so that the trained emergency braking decision model not only has a strong scene understanding ability, but can also make accurate and timely responses before potential collision risks occur, thereby improving the safety and reliability of the overall autonomous driving system.

[0082] According to a second aspect of the present invention, an emergency braking decision method is provided.

[0083] Figure 3 This is a schematic diagram of the main flow of the emergency braking decision-making method according to an embodiment of the present invention. Figure 3 As shown, the emergency braking decision method according to an embodiment of the present invention mainly includes the following steps S301 to S303.

[0084] Step S301: Obtain the user's point cloud data and image data to be decided.

[0085] Step S302: Input the point cloud data to be decided and the image data to be decided into the emergency braking decision model, and the emergency braking decision model outputs the emergency braking decision result; wherein, the emergency braking decision model is obtained by any of the methods in the first aspect of the present invention.

[0086] Step S303: Feedback the emergency braking decision result to the user.

[0087] First, the point cloud data and image data to be decided in the user's current scene are acquired. These data comprehensively reflect the three-dimensional spatial and visual semantic information of the current traffic environment. The point cloud data can be collected by LiDAR equipment, providing high-precision three-dimensional structural information. Image data can come from the vehicle's forward-facing camera, used to supplement information such as texture, color, and target recognition. The point cloud data and image data are used as input to the model, synchronously fed into a trained emergency braking decision model. This model can output an emergency braking decision result based on the user's current driving state. This model is obtained by fusing target motion state, vehicle features, and multimodal feature interaction modeling, possessing good scene understanding and risk prediction capabilities. The emergency braking decision result is fed back to the user or the autonomous driving system in real time to assist driving decisions or trigger active safety control. In addition to the above methods, prior rule screening or simple target filtering mechanisms can also be introduced into this process to predict whether there are high-risk targets before inputting into the model, thereby improving inference efficiency. At the same time, model inference can be combined with temporal information to dynamically assess the risk evolution trend, thereby improving the timeliness and accuracy of emergency braking response. This embodiment enables rapid identification and response to potential collision risks in complex traffic environments, improving the decision-making efficiency and proactive safety protection capabilities of intelligent driving systems at critical moments.

[0088] According to a preferred embodiment of the present invention, a multimodal, end-to-end automatic emergency braking decision model is proposed. This model has a simple and compact structure, capable of directly receiving point cloud data and image data input from sensors. It automatically mines features through neural networks, outputting the final emergency braking decision result without any manual rule splitting or intermediate processing steps. Through ingenious design of the model architecture, the interactive modeling of vehicle information and target information is deeply integrated with the braking decision process, avoiding the performance bottleneck caused by the independent feature and decision modules in traditional separate algorithms, and significantly improving the model's feature representation capability and decision accuracy.

[0089] Specifically, this model is a cross-modal algorithm. Its first part is used to perceive the surrounding environment and infer the motion state information of the target. Point cloud data is first processed through voxelization, then point cloud features are extracted through a neural network, and flattened along the height dimension (e.g., the z-axis) to obtain a point cloud feature map under BEV (Bird's Eye View). Simultaneously, image data is processed through a neural network to extract two-dimensional image features, and a depth estimation network is used in one branch to predict a dense depth map of the current scene. A mature model (such as DepthNet) can be used for the depth estimation network. The dense depth map allows the two-dimensional image features to be back-projected into three-dimensional space, thereby constructing a bird's-eye view feature map of the visual modality. This back-projection process can employ algorithms such as LSS (Lift, Splat, Shoot, object detection algorithms).

[0090] After acquiring the point cloud modality and the visual modality's bird's-eye view feature maps, the two are concatenated along the channel dimension to form a multimodal bird's-eye view feature map, achieving multimodal information fusion. Since single-frame information is often insufficient to fully depict the target's motion trend, the model maintains a historical frame feature cache to store the bird's-eye view feature maps from the previous few frames. The historical frame features are concatenated with the current frame features and input into the neural network to further extract target motion state information, including velocity, acceleration, and category information, providing support for subsequent decision-making modules.

[0091] In the second part of the model, namely the emergency braking decision part, the motion state information of the aforementioned targets is first decoded to obtain a set of target points for traffic participants. Each element in this set has seven dimensions of attribute features, including position (x, y), velocity (v), and velocity (v). x v y ), acceleration (a x a y The system collects target points and their category information (type), which can fully characterize their motion state and category features. This set of target points is input into a neural network to extract a target feature sequence of dimension [N, C], which is used as the key-value vector input. Here, N is the number of target points for traffic participants, and C is the dimension of the features. If each target point is described by a 256-dimensional feature vector, then the dimension of the feature sequence is [N, 256]. The value of C can be preset, such as 256 or 512. To enhance spatial perception, five key points are selected from the driver's frame: the four corner points and one center point of the rectangle. These key points have their own positions (x, y) but share unified velocity, acceleration, and category information. These are also encoded into a set of feature sequences, used as the query vector input. An attention mechanism is used to model the interaction between the driver and the target, and to mine features, thereby determining whether the driver is in a potential collision risk. The interaction features output by the neural network decoder are flattened and input into a fully connected network layer to generate the final emergency braking decision. The entire model achieves end-to-end inference from multimodal inputs to decision results. All network layers are differentiable and can be trained efficiently through backpropagation. Compared to traditional rule-based or stage-based emergency braking algorithms, this model has stronger feature extraction and fitting capabilities, enabling it to make more accurate and robust decisions in complex environments. It is suitable for high-performance active safety control modules in autonomous driving systems.

[0092] According to a third aspect of the present invention, an apparatus for generating an emergency braking decision model is provided.

[0093] Figure 4This is a schematic diagram of the main modules of the emergency braking decision model generation device according to an embodiment of the present invention, as shown below. Figure 4 As shown, an emergency braking decision model generation device 400 includes:

[0094] The determination module 401 is used to determine the motion state information of the target based on the acquired point cloud data and image data, and to construct a target feature sequence based on the motion state information of the target;

[0095] Module 402 is used to construct a main vehicle feature sequence based on the acquired key point information of the main vehicle;

[0096] Modeling module 403 is used to perform interactive modeling processing on the target feature sequence and the main vehicle feature sequence, and generate an emergency braking decision model based on the modeling results.

[0097] Optionally, the determining module 401 is also used for:

[0098] A first bird's-eye view feature map is generated based on point cloud data, and a second bird's-eye view feature map is generated based on image data.

[0099] The first bird's-eye view feature map and the second bird's-eye view feature map are stitched together to obtain a multimodal bird's-eye view feature map.

[0100] The multimodal bird's-eye view feature map is concatenated with the pre-acquired historical frame feature map to obtain the target feature map sequence;

[0101] The motion state information of the target is predicted based on the target feature map sequence and neural network.

[0102] Optionally, the determining module 401 is also used for:

[0103] Voxelization is performed on the point cloud data to obtain point cloud voxels;

[0104] Point cloud features are extracted from point cloud voxels based on neural networks;

[0105] The point cloud features are flattened along the height dimension to generate a feature map from the first bird's-eye view perspective.

[0106] Optionally, the determining module 401 is also used for:

[0107] Image features are extracted from image data based on neural networks, and dense depth maps are predicted based on depth estimation networks and image features.

[0108] Image features are back-projected into three-dimensional space based on dense depth maps to generate a second bird's-eye view feature map.

[0109] Optionally, the determining module 401 is also used for:

[0110] The motion state information of the target is decoded to obtain a set of target points;

[0111] The target point set is input into a neural network for feature extraction to obtain the target feature sequence.

[0112] Optionally, the modeling module 403 is also used for:

[0113] The target feature sequence is used as key information and the main vehicle feature sequence is used as query information. These are input into the neural network to perform feature interaction and obtain the target interaction features.

[0114] The target interaction features are flattened and then input into a fully connected network to obtain emergency braking decision information.

[0115] An emergency braking decision model is generated based on emergency braking decision information.

[0116] It should be noted that the specific implementation details of the emergency braking decision model generation device in this embodiment of the invention have been described in detail in the above emergency braking decision model generation method, so the details will not be repeated here.

[0117] According to a fourth aspect of the present invention, an emergency braking decision device is provided.

[0118] Figure 5 This is a schematic diagram of the main modules of the emergency braking decision device according to an embodiment of the present invention, as shown below. Figure 5 As shown, an emergency braking decision device 500 includes:

[0119] The acquisition module 501 is used to acquire the user's point cloud data and image data to be decided;

[0120] Decision module 502 is used to input the point cloud data to be decided and the image data to be decided into the emergency braking decision model, and the emergency braking decision model outputs the emergency braking decision result; wherein, the emergency braking decision model is obtained by any of the methods in the first aspect of the present invention;

[0121] Feedback module 503 is used to provide feedback on the emergency braking decision to the user.

[0122] It should be noted that the specific implementation details of the emergency braking decision device in the embodiments of the present invention have been described in detail in the above emergency braking decision method, so the details will not be repeated here.

[0123] According to a fifth aspect of the present invention, an electronic device is provided, comprising:

[0124] One or more processors;

[0125] Storage device for storing one or more programs.

[0126] When one or more programs are executed by one or more processors, the one or more processors implement the methods provided by the first aspect and / or the second aspect of the embodiments of the present invention.

[0127] According to a sixth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the methods provided in the first aspect and / or the second aspect of the present invention.

[0128] According to a seventh aspect of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods provided by the first aspect and / or the second aspect of the present invention.

[0129] Figure 6 An exemplary system architecture 600 is shown, in which the method or apparatus for generating an emergency braking decision model according to embodiments of the present invention can be applied.

[0130] Figure 6 An exemplary system architecture 600 is shown that can be applied to the emergency braking decision method or emergency braking decision device of the present invention.

[0131] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, and 603, a network 604, and a server 605. Network 604 serves as the medium for providing communication links between terminal devices 601, 602, and 603 and server 605. Network 604 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0132] Users can use terminal devices 601, 602, and 603 to interact with server 605 via network 604 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 601, 602, and 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0133] Terminal devices 601, 602, and 603 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0134] Server 605 can be a server that provides various services, such as a backend management server that supports shopping websites browsed by users using terminal devices 601, 602, and 603 (for example only). The backend management server can analyze and process data such as received model generation requests, and feed back the processing results (such as an emergency braking decision model - for example only) to the terminal devices.

[0135] It should be noted that the emergency braking decision model generation method provided in this embodiment of the invention is generally run by server 605, and correspondingly, the emergency braking decision model generation device is generally set in server 605.

[0136] It should be noted that the emergency braking decision-making method provided in this embodiment of the invention is generally run by server 605, and correspondingly, the emergency braking decision-making device is generally set in server 605.

[0137] It should be understood that Figure 6 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0138] The following is for reference. Figure 7 It shows a schematic diagram of the structure of a computer system 700 suitable for implementing a terminal device of the present invention. Figure 7 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0139] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 702 or programs loaded from storage section 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the system 700. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0140] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0141] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is run by central processing unit (CPU) 701, it performs the functions defined above in the system of this invention.

[0142] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more operable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually operate substantially in parallel, and they may sometimes operate in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0144] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor may include a determination module, a construction module, and a modeling module. The names of these modules do not necessarily limit the module itself; for example, the construction module may be described as "a module for constructing a main vehicle feature sequence based on acquired key point information of the main vehicle." Alternatively, a processor may include an acquisition module, a decision module, and a feedback module, where the names of these modules do not necessarily limit the module itself; for example, the feedback module may be described as "a module for feeding back the emergency braking decision result to the user."

[0145] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: determining the motion state information of a target based on acquired point cloud data and image data; constructing a target feature sequence based on the target's motion state information; constructing a main vehicle feature sequence based on acquired key point information of the main vehicle; performing interactive modeling processing on the target feature sequence and the main vehicle feature sequence; and generating an emergency braking decision model based on the modeling results. Alternatively, the device implements the following method: acquiring the user's point cloud data and image data to be decided; inputting the point cloud data and image data to be decided into the emergency braking decision model, and the emergency braking decision model outputting an emergency braking decision result; wherein the emergency braking decision model is obtained by any of the methods in the first aspect of the present invention; and feeding back the emergency braking decision result to the user.

[0146] The computer program product provided in the embodiments of the present invention includes a computer program that, when executed by a processor, implements the emergency braking decision model generation method in the first aspect embodiment of the present invention and / or the emergency braking decision method in the second aspect embodiment of the present invention.

[0147] According to the technical solution of the present invention, the following advantages or beneficial effects are achieved: The motion state information of the target is determined based on the acquired point cloud data and image data; a target feature sequence is constructed based on the target's motion state information; a main vehicle feature sequence is constructed based on the acquired key point information of the main vehicle; interactive modeling processing is performed on the target feature sequence and the main vehicle feature sequence; an emergency braking decision model is generated based on the modeling results. This embodiment efficiently integrates the decision-making part into the model architecture, directly mining features from the acquired relevant information, without manual splitting, thus realizing the interaction and decision-making between main vehicle information and target information, improving coupling and decision-making effectiveness. The user's point cloud data and image data to be decided are acquired; the point cloud data and image data to be decided are input into the emergency braking decision model, and the emergency braking decision model outputs the emergency braking decision result; wherein, the emergency braking decision model is obtained by any method in the first aspect of the present invention; the emergency braking decision result is fed back to the user; the final emergency braking decision result can be directly output based on the user's point cloud data and image data, with stronger feature mining capabilities and better decision-making effects.

[0148] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0149] It should be noted that the acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

Claims

1. A method for generating an emergency braking decision model, characterized in that, include: Based on the acquired point cloud data and image data, the motion state information of the target is determined, and a target feature sequence is constructed based on the motion state information of the target. Construct a main vehicle feature sequence based on the acquired key point information of the main vehicle; The target feature sequence and the main vehicle feature sequence are interactively modeled, and an emergency braking decision model is generated based on the modeling results.

2. The method according to claim 1, characterized in that, Based on the acquired point cloud data and image data, the motion state information of the target is determined, including: A first bird's-eye view feature map is generated based on the point cloud data, and a second bird's-eye view feature map is generated based on the image data. The first bird's-eye view feature map and the second bird's-eye view feature map are stitched together to obtain a multimodal bird's-eye view feature map. The multimodal bird's-eye view feature map is concatenated with the pre-acquired historical frame feature map to obtain the target feature map sequence; The motion state information of the target is predicted based on the target feature map sequence and the neural network.

3. The method according to claim 2, characterized in that, Based on the point cloud data, a first bird's-eye view feature map is generated, including: The point cloud data is voxelized to obtain point cloud voxels; Point cloud features are extracted from the point cloud voxels based on a neural network; The point cloud features are flattened along the height dimension to generate a first bird's-eye view feature map.

4. The method according to claim 2, characterized in that, Generating a second bird's-eye view feature map based on the image data includes: Image features are extracted from the image data based on a neural network, and a dense depth map is predicted based on the depth estimation network and the image features. Based on the dense depth map, the image features are back-projected into three-dimensional space to generate a second bird's-eye view feature map.

5. The method according to claim 1, characterized in that, Construct a target feature sequence based on the target's motion state information, including: The motion state information of the target is decoded to obtain a set of target points; The target point set is input into a neural network for feature extraction to obtain a target feature sequence.

6. The method according to claim 1, characterized in that, The target feature sequence and the main vehicle feature sequence are interactively modeled, and an emergency braking decision model is generated based on the modeling results, including: The target feature sequence is used as key information and the main vehicle feature sequence is used as query information. These are then input into a neural network to perform feature interaction and obtain the target interaction features. The target interaction features are flattened, and the flattened target interaction features are input into a fully connected network to obtain emergency braking decision information; An emergency braking decision model is generated based on the emergency braking decision information.

7. An emergency braking decision-making method, characterized in that, include: Acquire the user's point cloud data and image data to be decided; The point cloud data and image data to be decided are input into the emergency braking decision model, and the emergency braking decision model outputs the emergency braking decision result; wherein, the emergency braking decision model is obtained by the method described in any one of claims 1 to 6; The emergency braking decision result is fed back to the user.

8. A device for generating an emergency braking decision model, characterized in that, include: The determination module is used to determine the motion state information of the target based on the acquired point cloud data and image data, and to construct a target feature sequence based on the motion state information of the target; The module is used to construct the main vehicle feature sequence based on the acquired key point information of the main vehicle; The modeling module is used to perform interactive modeling processing on the target feature sequence and the main vehicle feature sequence, and generate an emergency braking decision model based on the modeling results.

9. An emergency braking decision-making device, characterized in that, include: The acquisition module is used to acquire the user's point cloud data and image data to be decided; A decision module is used to input the point cloud data and image data to be decided into an emergency braking decision model, wherein the emergency braking decision model outputs an emergency braking decision result; wherein the emergency braking decision model is obtained by the method described in any one of claims 1 to 6; The feedback module is used to feed back the emergency braking decision result to the user.

10. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.