Causal saliency mining and anti-interference semantic representation method in autonomous driving scene
By using a self-attention mechanism to filter key features and suppress noise, the problems of causal confusion and noise interference in autonomous driving scenarios are solved, improving the model's generalization ability in unknown scenarios and the safety and smoothness of trajectory planning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TONGJI UNIV
- Filing Date
- 2026-05-12
- Publication Date
- 2026-06-09
AI Technical Summary
Existing autonomous driving scenario representation and planning methods suffer from problems such as causal confusion, difficulty in suppressing noise interference, and insufficient applicability of contrastive learning in complex environments, resulting in insufficient generalization ability and safety of models in unknown scenarios.
We employ an attention-guided automated sampling mechanism and recoding strategy. We use a self-attention mechanism to rank global importance, filter out key features of causal relationships and suppress spurious noise, construct category-aware automated positive and negative samples, and use a recoding mechanism to perform context alignment. Finally, we construct a contrastive loss function to distinguish key causal information from interfering noise.
It significantly improves the model's generalization ability in unknown scenarios, enhances the safety and smoothness of trajectory planning, maintains the structural integrity of the driving scenario, and reduces the impact of causal confusion and noise interference.
Smart Images
Figure CN122173877A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of autonomous driving scene representation and artificial intelligence technology, and in particular to a method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios. Background Technology
[0002] With the rapid development of artificial intelligence technology, deep learning-based autonomous driving trajectory planning technology has become a core research direction in intelligent transportation systems. Among these, imitation learning has attracted significant attention due to its ability to learn complex strategies directly from human expert driving data without requiring cumbersome rule design. However, existing autonomous driving scene representation and planning methods still face the following significant technical bottlenecks in practical applications: First, there is a phenomenon of "causal confusion." Traditional deep learning models primarily rely on statistical correlations in data for fitting, rather than understanding the true causal logic behind driving behavior. In complex road environments, models are prone to learning spurious correlations. For example, a model might incorrectly associate the shape of a specific roadside green belt with a left-turn instruction, rather than making a decision based on lane topology. When a vehicle encounters a scenario with a similar green belt but requiring a straight-ahead maneuver, this erroneous perception based on spurious correlations can lead to serious decision-making errors, significantly reducing the model's generalization ability and safety in unknown scenarios.
[0003] Second, noise interference in high-dimensional scenarios is difficult to suppress effectively. Real driving scenarios contain massive amounts of heterogeneous information, including both "key causal factors" that are crucial for driving decisions (such as vehicles braking suddenly ahead or traffic lights in the current lane) and a large amount of irrelevant "background noise" (such as pedestrians in the distance, vehicles in the opposite lane, or stationary billboards on the roadside). Existing feature encoding methods (such as standard CNNs or Transformers) often lack clear mechanisms to distinguish between these two types of information, easily wasting computational resources on irrelevant features, or even allowing noisy features to dominate planning and generation, resulting in unsmooth trajectories or violations of traffic rules.
[0004] Third, existing contrastive learning methods lack applicability in driving scenarios. While contrastive learning is widely used in the image domain to enhance feature robustness, its direct application in autonomous driving scenarios presents challenges. Traditional contrastive learning typically constructs positive and negative samples through data augmentation techniques such as random cropping and rotation. This approach easily violates the strict spatial semantic constraints in driving scenarios (e.g., rotating a map can lead to incorrect navigation directions). Currently, there is a lack of a method that can automatically and intelligently select "positive samples" (causal features) and "negative samples" (noise features) based on scene semantics, and optimize scene representation accordingly.
[0005] In summary, how to automatically extract key features with causal significance from complex and ever-changing driving environments, effectively shield against the interference of spurious correlation noise, and establish robust scene semantic representations are key technical problems that urgently need to be solved in the field of autonomous driving trajectory planning. Summary of the Invention
[0006] This application provides a method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios. Through attention-guided automated sampling mechanism and recoding strategy, it selects key features with causal relationships from high-dimensional heterogeneous driving scenarios and suppresses spurious correlation noise in the environment.
[0007] To address the aforementioned technical issues, this application provides a method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios, comprising the following steps: First, multimodal scene elements are encoded separately, and the encoding results are concatenated to obtain a unified scene representation; then, the unified scene representation is input into a contrastive representation module, and global importance is ranked based on the internal weights of the self-attention mechanism; next, scene elements are divided into different categories, and the categories are filtered to construct category-aware automated positive and negative samples; based on a recoding mechanism, the category-aware automated positive and negative samples are context-aligned; finally, a contrastive loss function is constructed to enable the model to distinguish key causal information from interference noise in the feature space.
[0008] In some exemplary embodiments, the multimodal scene elements include: intelligent agents, vector maps, and static objects; wherein, intelligent agents represent the historical states of moving objects in the scene; vector maps represent road topology, lane lines, intersection structures, and navigation-related map information; and static objects represent roadblocks, cones, and fixed facilities.
[0009] In some exemplary embodiments, multimodal scene elements are encoded separately, and the encoding results are concatenated to obtain a unified scene representation, including: encoding the agent, vector map, and static objects separately to obtain dynamic obstacle features C. A Map Features C M Characteristics of static objects C O ; Dynamic obstacle features C A Map Features C M Characteristics of static objects C O By splicing the images together, a unified scene representation C is obtained. scene = [C A C M C O ].
[0010] In some exemplary embodiments, encoding the agent includes: acquiring historical state sequences of the vehicle and surrounding traffic participants; extracting temporal features using an MLP-Mixer network and superimposing learnable type embeddings to obtain dynamic obstacle features, as shown below:
[0011] in, This is a historical state sequence of the vehicle and surrounding traffic participants. Embed the type of dynamic obstacle.
[0012] In some exemplary embodiments, encoding the vector map includes: converting lane centerlines and road network topology into vectorized data; and generating a planning intent embedding in conjunction with navigation instructions. Different perception weights are assigned based on lane direction, and map features are extracted using MLP-Mixer, as shown below:
[0013] in, Generate planning intent embeddings for navigation instructions. This is vectorized data resulting from the transformation of lane centerlines and road network topology. Embed the type of vector map.
[0014] In some exemplary embodiments, encoding static objects includes: for static objects without temporal evolution, extracting geometric features using a multilayer perceptron and adding category labels to obtain static object features, as shown below:
[0015] Among them, S obj T represents data about static objects. obj It is a type embedding of static objects.
[0016] In some exemplary embodiments, global importance ranking is performed based on the internal weights of the self-attention mechanism, including: unifying the scene representation. The input is to the attention module, and the output, after contextual interaction, serves as the anchor point for contrastive learning, denoted as... ; Extract the attention matrix from the attention calculation process; Define the first i Importance score of each scene element For this element, from all other elements in the scene j The sum of the normalized attention weights obtained at the point; the calculation formula is:
[0017] in, Si This represents the global importance score of the i-th scene element; N The input represents the total number of scene elements fed into the self-attention module; Q and K represent the query matrix and key matrix in the self-attention mechanism, respectively; D represents the feature dimension of the key vector; softmax() represents the normalization exponential function; the summation operation is performed along the column dimension. j "to perform" indicates summarizing all other elements against the element. i Attention level; score The higher the value, the more central the element is in the interaction graph, indicating a higher causal relevance.
[0018] In some exemplary embodiments, constructing category-aware automated positive and negative samples includes: constructing a positive sample set separately. and negative sample set The construction of the positive sample set includes: selecting importance scores for each category. The top-k elements are used to construct a positive sample set, as shown below:
[0019] Constructing a negative sample set includes: selecting importance scores for each category. The lowest bottom-k elements are used to construct the negative sample set, as shown below:
[0020] Among them, C A C M C O These are dynamic obstacle features, map features, and static object features, respectively.
[0021] In some exemplary embodiments, context alignment of category-aware automated positive and negative samples is performed based on a recoding mechanism, including: adjusting the selected anchor points... Positive sample set and negative sample set The inputs are fed into a shared encoder; the interaction weights are recalibrated within their respective retained elements using a self-attention mechanism to generate context-aligned final feature representations, denoted as anchor representations E. a Positive sample characterization E p Negative sample representation E n .
[0022] In some exemplary embodiments, constructing a contrastive loss function includes: using a linear layer to recode the features E a E p E n Mapping to the normalized metric space, we get h a hp h n Training is performed using a triplet loss function based on Softmax; the optimization objective is to maximize the anchor point h. a and positive sample h p The cosine similarity between them, while minimizing the anchor point h. a and negative sample h n The similarity between them; the formula for the constructed contrastive loss function is shown below:
[0023] in, This is the temperature coefficient.
[0024] The technical solution provided in this application has at least the following advantages: This application provides a method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios, including the following steps: First, multimodal scene elements are encoded separately, and the encoding results are concatenated to obtain a unified scene representation; then, the unified scene representation is input into a contrastive representation module, and global importance is ranked based on the internal weights of the self-attention mechanism; next, scene elements are divided into different categories, and the categories are filtered to construct category-aware automated positive and negative samples; based on a recoding mechanism, the category-aware automated positive and negative samples are context-aligned; finally, a contrastive loss function is constructed to enable the model to distinguish key causal information from interference noise in the feature space. Compared with existing technologies, the feature learning method for complex autonomous driving scenarios proposed in this application has the following advantages: (1) This application solves the problem of "causal confusion" and significantly improves the generalization ability of the model in unknown scenarios. Existing deep learning models mostly rely on statistical correlation fitting, which makes it easy to learn pseudo features in the environment. This application abandons blind fitting and uses the internal weights of the self-attention mechanism to perform global importance ranking, forcing the model to accurately capture the real causal logic behind driving behavior, fundamentally reducing the decision error rate caused by causal confusion.
[0025] (2) This application can intelligently suppress high-dimensional heterogeneous noise, improving the safety and smoothness of trajectory planning. Real driving scenarios are filled with massive amounts of irrelevant background noise (such as pedestrians in the distance and vehicles in the opposite lane). Unlike traditional encoders that indiscriminately consume computing resources, this application innovatively introduces a category-aware automated positive and negative sample screening mechanism, focusing the computing attention on the core causal elements with the highest scores, effectively shielding irrelevant noise from interfering with the planning process and ensuring that the generated trajectory more strictly follows traffic rules.
[0026] (3) This application breaks through the limitations of traditional contrastive learning in driving scenarios and achieves strict spatial semantic alignment. Traditional contrastive learning relies on data augmentation methods such as random pruning and rotation, which can easily destroy the spatial constraints in driving scenarios that are highly sensitive to direction and topology. This application does not require any destructive data augmentation. It directly splits positive and negative samples based on attention scores and completes the re-alignment of the context through a recoding mechanism. This enables the model to not only effectively distinguish key information from interference noise in the feature space, but also perfectly maintain the structural integrity of the autonomous driving scenario. Attached Figure Description
[0027] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments, and unless otherwise stated, the figures in the drawings are not to be limited by scale.
[0028] Figure 1 This is a flowchart illustrating a method for causal saliency mining and anti-interference semantic representation in an autonomous driving scenario, provided in an embodiment of this application.
[0029] Figure 2 The overall architecture flowchart of the causal saliency mining and anti-interference semantic representation method in the autonomous driving scenario provided in the embodiments of this application is shown.
[0030] Figure 3 This is a data flow diagram illustrating the causal saliency mining and anti-interference semantic representation method for autonomous driving scenarios provided in this application embodiment.
[0031] Figure 4 This is a structural diagram of a causal saliency mining and anti-interference semantic representation system for autonomous driving scenarios provided in an embodiment of this application. Detailed Implementation
[0032] As can be seen from the background technology, the existing technology currently lacks a method that can automatically and intelligently filter out "positive samples" (causal features) and "negative samples" (noise features) based on scene semantics, and optimize scene representation accordingly.
[0033] To address this technical problem, this application provides a method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios, comprising the following steps: First, multimodal scene elements are encoded separately, and the encoding results are concatenated to obtain a unified scene representation; then, the unified scene representation is input into a contrastive representation module, and global importance is ranked based on the internal weights of a self-attention mechanism; next, scene elements are divided into different categories, and the categories are filtered to construct category-aware automated positive and negative samples; based on a recoding mechanism, the category-aware automated positive and negative samples are context-aligned; finally, a contrastive loss function is constructed to enable the model to distinguish key causal information from interference noise in the feature space. This application provides a method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios, which, through an attention-guided automated sampling mechanism and a recoding strategy, filters out key features with causal relationships from high-dimensional heterogeneous driving scenarios and suppresses spurious correlation noise in the environment.
[0034] The embodiments of this application will now be described in detail with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the embodiments of this application to facilitate a better understanding of the application. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.
[0035] See Figure 1 This application provides a method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios, including the following steps: Step S1: Encode the multimodal scene elements separately, and then concatenate the encoding results to obtain a unified scene representation.
[0036] Step S2: Input the unified scene representation into the comparison representation module, and perform global importance ranking based on the internal weights of the self-attention mechanism.
[0037] Step S3: Divide the scene elements into different categories, filter the categories, and construct category-aware automated positive and negative samples.
[0038] Step S4: Based on the recoding mechanism, perform context alignment on the category-aware automatic positive and negative samples.
[0039] Step S5: Construct a contrastive loss function to enable the model to distinguish key causal information from interference noise in the feature space.
[0040] This application is mainly applied to the scene understanding, feature extraction and trajectory planning system of autonomous vehicles in complex dynamic environments, and is particularly suitable for solving the problems of causal confusion and environmental noise interference in high-dimensional driving scenarios.
[0041] Step S1 primarily involves performing unified feature encoding for multimodal scene elements. First, the multi-source heterogeneous data perceived by the autonomous vehicle is preprocessed and its features are embedded to construct a unified scene context representation. .
[0042] In some embodiments, the multimodal scene elements in step S1 include: intelligent agents, vector maps, and static objects; wherein, intelligent agents represent the historical states of moving objects in the scene, such as vehicles, surrounding vehicles, pedestrians, etc.; vector maps represent road topology, lane lines, intersection structures, and navigation-related map information; and static objects represent roadblocks, cones, and fixed facilities.
[0043] In some embodiments, step S1 involves encoding the multimodal scene elements separately and concatenating the encoding results to obtain a unified scene representation, including: Step S101: Encode the intelligent agent, vector map, and static object respectively to obtain dynamic obstacle features C. A Map Features C M Characteristics of static objects C O .
[0044] Step S102: Transfer dynamic obstacle features C A Map Features C M Characteristics of static objects C O By splicing the images together, a unified scene representation C is obtained. scene = [C A C M C O ].
[0045] In some embodiments, encoding the agent in step S101 includes: acquiring the historical state sequence (such as position, speed, heading, etc.) of the vehicle and surrounding traffic participants; extracting temporal features using an MLP-Mixer network and superimposing learnable type embeddings to obtain dynamic obstacle features, as shown below:
[0046] in, This is a historical state sequence of the vehicle and surrounding traffic participants. Embed the type of dynamic obstacle.
[0047] In some embodiments, encoding the vector map in step S101 includes: converting lane centerlines and road network topology into vectorized data; and generating a planning intent embedding in conjunction with navigation instructions. Different perception weights are assigned based on lane direction (higher weight is given to lanes traveling in the same direction), and map features are extracted using MLP-Mixer, as shown below:
[0048] in, Generate planning intent embeddings for navigation instructions. This is vectorized data resulting from the transformation of lane centerlines and road network topology. Embed the type of vector map.
[0049] In some embodiments, encoding static objects in step S101 includes: for static objects such as traffic cones and roadblocks that do not evolve over time, extracting geometric features using a multilayer perceptron (MLP) and adding category labels to obtain static object features, as shown below:
[0050] Among them, S obj T represents data about static objects. obj It is a type embedding of static objects.
[0051] Unified Scene Construction: The above three types of features are concatenated to form a comprehensive scene representation C containing complete environmental semantics. scene = [C A C M C O ].
[0052] Step S2 is primarily a process of scoring causal importance based on the self-attention mechanism. To quantify the causal influence of each element in the scenario on driving decisions, this step utilizes the internal weights of the self-attention mechanism to perform a global importance assessment.
[0053] In some embodiments, step S2, which involves ranking global importance based on the internal weights of the self-attention mechanism, includes: Step S201: Unify scene representation The input is a self-attention module, and the output, after contextual interaction, serves as the anchor for contrastive learning, denoted as... .
[0054] Step S202: Extract the attention matrix from the attention calculation process; define the first... i Importance score of each scene element (Token) For this element, from all other elements in the scene j The sum of the normalized attention weights obtained at each location.
[0055] The calculation formula is as follows:
[0056] in, S i This represents the global importance score of the i-th scene element; N This represents the total number of scene elements input into the self-attention module; N This represents the sum of all independent environmental elements within the current perception range of the autonomous vehicle, including the total number of surrounding agents, static objects, and map topology features. Q and K represent the query matrix and key matrix in the self-attention mechanism, respectively, represented by a unified scene representation. The linear mapping is used to obtain Q, which represents the "information exploration intention" of each scene element at the current moment, and K represents the "objective state and attributes" of each scene element broadcast outward.
[0057] D represents the feature dimension of the key vector, used to scale the dot product result to prevent gradient vanishing; the dimension of the key vector K is the value divided by in the formula. This is used to scale the dot product result, representing the "information capacity" of the scene feature representation. (By...) Scaling is performed to prevent the gradient vanishing problem caused by excessively large inner products when interacting with high-dimensional features, thereby ensuring that the model can still stably and sensitively calculate causal weights when facing extremely complex traffic conditions, thus ensuring driving safety.
[0058] `softmax()` represents the normalized exponential function used to transform attention scores into a probability distribution; the nth element in the attention weight matrix... j line, number i The specific values in the column are normalized probability values, i.e., scene elements. j To what extent does the future state depend on factors? i The current state.
[0059] Summation operation along column dimension j "to perform" indicates summarizing all other elements against the element. i Attention level; score The higher the value, the more central the element is in the interaction graph, indicating a higher causal relevance.
[0060] Step S3 mainly involves the automated construction of category-aware positive and negative samples. To avoid the complete neglect of certain key elements (such as map information) due to direct global sorting, a selection strategy based on categories is adopted to construct a set of positive and negative samples for contrastive learning: scene elements are divided into categories such as "dynamic agents", "static objects", and "map elements", and sorted independently within each category.
[0061] In some embodiments, step S3, which involves constructing category-aware automated positive and negative samples, includes: Construct positive sample sets respectively and negative sample set .
[0062] The construction of the positive sample set includes: selecting importance scores for each category. The top-k elements are used to construct a positive sample set, as shown below:
[0063] It should be noted that importance scores are selected when constructing the positive sample set. The top-k elements represent key causal factors that have a substantial impact on driving behavior (such as the vehicle in front that brakes suddenly or the lane markings).
[0064] Constructing a negative sample set includes: selecting importance scores for each category. The lowest bottom-k elements are used to construct the negative sample set, as shown below:
[0065] Among them, C A C M C O These are dynamic obstacle features, map features, and static object features, respectively.
[0066] It should be noted that importance scores are selected when constructing the negative sample set. The lowest Bottom-k elements. These elements represent background noise or pseudo-related information in the scene (such as irrelevant vehicles in the distance, or vehicles in the opposite lane).
[0067] Step S4 is primarily based on context alignment for recoding.
[0068] Because the filtering operation in step S3 alters the composition of the feature set, the original contextual semantic relationships are broken. To eliminate residual noise and establish new dependencies between the filtered elements, a recoding mechanism is introduced.
[0069] In some embodiments, step S4 involves context alignment of category-aware automated positive and negative samples based on a recoding mechanism, including: Step S401: Select the anchor points Positive sample set and negative sample set Input into the shared encoder.
[0070] Step S402: Recalibrate the interaction weights within each retained element using a self-attention mechanism to generate context-aligned final feature representations, denoted as anchor representations E. a Positive sample characterization E p Negative sample representation E n .
[0071] Step S5 primarily involves constructing the causal perception contrastive optimization objective. By constructing a contrastive loss function, the model is forced to distinguish key causal information from interfering noise in the feature space. First, feature projection is performed to obtain h. a h p h n Then, triplet loss optimization is performed. By constructing a contrastive loss function, the model can learn scene representations with strong robustness and causal discriminative power.
[0072] In some embodiments, constructing the contrastive loss function in step S5 includes: Step S501: Use a linear layer to recode the feature E. a E p E n Mapping to the normalized metric space, we get h a h p h n .
[0073] Step S502: Training is performed using a triplet loss function based on Softmax. The optimization objective is to maximize the anchor point h. a and positive sample h p The cosine similarity between them, while minimizing the anchor point h. a and negative sample h n The similarity between them.
[0074] The formula for the constructed contrastive loss function is shown below:
[0075] in, This is the temperature coefficient.
[0076] The following section provides a detailed description of the causal saliency mining and anti-interference semantic representation method for autonomous driving scenarios provided in this application, with reference to the accompanying drawings.
[0077] like Figure 2 As shown, the system first receives a raw driving scenario as input. This scenario consists of three parts: intelligent agents (dynamic traffic participants), vector maps (map information), and static objects (static targets). Since these three types of input data have different formats, the model first encodes them separately: the dynamic participants are represented by C through an MLP-Mixer. AThe map is represented by C through an encoder. M The static target is represented by C through MLP. O Here, each C represents the encoding result of a class of scene elements in a unified feature space. agent T map T obj Type embeddings are used to explicitly tell the model "which type of object this token belongs to", avoiding confusion of information from different sources in a unified space; for example, T agent Tell the model that this part is a dynamic body, T map Tell the model that this part is a map element, T obj This tells the model that this part is a static object. After completing these encodings, the system concatenates the three types of results into a unified scene representation C. scene = [C A C M C O Next, C scene Upon entering the causal perception scene representation module, the initial scene encoding C is first obtained through self-attention. anc Here, the subscript anc represents the anchor, which can be understood as the baseline representation of the complete scene after global interactions. However, this representation still contains some irrelevant or weakly relevant information, so the model further automatically filters key elements and re-encodes them, ultimately obtaining E. a E p and E n Here, E represents the recoded embedding; the subscript a represents the anchor, p represents positive, and n represents negative, therefore E a It is the recoded result of the complete anchor.
[0078] like Figure 3 As shown, the starting point of the data flow is the unified scenario representation C. scene That is, by C A C M C O The resulting full-scene encoding is obtained by splicing. The system first processes C... scene Perform self-attention once and obtain C. anc = Self-Attention(C scene Here C anc This refers to the anchor context, representing the baseline context formed after all elements in the scene interact with each other. Simultaneously, the model extracts the attention matrix from the self-attention mechanism and calculates an importance score S for each token.i Here S i In this context, the subscript i represents the i-th scene element, and S represents the score; if an element receives more attention from other elements, its S... i The higher the value, the more likely it is to be a key influencing factor in the current scenario. (This is followed by a seemingly unrelated sentence about obtaining all S values.) i Afterwards, the model will not mix all tokens together and sort them uniformly, but will process them separately according to their categories, that is, in the corresponding C of the dynamic body. A C corresponding to the map M C corresponding to static targets O The selection process is performed within the range where the top-k elements with the highest scores in each category form the positive sample C. pos = [Top-k(C A Top-k(C) M Top-k(C) O [)], where pos represents positive; the lowest-scoring bottom-k elements in each class form the negative sample C. neg =[Bottom-k(C A ), Bottom-k(C M ), Bottom-k(C O [], where *neg* represents negative. Here, *k* is not a fixed number, but is controlled by the pruning ratio *ρ*, which is the proportion of elements retained or removed. The significance of this is that it can select key elements without completely deleting all information of a certain type due to uniform sorting. Next, C... anc C pos and C neg Each element will then enter the re-encoding block and be processed again by the shared encoder. This is because the contextual relationships have changed after filtering, and their internal connections must be remodeled to obtain a truly stable representation. The re-encoded output is E. a E p and E n Subsequently, these three representations enter the projection layer and are mapped to vector h in the contrastive learning space. a h p and h n Where h represents the projected embedding, and a / p / n still correspond to the anchor, positive, and negative, respectively. The model uses contrastive loss to make h... a Closer to hp At the same time, stay away from h n This means making the complete scene representation closer to the "truly crucial scene subset" and further away from the "irrelevant noise subset." Therefore, Figure 3 The complete data flow can be summarized as: C scene → C anc With attention score S i → Filter by category to get C pos C neg → Re-encode to obtain E a E p E n → Projection as h a , h p , h n → Calculate the contrastive loss to bring positive samples closer to the anchor point and push negative samples further away from the anchor point. Finally, E can be used to... a E p It is sent to different downstream modules for use.
[0079] Compared to existing technologies, the feature learning method for complex autonomous driving scenarios proposed in this application has the following advantages: (1) This application solves the problem of "causal confusion" and significantly improves the generalization ability of the model in unknown scenarios. Existing deep learning models mostly rely on statistical correlation fitting, which makes it easy to learn pseudo features in the environment. This application abandons blind fitting and uses the internal weights of the self-attention mechanism to perform global importance ranking, forcing the model to accurately capture the real causal logic behind driving behavior, fundamentally reducing the decision error rate caused by causal confusion.
[0080] (2) This application can intelligently suppress high-dimensional heterogeneous noise, improving the safety and smoothness of trajectory planning. Real driving scenarios are filled with massive amounts of irrelevant background noise (such as pedestrians in the distance and vehicles in the opposite lane). Unlike traditional encoders that indiscriminately consume computing resources, this application innovatively introduces a category-aware automated positive and negative sample screening mechanism, focusing the computing attention on the core causal elements with the highest scores, effectively shielding irrelevant noise from interfering with the planning process and ensuring that the generated trajectory more strictly follows traffic rules.
[0081] (3) This application breaks through the limitations of traditional contrastive learning in driving scenarios and achieves strict spatial semantic alignment. Traditional contrastive learning relies on data augmentation methods such as random pruning and rotation, which can easily destroy the spatial constraints in driving scenarios that are highly sensitive to direction and topology. This application does not require any destructive data augmentation. It directly splits positive and negative samples based on attention scores and completes the re-alignment of the context through a recoding mechanism. This enables the model to not only effectively distinguish key information from interference noise in the feature space, but also perfectly maintain the structural integrity of the autonomous driving scenario.
[0082] like Figure 4 As shown, this application also provides a causal saliency mining and anti-interference semantic representation system for autonomous driving scenarios, including a scene processing module 101 and a contrastive representation module 102. The scene processing module 101 is used to encode multimodal scene elements respectively and concatenate the encoding results to obtain a unified scene representation. Then, the unified scene representation is input into the contrastive representation module 102, and global importance ranking is performed based on the internal weights of the self-attention mechanism to obtain the recoded feature E. a E p E n Then, feature projection is performed to obtain h. a h p h n Finally, triplet loss optimization is performed. By constructing a contrastive loss function, the model can learn scene representations with strong robustness and causal discriminative power.
[0083] Based on the above technical solutions, this application provides a method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios, including the following steps: First, multimodal scene elements are encoded separately, and the encoding results are concatenated to obtain a unified scene representation; then, the unified scene representation is input into a contrastive representation module, and global importance is ranked based on the internal weights of the self-attention mechanism; next, scene elements are divided into different categories, and the categories are filtered to construct category-aware automated positive and negative samples; based on the recoding mechanism, the category-aware automated positive and negative samples are context-aligned; finally, a contrastive loss function is constructed to enable the model to distinguish key causal information from interference noise in the feature space. This application proposes a feature learning method for complex autonomous driving scenarios, which, through an attention-guided automated sampling mechanism and recoding strategy, filters out key features with causal relationships from high-dimensional heterogeneous driving scenarios and suppresses pseudo-correlation noise in the environment.
[0084] Those skilled in the art will understand that the above-described embodiments are specific examples of implementing this application, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of this application. Any person skilled in the art can make their own modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application should be determined by the scope defined in the claims.
Claims
1. A method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios, characterized in that, Includes the following steps: The multimodal scene elements are encoded separately, and the encoding results are concatenated to obtain a unified scene representation; The unified scene representation is input into the comparison representation module, and global importance is ranked based on the internal weights of the self-attention mechanism; The scene elements are divided into different categories, and the categories are filtered to construct category-aware automated positive and negative samples; Based on the recoding mechanism, context alignment is performed on category-aware automated positive and negative samples; Construct a contrastive loss function to enable the model to distinguish key causal information from interfering noise in the feature space.
2. The method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios according to claim 1, characterized in that, The multimodal scene elements include: intelligent agents, vector maps, and static objects; among them... Intelligent agents represent the historical states of moving objects in the scene; vector maps represent road topology, lane lines, intersection structures, and navigation-related map information; static objects represent roadblocks, cones, and fixed facilities.
3. The method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios according to claim 2, characterized in that, The multimodal scene elements are encoded separately, and the encoding results are concatenated to obtain a unified scene representation, including: Encoding the agent, vector map, and static object separately yields the dynamic obstacle feature C. A Map Features C M Characteristics of static objects C O ; Dynamic obstacle feature C A Map Features C M Characteristics of static objects C O By splicing the images together, a unified scene representation C is obtained. scene =[C A C M C O ].
4. The method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios according to claim 3, characterized in that, Encoding the agent includes: Obtain the historical state sequence of the vehicle and surrounding traffic participants; Temporal features are extracted using an MLP-Mixer network and then superimposed with type embeddings to obtain dynamic obstacle features, as shown below: in, This is a sequence of historical states of the vehicle and surrounding traffic participants. Embed the type of dynamic obstacle.
5. The method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios according to claim 3, characterized in that, Encoding vector maps includes: Convert lane centerlines and road network topology into vector data; Combine navigation instructions to generate planning intent embedding Different perception weights are assigned based on lane direction, and map features are extracted using MLP-Mixer, as shown below: in, Generate planning intent embeddings for navigation instructions. This is vectorized data resulting from the transformation of lane centerlines and road network topology. Embed the type of vector map.
6. The method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios according to claim 3, characterized in that, Encoding static objects includes: For static objects without temporal evolution, geometric features are extracted using a multilayer perceptron and category labels are added to obtain the static object features, as shown below: Among them, S obj T represents data about static objects. obj It is a type embedding of static objects.
7. The method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios according to claim 1, characterized in that, Global importance ranking is performed based on internal weights derived from the self-attention mechanism, including: Unified scene representation The input is to the attention module, and the output, after contextual interaction, serves as the anchor point for contrastive learning, denoted as... ; Extract the attention matrix from the attention calculation process; define the first... i Importance score of each scene element This is the sum of the normalized attention weights that the element obtains from all elements in the scene; the calculation formula is: in, S i This represents the global importance score of the i-th scene element; N The total number of scene elements input into the self-attention module is represented by Q; Q and K represent the query matrix and key matrix in the self-attention mechanism, respectively; D represents the feature dimension of the key vector; softmax() represents the normalization exponential function. Summation operation along column dimension j "to perform" indicates summarizing all elements from one element. i The level of attention; Fraction The higher the value, the more central the element is in the interaction graph, indicating a higher causal relevance.
8. The method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios according to claim 6, characterized in that, Constructing category-aware, automated positive and negative samples includes: Construct positive sample sets respectively and negative sample set ;in, Constructing a positive sample set includes: selecting importance scores for each category. The top-k elements are used to construct a positive sample set, as shown below: Constructing a negative sample set includes: selecting importance scores for each category. The lowest (Bottom-k) elements are used to construct the negative sample set, as shown below: Among them, C A C M C O These are dynamic obstacle features, map features, and static object features, respectively.
9. The method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios according to claim 7, characterized in that, Based on a recoding mechanism, context alignment is performed on category-aware automated positive and negative samples, including: Selected anchor points Positive sample set and negative sample set Input is fed into the shared encoder; By utilizing a self-attention mechanism, the interaction weights are recalibrated within their respective retained elements to generate context-aligned final feature representations, denoted as anchor representations E. a Positive sample characterization E p Negative sample representation E n .
10. The method for causal saliency mining and anti-interference semantic representation in autonomous driving scenarios according to claim 1, characterized in that, Constructing the contrastive loss function includes: The recoded feature E is processed using a linear layer. a E p E n Mapping to the normalized metric space, we get h a h p h n ; Training is performed using a triplet loss function based on Softmax; the optimization objective is to maximize the anchor point h. a and positive sample h p The cosine similarity between them is minimized while minimizing the anchor point h. a and negative sample h n The similarity between them; the formula for the constructed contrastive loss function is shown below: in, This is the temperature coefficient.