Yaw detection method, electronic equipment, vehicle and storage medium
By fusing features of navigation path and vehicle trajectory through Fourier coding and cross-attention mechanism, the problem of low yaw detection accuracy in existing systems is solved, and high-precision yaw detection is achieved in complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-02
- Publication Date
- 2026-04-03
AI Technical Summary
Existing yaw detection methods have low accuracy during vehicle navigation and are unable to accurately identify situations where the vehicle's actual trajectory deviates from the navigation path.
A Fourier encoder is used to encode the navigation path and the vehicle's actual trajectory sequence, which is then converted to the frequency domain. Multimodal information is fused by combining a cross-attention mechanism with local map features. The yaw detection result is determined by the decoder, and the decoder model is optimized in abnormal situations.
It improves the accuracy and robustness of yaw detection, enabling accurate identification of vehicle yaw in complex environments, reducing false alarms and false negatives, and enhancing the model's coverage in long-tail scenarios.
Smart Images
Figure CN121783181A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of vehicle navigation technology, and in particular to a yaw detection method, electronic device, vehicle, and storage medium. Background Technology
[0002] Currently, navigation software-provided routes for driving are becoming increasingly common. In vehicle navigation, yaw detection is crucial. Specifically, when the vehicle's actual trajectory deviates from the navigation path, early and accurate yaw detection is essential for accurately and promptly planning a new navigation route and guiding the user to control the vehicle along that route. Existing yaw detection methods suffer from low detection accuracy. Summary of the Invention
[0003] This application provides a yaw detection method, electronic device, vehicle, and storage medium, aiming to improve the problem of low detection accuracy in existing yaw detection methods.
[0004] A yaw detection method, comprising: Obtain navigation path sequences, vehicle real trajectory sequences, and local map features corresponding to the area where the vehicle is located; The navigation path sequence and the vehicle's actual trajectory sequence are feature-encoded to determine the navigation coding features corresponding to the navigation path sequence and the trajectory coding features corresponding to the vehicle's actual trajectory sequence; Multimodal information fusion is performed on the navigation coding features, the trajectory coding features, and the local map features to determine the fused features; Yaw detection is performed based on the fused features to determine the yaw detection result.
[0005] In this embodiment, the navigation path sequence and the vehicle's actual trajectory sequence are feature-encoded to convert the sequence data into coded features, aligning them with local map features to facilitate subsequent fusion operations and filter out interference information. Multimodal information fusion is performed on the navigation coded features, trajectory coded features, and local map features, so that the fused features not only contain deviation information reflecting the target navigation path and the vehicle's actual trajectory, but also contain constraint information of the local map features of the area where the vehicle is located, ensuring the accuracy of the yaw detection results determined by subsequent yaw detection based on the fused features.
[0006] In one embodiment, the step of obtaining the navigation path sequence, the vehicle's actual trajectory sequence, and the local map features corresponding to the area where the vehicle is located includes: The target navigation path and the vehicle's actual trajectory are time-aligned and normalized to determine the navigation path sequence corresponding to the target navigation path and the vehicle's actual trajectory sequence corresponding to the vehicle's actual trajectory. Feature extraction is performed on the high-precision map information to determine the local map features of the area where the vehicle is located.
[0007] In this embodiment, the target navigation path and the vehicle's actual trajectory are time-aligned and normalized to ensure the consistency of subsequent processing data and avoid data asynchrony affecting the accuracy of the detection results. Extracting local map features of the area where the vehicle is located from high-precision map information can reduce the amount of data for subsequent data analysis, which helps to ensure the efficiency and accuracy of the yaw detection process.
[0008] In one embodiment, the step of performing feature encoding on the navigation path sequence and the vehicle's actual trajectory sequence to determine the navigation encoding features corresponding to the navigation path sequence and the trajectory encoding features corresponding to the vehicle's actual trajectory sequence includes: A Fourier encoder is used to perform Fourier transform on the navigation path sequence and the vehicle's actual trajectory sequence to determine the navigation frequency domain features corresponding to the navigation path sequence and the trajectory frequency domain features corresponding to the vehicle's actual trajectory sequence. The navigation frequency domain features and the trajectory frequency domain features are feature-encoded to determine the navigation coding features corresponding to the navigation path sequence and the trajectory coding features corresponding to the vehicle's actual trajectory sequence.
[0009] In this embodiment, a Fourier encoder is used to perform Fourier transforms on the navigation path sequence and the vehicle's actual trajectory sequence, respectively. The sequence data (a series of path points or trajectory points) in the spatial domain is transformed into the frequency domain for feature extraction, determining the corresponding navigation frequency domain features and trajectory frequency domain features. Then, feature encoding is performed on the navigation frequency domain features and trajectory frequency domain features to generate navigation encoded features and trajectory encoded features that contain their global structure and periodic patterns. The Fourier encoder completes the feature extraction from the spatial domain to the frequency domain. Its low-pass filtering characteristics can effectively smooth high-frequency disturbances in the trajectory caused by sensor noise, ground bumps, or control jitter. At the same time, its global nature helps the model to focus more on the overall direction and intention of the path and trajectory, rather than local detail deviations. Subsequent consistency judgment provides a more robust and higher-level feature representation, which is the foundation for improving yaw detection accuracy.
[0010] In one embodiment, the step of fusing multimodal information from the navigation coding features, the trajectory coding features, and the local map features to determine the fused features includes: A cross-attention mechanism is used to perform cross-attention calculation on the navigation coding features, the trajectory coding features, and the local map features to determine multiple associated features and multiple differential features; Multiple related features and multiple differential features are fused to determine the fused feature.
[0011] In this embodiment, a cross-attention mechanism is used to perform cross-attention calculations on navigation coding features, trajectory coding features, and local map features to determine multiple associated features and multiple differential features. The multiple associated features can reflect the correlation of input features such as navigation coding features, trajectory coding features, and local map features, while the multiple differential features can reflect the differences of input features such as navigation coding features, trajectory coding features, and local map features. Then, the multiple associated features and multiple differential features are fused to determine the fused feature. This fused feature contains the key features for deep information interaction among the three input features, which helps to ensure the accuracy of subsequent yaw.
[0012] In one embodiment, the associated features include navigation associated features and trajectory associated features. The navigation associated features are features obtained by residually connecting the navigation matching features and the navigation coding features. The trajectory associated features are features obtained by residually connecting the trajectory matching features and the trajectory coding features. The navigation matching features are features related to the navigation coding features matched from the trajectory coding features and the local map features. The difference features include a first difference feature and a second difference feature. The first difference feature is a feature determined by distance calculation of the navigation matching feature and the trajectory coding feature. The second difference feature is a feature determined by distance calculation of the trajectory matching feature and the navigation coding feature. In this embodiment, a cross-attention mechanism is used to mine navigation association features, trajectory association features, first difference features, and second difference features between navigation coding features and trajectory coding features. These features can effectively reflect the differences between the target navigation path and the vehicle's actual trajectory. Since the above analysis process introduces local map features for constraints, it helps to ensure the accuracy of the determined association features and difference features.
[0013] In one embodiment, the step of performing yaw detection based on the fused features and determining the yaw detection result includes: The fused features are decoded using a decoder to determine the yaw probability; When the yaw probability is greater than a preset probability, the yaw detection result is determined to be yawed; When the yaw probability is not greater than a preset probability, the yaw detection result is determined to be no yaw.
[0014] In this embodiment, the yaw probability is determined by decoding the fused features, and the yaw detection result is determined by comparing the yaw probability with the preset probability. The entire architecture is end-to-end, from the input of actual measured data to the output of yaw detection results. There is no need to manually design complex rules, and the yaw detection results can be quickly output through simple comparison.
[0015] In one embodiment, after performing yaw detection based on the fused features and determining the yaw detection result, the yaw detection method further includes: When the yaw detection result is abnormal, the yaw label corresponding to the fused feature is determined; An optimized sample is formed based on the fusion features and the yaw label. The decoder is then optimized and updated based on the optimized sample to determine the optimized decoder.
[0016] In this embodiment, when the yaw detection result is abnormal, the decoder can form optimized samples based on the fused features and their corresponding yaw labels, so that it can learn from the optimized samples in the future, which significantly reduces the long-term maintenance cost and iteration difficulty, and makes the model performance of the decoder continuously improve over time.
[0017] An electronic device includes a processor and a memory, wherein, Memory, used to store computer programs; The processor is used to execute the program stored in the memory to implement the yaw detection method described above.
[0018] A vehicle comprising the aforementioned electronic equipment.
[0019] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned yaw detection method. Attached Figure Description
[0020] Figure 1 This is a flowchart of a yaw detection method provided in an embodiment of this application; Figure 2 yes Figure 1 A flowchart of step S101; Figure 3 yes Figure 1 A flowchart of step S102; Figure 4 yes Figure 1 A flowchart of step S103; Figure 5 yes Figure 1 A flowchart of step S104; Figure 6 yes Figure 1A flowchart following step S104; Figure 7 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0021] To make the technical problems, technical solutions, and beneficial effects solved by this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0022] This application provides a yaw detection method applicable to electronic devices, specifically vehicles, which can be, but are not limited to, vehicle controllers. This yaw detection method dynamically assesses whether the vehicle has yawed while traveling along a target navigation path, based on real-time acquired data, and outputs a highly accurate yaw detection result. Here, yaw refers to the situation where the vehicle's actual trajectory deviates from the target navigation path.
[0023] This application provides a yaw detection method, which will be described using an electronic device (hereinafter referred to as "device") as an example. Figure 1 As shown, the yaw detection method includes: S101: Obtain navigation path sequence, vehicle real trajectory sequence, and local map features corresponding to the area where the vehicle is located; S102: Perform feature encoding on the navigation path sequence and the vehicle's actual trajectory sequence to determine the navigation encoding features corresponding to the navigation path sequence and the trajectory encoding features corresponding to the vehicle's actual trajectory sequence; S103: Perform multimodal information fusion on navigation coding features, trajectory coding features, and local map features to determine the fused features; S104: Perform yaw detection based on fused features and determine the yaw detection result.
[0024] The navigation path sequence is a sequence of data formed based on the navigation path taken during vehicle navigation. As an example, the navigation path sequence can be a sequence related to the navigation path determined by preprocessing the target navigation path. The target navigation path refers to the planned navigation path during vehicle travel, primarily based on static road network information (a series of latitude and longitude coordinates or points), without considering real-time traffic dynamics, and calculating the optimal path between two points using algorithms. The navigation coding feature corresponding to the navigation path sequence refers to the features determined by feature encoding the navigation path sequence; this navigation coding feature can be represented by E_nav.
[0025] The vehicle's actual trajectory sequence is the sequence data formed during the vehicle's movement. As an example, the vehicle's actual trajectory sequence can be a sequence related to the actual trajectory determined by preprocessing the vehicle's actual trajectory. The vehicle's actual trajectory is the trajectory of the vehicle's actual driving process, specifically based on time-series vehicle pose data, which includes, but is not limited to, coordinates, velocity, and orientation angle. The trajectory encoding feature corresponding to the vehicle's actual trajectory sequence refers to the features determined by feature encoding the vehicle's actual trajectory sequence; this trajectory encoding feature can be represented by E_traj.
[0026] The local map features corresponding to the vehicle's location are features related to yaw detection extracted from map information of that area. As an example, these local map features can be features related to the vehicle's location determined by preprocessing high-precision map information, and can be represented as E_map. High-precision map information refers to map information output by navigation software, specifically including but not limited to relevant information from high-precision maps, including but not limited to vector elements such as lane lines and road topology.
[0027] As an example, in step S101, during vehicle operation, the device can acquire data such as the target navigation path, the vehicle's actual trajectory, and high-precision map information via the CAN bus or other communication methods. Then, based on pre-set preprocessing rules, the device preprocesses the target navigation path, the vehicle's actual trajectory, and the high-precision map information to obtain the navigation path sequence corresponding to the target navigation path, the vehicle's actual trajectory sequence corresponding to the vehicle's actual trajectory, and the local map features corresponding to the area where the vehicle is located, respectively, to ensure that the preprocessed data is comparable and facilitates subsequent analysis.
[0028] As an example, in step S102, the device can use a pre-set encoder or encoding model to perform feature encoding on the navigation path sequence and the vehicle's actual trajectory sequence, determining the navigation encoding features corresponding to the navigation path sequence and the trajectory encoding features corresponding to the vehicle's actual trajectory sequence. In this example, feature encoding is performed on the navigation path sequence and the vehicle's actual trajectory sequence to convert the sequence data into encoded features, aligning it with local map features to facilitate subsequent fusion operations. Furthermore, the pre-set encoder generally has filtering capabilities, effectively filtering noise interference caused by sensor noise, ground bumps, or handling vibrations in the sequence data, ensuring the accuracy of the encoded features. The neural network model here can be, but is not limited to, a hierarchical graph neural network (VectorNet) and a long short-term memory network (LSTM).
[0029] As an example, in step S103, the device may use a pre-set fusion algorithm or fusion model to perform multimodal information fusion on navigation coding features, trajectory coding features and local map features, and determine the fusion features, so as to integrate three different sources, different types or different dimensions of coding features into more discriminative fusion features.
[0030] As an example, in step S104, the device can use a pre-set decoder to decode the fused features, and then perform yaw detection based on the decoding result output by the decoder to determine the yaw detection result. In this example, since the fused features are abstract features resulting from the fusion of multiple codes, they lack semantic interpretability and cannot perform subsequent semantic understanding tasks. Therefore, it is necessary to first use a decoder to decode the fused features, and then perform yaw detection based on the decoding result to obtain a yaw detection result with higher detection accuracy.
[0031] Alternatively, the device can use a pre-trained yaw detection model to identify the fused features, and determine the yaw detection model based on the output of the yaw detection results. The training process of the pre-trained yaw detection model is as follows: Training sample data is acquired, including pre-determined fused features and sample labels, with labels including "yaw present" and "no yaw present"; the training sample data is input into a neural network model for training; when the neural network model meets pre-set convergence conditions, the trained yaw detection model is determined, enabling it to identify the input fused features and output a yaw detection result of "yaw present" or "no yaw present".
[0032] In this embodiment, the navigation path sequence and the vehicle's actual trajectory sequence are feature-encoded to convert the sequence data into coded features, aligning them with local map features to facilitate subsequent fusion operations and filter out interference information. Multimodal information fusion is performed on the navigation coded features, trajectory coded features, and local map features, so that the fused features not only contain deviation information reflecting the target navigation path and the vehicle's actual trajectory, but also contain constraint information of the local map features of the area where the vehicle is located, ensuring the accuracy of the yaw detection results determined by subsequent yaw detection based on the fused features.
[0033] In one embodiment, such as Figure 2 As shown, step S101, which involves obtaining the navigation path sequence, the vehicle's actual trajectory sequence, and the local map features corresponding to the area where the vehicle is located, includes: S201: Perform time alignment and normalization on the target navigation path and the vehicle's actual trajectory to determine the navigation path sequence corresponding to the target navigation path and the vehicle's actual trajectory sequence corresponding to the vehicle's actual trajectory. S202: Extract features from high-precision map information to determine the local map features of the area where the vehicle is located.
[0034] As an example, in step S201, the device can perform time alignment and normalization processing on the target navigation path and the vehicle's actual trajectory, so that the navigation path sequence corresponding to the processed target navigation path and the vehicle's actual trajectory sequence corresponding to the vehicle's actual trajectory are comparable in both time and space scales, avoiding at least one scale being out of sync and affecting the accuracy of the yaw detection results.
[0035] As an example, in step S202, the device can extract features from the high-precision map information. Specifically, it can determine the area where the vehicle is located based on the vehicle's current position, and then extract local map features of the area where the vehicle is located from the high-precision map information. In this example, extracting local map features of the area where the vehicle is located from the high-precision map information avoids interference from other information outside the vehicle's location, reduces the amount of data for subsequent data analysis, and ensures the efficiency and accuracy of the yaw detection process.
[0036] In this embodiment, the target navigation path and the vehicle's actual trajectory are time-aligned and normalized to ensure the consistency of subsequent processing data and avoid data asynchrony affecting the accuracy of the detection results. Extracting local map features of the area where the vehicle is located from high-precision map information can reduce the amount of data for subsequent data analysis, which helps to ensure the efficiency and accuracy of the yaw detection process.
[0037] In one embodiment, step S102, which involves feature encoding of the navigation path sequence and the vehicle's actual trajectory sequence to determine the navigation encoding features corresponding to the navigation path sequence and the trajectory encoding features corresponding to the vehicle's actual trajectory sequence, includes: S301: Use a Fourier encoder to perform Fourier transform on the navigation path sequence and the vehicle's actual trajectory sequence to determine the navigation frequency domain features corresponding to the navigation path sequence and the trajectory frequency domain features corresponding to the vehicle's actual trajectory sequence. S302: Perform feature encoding on navigation frequency domain features and trajectory frequency domain features to determine the navigation coding features corresponding to the navigation path sequence and the trajectory coding features corresponding to the vehicle's actual trajectory sequence.
[0038] As an example, in step S301, the device can use a Fourier encoder to perform Fourier transform on the navigation path sequence and the vehicle's actual trajectory sequence to convert them from the spatial domain to the frequency domain, thereby determining the navigation frequency domain features corresponding to the navigation path sequence and the trajectory frequency domain features corresponding to the vehicle's actual trajectory sequence. In this example, the Fourier transform can capture the global shape and periodicity features of the sequence, expressed by the formula: F(ω) = ∫ f(t) e^(-iωt) dt, where f(t) is the input sequence and F(ω) is the output frequency domain feature. Specifically, when f(t) is the navigation path sequence, F(ω) is the navigation frequency domain feature corresponding to the navigation path sequence; when f(t) is the vehicle's actual trajectory sequence, F(ω) is the trajectory frequency domain feature corresponding to the vehicle's actual trajectory sequence. Generally speaking, frequency domain features can keenly capture the overall topological trends, motion trends, and possible local abrupt changes in topological trends and rapidly changing motion trends of navigation path sequences and vehicle real trajectory sequences.
[0039] As an example, in step S302, the device can learn and embed the navigation frequency domain features corresponding to the navigation path sequence and the trajectory frequency domain features corresponding to the vehicle's actual trajectory sequence to complete the feature encoding process and determine the navigation encoding features corresponding to the navigation path sequence and the trajectory encoding features corresponding to the vehicle's actual trajectory sequence. In this example, the navigation encoding feature corresponding to the navigation path sequence is E_nav∈ R^d, and the trajectory encoding feature of the vehicle's actual trajectory is E_traj∈ R^d (d is the feature dimension).
[0040] Traditional spatial domain feature extraction processes typically involve directly calculating point-to-point Euclidean distances or angles in the spatial domain, or using models such as RNNs and LSTMs to capture sequence dependencies. However, these methods are sensitive to noise and local perturbations and are difficult to effectively capture long-range dependencies and global shape features.
[0041] Compared to the traditional methods described above, this example employs a Fourier encoder to perform Fourier transforms on both the navigation path sequence and the vehicle's actual trajectory sequence. The sequence data (a series of path points or trajectory points) in the spatial domain is transformed to the frequency domain for feature extraction, determining the corresponding navigation frequency domain features and trajectory frequency domain features. These features are then encoded to generate navigation-coded features E_nav and trajectory-coded features E_traj, which contain their global structure and periodic patterns. In this example, the Fourier encoder completes feature extraction from the spatial domain to the frequency domain. Its low-pass filtering effectively smooths high-frequency trajectory disturbances caused by sensor noise, ground bumps, or handling jitter. Furthermore, its global nature helps the model focus more on the overall direction and intent of the path and trajectory, rather than local detail deviations. Subsequent consistency judgments provide a more robust and higher-level feature representation, forming the basis for improving yaw detection accuracy.
[0042] In one embodiment, step S103, namely, fusing multimodal information from navigation coding features, trajectory coding features, and local map features to determine the fused features, includes: S401: A cross-attention mechanism is used to perform cross-attention calculations on navigation coding features, trajectory coding features, and local map features to determine multiple associated features and multiple differential features; S402: Merge multiple related features and multiple differential features to determine the fused features.
[0043] As an example, in step S401, the device may use a cross-attention mechanism to perform cross-attention calculation on the three features: navigation coding feature E_nav, trajectory coding feature E_traj, and local map feature E_map, to determine multiple associated features and multiple differential features between the navigation coding feature and the trajectory coding feature. The associated features are used to characterize the correlation between the navigation coding feature and the trajectory coding feature, and the differential features are used to characterize the differences between the navigation coding feature and the trajectory coding feature.
[0044] In this example, the core computation process of the cross-attention mechanism can be simplified to Attention(Q, K, V) = Where Q (Query) is the query term for which output needs to be calculated, K (Key) is the key that matches the query term, and V (Value) is the actual value corresponding to the key. The dimension of the key is used to scale the dot product result. In this example, either the navigation encoding feature E_nav or the trajectory encoding feature E_traj can be used as the query. The other can be combined with the local map feature E_map to determine the key and value for attention computation. This process aims to uncover multiple associated and dissimilar features between the navigation encoding feature E_nav and the trajectory encoding feature E_traj, ensuring that these features effectively reflect the differences between the target navigation path and the vehicle's actual trajectory. The local map feature E_map serves as the context-injected attention computation, providing environmental constraint information (specifically, spatial constraint semantics, such as lane lines or traffic rules), which helps ensure the accuracy of the multiple associated and dissimilar features it calculates.
[0045] As an example, in step S402, the device can perform feature fusion on multiple associated features and multiple differential features to obtain fused features. The fused features can be represented by F_fused, where F_fused ∈ R^d. In this example, multiple associated features and multiple differential features can be directly concatenated, or information can be exchanged in a neural network model through pre-set interaction logic to determine the corresponding fused features.
[0046] Traditional rule-based methods or simple feature splicing cannot effectively model the complex and nonlinear interactions between paths, trajectories, and the environment. The degree of information fusion is shallow, and "information silos" are easily formed.
[0047] In this example, a cross-attention mechanism is used to calculate the cross-attention of navigation coding features, trajectory coding features, and local map features, identifying multiple related features and multiple differential features. The related features reflect the correlation between the input features such as navigation coding features, trajectory coding features, and local map features, while the differential features reflect the differences between these input features. These related and differential features are then fused to determine a fused feature. This fused feature contains key features for deep information interaction among the three input features, helping to ensure subsequent deviation accuracy. This fusion method enables the determination of whether the vehicle's actual trajectory is executing the navigation intent of the target navigation path in a specific map environment, thus providing rich and highly correlated feature evidence for accurate deviation judgment and improving the model's understanding ability in complex intersections, roundabouts, and other scenarios.
[0048] In one embodiment, the associated features include navigation associated features and trajectory associated features. The navigation associated features are the features obtained by residual connection of navigation matching features and navigation coding features, and the trajectory associated features are the features obtained by residual connection of trajectory matching features and trajectory coding features. The navigation matching features are the features related to the navigation coding features matched from the trajectory coding features and local map features. The difference features include a first difference feature and a second difference feature. The first difference feature is a feature determined by distance calculation of navigation matching features and trajectory coding features, and the second difference feature is a feature determined by distance calculation of trajectory matching features and navigation coding features.
[0049] As an example, the device can perform a first round of cross-attention calculation on the navigation coding feature E_nav, the trajectory coding feature E_traj, and the local map feature E_map to determine the navigation matching feature Attn1. The navigation matching feature Attn1 is a feature related to the navigation coding feature E_nav that is matched from the trajectory coding feature E_traj and the local map feature E_map. Specifically, it can be calculated using the following formula: Attn1 = CrossAttention(Q=E_nav, K=E_traj+E_map, V=E_traj+E_map).
[0050] As an example, the device can perform a second round of cross-attention calculation on the navigation coding feature E_nav, the trajectory coding feature E_traj, and the local map feature E_map to determine the trajectory matching feature Attn2. The trajectory matching feature Attn2 is the feature related to the trajectory coding feature E_traj matched from the navigation coding feature E_nav and the local map feature E_map. Specifically, it can be calculated using the following formula: Attn2 = CrossAttention(Q=E_traj, K=E_nav+E_map, V=E_nav+E_map).
[0051] As an example, the device can perform residual join calculations on the navigation matching feature Attn1 and the navigation coding feature E_nav to determine the navigation associated feature E_nav', i.e., E_nav' = E_nav + Attn1; and the device can perform residual join calculations on the trajectory matching feature Attn2 and the trajectory coding feature E_traj to determine the trajectory associated feature E_traj', i.e., E_traj' = E_traj + Attn2. In this example, the navigation associated feature E_nav' and trajectory associated feature E_traj' obtained from the residual join calculations can enhance the consistency based on the constraints of the local map feature E_map.
[0052] As an example, the device can perform distance calculations on the navigation matching feature Attn1 and the trajectory coding feature E_traj to determine a first difference feature. This first difference feature can be, but is not limited to, the cosine distance between the navigation matching feature Attn1 and the trajectory coding feature E_traj. Furthermore, the device can perform distance calculations on the trajectory matching feature Attn2 and the navigation coding feature E_nav to determine a second difference feature. This second difference feature can be, but is not limited to, the L2 distance between the trajectory matching feature Attn2 and the navigation coding feature E_nav. In this example, the first and second difference features quantify the deviation between the expected navigation coding feature E_nav and the actual trajectory coding feature E_traj.
[0053] In this example, a cross-attention mechanism is used to mine navigation association features E_nav', trajectory association features E_traj', first difference features, and second difference features between navigation coding features E_nav and trajectory coding features E_traj. These features can effectively reflect the differences between the target navigation path and the vehicle's actual trajectory. Since the above analysis process introduces local map features E_map for constraints, it helps to ensure the accuracy of the determined association features and difference features.
[0054] In one embodiment, such as Figure 5 As shown, step S104, which involves yaw detection based on fused features and determining the yaw detection result, includes: S501: The decoder is used to decode the fused features to determine the yaw probability; S502: When the yaw probability is greater than the preset probability, the yaw detection result is determined to be yaw; S503: When the yaw probability is not greater than the preset probability, the yaw detection result is determined to be no yaw.
[0055] The preset probability is a pre-set threshold used to assess whether the yaw probability has reached a large standard. For example, the preset probability can be set to 0.5 or other values.
[0056] As an example, the device can use a pre-configured decoder to decode the fused feature F_fused, and the decoder outputs the decoded feature value. Then, the decoded feature value is processed using, but not limited to, a Sigmoid function to map it to a value in the range [0,1], which is then determined as the yaw probability P(yaw) of the vehicle yaw, i.e., P(yaw) = Sigmoid(Decoder(F_fused)). In this example, the decoder is pre-configured as a lightweight decoder, which can be, but not limited to, a multilayer perceptron (MLP) or a linear classifier, significantly reducing computational resource requirements while maintaining performance.
[0057] As an example, the device can compare the yaw probability with a preset probability. When the yaw probability is greater than the preset probability, it determines that its yaw detection result is yawed, that is, the vehicle is highly likely to deviate from the target navigation path. Conversely, when the yaw probability is not greater than the preset probability, it determines that its yaw detection result is not yawed, that is, the vehicle is highly likely to maintain normal driving based on the target navigation path.
[0058] In this embodiment, the yaw probability is determined by decoding the fused features, and the yaw detection result is determined by comparing the yaw probability with the preset probability. The entire architecture is end-to-end, from the input of actual measured data to the output of yaw detection results. There is no need to manually design complex rules, and the yaw detection results can be quickly output through simple comparison.
[0059] In one embodiment, such as Figure 6 As shown, after step S104, i.e., after yaw detection based on fused features and determining the yaw detection result, the yaw detection method further includes: S601: When there are anomalies in the yaw detection results, determine the yaw label corresponding to the fused features; S602: Based on the fusion features and yaw labels, an optimized sample is formed. The decoder is then optimized and updated based on the optimized sample to determine the optimized decoder.
[0060] As an example, in step S601, when the device detects an anomaly in the yaw detection result corresponding to the fused feature—that is, when the yaw detection result does not match the actual situation—it needs to determine the yaw label corresponding to the fused feature based on the actual situation. For example, if the vehicle's actual trajectory deviates from the target navigation path, and its yaw detection result is "no yaw," it indicates that the yaw probability obtained by the decoder is incorrect, resulting in a "missed detection." In this case, it is necessary to determine the yaw label of the fused feature as "yaw present." As another example, if the vehicle's actual trajectory follows the target navigation path, and its yaw detection result is "yaw present," it indicates that the yaw probability obtained by the decoder is incorrect, resulting in a "false detection." In this case, it is necessary to determine the yaw label of the fused feature as "no yaw present."
[0061] As an example, in step S602, the device will form optimized samples based on fused features and yaw labels, and add the optimized samples to the training set corresponding to the decoder to form an enhanced training set. Then, the device will periodically or online adjust the model parameters of the decoder based on the samples in the training set (including existing samples and optimized samples) so that the optimized decoder can adapt to new and unseen yaw scenarios, thereby realizing continuous learning and model iteration of the decoder.
[0062] Compared with traditional methods that require manual rule updates, this scheme can form optimized samples for the decoder based on the fusion features and their corresponding yaw labels when the yaw detection results are abnormal. This allows the decoder to learn from the optimized samples in the future, significantly reducing long-term maintenance costs and iteration difficulties, and enabling the decoder's model performance to continuously improve over time.
[0063] The yaw detection method provided in this application includes: determining the navigation path sequence, the vehicle's actual trajectory, and the local map features corresponding to the area where the vehicle is located, based on the target navigation path, the vehicle's actual trajectory, and high-precision map information; using a Fourier encoder to encode the navigation path sequence and the vehicle's actual trajectory sequence to determine the navigation coding features corresponding to the navigation path sequence and the trajectory coding features corresponding to the vehicle's actual trajectory sequence; using a cross-attention mechanism to fuse the navigation coding features, trajectory coding features, and local map features into multimodal information to determine the fused features; using a decoder to decode the fused features, performing yaw detection based on the decoding results, and determining the yaw detection result; and, when the yaw detection result is abnormal, optimizing and training the decoder based on the fused features and their corresponding yaw labels to obtain an optimized decoder. Overall, this method has the following beneficial effects: Firstly, it significantly improves detection accuracy and robustness.
[0064] The Fourier encoder transforms the navigation path sequence and the vehicle's actual trajectory sequence from the spatial domain to the frequency domain for feature extraction. Its low-pass filtering effectively smooths high-frequency trajectory disturbances caused by sensor noise, ground bumps, or handling jitter, allowing the model to focus more on the overall direction and macroscopic intent of the path and trajectory, rather than irrelevant local details. This provides a more stable and fundamental feature foundation for judgment. The cross-attention mechanism dynamically calculates the correlation weights between navigation-encoded features, trajectory-encoded features, and local map features, improving the accuracy of yaw detection using the final determined fused features. For example, at intersections, the model learns to pay more attention to the matching degree between the steering intent of the navigation path and the vehicle's actual turning action, and makes comprehensive decisions by referring to map semantics such as lane lines, rather than simply relying on fixed distance or angle thresholds. This deep, context-aware information fusion greatly improves the accuracy of judgment. Yaw detection is based on fusion features to accurately determine whether a vehicle has veered off course as intended. It exhibits higher robustness, especially in scenarios where traditional rule-based methods are prone to failure, such as complex intersections, tunnels, urban canyons, severe weather (such as rain, snow, and fog), and drastic changes in lighting. It significantly reduces the false positive and false negative rates.
[0065] Secondly, it enhances the coverage of complex and long-tail scenarios.
[0066] End-to-end data-driven models do not rely on manually preset fixed rules. Instead, they are trained on a large amount of diverse data (including a large amount of collected yaw scenario data) to learn the underlying patterns and characteristics of yaw. This means that as long as the data covers or approximately covers a certain scenario, the model has the potential to handle it, avoiding the passive situation and inevitable blind spots of rule-based methods that "create a rule for each case". In this example, the model supports continuous learning. When the system encounters new yaw cases in the real world that it has not handled correctly, these difficult samples can be included in the training set for iterative optimization of the model. This allows the model's performance to continuously evolve over time and with the accumulation of data, proactively adapting to new scenarios that may arise in the future, fundamentally solving the problems of rigidity and difficulty in updating traditional rule systems. In summary, optimizing the decoder model based on fused features and new yaw labels makes the optimized decoder more adaptable and capable of judging various rare, extreme, or unforeseen "long-tail" yaw scenarios (such as temporary construction detours, special traffic control, extreme obstacle avoidance behaviors, etc.) during autonomous driving.
[0067] Third, it has a high degree of automation, which greatly reduces the cost of manual intervention and maintenance.
[0068] Traditional methods require experienced engineers to manually design and debug numerous geometric rules, logical judgments, and thresholds (such as lateral distance tolerance and angle tolerance) for different scenarios. This process is time-consuming and labor-intensive, and conflicts between rules are prone to occur, resulting in extremely high debugging costs. This application provides an end-to-end model that transforms the task of "rule formulation" into "model training." Most of the work is automated through data collection, annotation, and model training processes, greatly improving development efficiency. Moreover, in later maintenance, when faced with new scenario problems, there is no need for manual code analysis and rule modification. Simply injecting new data and retraining the model achieves automated and sustainable operation and maintenance, significantly reducing long-term costs. In summary, the method provided in this application can greatly reduce the large amount of expert manpower required for yaw detection rule development, debugging, verification, and later maintenance, thereby reducing the overall development and long-term operation and maintenance costs of the system.
[0069] Fourth, real-time processing and computational efficiency optimization.
[0070] Fourier transform is an efficient and mature algorithm with manageable computational complexity. Furthermore, in cross-attention mechanisms, sparsity operations (such as Top-K filtering) can be introduced to actively ignore weakly correlated connections between features, concentrating computational resources on feature associations most likely to contain key information, thereby reducing unnecessary computation and improving inference speed. This optimization is particularly important for deploying end-to-end models on computationally limited in-vehicle platforms. While ensuring accuracy, model design helps control computational complexity, meeting the stringent real-time requirements of autonomous driving systems.
[0071] This application also provides an electronic device 700, such as... Figure 7 As shown, it includes a memory 701 and a processor 702, wherein the memory 701 is used to store computer programs; the processor 702 is used to execute the programs stored in the memory 701 to implement the yaw detection method described in any embodiment of this application.
[0072] This application also provides a vehicle including the electronic device 700 described in the above embodiments.
[0073] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the yaw detection method described in any embodiment of this application.
[0074] In this application, "multiple" refers to two or more.
[0075] In this application, unless otherwise expressly defined, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.
[0076] The terms “first,” “second,” “third,” “fourth,” etc., in this application (if present) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0077] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0078] Unless otherwise specified, all steps in this application may be performed sequentially or randomly. For example, if the method includes steps A and B, it means that the method may include steps A and B performed sequentially, or it may include steps B and A performed sequentially. For example, if the method may also include step C, it means that step C may be added to the method in any order. For example, the method may include steps A, B, and C, or it may include steps A, C, and B, or it may include steps C, A, and B, etc.
[0079] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A yaw detection method, characterized in that, include: Obtain navigation path sequences, vehicle real trajectory sequences, and local map features corresponding to the area where the vehicle is located; The navigation path sequence and the vehicle's actual trajectory sequence are feature-encoded to determine the navigation coding features corresponding to the navigation path sequence and the trajectory coding features corresponding to the vehicle's actual trajectory sequence; Multimodal information fusion is performed on the navigation coding features, the trajectory coding features, and the local map features to determine the fused features; Yaw detection is performed based on the fused features to determine the yaw detection result.
2. The method according to claim 1, characterized in that, The acquisition of the navigation path sequence, the vehicle's actual trajectory sequence, and the local map features corresponding to the area where the vehicle is located includes: The target navigation path and the vehicle's actual trajectory are time-aligned and normalized to determine the navigation path sequence corresponding to the target navigation path and the vehicle's actual trajectory sequence corresponding to the vehicle's actual trajectory. Feature extraction is performed on high-precision map information to determine the local map features of the area where the vehicle is located.
3. The method according to claim 1, characterized in that, The step of performing feature encoding on the navigation path sequence and the vehicle's actual trajectory sequence to determine the navigation encoding features corresponding to the navigation path sequence and the trajectory encoding features corresponding to the vehicle's actual trajectory sequence includes: A Fourier encoder is used to perform Fourier transform on the navigation path sequence and the vehicle's actual trajectory sequence to determine the navigation frequency domain features corresponding to the navigation path sequence and the trajectory frequency domain features corresponding to the vehicle's actual trajectory sequence. The navigation frequency domain features and the trajectory frequency domain features are feature-encoded to determine the navigation coding features corresponding to the navigation path sequence and the trajectory coding features corresponding to the vehicle's actual trajectory sequence.
4. The method according to claim 1, characterized in that, The process of fusing multimodal information from the navigation coding features, the trajectory coding features, and the local map features to determine the fused features includes: A cross-attention mechanism is used to perform cross-attention calculation on the navigation coding features, the trajectory coding features, and the local map features to determine multiple associated features and multiple differential features; Multiple related features and multiple differential features are fused to determine the fused feature.
5. The method according to claim 4, characterized in that, The associated features include navigation associated features and trajectory associated features. The navigation associated features are the features obtained by residually connecting the navigation matching features and the navigation coding features. The trajectory associated features are the features obtained by residually connecting the trajectory matching features and the trajectory coding features. The navigation matching features are features related to the navigation coding features matched from the trajectory coding features and the local map features. The difference features include a first difference feature and a second difference feature. The first difference feature is a feature determined by distance calculation of the navigation matching feature and the trajectory coding feature. The second difference feature is a feature determined by distance calculation of the trajectory matching feature and the navigation coding feature.
6. The method according to claim 1, characterized in that, The step of performing yaw detection based on the fused features and determining the yaw detection result includes: The fused features are decoded using a decoder to determine the yaw probability; When the yaw probability is greater than a preset probability, the yaw detection result is determined to be yawed; When the yaw probability is not greater than a preset probability, the yaw detection result is determined to be no yaw.
7. The method according to claim 6, characterized in that, After determining the yaw detection result based on the fused features, the yaw detection method further includes: When the yaw detection result is abnormal, the yaw label corresponding to the fused feature is determined; An optimized sample is formed based on the fusion features and the yaw label. The decoder is then optimized and updated based on the optimized sample to determine the optimized decoder.
8. An electronic device, characterized in that, Including processor and memory, among which, Memory, used to store computer programs; A processor for executing a program stored in memory to implement the yaw detection method according to any one of claims 1-7.
9. A vehicle, characterized in that, It includes the electronic device as described in claim 8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the yaw detection method according to any one of claims 1-7.