Vehicle abnormality early warning method and vehicle

Abstract

This application relates to the field of vehicle control technology, and in particular to a vehicle anomaly warning method and a vehicle. The method includes: acquiring road surface perception data and vehicle state data during vehicle operation; performing target object region extraction processing on the road surface perception data to obtain road surface perception data containing the target object region; inputting the road surface perception data containing the target object region into an object segmentation model to analyze the geometric properties of the target object and obtain its structural features; based on the object structural features, performing multimodal data fusion analysis combined with vehicle state data to obtain the object's sharpness; inputting the vehicle state data into an anomaly recognition model to analyze abnormal vehicle state data and obtain vehicle anomaly state features; and determining the vehicle anomaly warning result based on the vehicle anomaly state features, the object's sharpness, and the vehicle state data. This ensures that the obtained anomaly warning result is more accurate, providing early and precise warnings of potential mechanical damage to the vehicle.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a vehicle anomaly warning method and a vehicle. Background Technology

[0002] In the existing technology, traditional vehicle anomaly detection methods mainly rely on threshold judgment of a single sensor or simple fusion of multiple signals based on preset rules.

[0003] The current method, which simply integrates sensor detection results for judgment, is difficult to adapt to complex road conditions and diverse solid obstacle shapes. It is prone to detection errors or delays due to complex road conditions. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a vehicle anomaly warning method and vehicle to solve the technical problem that current vehicle anomaly detection relies on data from a single sensor, which is prone to detection errors or delays due to complex road conditions.

[0005] To achieve the above objectives, this application provides a vehicle anomaly warning method, the method comprising:

[0006] Acquire road perception data and vehicle status data while the vehicle is in motion; The road surface perception data is processed to extract the target object region, resulting in road surface perception data containing the target object region. The road surface perception data containing the target object region is input into the object segmentation model to analyze the geometric properties of the target object and obtain the object structure features of the target object. Based on the structural features of the object, multimodal data fusion analysis is performed in conjunction with the vehicle state data to obtain the sharpness of the object; The vehicle status data is input into the anomaly identification model, and the abnormal vehicle status data is analyzed to obtain the vehicle abnormal status characteristics. Based on the abnormal vehicle state characteristics, the sharpness of the object, and the vehicle state data, the vehicle abnormality warning result is determined.

[0007] Based on the same inventive concept, this application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0008] Based on the same inventive concept, this application also provides a vehicle including the above-mentioned electronic devices.

[0009] As can be seen from the above, the vehicle anomaly warning method and vehicle provided in this application, by acquiring road perception data and vehicle status data during vehicle operation, achieve simultaneous collection of external environmental risks and the vehicle's own operating status data, providing a multimodal data foundation for comprehensive causal correlation analysis. By extracting target object regions from the road perception data, road perception data containing target object regions is obtained, enabling precise location and preliminary screening of key road targets that may pose a mechanical impact risk. By inputting the road perception data containing target object regions into an object segmentation model, the geometric attributes of the target objects are analyzed, and the object structural features are obtained, achieving quantitative geometric analysis of solid road obstacles. This allows for the accurate extraction of key structural attributes such as shape and contour, transforming unstructured visual perception into calculable geometric risk indicators. Based on the object structural features, multimodal data fusion analysis is performed using vehicle status data to obtain the object's sharpness. By integrating external environment and vehicle dynamics, a dynamic and precise quantitative assessment of the potential mechanical damage risk of road obstacles is achieved. The vehicle status data is then input into the anomaly recognition model to analyze the abnormal vehicle status data and obtain vehicle anomaly status characteristics. This allows for the extraction of intrinsic feature patterns representing potential or already occurring anomalies from complex time-series data. In this way, based on vehicle anomaly status characteristics, object sharpness, and vehicle status data, vehicle anomaly warning results are determined, and vehicle anomalies are correlated with specific external risk events, significantly improving the accuracy, interpretability, and targeted nature of the detection results and warnings. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating the vehicle anomaly warning method according to an embodiment of this application; Figure 2 This is a schematic diagram of the vehicle anomaly warning device according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0013] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0014] CAN: Controller Area Network. The text mentions a vehicle bus type used for data transmission. As an in-vehicle network standard, it is used for real-time, reliable communication between electronic control units (ECUs) within the vehicle.

[0015] RGB: Red, Green, Blue. This refers to the image data format captured by a camera. It's an additive color model that uses different intensities of the red, green, and blue color channels to create a color image.

[0016] YOLO v5: You Only Look Once version 5, the article mentions a neural network model for image object detection. It's a single-stage object detection algorithm, known for its speed and high accuracy, used to quickly locate objects in images.

[0017] DBSCAN: Density-Based Spatial Clustering of Applications with Noise. The paper mentions an algorithm for clustering point cloud data. It's a density-based clustering algorithm capable of discovering clusters of arbitrary shapes and effectively identifying and handling noisy points (outliers).

[0018] IMU: Inertial Measurement Unit. The text mentions sensors used to measure the motion of a vehicle. These typically include accelerometers and gyroscopes, used to measure the angular velocity and acceleration of an object, and then to calculate its attitude, velocity, etc.

[0019] GNSS: Global Navigation Satellite System. The text mentions a system used for vehicle positioning.

[0020] PCA: Principal Component Analysis. The text mentions algorithms used for boundary point extraction or orientation fitting. It's a statistical method that transforms a set of potentially correlated variables into a set of linearly uncorrelated variables (principal components) through orthogonal transformation, often used for dimensionality reduction, feature extraction, and data compression.

[0021] RANSAC: Random Sampling Consensus. The paper mentions an algorithm for fitting curves / surfaces in three-dimensional space. It's an iterative method for estimating mathematical model parameters from a set of observations containing "outliers" (noise, outliers).

[0022] MLP: Multilayer Perceptron. The text mentions a neural network model used for feature extraction or decision-making. It's a feedforward artificial neural network composed of multiple layers of neurons, with each layer fully connected to the next, capable of learning complex nonlinear relationships.

[0023] ReLU: Rectified Linear Unit. The text mentions the activation function used in neural networks. A commonly used nonlinear activation function is defined as f(x) = max(0, x), used to introduce nonlinearity into neural networks.

[0024] GELU: Gaussian Error Linear Unit. It's an activation function mentioned in the text, used in neural networks. It's a smoother activation function than ReLU, based on a Gaussian distribution, and commonly used in advanced models such as the Transformer.

[0025] FFN: Feed-Forward Network. It's mentioned in the text as a component of the Transformer architecture. It typically refers to a sub-layer in a Transformer encoder or decoder, consisting of two linear transformations and an activation function, used to independently process the representation at each location.

[0026] AUC stands for Area Under the Curve. It's a model evaluation metric mentioned in the text. Specifically, it refers to the area under the ROC (Receiver Operating Characteristic Curve) curve, used to evaluate the performance of binary classification models. A value closer to 1 indicates better model performance.

[0027] ROC: Receiver Operating Characteristic. The text mentions a curve related to model evaluation. This curve, with the false positive rate on the horizontal axis and the true positive rate on the vertical axis, visually demonstrates the classification model's performance at different decision thresholds.

[0028] MSE: Mean Squared Error. The loss function mentioned in the text for regression tasks. A commonly used metric to measure the difference between model predictions and actual values, calculated as the average of the squared errors.

[0029] MAE: Mean Absolute Error. A metric mentioned in the text for evaluating regression tasks. It measures the average absolute difference between the model's predicted values ​​and the actual values.

[0030] RMSE: Root Mean Squared Error. This is an evaluation metric mentioned in the text for regression tasks. The square root of the mean squared error has the same dimensions as the original data and is often used to assess the magnitude of prediction error.

[0031] TPMS: Tire Pressure Monitoring System. The text mentions the source of tire pressure data. It is an electronic system used to monitor tire pressure and temperature in real time.

[0032] ECU: Electronic Control Unit. The vehicle control module mentioned in the text. An embedded system in a vehicle used to control one or more electrical systems or subsystems.

[0033] GPS: Global Positioning System. One of the satellite navigation systems mentioned in the text (belonging to the GNSS category). A global satellite navigation system operated by the United States.

[0034] LSTM: Long Short-Term Memory. A type of recurrent neural network mentioned as background information. A special type of RNN capable of learning long-term dependencies, often used for processing time-series data.

[0035] ARIMA: Autoregressive Integrated Moving Average.

[0036] DeepAR: Deep Autoregressive.

[0037] OTA: Over-The-Air. A method of remote data transmission. It typically refers to the wireless transmission and updating of software, data, or configurations via mobile networks.

[0038] Adam: Adaptive Moment Estimation. An iterative optimization algorithm for training deep learning models, combining the advantages of momentum estimation and RMSprop.

[0039] RBF-NN: Radial Basis Function Neural Network. A feedforward neural network that uses radial basis functions as activation functions, often used for function approximation and classification.

[0040] AUC (Area Under the Curve) is a model evaluation metric mentioned in the paper. Specifically, it refers to the area under the ROC curve (Receiver Operating Characteristic Curve) and is used to evaluate the performance of binary classification models. The closer the value is to 1, the better the model performance.

[0041] In related technologies, vehicle anomaly detection methods mainly rely on setting static thresholds for single sensor signals (such as vibration and temperature) or simple fusion of multiple signals based on fixed rules. These methods are poorly adaptable to complex driving environments and diverse obstacle morphologies, making it difficult to accurately distinguish between disturbances caused by normal road bumps and steering maneuvers and real anomalies caused by impacts from solid road obstacles (such as stones or metal fragments). Furthermore, existing methods typically treat road risk and vehicle status as independent factors, lacking effective modeling of the dynamic relationship between the two. This results in delayed responses to early, subtle potential impacts, high false alarm rates, and an inability to accurately trace the source of risk events, impacting the real-time performance, accuracy, and early warning value of the detection system.

[0042] To improve anomaly detection capabilities, some existing solutions have introduced visual sensors to identify road obstacles or combined vehicle status data for simple rule-based judgments. However, these methods typically only classify or roughly locate obstacles, failing to provide a refined and quantifiable assessment of their specific geometric attributes (such as sharpness). Even when status data is incorporated, it is mostly based on post-hoc comparisons, failing to achieve a deep fusion analysis of external threats and internal responses from the perspective of physical interaction and risk evolution. Therefore, in complex and ever-changing real-world road conditions, existing methods have weak generalization capabilities and struggle to provide reliable early warnings for vehicle anomalies caused by obstacles of specific geometric shapes that exhibit progressive or delayed characteristics.

[0043] One intuitive approach to improvement is to increase the types and number of sensors and establish a more complex expert rule base. However, this would drastically increase system complexity, and the rules would struggle to cover all potential interaction scenarios. Another solution is to attempt to build a physics-based vehicle-road interaction simulation model. However, such models heavily rely on precise parameters, are computationally complex, and are difficult to meet the needs of real-time vehicle diagnostics.

[0044] Based on this, this application provides a vehicle anomaly detection scheme, which aims to solve how to accurately quantify external impact risks and identify abnormal vehicle states in complex driving environments by performing refined geometric analysis of solid obstacles on the road surface, deep temporal modeling of the vehicle's own state, and dynamic multimodal fusion between the two. Ultimately, it aims to effectively correlate external risk events with internal anomaly representations, thereby improving detection accuracy, robustness, and interpretability of early warnings.

[0045] The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0046] The vehicle anomaly warning method proposed in this application embodiment, such as Figure 1 As shown, it includes: S101, acquire road perception data and vehicle status data while the vehicle is in motion.

[0047] In practice, road perception data can be simultaneously acquired by an onboard forward-facing camera and a LiDAR. The camera provides high-resolution RGB images, while the LiDAR generates 3D point cloud data for the corresponding area. Vehicle status data is acquired in real time via the CAN bus, including timing signals such as vehicle speed, steering angle, longitudinal / lateral acceleration, wheel speed pulse signals, and drive motor current / voltage. During the data preprocessing stage, the two types of data are synchronized spatiotemporally through timestamp alignment and coordinate system unification. Image data is sampled at a preset frame rate, point cloud data is downsampled through voxel filtering, and CAN signals are filtered using a moving average to eliminate high-frequency noise, providing a stable and reliable data source for subsequent processing.

[0048] S102, the target object region is extracted from the road surface perception data to obtain road surface perception data containing the target object region.

[0049] In practice, the target object is a solid object with a specific shape, and it may have sharp parts that could affect vehicle movement. The target object region is an area containing one or more target objects. The road perception data includes image-point cloud paired data obtained from cameras and / or lidar detectors.

[0050] First, the RGB images captured by the camera can be initially identified using the YOLOv5 object detection network to quickly locate candidate regions with regular hard contours (such as stones, bricks, metal parts, etc.). Simultaneously, the LiDAR point cloud data can be processed using a density-based spatial clustering algorithm (DBSCAN) to separate the ground point cloud from outlier clusters. Next, using a calibrated camera-LiDAR extrinsic parameter matrix, the candidate regions in the image and outlier clusters in the point cloud are correlated and fused for verification, eliminating soft interference targets such as vegetation shadows and water stains. Finally, for regions identified as target objects, corresponding bounding boxes are generated on the image, and corresponding sub-point clouds are extracted from the point cloud, forming "image-point cloud" paired data containing precise spatial location and appearance information. This data serves as input for subsequent processing. The target object is a hard foreign object that poses a potential risk of puncture, cut, or impact damage to vehicle tires, including but not limited to stones, broken glass, metal parts, and construction debris.

[0051] S103, the road surface perception data containing the target object region is input into the object segmentation model to analyze the geometric properties of the target object and obtain the object structure features of the target object.

[0052] In practice, the obtained image-point cloud pairing data is input into a pre-trained object segmentation model. This model can use the PointNet++ 3D instance segmentation architecture, supplemented by 2D image features as prior guidance. First, semantic segmentation and instance segmentation are performed on the input target object point cloud, generating a corresponding point-level mask, a precise set of edge points, and the centroid coordinates in 3D space for each independent target object instance. For each segmented target object, based on its point cloud mask and edge point set, the local curvature distribution, edge angle change rate, and boundary point spatial density of its surface in 3D space are calculated to obtain preliminary geometric structural features describing its sharpness and irregularity. Subsequently, cross-frame tracking is performed using synchronously acquired multi-frame continuous data. Combined with the Kalman filter algorithm, the geometric features of a single frame are smoothed and correlated in the time dimension to form a stable feature sequence. Finally, this feature sequence is integrated with the centroid motion trajectory to output a structural feature vector containing the stable 3D shape, dynamic change trend, and spatial position of the target object.

[0053] S104. Based on the structural features of the object, multimodal data fusion analysis is performed in conjunction with the vehicle state data to obtain the sharpness of the object.

[0054] In practice, based on the obtained object structural features (including edge curvature, centroid trajectory, etc.) and the acquired real-time vehicle state data (especially vehicle velocity vector and frontal attitude angle), cross-frame tracking is first performed on the same target object. During cross-frame tracking, the following quantitative analysis is performed on each target object: the distribution density of local curvature extreme points, the average radius of curvature, and the frequency of edge line turning angle changes are calculated from its edge point set to form a morphological structural index characterizing its physical sharpness. Simultaneously, combined with the relative velocity vector between the vehicle and the object, the three-dimensional spatial angle between the vehicle's forward direction vector and the direction vector of the "potential impact ridge" fitted through the minimum curvature point set is calculated. Finally, the morphological index is used as the basic structural value, and this spatial angle (the smaller the angle, the greater the impact risk) is used as a correction coefficient to dynamically weight the basic structural value. After normalization, a continuous value between 0 and 1 is output, which represents the quantified sharpness of the target object in the current relative motion state.

[0055] S105, input the vehicle status data into the anomaly recognition model, analyze the abnormal vehicle status data, and obtain the vehicle abnormal status characteristics.

[0056] In practice, the preprocessed vehicle state data (vehicle speed, steering angle, acceleration, etc.) is slide-segmented according to a fixed time window length to form a data sample sequence with temporal continuity. These sequences are then input into a pre-trained anomaly detection model, which can be constructed using a Transformer encoder architecture. The anomaly detection model first performs linear transformation and positional encoding on the input multidimensional state data, converting it into an embedded vector sequence with temporal positional information. Subsequently, through a multi-head self-attention mechanism layer, the model calculates the dependencies between any two time points in the sequence in parallel, thereby capturing long-distance temporal patterns and associated features inherent in state changes. Based on this, the anomaly detection model further performs nonlinear transformation and feature extraction through a feedforward neural network layer, ultimately aggregating the contextual information of the entire sequence and outputting a high-dimensional feature vector that comprehensively characterizes whether the vehicle's operating state within the current window exhibits anomalies and their patterns—that is, the vehicle anomaly state feature.

[0057] S106, Based on the abnormal vehicle state characteristics, the sharpness of the object, and the vehicle state data, determine the vehicle abnormality warning result.

[0058] In practice, the extracted vehicle anomaly features, calculated object sharpness, and real-time vehicle state data (such as speed and acceleration) are concatenated and fused at the feature level to form a joint feature vector that comprehensively represents external risk, internal state, and immediate operating conditions. This joint feature vector is standardized to eliminate the influence of dimensions before being input into a lightweight multilayer perceptron network. This lightweight multilayer perceptron network first performs nonlinear transformation and deep feature extraction on the fused features through fully connected layers and activation functions. Then, an attention mechanism layer is applied to weight the feature sub-dimensions based on their importance, focusing on information most relevant to the impact anomaly. The weighted features are mapped to a continuous vehicle anomaly risk value through a regression layer. Finally, based on a preset risk threshold range (such as low risk, medium risk, and high risk), this vehicle anomaly risk value is mapped to a specific vehicle anomaly level, and the detection result, including the anomaly level, the associated target object ID (if any), and a timestamp, is output.

[0059] The above scheme enables the acquisition of road surface perception data and vehicle status data during vehicle operation, achieving simultaneous collection of external environmental risks and vehicle operating status data, providing a multimodal data foundation for comprehensive causal correlation analysis. By extracting target object regions from the road surface perception data, road surface perception data containing target object regions is obtained, enabling precise location and preliminary screening of key road targets that may pose a mechanical impact risk. By inputting the road surface perception data containing target object regions into an object segmentation model, the geometric attributes of the target objects are analyzed to obtain their structural features, achieving quantitative geometric analysis of solid road obstacles. This allows for the accurate extraction of key structural attributes such as shape and contour, transforming unstructured visual perception into calculable geometric risk indicators. Based on the object structural features, multimodal data fusion analysis is performed using vehicle status data to obtain the sharpness of the objects. By integrating external environment and vehicle dynamics, a dynamic and accurate quantitative assessment of the potential mechanical damage risk of road obstacles is achieved. Finally, vehicle status data is input into an anomaly recognition model to analyze abnormal vehicle status data, obtaining abnormal vehicle status characteristics. This allows for the extraction of intrinsic feature patterns representing potential or already occurring anomalies from complex time-series data. In this way, based on the characteristics of abnormal vehicle conditions, the sharpness of objects, and vehicle condition data, the vehicle abnormality warning results are determined, and the vehicle abnormality is associated with specific external risk events, which significantly improves the accuracy, interpretability, and pertinence of the detection results and the pertinence of the warnings.

[0060] In some embodiments, S103 includes: S301, the road surface perception data containing the target object region is input into the object segmentation model, and the target object instance segmentation is performed on the road surface perception data containing the target object region to obtain the mask, edge point set and centroid position of each target object.

[0061] In practice, the extracted "image-point cloud" pairing data is used as input and fed into the inference pipeline of the deployed object segmentation model. First, features are extracted from the RGB image through its backbone network, and a region proposal network is used to generate candidate bounding boxes for targets. Simultaneously, the corresponding subset of point clouds is fed into the point cloud processing branch for voxelization and geometric feature learning. Then, a feature fusion layer based on an attention mechanism is used to perform cross-modal alignment and depth fusion of the semantic information (color, texture) of the image domain and the spatial geometric information (depth, shape) of the point cloud domain. Based on this, precise instance-level segmentation is performed: for the image modality, a pixel-level precise binary mask is generated for each candidate target object; for the point cloud modality, a 3D point cloud subset (i.e., a point cloud mask) corresponding to the image mask space and belonging to the same physical entity is output. Furthermore, a boundary point extraction algorithm based on principal component analysis is applied to the point cloud mask to obtain its set of edge points in 3D space; simultaneously, the mean of the 3D coordinates of all points within the point cloud mask is calculated and used as the centroid position of the target object in the current coordinate system. Ultimately, each successfully segmented target object will obtain four basic outputs: its unique image mask, 3D point cloud mask, 3D edge point set, and 3D centroid position coordinates.

[0062] S302, for each target object, based on the mask and the edge point set, determine the local curvature distribution, edge angle change rate and boundary point density of the target object edge, and obtain the first geometric structure feature corresponding to each target object.

[0063] In practice, for each target object's 3D edge point set and point cloud mask, refined geometric analysis is performed to quantify its edge morphology. First, a local surface fitting algorithm based on k-nearest neighbor search is used to calculate the local surface formed by the neighboring points of each point in the edge point set, thereby solving for the normal vector and principal curvature at that point. By statistically analyzing the distribution of principal curvature values ​​(such as mean, variance, and maximum value) of all edge points, the local curvature distribution characteristics describing the overall sharpness and irregularity of the edge are obtained. Second, edge points are connected in spatial order to form an ordered boundary contour, and the directional angle changes between adjacent contour segments are calculated. By analyzing the statistical characteristics of these angle changes (such as frequency of change and abrupt change amplitude), the tortuosity and sharpness of the edge are quantified, i.e., the edge angle change rate. Simultaneously, the distribution density of the edge point set in 3D space is calculated, i.e., the number of edge points per unit length or unit surface area. This indicator reflects the richness of edge detail and potential discontinuities. Finally, the calculated local curvature distribution (multiple statistics), edge angle change rate (multiple statistics), and boundary point density are encoded and concatenated to form a multidimensional numerical vector. This multidimensional numerical vector is the first geometric structure feature that characterizes the basic edge geometric properties of the target object.

[0064] S303, combining the road surface perception data containing the target object area, the first geometric structure feature is corrected to obtain the second geometric structure feature corresponding to each target object.

[0065] In practice, road surface perception data containing the target object area can be used to accurately correct the first geometric structure feature, thereby correcting the shape and size of the first geometric structure feature and obtaining the accurate second geometric structure feature corresponding to each target object.

[0066] S304, perform cross-frame tracking on the second geometric structure features to obtain the second geometric structure feature sequence.

[0067] In practice, the system utilizes continuous timestamps and location information from vehicle status data, combined with a target association algorithm, to perform cross-frame tracking and identity maintenance for the same target object appearing in multiple consecutive frames of perception data. The main steps of the tracking process are as follows: After acquiring the second geometric structure features and three-dimensional centroid position of a target object in the current frame, the system first performs data association in its historical trajectory database based on the centroid position, motion velocity (calculated from the centroid displacement of consecutive frames), and similarity of geometric features (such as calculating the cosine similarity between feature vectors) to determine whether it is a tracked object. If the association is successful, the ID of the target object is carried over to the current frame, and the second geometric structure features (such as multi-dimensional vectors) calculated in the current frame are used as the observation value of the target object at that moment, appended to the end of its existing feature sequence, thus forming a second geometric structure feature sequence arranged in chronological order that reflects the dynamic changes of the target object's geometric features. If no historical target is associated (i.e., a newly appearing object), a new unique ID is assigned to it, and a new feature sequence is constructed starting from the current feature as the starting point of the sequence.

[0068] Throughout the process, Kalman filtering or more advanced nonlinear filters (such as extended Kalman filtering) are employed to predict the centroid position and motion state of the object in the next frame, assisting and optimizing the cross-frame data association process. Simultaneously, the feature sequence is smoothed to reduce the impact of single-frame measurement noise. Finally, for each continuously tracked target object, a sequence of its second geometric features evolving over time is output.

[0069] S305, based on the second geometric structure feature sequence, determine the edge features of the target object, and combine the centroid position of the target object to obtain the object structure features of the target object.

[0070] In practice, a time-series analysis is performed on the "second geometric structure feature sequence" of each tracked target object. Specifically, the second geometric structure feature sequence is segmented using a sliding window. Within each segment, the statistical quantities (such as mean, variance, and extreme values) and trends of change (such as slope extracted through linear fitting or more complex time-series models) of the features are calculated. This allows for the extraction of time-series aggregated features, known as "edge features," from the dynamic evolution that better characterize the object's stability attributes and risk patterns. For example, the variance of the edge angle change rate feature sequence of a rolling stone may be large, while the local curvature distribution feature of a stationary metal fragment may remain stable.

[0071] Simultaneously, the centroid position sequence of the target object is processed to calculate its average position, trajectory (such as fitted velocity and direction), and relative positional relationship with the vehicle (such as lateral / longitudinal distance). Finally, the extracted "edge feature" vector, which incorporates temporal information, is fused with "centroid position features" (such as average 3D coordinates and relative velocity vectors) containing spatial motion information at the feature level (e.g., by splicing or through a lightweight feature fusion network) to generate a composite feature vector that comprehensively represents the target object's 3D geometric shape, temporal dynamic characteristics, and spatial position information. This is the final "object structural feature" of the target object.

[0072] The above approach utilizes a hybrid 2D-3D instance segmentation model to accurately identify target objects and extract geometric features from road perception data. Through cross-frame tracking and temporal analysis, structured features comprehensively characterize the object's three-dimensional morphology, dynamic changes, and spatial location. These structured features not only include static geometric sharpness indices but also integrate the object's motion state and relative position information. This allows the system to dynamically and quantitatively assess the potential mechanical damage risk (such as tire puncture or cut risk) of solid road obstacles. This enables it to accurately and interpretably trace the source of vehicle anomalies caused by obstacles of specific geometric shapes, providing high-precision external risk quantification input for subsequent comprehensive anomaly detection and early warning.

[0073] In practical implementation, the corresponding object segmentation model can be a hybrid 2D-3D instance segmentation model based on deep learning. Among them, a cascaded model based on the fusion of Mask R-CNN (for 2D images) and PointNet++ (for 3D point clouds) is preferred. This cascaded model can perform end-to-end joint inference and accurate instance segmentation on image and point cloud data simultaneously, and is particularly suitable for extracting the accurate three-dimensional geometric contours and spatial location information of target objects from complex road scenes.

[0074] The system collects massive amounts of road perception data (including images and corresponding point clouds) gathered by vehicles under various road and weather conditions. This data is then precisely manually annotated, including generating image-level masks, point cloud-level segmentation labels, and centroid coordinates for each target object instance, thus constructing a structured training dataset. During training, the annotated data is used to iteratively optimize the initial Mask R-CNN and PointNet++ network parameters. A joint loss function (combining bounding box regression, semantic segmentation, instance masking, and point cloud segmentation losses) is employed to simultaneously improve 2D and 3D recognition accuracy. This training process is repeated, and cross-validation is performed on independent validation sets, ultimately resulting in a highly accurate and robust object segmentation model.

[0075] Thus, after acquiring image-point cloud pairing data containing the target object region, the data is first preprocessed and aligned in the same manner as during the training phase. The object segmentation model receives this pairing data and internally extracts image features and generates 2D candidate regions and an initial mask using a MaskR-CNN network, while simultaneously extracting geometric features of the point cloud using a PointNet++ network. Then, a feature fusion module deeply fuses and calibrates the 2D semantic information and 3D spatial geometric information, performing accurate 3D instance segmentation on this basis. Finally, the model outputs the accurate 3D point cloud mask, edge point set, and 3D centroid coordinates for each segmented target object instance, and further calculates features describing its geometric attributes (such as local curvature distribution, edge angle change rate, boundary point density, etc.) using a built-in geometric analysis unit. These features collectively constitute an object structure feature vector. This object structure feature vector encodes the target object's accurate 3D morphology, spatial location, and key geometric risk attributes, providing accurate input for subsequent risk quantification assessment.

[0076] In some embodiments, S303 includes: S3031, Based on the road surface perception data, determine the three-dimensional point cloud set of the target object region.

[0077] In practice, the generated "image-point cloud" pairing data is used as the direct input for this step. The point cloud subset corresponding to the confirmed target object is read. This subset is a set of three-dimensional spatial points aligned with the image region of the target object, obtained by LiDAR acquisition and separation and association in previous steps. This point cloud set contains high-precision three-dimensional coordinate information of the target object's surface in the LiDAR coordinate system, providing a data foundation for subsequent detailed three-dimensional geometric analysis.

[0078] S3032, Based on the three-dimensional point cloud set, perform three-dimensional spatial fitting on the edge point set of the target object to obtain the three-dimensional edge contour of the target object.

[0079] In practice, the obtained 3D point cloud set is used in conjunction with the initial edge point set of the target object. A 3D curve / surface fitting technique based on moving least squares or RANSAC algorithm is employed to optimize and reconstruct this initial edge point set. Within 3D space, the algorithm smoothly fits the discrete edge points, generating a set of 3D spatial curves or surface patches that more accurately reflect the actual physical boundaries of the object, thus obtaining the optimized, continuous, and accurate 3D edge contour of the target object.

[0080] S3033, Based on the three-dimensional edge contour, determine the local curvature distribution, edge angle change rate, and boundary point density of the target object's edge.

[0081] In practice, dense sampling is performed again on the obtained optimized 3D edge contour to obtain a more accurate sequence of 3D contour points. Based on this sequence of 3D contour points, the local curvature at each sampling point is recalculated (e.g., by calculating the deflection of the chord formed by adjacent points or fitting a local arc), and its distribution is statistically analyzed. At the same time, the rate of change of the contour tangent direction in 3D space is calculated as the rate of change of the edge angle. In addition, based on the optimized contour length and the number of sampling points, a more accurate spatial density of boundary points can be calculated.

[0082] S3034, using the local curvature distribution of the target object's edge, the rate of change of the edge angle, and the density of boundary points, the first geometric structure feature is modified to obtain the second geometric structure feature.

[0083] In practice, these more accurate edge local curvature distribution statistics, edge angle change rate statistics, and boundary point density of the target object, calculated based on the optimized three-dimensional contour, are used to re-encode and fuse the first geometric structure to form a corrected multi-dimensional numerical vector that better reflects the true three-dimensional sharpness and irregularity of the object, which is the second geometric structure feature of the target object.

[0084] The above scheme can determine an accurate three-dimensional point cloud set based on road surface perception data. Since the three-dimensional point cloud set contains the three-dimensional shapes of each object, the three-dimensional edge contour of the target object obtained by fitting the three-dimensional space based on the three-dimensional point cloud set is more accurate. In this way, the three-dimensional edge contour of the target object is used to correct the local curvature distribution, edge angle change rate and boundary point density of the first set of structural features, thereby obtaining an accurate second geometric structural feature and improving the accuracy of the second geometric structural feature.

[0085] As a preferred embodiment, the training process of this object segmentation model is as follows: (1) Construct the initial object segmentation model. MaskR-CNN (for processing 2D images) and PointNet++ (for processing 3D point clouds) can be used as the dual backbones to construct a 2D-3D fusion instance segmentation architecture. The model design includes feature alignment and fusion modules, and its output design is to synchronously generate a 2D pixel mask, a 3D point cloud mask, and three-dimensional centroid coordinates for each target object instance.

[0086] (2) An initial training dataset was constructed based on the actual driving data collected from a large number of vehicles operated by automakers under various road conditions (such as urban, highway, and off-road). Key road perception data (RGB images and corresponding point clouds) and vehicle status data were extracted simultaneously through vehicle-mounted front-view cameras, LiDAR, and CAN bus. For these data samples, a professional annotation team labeled each target object instance (such as stone or metal fragment) in the image with its precise pixel-level mask, and through manual comparison and projection, labeled its point cloud-level segmentation label and three-dimensional centroid coordinates on the corresponding point cloud data. The labeled samples constituted the basic training sample set.

[0087] (3) Perform preprocessing on each image-point cloud pairing data sample. Normalize the color and scale the image; voxelize and downsample the point cloud and normalize its coordinates. Use the calibrated camera-LiDAR extrinsic parameter matrix to generate a rough correspondence between image pixels and point cloud. Use the preprocessed image, point cloud and their correspondence as the input tensor of the model.

[0088] (4) Input the constructed input tensor into the initial object segmentation model. The Mask R-CNN branch can be used to extract 2D semantic features of the image and generate candidate regions, and the PointNet++ branch can be used to extract 3D geometric features of the point cloud. The fusion module aligns and deeply fuses 2D semantic and 3D geometric information through a cross-modal attention mechanism, and performs accurate 3D instance segmentation prediction on this basis, outputting the 2D mask, 3D point cloud mask and centroid position of each instance.

[0089] (5) A joint loss function is used as the objective function for optimization. This joint loss function combines bounding box regression loss (2D branch only), semantic segmentation loss, 2D instance mask loss, 3D point cloud segmentation loss, and centroid coordinate regression loss to comprehensively supervise the multi-task learning of the model. The total loss value is calculated, and all weight parameters of the model are updated through backpropagation algorithm combined with Adam optimizer.

[0090] (6) Repeat steps (4) and (5) to train the model iteratively multiple times using the basic training sample set. After each training round, evaluate the model performance using an independent validation set that covers various weather and lighting conditions, and monitor changes in metrics such as average precision (AP) and intersection-over-union (IoU). When the model's performance on the validation set stabilizes, end this stage of training to obtain the pre-trained model.

[0091] (7) To improve the segmentation accuracy and robustness of the model under complex, harsh or extreme road conditions (such as wet and slippery road surface reflections, low light at night, and dense gravel on off-road roads), an additional challenging working condition sample set is constructed. This sample set focuses on the above-mentioned difficult scenarios and is annotated more finely.

[0092] (8) Fine-tune the pre-trained model using a challenging work condition sample set. In this stage, a significantly reduced learning rate is used for training, so that the model parameters are further refined based on the learned general patterns to improve the perception and segmentation capabilities of difficult scenarios.

[0093] (9) The fine-tuned model was finally evaluated on a completely independent comprehensive test set covering multiple seasons and time periods. Evaluation metrics included the average accuracy of various 2D / 3D instance segmentation metrics, mask quality, centroid localization error, etc. Finally, the model whose performance met the preset threshold was determined as the deployable object segmentation model.

[0094] In some embodiments, S104 includes: S401, based on the structural features of the target object, and combined with the vehicle velocity vector and vehicle front attitude angle in the vehicle state data, perform cross-frame tracking of the target object.

[0095] In practice, the three-dimensional centroid position of the target object in the current frame (derived from the object's structural features) is correlated with the historical trajectory database. During correlation, the vehicle's displacement and heading changes since the previous frame are calculated by combining the velocity vector and frontal attitude angle from the vehicle state data (calculated via IMU or GPS), thus projecting the predicted historical trajectory points onto the coordinate system of the current frame. By comparing the distance between the projected predicted position and the centroid position detected in the current frame, and combining the similarity of object structural features (such as geometric feature vectors) (e.g., calculating cosine similarity), it can be determined whether the currently detected object is a continuation of the same target. If the correlation is successful, the motion trajectory (position, velocity) of the target object is updated, and a unique tracking ID is maintained for it; if it is a newly appearing object, a new ID is assigned and its trajectory is initialized. This cross-frame tracking process ensures the consistency of each target object's identity in the time dimension, providing a continuous data foundation for subsequent analysis of its motion state and relative relationships.

[0096] S402, during cross-frame tracking, based on the set of edge points in the object's structural features, determines the local curvature distribution, edge angle change rate, and boundary point density of the target object's edge, thereby obtaining the morphological structural indices of the target object.

[0097] In practice, for the currently tracked target object, its three-dimensional edge point set is extracted from the object's structural features. First, based on this three-dimensional edge point set, a moving surface fitting algorithm (such as the k-nearest neighbor-based PCA method) is used to calculate the local principal curvature value at each edge point, and the distribution characteristics of all point principal curvature values ​​are statistically analyzed. For example, the mean, variance, maximum value, and high quantile (such as the 95th percentile) of curvature are calculated to quantify the overall sharpness and irregularity of the edges, forming a local curvature distribution index. Second, the edge points are sorted according to spatial proximity to form an ordered contour sequence, and the change in orientation angle between adjacent contour segments is calculated. The absolute values ​​of these angle changes (mean, variance, and the number of abrupt changes exceeding a specific threshold (such as 45 degrees)) are statistically analyzed to quantify the tortuosity and sharpness of the edge contour, obtaining an edge angle change rate index. Simultaneously, the number of edge points per unit length of the three-dimensional contour line is calculated, i.e., the spatial density of boundary points. Finally, the calculated local curvature distribution (multiple statistics), edge angle change rate (multiple statistics), and boundary point density are normalized and spliced ​​to form a comprehensive, multidimensional numerical vector, which is the morphological structural index characterizing the static geometric risk of the target object.

[0098] S403, combining the vehicle status data, obtain the relative motion state between the vehicle and the target object, and determine the angle between the vehicle's direction of travel and the direction of the sharp edge of the target object.

[0099] S404, Based on the morphological structural indicators, determine the basic structural values ​​of the target object.

[0100] In practice, the obtained multidimensional morphological structural index vector (including multiple statistics such as curvature distribution, rate of change of angle, and boundary point density) is input into a scoring function. This scoring function can be a pre-trained lightweight regression model (such as a linear model or a shallow neural network) or a weighted summation formula based on expert knowledge. The core idea is to comprehensively map the various indicators representing the static geometric risk of an object into a scalar score. For example, the smaller the mean curvature (sharper), the larger the variance of curvature (more prominent local extrema), the larger the mean rate of change of angle (more sharp edges), and the higher the boundary point density (more complex edges), the higher the calculated structural base value S_base, indicating that the target object has a higher potential sharpness or danger based on its own geometry.

[0101] S405, Based on the included angle, the basic structural value is corrected.

[0102] In practical implementation, dynamic interactive risks (characterized by the included angle θ) need to be integrated into static risks (characterized by S_base). To this end, a correction function f(θ) is defined. This correction function reaches its maximum value when the included angle θ is 0 (the direction of movement is completely parallel to the sharp edge), where the risk amplification effect is strongest. As θ increases, the correction coefficient should monotonically decrease. When θ is π / 2 (90 degrees), the correction coefficient is 1 (i.e., neither amplifying nor reducing the static risk). When θ approaches π (180 degrees, i.e., the vehicle is reversing towards the object), the correction coefficient may further decrease, but since forward movement is usually the focus, it can be handled symmetrically or a lower limit can be set. A typical implementation uses a cosine function or an exponential decay function to construct f(θ), for example: f(θ) = 1 + α × × exp(-β × θ), where α and β are adjustment parameters that control the magnitude of risk amplification and the rate of decay. The corrected structural base value S_corrected is calculated as: S_corrected = S_base × f(θ).

[0103] S406, normalize the corrected structural baseline values ​​to obtain the sharpness of the target object.

[0104] In practice, the corrected structural baseline value S_corrected is normalized. Based on a large number of sample statistics or theoretical analysis, a pre-set upper limit for the structural baseline value S_max is established (which may correspond to the theoretical value of the most extreme dangerous situation or the very high percentile of historical observations, such as the 99.9th percentile). Finally, the sharpness S_sharpness of the target object is calculated as follows: S_sharpness = min(S_corrected / S_max, 1.0). Here, the min function is used to limit the result to the interval [0, 1.0]. A value of 0 represents no sharpness risk, and 1.0 represents the highest level of sharpness risk. This S_sharpness is a standardized index that integrates the object's own geometric characteristics and its relative motion direction with respect to the vehicle, used to quantify the risk of puncture or cut to the tire, i.e., the "sharpness of the object".

[0105] The above-described scheme enables a dynamic and quantitative assessment of the potential mechanical damage risk from solid obstacles on the road surface. It not only considers the inherent geometric sharpness of the object (through morphological indicators) but also introduces the key dynamic factor of the relative motion direction between the vehicle and the object. By calculating the angle between the direction of travel and the orientation of the sharp edge, the risk value is corrected, thus more accurately reflecting the instantaneous impact risk.

[0106] In some embodiments, S403 includes: S4031, based on the vehicle velocity vector and the vehicle front attitude angle in the vehicle state data, determine the vehicle's travel direction vector in the world coordinate system.

[0107] In practice, vehicle state data is acquired from a vehicle integrated navigation system (such as GNSS / IMU) or estimated through wheel speed and steering angle. This data includes the vehicle's current speed magnitude (scalar) and heading angle (typically the angle between the vehicle's longitudinal axis and true north or the global coordinate system's X-axis). Using this core information, the speed scalar is first decomposed into a global coordinate system (such as the ENU coordinate system) based on the heading angle, resulting in the vehicle velocity vector V_vehicle=[v_east,v_north,v_up]. Under normal road driving assumptions, the vertical speed v_up is typically close to zero. The horizontal direction of this vehicle velocity vector (i.e., the direction indicated by [v_east,v_north]) is the vehicle's instantaneous direction of travel in the world coordinate system. This vector dynamically reflects the vehicle's actual direction of motion and is the basis for assessing its relative motion with obstacles.

[0108] S4032, Based on the continuous change in the centroid position of the target object in its structural features, determine the trajectory of the target object relative to the vehicle, and combine it with the travel direction vector to obtain the relative velocity vector between the vehicle and the target object.

[0109] In practice, the system utilizes the 3D centroid position (located in the world coordinate system) of the target object tracked over multiple consecutive frames. By performing time-series analysis on these historical position points, such as using the finite difference method or velocity filtering algorithm, the instantaneous velocity vector of the target object in the world coordinate system can be calculated, denoted as V_object_world. To obtain the relative motion relationship between the vehicle and the object, coordinate transformation is required. The system uses the known instantaneous velocity vector of the vehicle itself in the world coordinate system, V_vehicle (obtained from S4031). Then, the relative velocity vector is calculated through vector subtraction: V_relative = V_object_world - V_vehicle. This V_relative vector describes the motion state (magnitude and direction) of the target object relative to the vehicle. More commonly, and more directly in actual perception, since the target object (such as a road stone) is usually considered stationary or moving at low speed, its velocity V_object_world relative to the ground is approximately zero. In this case, the velocity vector of the vehicle itself, V_vehicle, can be directly inverted to obtain the velocity vector of the object relative to the vehicle, i.e., V_relative ≈ -V_vehicle. The velocity vector points from the object towards the vehicle, in the opposite direction to the vehicle's direction of travel, and its magnitude is equal to the vehicle's speed. The system selects or weights and fuses these two calculation methods based on the confidence level of the object's motion (e.g., determined by trajectory smoothness) to ultimately obtain the relative velocity vector between the vehicle and the target object, providing crucial input for subsequent collision risk angle calculations.

[0110] S4033, based on the morphological structure index of the target object, identify the set of points with minimum local curvature on the edge of the target object, and fit the tangent direction of the set of points as the sharp edge orientation vector of the target object.

[0111] In practice, based on the obtained set of 3D edge points of the target object and their corresponding local curvature values ​​(e.g., the minimum principal curvature of each point is used as the "sharpest curvature" of that point), a curvature threshold is set (or points ranked in the top N% by curvature value are selected). All edge points with local curvature values ​​less than this threshold are filtered out, forming a "high curvature point set" or "candidate sharp edge set". Since the sharp edges of the target object (such as a stone) are usually composed of a series of high curvature points, and these points are often continuously distributed in space, spatial clustering analysis (e.g., using the DBSCAN algorithm) is performed on these selected high curvature point sets to group spatially adjacent points into the same "potential sharp edge cluster". For each identified cluster containing a sufficient number of points, the system uses linear regression or principal component analysis (PCA) to fit a spatial straight line or curve to all 3D points within the potential sharp edge cluster. The direction of the fitted straight line (or the tangent direction of the curve at the cluster center) is defined as the orientation of the "potential sharp edge". Typically, the orientation of the edge cluster with the smallest mean curvature (i.e., the sharpest) or the longest spatial extension is selected as the most threatening "sharp edge orientation" of the target object, and represented by a three-dimensional unit vector to obtain the sharp edge orientation vector of the target object. If multiple significantly different sharp edges exist, the system may retain multiple orientation vectors for subsequent analysis.

[0112] S4034, determine the spatial angle between the vehicle's travel direction vector and the sharp edge orientation vector of the target object, and use it as the angle between the vehicle's travel direction and the sharp edge orientation of the target object.

[0113] In practice, the calculated unit vector V_dir of the vehicle's direction of travel (usually its horizontal component is taken and normalized) and the unit vector E_dir of the target object's sharp edge orientation are obtained. Then, the spatial angle θ between these two three-dimensional vectors is calculated. This is done by taking the dot product of the two vectors and applying the inverse cosine function: θ = arccos. Since the dot product result is affected by the vector length, and these are all unit vectors, the calculation is direct and effective. The spatial angle θ ranges from 0 to π radians (i.e., 0 to 180 degrees). The smaller the angle θ, the closer the vehicle's direction of travel is to the direction of the object's sharp edge, and the higher the risk of the tire "cutting" or "scratching" the sharp edge during a collision. The larger the angle (especially when it is close to perpendicular), the more likely the collision is to "impact" or "run over," with a relatively lower direct risk of tire puncture. This spatial angle θ quantifies the alignment between the direction of motion and the geometric direction of the threat, and is a key dynamic parameter for correcting risk assessment.

[0114] The above method can accurately determine the trajectory of the target object relative to the vehicle by analyzing the change in the centroid position of the target object's structural features. This trajectory determines the positional change of the target object relative to the vehicle. Combined with the vehicle's direction of travel vector, the relative velocity vector of the target object relative to the vehicle is determined. Then, the spatial angle between this relative velocity vector and the determined sharp edge orientation vector of the target object is used. This spatial angle quantifies the alignment between the direction of travel and the sharp geometric direction of the target object that poses a threat to the vehicle's movement. Using this spatial angle as the angle between the vehicle's direction of travel and the sharp edge orientation of the target object provides higher accuracy.

[0115] In some embodiments, in S105, the vehicle status data is input into the anomaly recognition model to analyze the abnormal vehicle status data and obtain the vehicle abnormal status characteristics.

[0116] In practical implementation, the corresponding anomaly recognition model can use the temporal feature extraction architecture of the Transformer encoder. This anomaly recognition model can use a multi-head self-attention mechanism and positional encoding, which can effectively capture long-term dependencies and complex temporal patterns in vehicle state data. It is especially suitable for accurately identifying anomaly patterns related to vehicle impact, component wear, or jamming from multi-dimensional, high-noise vehicle dynamic signals (such as wheel speed pulses, acceleration, motor current, etc.).

[0117] It collects massive amounts of CAN bus status data sequences gathered during vehicle operation under various typical and extreme conditions. Based on expert rules, historical fault logs, or synchronous measurements from high-precision sensors (such as vibration sensors), it precisely labels abnormal periods in the data sequences, thereby constructing a structured training dataset. During the training phase, a strategy combining masked language model (MLM)-style self-supervised pre-training with labeled fine-tuning can be employed, using cross-entropy loss or contrastive loss functions to iteratively optimize the model parameters. This training process is repeated, and cross-validation is performed on independent validation sets, ultimately yielding an anomaly recognition model with high accuracy and strong generalization ability.

[0118] Thus, after acquiring the preprocessed vehicle state data sequence, the sequence is first normalized and segmented using a sliding window, consistent with the training phase, to form a fixed-length time-series sample. The anomaly detection model receives this time-series sample. Internally, it first maps the multi-dimensional state data into a high-dimensional feature vector through an embedding layer, and adds positional encoding to preserve temporal information. Then, through stacked multi-head self-attention layers and feedforward neural network layers, the anomaly detection model models the global dependencies between all time points within the sequence, extracting deep temporal features that characterize the evolution of normal states and deviation patterns of abnormal states. Finally, the anomaly detection model outputs a high-dimensional vehicle abnormal state feature vector through a classification or feature aggregation head. This vehicle abnormal state feature vector encodes the degree to which the vehicle's operating state deviates from the normal pattern and the information on potential anomaly types within the current time window, providing crucial internal state representation input for subsequent comprehensive risk decision-making.

[0119] S105 includes: S501, the vehicle status data is divided into sliding window segments to obtain a status data sequence window.

[0120] In practice, multiple vehicle status data streams are received from the CAN bus, wheel speed sensors, etc. A sliding window of fixed time length is used to segment the continuous data stream. For example, the window length is set to T seconds (e.g., 2 seconds), and the window sliding step is Δt seconds (e.g., 0.1 seconds). Starting from the current moment, historical data of length T is extracted backwards to form a multi-dimensional time series, i.e., a "status data sequence window". The data dimension within each window is [time series length, feature dimension], where the time series length is determined by the sampling frequency and the window duration. To ensure the timeliness of the data within the window and to cover all potential abnormal events, a continuous window sequence with overlapping parts (determined by the step size Δt) is continuously generated, providing context-continuous input samples for subsequent anomaly identification models.

[0121] S502, the state data sequence window is input into the anomaly recognition model, and the vehicle state data in the state data sequence window is dimensionally embedded and positionally encoded to obtain an embedding vector sequence.

[0122] In practice, the anomaly recognition model can be used to embed the vehicle status data in the status data sequence window according to the corresponding dimensions. This ensures that the data after each dimension embedding is more accurate. In addition, position encoding is performed to ensure that it conforms to the corresponding positional relationship. This ensures that the resulting embedding vector sequence conforms to both the dimensional and positional conditions.

[0123] S503, using the anomaly recognition model based on the embedded vector sequence, extracts the temporal dependency features from the vehicle state data through a multi-head self-attention mechanism.

[0124] In practice, after obtaining the embedded vector sequence, it is input into the core multi-head self-attention mechanism layer. The sequence is simultaneously mapped to multiple (e.g., eight) different "representation subspaces" (i.e., multiple attention heads). In each subspace, the "attention weight" between any two time-step vectors in the sequence is calculated in parallel. This attention weight reflects the importance of information at one time step for understanding information at another time step from the semantic perspective of the current subspace. For example, an abnormal wheel velocity pulse caused by a stone impact may have a strong correlation between multiple time steps before the impact (precursor), during the impact (peak), and after the impact (decay). The self-attention mechanism can automatically capture this cross-temporal dependency.

[0125] For each attention head, the embedded vector sequence is linearly transformed into a query vector, a key vector, and a value vector, respectively. The attention weight matrix is ​​obtained by calculating the dot product of the query vector and all key vectors, scaling it, and then normalizing it using the Softmax function. Then, the value vectors are weighted and summed using this weight matrix to obtain a new sequence representation output by that attention head that incorporates global context information. Finally, the outputs of all attention heads are concatenated and integrated through a linear transformation layer to output a new feature sequence.

[0126] S504, using the anomaly recognition model based on the temporal dependency features, generate the vehicle abnormal state features.

[0127] In practice, the sequence of "time-dependent features" obtained after processing by a multi-head self-attention layer is input into a feed-forward network (FFN) layer for further point-by-point nonlinear transformation and feature enhancement. An FFN typically consists of two linear transformation layers and an intermediate activation function (such as ReLU or GELU), which processes the feature vector at each time step of the sequence independently, aiming to extract higher-order abstract features. Subsequently, a hierarchical structure is adopted, stacking multiple "encoder layers" consisting of self-attention layers and FFN layers. Each layer performs deeper and more abstract feature extraction on the output of the previous layer. After the last encoder layer, the model needs to aggregate the information from the entire sequence to generate a fixed-dimensional "vehicle abnormal state feature" that can summarize the state of the entire time window.

[0128] As a preferred embodiment, the training process of this anomaly feature recognition model is as follows: (1) Constructing an initial anomaly recognition model. A Transformer encoder can be selected as the core to construct a multi-task temporal recognition architecture. The input is designed as multi-dimensional heterogeneous vehicle state temporal data (such as tire pressure, vertical acceleration, wheel speed, steering angle, etc.), and the output layer is designed as three branches: a regression branch for predicting future tire pressure changes, a binary classification branch for identifying whether there is an impact event, and a joint confidence rating branch for evaluating the probability of the occurrence of a joint tire pressure and impact anomaly event.

[0129] (2) Based on the massive amount of vehicle sensor data (including TPMS tire pressure values, IMU acceleration, wheel speed encoder signals, CAN bus dynamic parameters, etc.) collected from historical off-road tests and actual operations of car manufacturers, an initial training dataset is constructed. Through expert review and high-precision synchronized fault / event records (such as manually labeled puncture, impact, and tire pressure drop event times), abnormal time periods and preceding and following segments in the data sequence are accurately labeled to form a structured sample set with tire pressure change labels, impact event labels, and joint abnormal event labels.

[0130] (3) For each multi-sensor time series data sample, perform the same preprocessing as in the inference stage, including sliding window segmentation, Z-score normalization of each signal source (based on historical statistics), etc. Input the normalized multi-dimensional time series variables into the model, map different physical signals to a unified time series feature space through the embedding layer, and add sine / cosine position encoding to inject time series sequence information.

[0131] (4) The preprocessed and embedded temporal tensors are input into the initial anomaly detection model. Through stacked multi-head self-attention encoder layers and feedforward network layers, the model learns and extracts long-range temporal dependencies and complex anomaly patterns in the sequence. The output features of the encoder are input into the three output branches for prediction.

[0132] (5) A multi-task joint loss function is used as the objective function for optimization. This multi-task joint loss function is a weighted sum of the losses of the three tasks, including: the mean squared error loss for tire pressure change prediction, the binary cross-entropy loss for impact event identification, and the focal loss for joint abnormal event identification. All weight parameters of the model are updated by combining the backpropagation algorithm with the Adam optimizer.

[0133] (6) Repeat steps (4) and (5) to train the model iteratively multiple times using the initial training dataset. After each training round, evaluate the model performance using an independent validation set containing various typical off-road conditions (such as gravel, wading, and steep slopes), and monitor the loss and accuracy / error metrics for each task. When the model's performance on the validation set stabilizes, end this stage of training to obtain the pre-trained model.

[0134] (7) To improve the model's recognition accuracy and robustness under extreme, rare, or compound abnormal conditions (such as rapid tire blowout accompanied by severe impact, early identification of slow tire deflation on bumpy roads, etc.), an additional challenging condition sample set is constructed. This challenging condition sample set focuses on the above-mentioned difficult scenarios and is annotated more finely.

[0135] (8) Fine-tune the pre-trained model using a challenging set of working conditions. At this stage, a lower learning rate is used to allow the model to further refine its sensitivity to complex, weak, and compound anomalies based on the general anomaly patterns it has already mastered.

[0136] (9) The fine-tuned model was finally evaluated on a completely independent comprehensive test set covering multiple vehicle models and environments. Evaluation metrics included tire pressure prediction error (MAE, RMSE), precision and recall rate of impact event detection, and AUC of joint anomaly detection. Finally, the model whose performance met the preset threshold was determined as the deployable anomaly recognition model.

[0137] Using the above approach, an anomaly detection model based on the Transformer architecture can be used for in-depth analysis and modeling of vehicle state time-series data, extracting vehicle anomaly state features that can accurately represent complex temporal dependencies and anomaly patterns. These vehicle anomaly state features not only include traditional statistical information but also capture long-term dependencies and contextual relationships across time steps through a self-attention mechanism. This allows for the effective differentiation between normal disturbances caused by road bumps and genuine anomaly signals caused by component wear, jamming, or impacts from high-noise, multi-interference signals.

[0138] In some embodiments, S502 includes: S5021, the state data sequence window is input into the anomaly recognition model, and the original multidimensional vehicle state data of each time step in the state data sequence window is mapped to the semantic feature space using the linear transformation layer in the anomaly recognition model to obtain the initial feature vector.

[0139] In practice, the input layer of the anomaly detection model receives a window of state data sequences of fixed dimensions. First, through a learnable linear transformation layer (dimensional embedding layer), the original multidimensional vehicle state data (such as speed, acceleration, wheel speed difference, etc., which have different physical units and numerical ranges) at each time step in the window are mapped to a unified, higher-dimensional semantic feature space, forming an initial feature vector.

[0140] S5022, The initial feature vector is position-encoded to generate a position vector.

[0141] In practice, the initial feature vector at each time step is "positionally encoded". Positional encoding generates a position vector with the same dimension as the feature vector and unique at each position through a predefined function (such as using sine and cosine functions). This position vector encodes the absolute or relative position of that time step in the entire sequence.

[0142] S5023, the position vector is added to the initial feature vector to obtain the embedding vector corresponding to each time step, and the embedding vectors corresponding to each time step are combined to form the embedding vector sequence.

[0143] In practice, the position vector is added element-wise to the initial feature vector after dimensional embedding. Each time step yields an "embedding vector" that integrates its original state information, high-dimensional semantic information, and its precise position information in the sequence. The embedding vectors from all time steps are arranged in order to form the "embedding vector sequence" for subsequent processing by the anomaly detection model.

[0144] The above scheme can be used to embed the vehicle status data in the status data sequence window according to the corresponding dimensions using the anomaly recognition model. This ensures that the data after each dimension embedding is more accurate. In addition, position encoding is performed to ensure that it conforms to the corresponding positional relationship. This ensures that the resulting embedding vector sequence conforms to both the dimensional and positional conditions.

[0145] In some embodiments, in S106, determining the vehicle anomaly warning result based on the vehicle abnormality characteristics, the sharpness of the object, and the vehicle status data includes: S601, the abnormal vehicle state features, the sharpness of the object, and the vehicle state data are fused to obtain fused features.

[0146] In practice, the system first aligns and concatenates the multi-dimensional input data. Vehicle abnormal state features are typically high-dimensional vectors (e.g., 128-dimensional) and are directly used as one component. Object sharpness is a scalar value (between 0 and 1), which is expanded into a one-dimensional feature. Vehicle state data selects key state variables (such as vehicle speed, longitudinal / lateral acceleration, steering angle, etc.) within the current moment or a short time window to form a low-dimensional vector. Since the dimensions and numerical ranges of all features may differ significantly, the system performs preliminary scaling or simple normalization on the vehicle state data vector before concatenation to ensure its numerical range is roughly on the same order of magnitude as other feature vectors. Then, the processed vehicle abnormal state feature vector, the expanded sharpness scalar, and the pre-processed vehicle state data vector are concatenated along the feature dimensions to form a higher-dimensional comprehensive feature vector, i.e., the fused feature.

[0147] S602, the fused features are normalized to obtain a standardized feature vector.

[0148] In practice, the fused features are standardized using the Z-score standardization method. Specifically, in the offline phase, based on a large amount of historical normal operation data and abnormal data samples, the mean (μ) and standard deviation (σ) of each dimension of the fused feature vector are pre-calculated. In the online inference phase, for the real-time generated fused feature vector X, the formula x_i_normalized=(x_i-μ_i) / σ_i is applied to each element x_i of its dimension i. After this operation, the values ​​of each dimension are converted into a standard normal distribution (approximately) with a mean of 0 and a standard deviation of 1, thus obtaining a standardized feature vector. This step is a crucial preprocessing step to ensure the robustness and generalization ability of the model.

[0149] S603, perform nonlinear transformation and feature extraction on the standardized feature vector to obtain the extracted features.

[0150] In practice, the standardized feature vector is input into a multilayer perceptron (MLP) or a small deep neural network. This small deep neural network typically consists of several fully connected layers, with non-linear activation functions (such as ReLU or GELU) used between layers. The goal is to extract more abstract, discriminative, and lower-dimensional (or compressed) deep feature representations from the high-dimensional, potentially redundant, standardized fused features through a series of non-linear transformations. For example, the first layer might learn simple combinations of features, while deeper layers can capture complex interaction patterns, such as a composite pattern like "the appearance of a highly sharp object is accompanied by a specific frequency of wheel speed fluctuations and slight lateral acceleration anomalies." The output is the "extracted feature," a new, dense vector more suitable for the final risk decision.

[0151] S604, Based on the extracted features, determine the vehicle anomaly level, including: S6041, Apply an attention weighting mechanism to the different sub-extracted features in the extracted features to perform weighted processing, and obtain the weighted extracted features.

[0152] In specific implementation, the extracted feature vector can be regarded as composed of multiple "sub - features" or "feature channels". Not all sub - features are equally important for the final abnormal risk judgment. An attention mechanism layer is introduced (for example, a simple fully - connected layer plus a Softmax function, or a variant of a more complex multi - head attention mechanism). It receives the extracted features as input and learns to generate an attention weight vector with the same dimension as the extracted features. Each element in the attention weight vector corresponds to the importance score of a sub - feature in the extracted features. All scores are normalized by Softmax, and their sum is 1. Then, the extracted feature vector is multiplied element - by - element (Hadamard product) with this attention weight vector. In this way, important sub - features are enhanced and unimportant sub - features are suppressed, thus obtaining the "weighted - processed extracted features".

[0153] S6042, Based on the weighted - processed extracted features, generate the continuous risk value of the vehicle through regression calculation.

[0154] In specific implementation, the weighted - processed extracted features are input into a regression layer. This regression layer is usually one (or several) fully - connected layers, and finally outputs a scalar value. This scalar value is the "continuous risk value of the vehicle". This continuous risk value is an unbounded or semi - bounded real number. In theory, the higher the score, the higher the risk or severity of abnormalities (such as impact damage, potential puncture, component looseness, etc.) occurring at the vehicle wheel end (including tires, wheels, bearings, etc.). In the training phase, this regression layer learns by minimizing the error (such as mean squared error MSE) between the predicted continuous risk value and the true risk label (for example, a 0 - 1 continuous risk value based on post - event diagnosis or expert evaluation).

[0155] S6043, Based on the continuous risk value, determine the vehicle abnormal level.

[0156] In specific implementation, the system presets several risk - level intervals and their corresponding thresholds. For example, three levels can be set: low risk (0 ≤ risk value < T1), medium risk (T1 ≤ risk value < T2), high risk (risk value ≥ T2). The thresholds T1 and T2 are determined by analyzing the correspondence between the risk - value distribution in historical data and true abnormal events, such as using the ROC curve or precision - recall curve to select the optimal cut - off point. During online operation, the system compares the continuously calculated risk value with these preset thresholds and maps it to the corresponding discrete "vehicle abnormal level".

[0157] S605, Based on the vehicle abnormal level, determine the vehicle abnormal warning result.

[0158] In practice, vehicle anomaly warning results are structured information ultimately output to the driver or upper-level control system. This includes not only the vehicle anomaly level (e.g., "high risk"), but also key risk sources and contextual information. For example, the results may include the target object ID that triggered the risk (if any), the sharpness of the object, the estimated time and location of the anomaly (e.g., GNSS coordinates), and possible anomaly types interpreted from the vehicle's abnormal state characteristics (e.g., "impact anomaly," "periodic wear," etc.) or the affected wheel end location (e.g., "left front wheel area"). Depending on the anomaly level, different levels of warnings may be triggered (e.g., dashboard icons, audible alerts, suggested speed limits, suggested stops for inspection, etc.), and detailed results are recorded in the vehicle log for subsequent analysis. This completes the entire anomaly detection process from multi-source data perception to comprehensive decision-making.

[0159] The above scheme deeply integrates deep temporal features characterizing the state of internal components, dynamic geometric risk values ​​quantifying external physical threats, and vehicle state data reflecting real-time driving conditions. By introducing a feature extraction network and attention mechanism, it can automatically learn and focus on the feature combinations that best indicate abnormal risks. The final output not only includes intuitive anomaly levels but also provides quantitative trends in risk through continuous risk values. Furthermore, by associating with information about external objects, the interpretability of the results is enhanced, significantly improving the vehicle's active safety and predictive maintenance capabilities.

[0160] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0161] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0162] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a vehicle abnormality warning device.

[0163] refer to Figure 2 The device includes: The data acquisition module 210 is configured to acquire road perception data and vehicle status data when the vehicle is in motion. The region extraction module 220 is configured to perform target object region extraction processing on the road surface perception data to obtain road surface perception data containing the target object region. The geometric analysis module 230 is configured to input the road surface perception data containing the target object region into the object segmentation model, analyze the geometric properties of the target object, and obtain the object structure features of the target object. The risk fusion module 240 is configured to perform multimodal data fusion analysis based on the structural features of the object and the vehicle state data to obtain the sharpness of the object; The status analysis module 250 is configured to input the vehicle status data into the anomaly identification model, analyze the abnormal vehicle status data, and obtain the abnormal vehicle status characteristics. The result decision module 260 is configured to determine the vehicle anomaly warning result based on the vehicle anomaly state characteristics, the sharpness of the object, and the vehicle state data.

[0164] In some embodiments, the geometric analysis module 230 is specifically configured as follows: The road surface perception data containing the target object region is input into the object segmentation model, and the target object instance segmentation is performed on the road surface perception data containing the target object region to obtain the mask, edge point set and centroid position of each target object. For each target object, based on the mask and the edge point set, the local curvature distribution, edge angle change rate and boundary point density of the target object edge are determined to obtain the first geometric structure feature corresponding to each target object; By combining the road surface perception data containing the target object region, the first geometric structure feature is corrected to obtain the second geometric structure feature corresponding to each target object. The second geometric structure features are tracked across frames to obtain the second geometric structure feature sequence. Based on the second geometric structure feature sequence, the edge features and centroid position corresponding to each target object are determined, and the edge features and centroid position are combined to obtain the object structure features corresponding to each target object.

[0165] In some embodiments, the geometric analysis module 230 is further configured to: Based on the road surface perception data, a three-dimensional point cloud set of the target object region is determined; Based on the three-dimensional point cloud set, the edge point set of the target object is fitted in three-dimensional space to obtain the three-dimensional edge contour of the target object. Based on the three-dimensional edge contour, determine the local curvature distribution, edge angle change rate, and boundary point density of the target object's edge; The first geometric structure feature is modified by utilizing the local curvature distribution, edge angle change rate, and boundary point density of the target object's edge to obtain the second geometric structure feature.

[0166] In some embodiments, the risk fusion module 240 is specifically configured as follows: Based on the structural features of the target object, and combined with the vehicle velocity vector and vehicle front attitude angle in the vehicle state data, the target object is tracked across frames. During cross-frame tracking, based on the edge point set in the object's structural features, the local curvature distribution, edge angle change rate, and boundary point density of the target object's edge are determined, thus obtaining the morphological structural indices of the target object. Based on the vehicle status data, the relative motion state between the vehicle and the target object is obtained, and the angle between the vehicle's direction of travel and the direction of the sharp edge of the target object is determined. Based on the morphological structural indicators, the basic structural values ​​of the target object are determined; Based on the included angle, the basic structural values ​​are corrected; The corrected structural baseline values ​​are normalized to obtain the sharpness of the target object.

[0167] In some embodiments, the risk fusion module 240 is further configured to: Based on the vehicle velocity vector and the vehicle front attitude angle in the vehicle state data, the vehicle's travel direction vector in the world coordinate system is determined. Based on the continuous change in the centroid position of the target object in its structural features, the motion trajectory of the target object relative to the vehicle is determined, and combined with the travel direction vector, the relative velocity vector between the vehicle and the target object is obtained. Based on the morphological structure indicators of the target object, the set of points with minimum local curvature in the edge of the target object is identified, and the tangent direction of the set of points is fitted as the sharp edge orientation vector of the target object. The spatial angle between the vehicle's travel direction vector and the sharp edge orientation vector of the target object is determined as the angle between the vehicle's travel direction and the sharp edge orientation of the target object.

[0168] In some embodiments, the state analysis module 250 is specifically configured as follows: The vehicle status data is segmented using a sliding window to obtain a status data sequence window; The state data sequence window is input into the anomaly recognition model, and the vehicle state data in the state data sequence window is subjected to dimensional embedding and position encoding to obtain an embedding vector sequence. The anomaly detection model is used to extract temporal dependency features from the vehicle state data based on the embedded vector sequence through a multi-head self-attention mechanism. The abnormal vehicle state features are generated using the anomaly recognition model based on the temporal dependency features.

[0169] In some embodiments, the state analysis module 250 is further configured to: The state data sequence window is input into the anomaly recognition model. The linear transformation layer in the anomaly recognition model is used to map the original multidimensional vehicle state data of each time step in the state data sequence window to the semantic feature space to obtain the initial feature vector. The initial feature vector is positionally encoded to generate a position vector; The position vector is added to the initial feature vector to obtain the embedding vector corresponding to each time step, and the embedding vectors corresponding to each time step are combined to form the embedding vector sequence.

[0170] In some embodiments, the outcome decision module 260 is configured to: The abnormal vehicle state features, the sharpness of the object, and the vehicle state data are fused to obtain fused features; The fused features are normalized to obtain a standardized feature vector; The standardized feature vector is subjected to nonlinear transformation and feature extraction to obtain the extracted features; Based on the extracted features, the vehicle anomaly level is determined; Based on the vehicle anomaly level, the vehicle anomaly warning result is determined.

[0171] In some embodiments, the result decision module 260 is further configured to: The different sub-extracted features in the extracted features are weighted by an attention weighting mechanism to obtain the weighted extracted features; Based on the extracted features after weighted processing, continuous risk values ​​for vehicles are generated through regression calculation. Based on the continuous risk values, the vehicle anomaly level is determined.

[0172] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0173] The apparatus of the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0174] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, the electronic device including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the methods described in any of the above embodiments.

[0175] Figure 3 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0176] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0177] The memory 1020 can be implemented in the form of ROM (Read-Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0178] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0179] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0180] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0181] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0182] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0183] Based on the same inventive concept, this application also provides a vehicle including the device or electronic device described in the above embodiments. The beneficial effects of embodiments having corresponding devices or electronic devices will not be elaborated further here.

[0184] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0185] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0186] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0187] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0188] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0189] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0190] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0191] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for early warning of vehicle anomalies, characterized in that, include: Acquire road perception data and vehicle status data while the vehicle is in motion; The road surface perception data is processed to extract the target object region, resulting in road surface perception data containing the target object region. The road surface perception data containing the target object region is input into the object segmentation model to analyze the geometric properties of the target object and obtain the object structure features of the target object. Based on the structural features of the object, multimodal data fusion analysis is performed in conjunction with the vehicle state data to obtain the sharpness of the object; The vehicle status data is input into the anomaly identification model, and the abnormal vehicle status data is analyzed to obtain the vehicle abnormal status characteristics. Based on the abnormal vehicle state characteristics, the sharpness of the object, and the vehicle state data, the vehicle abnormality warning result is determined.

2. The method according to claim 1, characterized in that, The step of inputting the road surface perception data containing the target object region into the object segmentation model, analyzing the geometric attributes of the target object, and obtaining the object structure features of the target object includes: The road surface perception data containing the target object region is input into the object segmentation model, and the target object instance segmentation is performed on the road surface perception data containing the target object region to obtain the mask, edge point set and centroid position of each target object. For each target object, based on the mask and the edge point set, the local curvature distribution, edge angle change rate and boundary point density of the target object edge are determined to obtain the first geometric structure feature corresponding to each target object; By combining the road surface perception data containing the target object region, the first geometric structure feature is corrected to obtain the second geometric structure feature corresponding to each target object. The second geometric structure features are tracked across frames to obtain the second geometric structure feature sequence. Based on the second geometric structure feature sequence, the edge features and centroid position corresponding to each target object are determined, and the edge features and centroid position are combined to obtain the object structure features corresponding to each target object.

3. The method according to claim 2, characterized in that, The step of combining the road surface perception data to correct the first geometric structure features to obtain the second geometric structure features of the target object includes: Based on the road surface perception data, a three-dimensional point cloud set of the target object region is determined; Based on the three-dimensional point cloud set, the edge point set of the target object is fitted in three-dimensional space to obtain the three-dimensional edge contour of the target object. Based on the three-dimensional edge contour, determine the local curvature distribution, edge angle change rate, and boundary point density of the target object's edge; The first geometric structure feature is modified by utilizing the local curvature distribution, edge angle change rate, and boundary point density of the target object's edge to obtain the second geometric structure feature.

4. The method according to claim 1, characterized in that, The method of performing multimodal data fusion analysis based on the object's structural features and the vehicle's state data to obtain the object's sharpness includes: Based on the structural features of the target object, and combined with the vehicle velocity vector and vehicle front attitude angle in the vehicle state data, the target object is tracked across frames. During cross-frame tracking, based on the edge point set in the object's structural features, the local curvature distribution, edge angle change rate, and boundary point density of the target object's edge are determined, thus obtaining the morphological structural indices of the target object. Based on the vehicle status data, the relative motion state between the vehicle and the target object is obtained, and the angle between the vehicle's direction of travel and the direction of the sharp edge of the target object is determined. Based on the morphological structural indicators, the basic structural values ​​of the target object are determined; Based on the included angle, the basic structural values ​​are corrected; The corrected structural baseline values ​​are normalized to obtain the sharpness of the target object.

5. The method according to claim 4, characterized in that, The step of combining the vehicle state data to obtain the relative motion state between the vehicle and the target object, and determining the angle between the vehicle's direction of travel and the orientation of the sharp edge of the target object, includes: Based on the vehicle velocity vector and the vehicle front attitude angle in the vehicle state data, the vehicle's travel direction vector in the world coordinate system is determined. Based on the continuous change in the centroid position of the target object in its structural features, the motion trajectory of the target object relative to the vehicle is determined, and combined with the travel direction vector, the relative velocity vector between the vehicle and the target object is obtained. Based on the morphological structure indicators of the target object, the set of points with minimum local curvature in the edge of the target object is identified, and the tangent direction of the set of points is fitted as the sharp edge orientation vector of the target object. The spatial angle between the vehicle's travel direction vector and the sharp edge orientation vector of the target object is determined as the angle between the vehicle's travel direction and the sharp edge orientation of the target object.

6. The method according to claim 1, characterized in that, The process of inputting the vehicle status data into the anomaly recognition model, analyzing the abnormal vehicle status data, and obtaining vehicle abnormal status features includes: The vehicle status data is segmented using a sliding window to obtain a status data sequence window; The state data sequence window is input into the anomaly recognition model, and the vehicle state data in the state data sequence window is subjected to dimensional embedding and position encoding to obtain an embedding vector sequence. The anomaly detection model is used to extract temporal dependency features from the vehicle state data based on the embedded vector sequence through a multi-head self-attention mechanism. The abnormal vehicle state features are generated using the anomaly recognition model based on the temporal dependency features.

7. The method according to claim 6, characterized in that, The step of inputting the state data sequence window into the anomaly detection model, and performing dimensional embedding and position encoding on the vehicle state data in the state data sequence window to obtain an embedding vector sequence includes: The state data sequence window is input into the anomaly recognition model. The linear transformation layer in the anomaly recognition model is used to map the original multidimensional vehicle state data of each time step in the state data sequence window to the semantic feature space to obtain the initial feature vector. The initial feature vector is positionally encoded to generate a position vector; The position vector is added to the initial feature vector to obtain the embedding vector corresponding to each time step, and the embedding vectors corresponding to each time step are combined to form the embedding vector sequence.

8. The method according to claim 1, characterized in that, The process of determining the vehicle anomaly warning result based on the vehicle's abnormal state characteristics, the sharpness of the object, and the vehicle's state data includes: The abnormal vehicle state features, the sharpness of the object, and the vehicle state data are fused to obtain fused features; The fused features are normalized to obtain a standardized feature vector; The standardized feature vector is subjected to nonlinear transformation and feature extraction to obtain the extracted features; Based on the extracted features, the vehicle anomaly level is determined; Based on the vehicle anomaly level, the vehicle anomaly warning result is determined.

9. The method according to claim 8, characterized in that, The process of determining the vehicle anomaly level based on the extracted features includes: The different sub-extracted features in the extracted features are weighted by an attention weighting mechanism to obtain the weighted extracted features; Based on the extracted features after weighted processing, continuous risk values ​​for vehicles are generated through regression calculation. Based on the continuous risk values, the vehicle anomaly level is determined.

10. A vehicle, the vehicle including electronic equipment, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the method as described in any one of claims 1 to 9.

Images