A road condition adaptive vehicle dynamics state monitoring method, device and electronic equipment

Abstract

The application provides a road condition adaptive vehicle dynamics state monitoring method, device and electronic equipment. The method comprises: acquiring multi-modal data in the vehicle driving process; performing feature extraction on the multi-modal data, fusing the extracted road surface visual features, tire vibration features, vehicle body posture disturbance features and vehicle state features, obtaining multi-modal road surface state features and performing road surface category prediction to obtain the road surface category probability distribution of the vehicle driving; and fusing the tire vibration features and the vehicle body posture disturbance features to obtain the measured dynamics response; constructing a reference dynamics response based on a normal state reference template library; calculating the response deviation of the measured dynamics response relative to the reference dynamics response, and constructing state discrimination features according to the response deviation and preset vehicle state information and inputting the state discrimination features into a state evaluation network to determine the vehicle dynamics state monitoring result based on the state score.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, specifically to a method, device, and electronic equipment for monitoring vehicle dynamics state in road condition adaptive manner. Background Technology

[0002] During vehicle operation, dynamic responses such as tire radial acceleration and vehicle attitude angular velocity are affected not only by tire health but also by factors such as road surface roughness, pothole impacts, water accumulation, vehicle speed, tire pressure, and tire temperature. Healthy tires may experience significant transient impacts and vehicle attitude disturbances when driving on damaged roads; tires with minor abnormalities may only exhibit continuous, small-amplitude vibrations when driving on smooth roads. Therefore, dynamic responses of the same amplitude or similar shape may have different health implications under different road environments and vehicle operating conditions.

[0003] Existing tire condition monitoring methods typically employ fixed thresholds, preset rules, or directly train binary classification models for normal and abnormal conditions. Fixed threshold methods often fail to consider the differences in the normal fluctuation range of healthy tires under various road surface conditions, easily misclassifying normal road stimuli such as potholes and damage as tire abnormalities. While direct binary classification methods can utilize various information such as tire vibration, tire pressure, and tire temperature, if the model does not explicitly establish the correspondence between road environment, vehicle operating conditions, and healthy dynamic response, its judgment results are easily affected by the road condition distribution in the training data, resulting in false positives or false negatives under insufficiently covered road conditions.

[0004] Some existing technologies acquire road images, tire vibration, and vehicle attitude information simultaneously, but typically only perform feature stitching, classification result fusion, or risk score weighting on this information. These various types of information are primarily used as parallel inputs, without generating corresponding health dynamic response benchmarks based on the current road environment and vehicle operating conditions. Therefore, when the vehicle's dynamic response changes, existing methods struggle to determine whether the change is caused by normal road excitation or by degradation of tire or vehicle health. Summary of the Invention

[0005] In view of this, the present invention proposes a road condition adaptive vehicle dynamics state monitoring method, device, and electronic device. Specifically, the present invention is achieved through the following technical solution:

[0006] According to a first aspect of the embodiments of this specification, a road condition adaptive vehicle dynamics state monitoring method is provided, the method comprising the following steps:

[0007] Step S1: Acquire multimodal data during vehicle operation. The multimodal data includes road surface images, tire radial acceleration signals, vehicle body attitude angular velocity signals, and preset vehicle state information.

[0008] Step S2 involves extracting features from the multimodal data to obtain road surface visual features, tire vibration features, vehicle body posture disturbance features, and vehicle state features. These features are then fused to obtain multimodal road surface state features, and road surface category prediction is performed to obtain the probability distribution of the road surface category for vehicle travel. Finally, the tire vibration features and vehicle body posture disturbance features are fused to obtain the measured dynamic response.

[0009] Step S3: Based on the multimodal road surface state features, the road surface category probability distribution, and the vehicle state features, generate a road condition vector, and based on the road condition vector, match and generate a reference dynamic response from a preset normal state reference template library;

[0010] Step S4: Calculate the response deviation of the measured dynamic response relative to the reference dynamic response, construct state discrimination features based on the response deviation and the preset vehicle state information, input them into the state evaluation network, and determine the vehicle dynamic state monitoring result based on the state score.

[0011] According to a second aspect of the embodiments of this specification, a road condition adaptive vehicle dynamics state monitoring device is provided, the device comprising:

[0012] The data preprocessing unit is used to acquire multimodal data during vehicle operation, including road surface images, tire radial acceleration signals, vehicle body attitude angular velocity signals, and preset vehicle state information.

[0013] The feature fusion unit is used to extract features from the multimodal data to obtain road surface visual features, tire vibration features, vehicle body posture disturbance features, and vehicle state features; fuse the road surface visual features, tire vibration features, vehicle body posture disturbance features, and vehicle state features to obtain multimodal road surface state features and perform road surface category prediction to obtain the probability distribution of road surface categories for vehicle travel; and fuse the tire vibration features and the vehicle body posture disturbance features to obtain the measured dynamic response.

[0014] The reference dynamic response construction unit is used to generate a road condition vector based on the multimodal road surface state features, the road surface category probability distribution and the vehicle state features, and to match and generate a reference dynamic response from a preset normal state reference template library based on the road condition vector.

[0015] The dynamic state discrimination unit is used to calculate the response deviation of the measured dynamic response relative to the reference dynamic response, construct state discrimination features based on the response deviation and the preset vehicle state information, input them into the state evaluation network, and determine the vehicle dynamic state monitoring result based on the state score.

[0016] According to a third aspect of the embodiments of this specification, an electronic device is provided, comprising: a processor; and a computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in the first aspect.

[0017] The embodiments of the present invention have at least the following technical effects:

[0018] (1) This invention acquires road surface images, tire radial acceleration signals, vehicle body attitude angular velocity signals and preset vehicle state information simultaneously, and extracts road surface visual features, tire vibration features, vehicle body attitude disturbance features and vehicle state features respectively. Then, it integrates the above features into multimodal road surface state features for road surface category prediction. It can comprehensively utilize multi-dimensional information such as road appearance, tire road surface contact dynamics, vehicle body attitude response and vehicle speed, tire pressure and tire temperature, and make up for the shortcomings of single visual information being easily affected by illumination and occlusion, and single dynamic information being easily affected by operating conditions. It improves the accuracy and stability of road surface state perception in complex road environments. Furthermore, by integrating tire vibration features and vehicle body attitude disturbance features into measured dynamic response features, it fully preserves the current real dynamic response information of the vehicle and provides reliable measured basis for the judgment of vehicle dynamic state.

[0019] (2) This invention generates a road condition vector based on multimodal road surface state characteristics, road surface category probability distribution, and vehicle state characteristics. Based on this road condition vector, a reference dynamic response adapted to the current road environment and vehicle operating conditions is generated from a pre-set normal state reference template library. Since the normal state reference template library stores the dynamic response benchmarks of vehicles under normal dynamic conditions for different road surface categories (such as asphalt, cement, waterlogged, and damaged road surfaces) and under different vehicle speeds, tire pressures, and tire temperatures, the generated reference dynamic response can dynamically change with road and operating conditions. The larger dynamic response generated when a vehicle travels on a damaged road surface under normal dynamic conditions can be covered by the normal reference range under the corresponding road conditions and will not be misjudged as abnormal. Conversely, when a vehicle travels on a flat road surface, if its dynamic state deviates, even if it only exhibits continuous small vibrations, the small vibrations can be effectively identified because the fluctuation range of the normal dynamic response under this road surface condition is relatively narrow. Therefore, this invention overcomes the defect of existing fixed threshold methods that easily misjudge normal road excitations such as potholes and damage as abnormal vehicle states. Furthermore, by introducing the probability distribution of road surface categories for template matching, even if there is some uncertainty in road surface recognition, the template responses under multiple possible categories can be integrated, avoiding a serious mismatch between the reference dynamic response and the actual road conditions caused by misjudgment of a single category, thus enhancing the fault tolerance and robustness of the method.

[0020] (3) Since the response deviation is calculated under a dynamic reference dynamic response benchmark, it can truly reflect whether the measured dynamic response exceeds the range of changes in the normal dynamic state under the current road environment and operating conditions, thereby effectively distinguishing the response changes caused by the deviation of the vehicle's dynamic state from the normal road excitation. At the same time, the state discrimination features incorporate preset state information (such as tire pressure, tire temperature, and vehicle speed), enabling the state evaluation network to eliminate interference from non-dynamic state deviation factors such as insufficient tire pressure, temperature changes, or vehicle speed fluctuations, further improving the accuracy of vehicle dynamic state discrimination. Attached Figure Description

[0021] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0022] Figure 1 This is a schematic flowchart illustrating a road condition adaptive vehicle dynamics state monitoring method according to an exemplary embodiment of the present invention;

[0023] Figure 2This is a schematic diagram illustrating a multimodal data acquisition method according to an exemplary embodiment of the present invention;

[0024] Figure 3 This is a block diagram illustrating an electronic device according to an exemplary embodiment of the present invention;

[0025] Figure 4 This is a block diagram illustrating a road condition adaptive vehicle dynamics state monitoring device according to an exemplary embodiment of the present invention. Detailed Implementation

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0028] This invention discloses a road condition adaptive vehicle dynamics state monitoring scheme, belonging to the field of vehicle safety monitoring and intelligent transportation risk early warning technology. This scheme simultaneously acquires road surface images, tire radial acceleration, vehicle attitude angular velocity, and preset vehicle state information such as tire pressure, tire temperature, and vehicle speed, and forms synchronous multimodal samples through timestamp alignment and data preprocessing. It extracts road surface visual features, tire vibration features, vehicle attitude disturbance features, and vehicle state features, respectively, and obtains multimodal road surface state features through multimodal fusion, and obtains road surface category probability distributions through road surface category prediction. Based on the multimodal road surface state features, road surface category probability distributions, and vehicle state features, a road condition vector is generated, and a reference dynamic response under the current road environment and vehicle operating conditions is generated through a normal state reference template library. The tire vibration features and vehicle attitude disturbance features are fused through independent temporal feature extraction branches to obtain the current measured dynamic response of the vehicle. A state discrimination feature is constructed based on the response deviation of the measured dynamic response relative to the reference dynamic response and the preset vehicle state information. State prediction is performed based on the state discrimination feature, thereby determining the current vehicle dynamics state based on the state score. This invention can dynamically adjust the discrimination criteria based on the multimodal road environment perception results and vehicle operating conditions, reduce false alarms caused by normal excitation on complex road surfaces, and improve the adaptability and reliability of vehicle dynamic state discrimination under different road environments and operating conditions.

[0029] The embodiments described in this specification will now be described in detail.

[0030] This invention provides a road condition adaptive vehicle dynamics state monitoring method. This method can be deployed in an on-board computing unit, a cloud server, or a roadside unit. It can either be used by the vehicle to independently complete real-time road condition recognition or complete the perception task through vehicle-road-cloud collaboration.

[0031] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of the present invention for a road condition adaptive vehicle dynamics state monitoring method, as shown below. Figure 1 As shown, the vehicle dynamics state monitoring method includes the following steps:

[0032] Step S1: Acquire multimodal data during vehicle operation. The multimodal data includes road surface images, tire radial acceleration signals, vehicle body attitude angular velocity signals, and preset vehicle state information.

[0033] The tire radial acceleration signal reflects the change in tire acceleration in the direction perpendicular to the tread after being excited by the road surface. The preset vehicle status information includes vehicle status such as tire pressure, tire temperature, and vehicle speed.

[0034] This embodiment can acquire road surface images along the vehicle's driving direction using an onboard camera; acquire tire radial acceleration signals using sensors installed on the tires; synchronously acquire preset vehicle status information using a status acquisition unit; and assign timestamps to the road surface images, tire radial acceleration signals, and preset vehicle status information using a time synchronization unit. Based on the timestamps of the road surface images, the tire radial acceleration signals and preset vehicle status information within a preset time window are extracted to obtain the multimodal data.

[0035] Step S2 involves extracting features from the multimodal data to obtain road surface visual features, tire vibration features, vehicle body posture disturbance features, and vehicle state features. These features are then fused to obtain multimodal road surface state features, and road surface category prediction is performed to obtain the probability distribution of road surface categories for vehicle travel. Additionally, the tire vibration features and vehicle body posture disturbance features are fused to obtain the measured dynamic response.

[0036] Step S3: Based on the multimodal road surface state features, the road surface category probability distribution, and the vehicle state features, generate a road condition vector, and based on the road condition vector, match and generate a reference dynamic state response from a preset normal state reference template library.

[0037] Step S4: Calculate the response deviation of the measured dynamic response relative to the reference dynamic response, construct state discrimination features based on the response deviation and the preset vehicle state information, input them into the state evaluation network, and determine the vehicle dynamic state monitoring result based on the state score.

[0038] based on Figure 1 The vehicle dynamics state monitoring method shown in this embodiment acquires current driving conditions through multimodal road environment perception. Based on this, it generates a dynamic reference dynamic response tailored to road conditions. Then, it discriminates the measured dynamic response against this reference dynamic response to form a complete vehicle dynamics state monitoring path. This scheme enables vehicle dynamics state monitoring to adapt to different road surface types, vehicle speeds, tire pressures, and tire temperatures, maintaining reliable discrimination performance in diverse road environments. It significantly reduces false alarms and false negatives caused by complex road conditions, and substantially improves the adaptability and reliability of vehicle dynamics state monitoring in real-world driving scenarios.

[0039] Next, the process of vehicle dynamics state monitoring by the vehicle system will be explained in detail.

[0040] First, perform step S1 to preprocess the data.

[0041] Multi-source heterogeneous data related to road environment, tire-road contact dynamics, vehicle attitude response, and vehicle operating status are collected in real time during vehicle operation. Through timestamp alignment and data preprocessing, unified, synchronized, and standardized multimodal data is formed. This multimodal data is used for subsequent extraction of road environment conditions, generation of road condition-based health benchmarks, extraction of measured dynamic responses, and determination of vehicle health status, providing a data foundation for vehicle dynamics monitoring.

[0042] In this invention, raw signals are acquired during the vehicle tracing process. These raw signals include at least road surface images, tire radial acceleration signals, vehicle body attitude angular velocity signals, and preset vehicle state information. The expression is as follows:

[0043] (1)

[0044] in, Represents the original road surface image. This represents the original tire radial acceleration sequence. This represents the original vehicle body attitude angular velocity sequence collected by the gyroscope. This represents the original preset vehicle state vector.

[0045] For example, such as Figure 2 As shown, the road surface image is acquired by an onboard camera installed at the front of the vehicle, reflecting the texture, color, cracks, water reflection, potholes, damage, and macroscopic appearance differences of different paving materials on the road surface in front of the vehicle; the tire radial acceleration signal is acquired by an acceleration acquisition device installed on the inner sidewall of the tire, reflecting the high-frequency dynamic response generated during the contact between the tire and the road surface; the vehicle attitude angular velocity signal is acquired by a gyroscope installed on the vehicle body, chassis, or frame, reflecting the attitude disturbance of the vehicle in the pitch, roll, and yaw directions; preset vehicle status information includes tire pressure, tire temperature, and vehicle speed, used to characterize the tire working state and vehicle operating state.

[0046] Because the aforementioned signals have different data formats and sampling frequencies, directly inputting them into the model for unified processing can easily lead to temporal inconsistencies between modes. Therefore, it is necessary to further establish a unified timestamp marker and a synchronization alignment mechanism. Specifically, based on the acquisition time of the road surface image frame, sequential data within the corresponding time window are extracted from the tire radial acceleration signal and the vehicle attitude angular velocity signal. Window statistical processing is then performed on low-frequency scalar signals such as tire pressure, tire temperature, and vehicle speed to form multimodal data for the same vehicle driving segment.

[0047] Let the acquisition time of the k-th frame of the road surface image be . The corresponding time window is Then the original synchronization signal can be expressed as:

[0048] (2)

[0049] in, Indicates in Raw road surface images captured at all times. Indicates time window The original tire radial acceleration sequence acquired internally. This indicates that the data was collected by the gyroscope within the time window. The original vehicle body attitude angular velocity sequence acquired internally. Indicates time window The original vehicle state vector collected internally.

[0050] After timestamp alignment is completed, necessary data preprocessing is performed on various signals. For example, for road surface images, image cropping, scaling, pixel normalization, or standardization are performed to convert the images into fixed-size input images; for tire radial acceleration sequences, zero-bias removal, low-pass filtering, and zero-mean standardization are performed to suppress sensor noise and high-frequency interference; for vehicle attitude angular velocity sequences, outlier processing, zero-bias removal, smoothing filtering, and channel standardization are performed to reduce the impact of gyroscope random noise and zero-bias drift; for preset vehicle state information such as tire pressure, tire temperature, and vehicle speed, maximum and minimum normalization or standardization are performed to eliminate dimensional differences between different physical quantities.

[0051] After preprocessing, the k-th multimodal data It can be represented as:

[0052] (3)

[0053] in, This represents the preprocessed k-th frame of the road surface image. This represents the preprocessed radial acceleration sequence of the k-th tire. This represents the preprocessed k-th vehicle body attitude angular velocity sequence. This represents the preprocessed state vector of the kth vehicle.

[0054] In one implementation, the road surface image can be scaled to a 640×640×3 RGB image; the tire radial acceleration signal is sampled at a frequency of 2000Hz with a time window of 0.5s, corresponding to a sequence length of L=1000; the vehicle body attitude angular velocity signal is acquired by a gyroscope with a sampling frequency of 100Hz and a time window of 0.5s, corresponding to a 50×3 three-axis angular velocity sequence; the tire pressure, tire temperature, and vehicle speed signals can be averaged within the corresponding time window to form a three-dimensional vehicle state vector.

[0055] The above parameters can be adjusted according to the actual sensor configuration and model input requirements, and the embodiments of the present invention do not impose any special limitations on this.

[0056] Through the above data processing, the embodiments of the present invention can simultaneously obtain macroscopic appearance information of the road surface, dynamic information of tire-road contact, information of vehicle posture disturbance, and information of vehicle operation status, and convert signals with different data formats and sampling frequencies into unified, synchronous and standardized multimodal data under the same vehicle driving segment.

[0057] Then, step S2 is performed to perform multimodal feature fusion.

[0058] This step aims to comprehensively analyze road surface images, tire radial acceleration, vehicle attitude angular velocity, and vehicle state scalars based on multimodal data to extract multimodal road environment features of the current vehicle driving area, and output multimodal road surface state representation and road surface category probability distribution, providing road environment conditions for subsequent generation of road condition-based health benchmarks.

[0059] Road surface images are used to characterize macroscopic appearance information such as road texture, color, cracks, water accumulation and reflection, potholes and damage, and paving materials; tire radial acceleration is used to characterize the dynamic response caused by road surface roughness, unevenness, and local impacts during tire-road contact; vehicle attitude angular velocity is used to characterize changes in vehicle pitch, roll, and yaw caused by road excitation; vehicle state scalars such as tire pressure, tire temperature, and vehicle speed are used to constrain the differences in dynamic response under different operating conditions. By fusing the above information, the problems of single visual information being easily affected by illumination, occlusion, and reflection, and single dynamic information being easily affected by vehicle operating conditions can be overcome, thereby improving the accuracy and stability of road surface condition recognition in complex road environments.

[0060] It should be noted that the tire radial acceleration signal and the vehicle attitude angular velocity signal are used in this step to extract road condition features related to road type and road excitation mode, and to assist in generating the road surface category probability distribution. In subsequent steps, the tire radial acceleration signal and the vehicle attitude angular velocity signal are reprocessed through an independent dynamic response encoding path to obtain the measured dynamic response features used for vehicle dynamic state comparison. Thus, this method, while retaining the multimodal road perception strategy, achieves a functional distinction between road condition recognition and vehicle dynamic response comparison.

[0061] In some embodiments, image feature extraction branches, tire radial acceleration feature extraction branches, vehicle attitude angular velocity feature extraction branches, and vehicle state feature extraction branches can be set to obtain road surface visual features respectively. Tire vibration characteristics Vehicle body attitude disturbance characteristics and vehicle status characteristics The above process can be represented as:

[0062] (4)

[0063] in, , , and These represent the image feature extraction branch, the tire radial acceleration feature extraction branch, the vehicle attitude angular velocity feature extraction branch, and the vehicle state feature extraction branch, respectively.

[0064] The image feature extraction branch is used to extract the macroscopic appearance and local visual semantics of the road surface; the tire radial acceleration feature extraction branch can extract transient impact, road vibration, and periodic change features through one-dimensional convolution and temporal convolution residual blocks with different dilation rates; the vehicle attitude angular velocity feature extraction branch can extract the attitude change features of vehicle pitch, roll, and yaw directions through one-dimensional convolution and temporal convolution residual blocks; the vehicle state feature extraction branch is used to characterize the influence of tire pressure, tire temperature, and vehicle speed on the dynamic response of tires and vehicle body. Each branch can use a feature-stitched multilayer perceptron, gating network, attention network, Transformer encoder, or hybrid expert network, etc., and this embodiment does not make specific limitations on this.

[0065] After obtaining the features of each modality, the features are mapped to a unified feature space, and multimodal road surface state features are formed through feature concatenation, weighted fusion, or attention fusion. :

[0066] (5)

[0067] in, This embodiment represents a multimodal feature fusion network. However, it is not limited to multimodal fusion networks. Those skilled in the art can refer to relevant solutions for implementation.

[0068] During the fusion process, road surface visual features provide information on road type and surface morphology, tire vibration features and vehicle posture disturbance features provide information on the dynamic response generated when the vehicle passes over the current road surface, and vehicle state features are used to constrain the differences in dynamic response under different vehicle speeds, tire pressures, and tire temperatures. Therefore, the fused road surface state representation can simultaneously reflect road appearance, tire-road contact excitation, vehicle posture changes, and vehicle operating conditions, improving the accuracy and stability of road surface state recognition in complex environments.

[0069] Multimodal road surface state characteristics Input the road surface category prediction branch to obtain the probability distribution of the current road surface belonging to each of the various road surface states:

[0070] (6)

[0071] in, Represents the probability distribution of road surface categories. The road surface category prediction branch is represented by the road surface category prediction branch. Those skilled in the art can construct the road surface category prediction branch of this embodiment using relevant classification prediction networks.

[0072] In one specific implementation, road surface categories include asphalt pavement, cement pavement, flooded pavement, and damaged pavement, and the probability distribution of road surface categories is expressed as follows:

[0073] (7)

[0074] in, , , and These represent the predicted probabilities of asphalt pavement, cement pavement, waterlogged pavement, and damaged pavement, respectively.

[0075] The multimodal road surface state characteristics can be obtained through the above embodiments. and road surface category probability distribution , It integrates road surface vision, tire vibration, vehicle posture and vehicle status characteristics to characterize the current road environment and operating conditions; Used to represent the probability and identification uncertainty of each road surface category. Based on and It can generate reference dynamic responses that are adapted to the current road environment and vehicle operating conditions.

[0076] This step can use a dual-branch temporal convolutional network to generate the current measured dynamic response characteristics of the vehicle.

[0077] For example, the tire radial acceleration branch receives a one-dimensional sequence of length 1000. The tire vibration features are obtained by sequentially passing the initial one-dimensional convolutional layer, temporal convolutional residual blocks with dilation rates of 1, 2, and 4, and a global average pooling layer. Its network structure can be represented as:

[0078] Conv1D(1,32,7,2)→TCNBlock(32,64,1)→TCNBlock(64,96,2)→TCNBlock(96,128,4);

[0079] The vehicle attitude angular velocity feature extraction branch receives vehicle attitude angular velocity sequences with a size of 50×3. The vehicle body attitude perturbation features are obtained by sequentially passing through a one-dimensional convolutional layer, two temporal convolutional residual blocks, and a global average pooling layer. The outputs of the two branches are concatenated, and the measured dynamic response is obtained through a feature adaptation layer from 192 to 128. :

[0080] (8)

[0081] in, This represents the feature adaptation layer.

[0082] This embodiment addresses the differences in sampling frequency, data dimension, and physical meaning between tire radial acceleration and vehicle attitude angular velocity, employing a dual-branch temporal convolutional network for feature extraction. In other embodiments, one-dimensional convolutional networks, multi-scale temporal convolutional networks, convolutional networks combined with GRU or LSTM, temporal Transformers, frequency domain networks, or time-frequency joint networks can also be used.

[0083] Next, step S3 is performed to perform feature modulation and construct the reference dynamic state.

[0084] Tire radial acceleration and vehicle attitude angular velocity are affected not only by tire condition but also by factors such as road surface roughness, pothole impact, vehicle speed, tire pressure, and tire temperature. A normal tire may experience a strong impact when traversing a damaged road surface, while an abnormal tire may only exhibit continuous, small fluctuations on a smooth road. Therefore, this step generates a dynamic reference dynamic response based on the extracted road environment conditions and vehicle operating conditions, avoiding the use of fixed thresholds for all road conditions.

[0085] In some embodiments, the road condition vector is generated through the following sub-steps:

[0086] S3.1 The road surface category probability distribution and the preset vehicle state information are spliced ​​together, and the spliced ​​features are input into the conditional parameter generation network to obtain the scaling vector and the offset vector;

[0087] S3.2 Based on the scaling vector and offset vector, the multimodal road surface state features are linearly modulated to obtain the modulated multimodal road surface state features;

[0088] S3.3 The modulated multimodal road surface state features, the road surface category probability distribution, and the preset vehicle state information are concatenated, and a road condition vector is generated through a fully connected network.

[0089] Specifically, the probability distribution of road surface categories is concatenated with a preset vehicle state vector to form a concatenated feature. :

[0090] (9)

[0091] splicing features The data is fed into a conditional parameter generation network to obtain a scaling vector that matches the dimension of the multimodal road surface state representation. and offset vector :

[0092] (10)

[0093] in, It is a conditional parameter generation network.

[0094] In one specific implementation, if the road surface category probability is 4-dimensional and the vehicle state scalar is 3-dimensional, the condition parameter generation network adopts a 7→128→512 fully connected network, and the output is divided into two 256-dimensional scaling vectors and offset vectors.

[0095] Subsequently, the 256-dimensional multimodal pavement state characterization was linearly modulated channel by channel:

[0096] (11)

[0097] Here, ⊙ represents element-wise multiplication. This represents the modulated multimodal road surface state characteristics.

[0098] In this way, the modulated multimodal road conditions can represent different conditions under different vehicle speeds, tire pressures, and tire temperatures, thereby establishing the correspondence between road environment, vehicle operating conditions, and tire dynamic response.

[0099] The modulated multimodal road state representation, road category probability, and preset vehicle state vector are concatenated and then passed through a fully connected network of 263→256→128 to obtain the road condition vector. :

[0100] (12)

[0101] In this embodiment of the invention, a batch normalization layer, a ReLU activation function, and a dropout layer are set between each fully connected layer of the fully connected network, and the dropout rate can be set to 0.2.

[0102] The vehicle system has a pre-established normal state reference template library. :

[0103] (13)

[0104] in, Index representing the road surface category, This represents the index of the normal state reference template under the corresponding road surface category. and These represent the mean and standard deviation of the normal vehicle dynamics response corresponding to the template, respectively.

[0105] In one implementation, four normal state reference templates are set for asphalt, cement, water accumulation and damaged road surface respectively, forming a total of 16 128-dimensional templates.

[0106] The normal state reference template library is used to characterize the typical dynamic response of a normal tire under different vehicle speeds and road surface conditions. Considering that vehicle speed significantly affects tire radial acceleration and vehicle attitude angular velocity, speed conditions are divided according to vehicle speed, and normal state reference templates are then established for each speed condition, including asphalt pavement, cement pavement, flooded pavement, and damaged pavement.

[0107] For each synchronous sampling time window, the average value of the vehicle speed samples within that window is used as the representative vehicle speed for the current window. Vehicle speed is divided into four intervals: low speed, low-medium speed, medium-high speed, and high speed. Under each speed interval, four normal road surface reference templates are set, forming a total of 16 128-dimensional health reference templates, as shown in the table below:

[0108]

[0109] When the average vehicle speed is below 5 km / h, the vehicle is stationary or nearly stationary. The radial acceleration of the tires is mainly affected by starting, braking, and static noise from the sensors, making it difficult to fully reflect the vehicle's dynamic state under road excitation. Therefore, within this speed range, the determination of the vehicle's dynamic state can be suspended, or static state monitoring can be performed solely based on tire pressure and temperature.

[0110] Each normal state reference template includes the template's central feature. and normal fluctuation range Let the normal state reference template for the g-th speed range and the i-th road surface be represented as:

[0111] (14)

[0112] in, This indicates the central feature of the template. This indicates the normal fluctuation range of the template across each feature dimension.

[0113] The normal-state reference template is built using data collected under normal tire conditions. For example, the tire radial acceleration and vehicle attitude angular velocity under normal conditions are input into the dynamic response encoding network to obtain the dynamic response characteristics of normal samples. Subsequently, normal samples are divided according to vehicle speed range and road surface category. Healthy samples belonging to the same speed range and the same road surface category are used to build a normal-state reference template, with their feature mean serving as the central feature of the template, and the standard deviation or variance of each feature dimension serving as the normal fluctuation range of the template.

[0114] In some embodiments, a reference dynamic state is generated through the following sub-steps:

[0115] S3.4 Determine candidate templates corresponding to the vehicle speed from the normal state reference template library. Each template in the normal state reference template library is configured with road surface category probability and the mean and standard deviation of the normal vehicle dynamics response corresponding to the template.

[0116] S3.5 Based on the similarity between the road condition vector and each candidate template, and the road surface category probability of each candidate template, determine the weight of each candidate template.

[0117] S3.6 The mean values ​​of each candidate template are weighted and summed based on their respective weights to obtain the reference mean value of the reference dynamic response;

[0118] S3.7 Calculate the difference between the mean of each candidate template and the reference mean, and obtain the reference standard deviation of the reference dynamic response based on the difference, the standard deviation of the corresponding candidate template, and the weight.

[0119] During vehicle operation, the vehicle system first determines the speed range based on the average speed of the current window. After determining the speed range, it calls up the four normal state reference templates within that speed range and no longer matches them with templates from other speed ranges.

[0120] For example, if the current average vehicle speed is 35 km / h, the vehicle is in a low-to-medium speed operating condition. The vehicle system only uses low-to-medium speed asphalt templates, low-to-medium speed cement templates, low-to-medium speed water accumulation templates, and low-to-medium speed damaged templates. Other low-speed, medium-to-high-speed, and high-speed templates do not participate in the generation of the current reference dynamic state.

[0121] The reason for selecting the template group based on vehicle speed in this embodiment is that the dynamic response of a normal tire differs significantly at different vehicle speeds. When the current vehicle speed is determined, templates for other speed ranges cannot accurately describe the normal response of the current vehicle. If all 16 templates are used in the calculation simultaneously, templates for other speed conditions may interfere with the current reference dynamic response.

[0122] Since step S2 determines the probability distribution of four road surfaces, rather than a single, absolutely determined road surface category, the vehicle system does not directly select one of the templates. For example, the current road may primarily be an asphalt surface, while also containing a certain degree of damage. In this case, the low-to-medium speed asphalt template should play a major role, and the low-to-medium speed damaged template can also appropriately modify the reference dynamic response.

[0123] Therefore, in this embodiment, after determining the current speed range, the vehicle system compares the similarity between the road condition vector and four templates within that speed range. Based on the similarity between the four templates and the road condition vector, and combined with the predicted probability of the corresponding road surface category for each template, a contribution weight is assigned to each template. The template that is closer to the current road condition and has a higher probability of corresponding road surface category receives a larger weight; the template that differs more from the current road condition receives a smaller weight. The contribution weights of the four templates are all non-negative, and the sum of the weights is 1.

[0124] In this embodiment, the similarity comparison is used to determine how close each template is to the current operating condition, while the weighted combination is used to convert the four candidate templates into a reference dynamic response under the current operating condition. Assuming the current vehicle speed belongs to the g-th speed range, the reference mean of the current reference dynamic response is expressed as:

[0125] (15)

[0126] in, This indicates the weight of the candidate template.

[0127] Based on formula (15), the vehicle system first determines the speed range g based on the average vehicle speed of the current window, and then selects four candidate templates for this speed range from the template library. Subsequently, based on the matching degree between the current road conditions and the four candidate templates, as well as the road surface category prediction probability of each candidate template, a contribution weight is assigned to each template. The central features of the four candidate templates are weighted and summed to obtain the reference mean of the reference dynamic response under the current working condition. .

[0128] This embodiment also needs to determine the allowable range of normal variation in the normal dynamic response under the current operating conditions, and the reference variance of the reference dynamic response on the j-th feature dimension. Represented as:

[0129] (16)

[0130] Where j represents the j-th dimension in the 128-dimensional dynamic response characteristics; This represents the normal fluctuations that are allowed to exist in the i-th candidate template within the g-th velocity interval; This represents the value of the corresponding template center in the j-th feature dimension; express The value in the j-th feature dimension.

[0131] The first term in formula (17) This represents the normal fluctuations allowed within the i-th candidate template within the g-th speed range. Since the dynamic response of a normal tire will not be completely consistent during multiple runs under the same vehicle speed and road conditions, each template has a certain normal range of variation. This embodiment considers two factors: first, the normal response fluctuations within each template; and second, the differences between the multiple candidate templates currently participating in the combination.

[0132] The second term in formula (17) This represents the difference between the center of a single candidate template and the center of the final reference dynamic response. This term reflects whether there are significant response differences among the four candidate templates currently participating in the weighted combination. When the centers of the four templates are relatively close, this term is small; when the centers of multiple candidate templates with higher weights differ significantly, this term increases, thereby correspondingly expanding the allowable fluctuation range of the normal vehicle dynamic response under the current operating conditions.

[0133] When the road surface recognition result is relatively clear, the weight of a certain candidate template dominates, and the differences between high-weight candidate templates are small. At this time, the normal fluctuation range generated is relatively narrow. When the current road has multiple road surface features at the same time, or when the road surface recognition result has a large degree of uncertainty, multiple candidate templates with large differences participate in the generation of reference dynamic response, and the normal fluctuation range increases accordingly.

[0134] For example, when driving steadily on a flat asphalt road, the asphalt template usually has the main weight, and the normal fluctuation range of the generated reference dynamic response is relatively concentrated. At the transition position between the asphalt road and the damaged area, the asphalt template and the damaged template may have high weight at the same time, and the normal fluctuation range of the generated reference dynamic response is correspondingly expanded, thereby avoiding the vehicle system from misjudging normal road changes as vehicle abnormalities.

[0135] In practical applications, to avoid frequent switching of candidate template groups due to slight fluctuations in average vehicle speed at the speed condition boundary, this embodiment can set a speed hysteresis range. For example, the hysteresis width can be set to 2 km / h, and the measurement system can only switch to the candidate template group of the adjacent speed condition after the average vehicle speed has crossed the corresponding boundary for three consecutive time windows.

[0136] It should be noted that the speed range and number of templates in the above embodiments are specific implementation methods. In actual applications, the speed range boundaries and number of templates can be adjusted according to the vehicle model, tire model, road test speed range and sample distribution, which does not constitute a limitation on the scope of protection of this invention.

[0137] It should be understood that vehicle speed plays a two-fold role in this step. First, the average vehicle speed of the current window is used to filter corresponding candidate template groups from the template library to eliminate template interference from other speed ranges. Second, the current vehicle speed still participates in the generation of conditional parameters and the construction of road condition vectors as part of the vehicle state scalar, used to describe continuous speed changes within the same speed range. For example, although 25 km / h and 38 km / h are both low-to-medium speed conditions, their vehicle dynamic states may still differ, thus requiring further fine-grained correction through conditional modulation.

[0138] In this embodiment, the normal state reference template library is used to characterize the normal dynamic response mode of a normal tire under different road environments and vehicle operating conditions.

[0139] This step ultimately outputs the reference mean of the reference dynamic response. and reference standard deviation .

[0140] In this embodiment, step S3 generates a road condition vector based on multimodal road surface state features, road surface category probability distribution, and preset state features. This road condition vector integrates the current road surface's appearance attributes, dynamic excitation mode, and operating condition information such as vehicle speed, tire pressure, and tire temperature, comprehensively depicting the vehicle's current road environment and operating state. Specifically, the road surface category probability distribution and preset state features are concatenated and input into a condition parameter generation network to obtain scaling and offset parameters. These parameters are then used to modulate the multimodal road surface state features channel-by-channel, resulting in differentiated conditional expressions under different vehicle speeds, tire pressures, and tire temperatures. The modulated multimodal road surface state features, road surface category probability distribution, and preset vehicle state information are then further fused to obtain the road condition vector.

[0141] This step also involves matching and generating a reference dynamic response from a pre-defined normal state reference template library based on the road condition vector. The normal state reference template library pre-stores the average dynamic response and normal fluctuation range of a normal tire under different road surface categories and vehicle operating conditions. By matching the road condition vector with each template (e.g., through an attention mechanism or distance metric), and weighting the templates according to matching weights, a reference dynamic response adapted to the current road environment and vehicle operating conditions is generated.

[0142] Through the above methods, this invention transforms the vehicle dynamics state discrimination criterion from a fixed threshold or a single mode into a response criterion that can dynamically adjust with road surface type, vehicle speed, tire pressure, and tire temperature. Therefore, this embodiment avoids the problem of normal road stimuli such as potholes and damage being misjudged as tire abnormalities due to the use of a fixed threshold for all road conditions, while ensuring sufficient sensitivity to minor anomalies on flat roads. Furthermore, by employing road surface category probability distribution for matching, when there is some uncertainty in road surface recognition, it can integrate template responses from multiple possible categories, preventing deviations between the reference dynamics response criterion and actual road conditions due to misjudgment of a single category, thereby enhancing the method's fault tolerance in complex scenarios.

[0143] Finally, step S4 is performed to determine the vehicle's dynamic state.

[0144] This step is based on measured dynamic response. Reference mean of reference dynamic response and reference standard deviation The vehicle's current dynamic response is calculated relative to a reference dynamic response, and a state score is calculated accordingly to achieve vehicle dynamic state monitoring.

[0145] The response bias in this step includes the conditionally normalized response bias and the overall response bias score.

[0146] Since the normal fluctuation range of tire dynamic response varies under different road conditions, this step first utilizes the reference state standard deviation of the reference dynamic state. The measured dynamic response characteristics and the mean of the reference state Conditional normalization is performed on the differences between them:

[0147] (17)

[0148] Where r represents the conditionally normalized response bias. To prevent extremely small positive numbers with a denominator of zero.

[0149] Through this normalization process, the same amplitude of dynamic change can yield different deviation results under different road conditions. For example, the normal dynamic response fluctuation range corresponding to damaged road surfaces is usually wider, while the normal dynamic response fluctuation range corresponding to flat road surfaces is relatively narrower. Therefore, this step can determine whether the measured dynamic response characteristics exceed the corresponding normal variation range based on the current road conditions.

[0150] Based on the conditionally normalized response deviation, calculate the overall deviation of the vehicle's current dynamic response:

[0151] (18)

[0152] in, The dimension representing the measured dynamic response, This represents the overall response deviation score.

[0153] when When the value is small, it indicates that the current measured dynamic response characteristics can be explained by the normal dynamic response benchmark under the corresponding road conditions and vehicle operating conditions; when A large value indicates that the measured dynamic response characteristics exceed the normal range of change of a normal tire under current conditions, and the vehicle is more likely to have deteriorated or malfunctioned.

[0154] To comprehensively extract the abnormal combination relationships among various response dimensions, this step concatenates the absolute value of the conditionally normalized response deviation, the square of the deviation, the overall response deviation score, and the preset vehicle state information to form state discrimination features. :

[0155] (19)

[0156] in, This represents the absolute value of the response deviation in each dimension, used to indicate the magnitude of the deviation. This represents the square of the response deviation in each dimension, used to amplify the effect of larger deviations. It is used to help determine the impact of abnormal tire pressure, temperature changes, and speed changes on the vehicle's dynamics.

[0157] In one specific implementation, The state evaluation network has 128 dimensions, with an input dimension of 260. The network employs a multilayer perceptron structure.

[0158] 260 → 128 → 64 → 1

[0159] After the first and second fully connected layers, a batch normalization layer, a ReLU activation function, and a dropout layer are applied, with a dropout rate of 0.3. The output layer uses a Sigmoid activation function to obtain the state score. :

[0160] (20)

[0161] in, Represents the state evaluation network. Status rating The closer the result is to 0, the closer the measured dynamic response is to the reference dynamic response; State score The closer it is to 1, the more significant the deviation of the current measured dynamic response from the reference dynamic response.

[0162] To reduce misjudgments caused by single pothole impacts, steering maneuvers, or instantaneous sensor disturbances, an exponential moving average is applied to the state scores obtained over a continuous time window:

[0163] (twenty one)

[0164] in, Represents the smoothing coefficient. This represents the smoothed result of the previous time window; the first time window can set the smoothed result as the initial state score.

[0165] Determine the vehicle dynamics state based on the smoothed state score:

[0166] (twenty two)

[0167] in, and These represent the threshold values ​​that define the boundaries between the normal, cautious, and warning states, respectively. These thresholds are determined through a validation set search, with the macro-average F1 score serving as the primary evaluation metric, prioritizing recall in the warning state.

[0168] In one specific implementation, it can be , As the initial value, the warning status is officially output when the warning conditions are met for three consecutive time windows.

[0169] In this embodiment, the normal state indicates that the current measured dynamic response is within the normal fluctuation range of a normal tire under the same road environment and operating conditions. At this time, the vehicle tires, suspension, and chassis system do not show any signs of response degradation that significantly exceeds the normal level of road excitation. The vehicle system only performs routine data recording and background monitoring, without triggering active intervention or driver prompts.

[0170] The "Caution" state indicates that the measured dynamic response has exceeded the normal fluctuation range of the reference dynamic response benchmark, but the deviation is still at a low level. This state may correspond to slight uneven tire wear, minor deviation of tire pressure from the standard value, or early changes in suspension damping characteristics. In this state, the vehicle system prompts the driver via the in-vehicle interface or mobile terminal to "recommend a routine inspection of the tires and chassis in the near future," but does not require immediate deceleration or stopping, thus providing the driver with a timely warning.

[0171] A warning status indicates that the measured dynamic response deviates significantly from the normal response benchmark under current operating conditions, and the vehicle has a high probability of safety risks such as tire structural damage (e.g., punctures, bulges, or severe uneven wear), rapid drop in tire pressure, excessively high tire temperature, or abnormal chassis connection components. In this state, the vehicle system immediately outputs audible and visual warning signals, prompting the driver to "immediately slow down and move to a safe area for inspection," and, if necessary, simultaneously uploads the warning information to the cloud management platform.

[0172] By classifying the vehicle health status into three levels as described above, the vehicle status degradation score of this invention is mapped into a graded early warning output with clear physical meaning and operational suggestions. This not only avoids frequent false alarms caused by single threshold alarms, but also provides drivers with hierarchical and interpretable decision support information, thereby improving the practicality of the early warning system and the user-friendliness of human-computer interaction.

[0173] As can be seen from the above embodiments, this step constructs state discrimination features based on the response deviation of the measured dynamic response relative to the reference dynamic response under the current road surface and vehicle operating conditions. This can distinguish between normal dynamic changes caused by complex road surfaces and abnormal responses that exceed the corresponding normal range, thereby improving the adaptability and reliability of vehicle response state discrimination under different road environments and vehicle operating conditions.

[0174] Figure 3 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 3 At the hardware level, the device includes a processor 310, an internal bus 320, a network interface 330, memory 340, a hardware acceleration device 350, and non-volatile memory 360, and may also include other hardware required for its functions. One or more embodiments of the present invention can be implemented in software, for example, the processor 310 reads the corresponding computer program from the non-volatile memory 360 into the memory 340 and then runs it. Of course, in addition to software implementation, one or more embodiments of the present invention do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0175] Figure 4 This is a structural block diagram illustrating a road condition adaptive vehicle dynamics state monitoring device according to an exemplary embodiment of the present invention. The vehicle dynamics state monitoring device can be applied to, for example... Figure 3 The electronic device shown implements the technical solution of the present invention. The vehicle dynamics state monitoring device includes:

[0176] The data preprocessing unit 410 is used to acquire multimodal data during vehicle driving, including road surface images, tire radial acceleration signals, vehicle body attitude angular velocity signals, and preset vehicle state information.

[0177] The feature fusion unit 420 is used to extract features from the multimodal data to obtain road surface visual features, tire vibration features, vehicle body posture disturbance features, and vehicle state features; fuse the road surface visual features, tire vibration features, vehicle body posture disturbance features, and vehicle state features to obtain multimodal road surface state features and perform road surface category prediction to obtain the probability distribution of road surface categories for vehicle travel; and fuse the tire vibration features and the vehicle body posture disturbance features to obtain the measured dynamic response.

[0178] The reference dynamic response construction unit 430 is used to generate a road condition vector based on the multimodal road surface state features, the road surface category probability distribution and the vehicle state features, and to match and generate a reference dynamic response from a preset normal state reference template library based on the road condition vector.

[0179] The dynamic state discrimination unit 440 is used to calculate the response deviation of the measured dynamic response relative to the reference dynamic response, construct state discrimination features based on the response deviation and the preset vehicle state information, input them into the state evaluation network, and determine the vehicle dynamic state monitoring result based on the state score.

[0180] In some embodiments, the reference dynamic response construction unit 430 is used to concatenate the road surface category probability distribution and the preset vehicle state information, input the concatenated features into a conditional parameter generation network to obtain a scaling vector and an offset vector; linearly modulate the multimodal road surface state features based on the scaling vector and the offset vector to obtain modulated multimodal road surface state features; concatenate the modulated multimodal road surface state features, the road surface category probability distribution and the preset vehicle state information, and generate a road condition vector through a fully connected network.

[0181] In some embodiments, the preset vehicle state information includes vehicle speed. The reference dynamic response construction unit 430 is used to determine candidate templates corresponding to the vehicle speed from the normal state reference template library. Each template in the normal state reference template library is configured with road surface category probability and the mean and standard deviation of the normal vehicle dynamic response corresponding to the template. Based on the similarity between the road condition vector and each candidate template and the road surface category probability of each candidate template, the weight of each candidate template is determined. Based on the weight of each candidate template, the mean of each candidate template is weighted and summed to obtain the reference mean of the reference dynamic response. The difference between the mean of each candidate template and the reference mean is calculated, and based on the difference, the standard deviation of the corresponding candidate template, and the weight, the reference standard deviation of the reference dynamic response is obtained.

[0182] In some embodiments, the dynamic state discrimination unit 440 is used to compare the state score with a preset state boundary threshold, and determine the dynamic state of the vehicle based on the comparison result.

[0183] In some embodiments, the dynamic state discrimination unit 440 is used to perform an exponential moving average on the state score under a continuous time window, and compare the averaged state score with a preset state boundary threshold.

[0184] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0185] Accordingly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the above embodiments.

[0186] Accordingly, embodiments of the present invention also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0187] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0188] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0189] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0190] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0191] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0192] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0193] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0194] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0195] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A road condition adaptive vehicle dynamics state monitoring method, characterized in that, The method includes the following steps: Step S1: Acquire multimodal data during vehicle operation. The multimodal data includes road surface images, tire radial acceleration signals, vehicle body attitude angular velocity signals, and preset vehicle state information. Step S2 involves extracting features from the multimodal data to obtain road surface visual features, tire vibration features, vehicle body posture disturbance features, and vehicle state features. These features are then fused to obtain multimodal road surface state features, and road surface category prediction is performed to obtain the probability distribution of the road surface category for vehicle travel. Finally, the tire vibration features and vehicle body posture disturbance features are fused to obtain the measured dynamic response. Step S3: Based on the multimodal road surface state features, the road surface category probability distribution, and the vehicle state features, generate a road condition vector, and based on the road condition vector, match and generate a reference dynamic response from a preset normal state reference template library; Step S4: Calculate the response deviation of the measured dynamic response relative to the reference dynamic response, construct state discrimination features based on the response deviation and the preset vehicle state information, input them into the state evaluation network, and determine the vehicle dynamic state monitoring result based on the state score.

2. The method according to claim 1, characterized in that, The step S3 of generating the road condition vector includes: The road surface category probability distribution and the preset vehicle state information are concatenated, and the concatenated features are input into the conditional parameter generation network to obtain the scaling vector and the offset vector. The multimodal road surface state features are linearly modulated based on the scaling vector and the offset vector to obtain the modulated multimodal road surface state features. The modulated multimodal road surface state features, the road surface category probability distribution, and the preset vehicle state information are concatenated and a road condition vector is generated through a fully connected network.

3. The method according to claim 1, characterized in that, The preset vehicle state information includes vehicle speed, and the generation of reference dynamic state in step S3 includes: Candidate templates corresponding to the vehicle speed are determined from the normal state reference template library. Each template in the normal state reference template library is configured with road surface category probability and the mean and standard deviation of the normal vehicle dynamics response corresponding to the template. The weight of each candidate template is determined based on the similarity between the road condition vector and each candidate template, as well as the road surface category probability of each candidate template. The mean values ​​of each candidate template are weighted and summed based on their respective weights to obtain the reference mean value of the reference dynamic response. The difference between the mean of each candidate template and the reference mean is calculated. Based on the difference, the standard deviation of the corresponding candidate template, and the weight, the reference standard deviation of the reference dynamic response is obtained.

4. The method according to claim 1, characterized in that, The response bias in step S4 includes the conditionally normalized response bias and the overall response bias score, wherein: The conditionally normalized response bias ; The overall response deviation score ; in, The measured dynamic response is... The reference mean of the reference dynamic response, The reference standard deviation is the reference dynamic response. To prevent the use of a pre-defined minimum positive number with a denominator of zero, Let be the dimension of the measured dynamic response.

5. The method according to claim 4, characterized in that, The expression for the state discrimination feature in step S4 is: ； in, The state discrimination feature, This refers to the preset vehicle status information.

6. The method according to claim 1, characterized in that, The step S4, which determines the vehicle dynamics state monitoring results based on the state score, includes: The state score is compared with a preset state boundary threshold, and the dynamic state of the vehicle is determined based on the comparison result.

7. The method according to claim 6, characterized in that, Step S4 further includes: An exponential moving average is applied to the state scores within a continuous time window, and the averaged state scores are compared with preset state boundary thresholds.

8. A vehicle dynamics state monitoring device that adapts to road conditions, characterized in that, The device includes: The data preprocessing unit is used to acquire multimodal data during vehicle operation, including road surface images, tire radial acceleration signals, vehicle body attitude angular velocity signals, and preset vehicle state information. The feature fusion unit is used to extract features from the multimodal data to obtain road surface visual features, tire vibration features, vehicle body posture disturbance features, and vehicle state features; fuse the road surface visual features, tire vibration features, vehicle body posture disturbance features, and vehicle state features to obtain multimodal road surface state features and perform road surface category prediction to obtain the probability distribution of road surface categories for vehicle travel; and fuse the tire vibration features and the vehicle body posture disturbance features to obtain the measured dynamic response. The reference dynamic response construction unit is used to generate a road condition vector based on the multimodal road surface state features, the road surface category probability distribution and the vehicle state features, and to match and generate a reference dynamic response from a preset normal state reference template library based on the road condition vector. The dynamic state discrimination unit is used to calculate the response deviation of the measured dynamic response relative to the reference dynamic response, construct state discrimination features based on the response deviation and the preset vehicle state information, input them into the state evaluation network, and determine the vehicle dynamic state monitoring result based on the state score.

9. An electronic device, characterized in that, include: processor; as well as A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a processor according to any one of claims 1 to 7.

Images