Data processing method, storage medium, program product, device, system and vehicle

Abstract

The invention relates to a data processing method, a storage medium, a program product, equipment, a system and a vehicle. The data processing method comprises the steps of estimating parameter values of pavement parameters of a road where a vehicle is located according to driving state data of the vehicle; and associating the parameter values of the road surface parameters with the driving environment data of the vehicle to construct a training data set of the road surface perception model. The method can efficiently construct the training data set, facilitates the improvement of the training efficiency and accuracy of the pavement perception model, and achieves the efficient deployment of the pavement perception model.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a data processing method, storage medium, program product, device, system and vehicle. Background Art

[0002] To improve vehicle safety and stability, real-time perception of the physical properties of the road surface is required. This allows intelligent driving systems and drivers to adjust the vehicle's driving strategy and control parameters based on these perceptions. Artificial intelligence-based road perception models are gaining increasing attention due to their ability to identify the physical properties of various road surfaces. However, training road perception models to improve their training efficiency and accuracy, thereby enabling their efficient deployment, remains a pressing technical challenge. Summary of the Invention

[0003] The embodiments of the present application provide a data processing method, storage medium, program product, device, system and vehicle, which can efficiently construct a training data set, help improve the training efficiency and accuracy of the road perception model, and achieve efficient deployment of the road perception model, so as to at least partially solve the above-mentioned technical problems.

[0004] In order to achieve the above-mentioned purpose, according to the first aspect of the present application, a data processing method is provided, including: estimating the parameter values ​​of the road surface parameters of the road where the vehicle is located based on the vehicle's driving status data; associating the parameter values ​​of the road surface parameters with the vehicle's driving environment data to construct a training data set for a road surface perception model.

[0005] Optionally, the driving status data includes at least one of the following: motion data and control data.

[0006] Optionally, the motion data includes at least one of the following: speed, acceleration, wheel speed, and driving trajectory.

[0007] Optionally, the control data includes at least one of the following: throttle data, steering wheel angle data, and front wheel angle data.

[0008] Optionally, the road surface parameters include at least one of the following: traction coefficient, adhesion coefficient, slip rate, and friction coefficient.

[0009] Optionally, estimating the parameter values ​​of the pavement parameters of the road where the vehicle is located based on the vehicle's driving status data includes: solving an optimization objective function based on the vehicle's driving status data to obtain the parameter values ​​of the pavement parameters of the road where the vehicle is located.

[0010] Optionally, the optimization objective function includes minimizing at least one of the following: the difference between the estimated value of the vehicle's state at the current moment and the estimated value at the previous moment; the difference between the estimated value of the road surface parameter at the current moment and the estimated value at the previous moment; the difference between the estimated value of the vehicle's state in the current time period and the observed value.

[0011] Optionally, solving the optimization objective function based on the vehicle's driving status data to obtain parameter values ​​of the pavement parameters of the road where the vehicle is located includes: solving the optimization objective function based on the vehicle's driving status data to obtain parameter values ​​of the pavement parameters of the road where the vehicle is located while satisfying constraints.

[0012] Optionally, the method further includes: inputting an estimated value of the vehicle's state at the current moment into the vehicle's dynamic model to obtain a predicted value of the vehicle's state at the next moment; and constructing the constraint condition based on the predicted value and estimated value of the vehicle's state at the next moment.

[0013] Optionally, associating the parameter values ​​of the road parameters with the driving environment data of the vehicle to construct a training data set for the road perception model includes: marking the parameter values ​​of the road parameters in the driving environment data of the vehicle to construct a training data set for the road perception model.

[0014] Optionally, the parameter values ​​of the road parameters are labeled in the driving environment data of the vehicle to construct a training data set for the road perception model, including: labeling the parameter values ​​of the road parameters in the driving environment data of the vehicle to obtain a first labeled sample; determining a training sample based on the first labeled sample and the perspective type of the road perception model; and constructing a training data set for the road perception model based on the training sample.

[0015] Optionally, marking the parameter value of the road surface parameter in the driving environment data of the vehicle to obtain a first marked sample includes: in the driving environment data of the vehicle, aligning the parameter value of the road surface parameter and the driving trajectory of the vehicle according to timestamps to obtain the first marked sample.

[0016] Optionally, determining the training sample based on the first labeled sample and the perspective type of the road surface perception model includes: if the perspective type of the road surface perception model does not match the first labeled sample, performing perspective conversion processing on the first labeled sample to obtain the training sample.

[0017] Optionally, when the perspective type of the road surface perception model is a first-person perspective and the first labeled sample is based on a bird's-eye view, the perspective type of the road surface perception model does not match the first labeled sample.

[0018] Optionally, performing perspective conversion processing on the first labeled sample to obtain a training sample includes: performing coordinate system conversion processing on the first labeled sample to obtain a second labeled sample; and performing foreground filtering processing on the second labeled sample to obtain a training sample.

[0019] Optionally, the foreground filtering processing is performed on the second labeled sample to obtain a training sample, including: if the driving trajectory in the first labeled sample has foreground occlusion in the second labeled sample, then from the second labeled sample, the trajectory portion with the foreground occlusion and the parameter value of the road surface parameter associated with the trajectory portion are removed to obtain a training sample.

[0020] Optionally, the method further includes: training the road surface perception model based on the training data set.

[0021] Optionally, training the road surface perception model based on the training data set includes: determining a predicted loss value of the road surface perception model based on the training data set; and adjusting parameters of the road surface perception model based on the predicted loss value.

[0022] Optionally, determining the predicted loss value of the road surface perception model based on the training data set includes: inputting the driving environment data in the training data set into the road surface perception model so that the road surface perception model outputs the predicted value of the road surface parameter; determining the predicted loss value of the road surface perception model based on the predicted value of the road surface parameter and the parameter value of the road surface parameter in the training data set.

[0023] Optionally, determining the predicted loss value of the road surface perception model based on the predicted value of the road surface parameter and the parameter value of the road surface parameter in the training data set includes: for each training sample in the training data set, determining the predicted loss value of the training sample based on the predicted value and parameter value of the road surface parameter under the training sample; determining the predicted loss value of the road surface perception model based on the predicted loss values ​​and weight values ​​of multiple training samples.

[0024] Optionally, the method further includes: determining a plurality of value intervals of the pavement parameter based on the parameter value of the pavement parameter in the training data set; for each of the training samples, determining the value interval corresponding to the training sample based on the parameter value of the pavement parameter under the training sample; and determining the weight value of the training sample based on the number of the plurality of value intervals and the probability that the parameter value of the pavement parameter is within the value interval corresponding to the training sample.

[0025] Optionally, adjusting the parameters of the road surface perception model according to the predicted loss value includes: determining a target loss value of the road surface perception model according to the predicted loss value and the depth loss value of the road surface perception model; and adjusting the parameters of the road surface perception model according to the target loss value.

[0026] According to a second aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned data processing method is implemented.

[0027] According to a third aspect of the present application, a computer program product is provided, comprising a computer program, wherein the computer program implements the above-mentioned data processing method when executed by a processor.

[0028] According to a fourth aspect of the present application, an electronic device is provided, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the above-mentioned data processing method.

[0029] According to a fifth aspect of the present application, a data processing system is provided, which includes a data labeling module; wherein the data labeling module is used to: estimate the parameter values ​​of the road surface parameters of the road where the vehicle is located based on the vehicle's driving status data; and associate the parameter values ​​of the road surface parameters with the vehicle's driving environment data to construct a training data set for a road surface perception model.

[0030] Optionally, the data processing system further includes a first acquisition module; wherein the first acquisition module is used to send the driving status data to the data labeling module.

[0031] Optionally, the first acquisition module includes at least one of the following: an inertial measurement unit, a positioning unit, a wheel speed meter, and a sensor.

[0032] Optionally, the data processing system further includes a second acquisition module; wherein the second acquisition module is used to send the driving environment data to the data labeling module.

[0033] Optionally, the second acquisition module includes at least one of the following: a camera, a radar.

[0034] Optionally, the data processing system further includes a model training module; wherein the model training module is used to train the road surface perception model based on the training data set.

[0035] According to a sixth aspect of the present application, a vehicle is provided, comprising the above-mentioned electronic device or the above-mentioned data processing system.

[0036] In summary, the technical solution provided by the embodiment of the present application estimates the parameter values ​​of the road parameters of the road where the vehicle is located based on the vehicle's driving state data; then the parameter values ​​of the road parameters are associated with the vehicle's driving environment data to construct a training data set. Among them, the training data set can be used to train the road perception model so that the road perception model predicts the parameter values ​​of the road parameters corresponding to the driving environment data based on the driving environment data acquired in real time, thereby realizing real-time and accurate perception of the physical characteristics of the road surface. The embodiment of the present application separately models the driving state data based on the chassis model and the driving environment data based on the visual features, establishes a road parameter estimation problem based on the driving state data to estimate the parameter values ​​of the road parameters; then performs data annotation based on the parameter estimation results, establishes an association between the driving environment data and the estimated parameter values ​​of the road parameters to construct a training data set. The data annotation method provided by the embodiment of the present application is simple and accurate, and can efficiently construct a training data set, which helps to improve the training efficiency and accuracy of the road perception model, reduces the development difficulty of the road perception model, shortens the development cycle of the road perception model, and enhances the training interpretability of the road perception model, which helps to efficiently deploy the road perception model. In addition, the road perception model trained according to the training data set constructed according to the embodiment of the present application can predict the parameter values ​​of the road parameters in real time based on the input driving environment data. The input content required for model inference and prediction is simple, which avoids the encoding of driving status data and the fusion of driving status data and driving environment data when the road perception model is used, and helps to reduce the computing resource consumption of model inference and prediction.

[0037] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0039] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same drawing numbers represent the same parts in the following description.

[0040] Figure 1 This is a flow chart of a data processing method provided by an embodiment of the present application;

[0041] Figure 2 is a schematic diagram of a prediction result of a road surface perception model provided in an embodiment of the present application;

[0042] Figure 3 This is a schematic diagram of data annotation provided in an embodiment of the present application;

[0043] Figure 4 is a flow chart of another data processing method provided in an embodiment of the present application;

[0044] Figure 5 is a flow chart of another data processing method provided in an embodiment of the present application;

[0045] Figure 6 is a schematic diagram of a data processing system provided in an embodiment of the present application;

[0046] Figure 7 It is a schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0048] Currently, research on environmental perception for Advanced Driving Assistance Systems (ADAS) mainly focuses on urban traffic environments, where the detected objects (such as lanes, traffic signs, buildings, and roads) mainly originate from man-made scenes. These data usually have rich annotations and can be used to train AI-based perception models.

[0049] Due to its unstructured nature, autonomous driving in off-road environments still faces the challenge of a lack of large-scale, densely annotated datasets, hindering the development and application of AI models in these environments. In complex off-road conditions, due to the lack of manually maintained paved roads like urban roads, vehicles must additionally assess the physical permeability of the road surface and the changes in their operating state as they interact with the environment. Therefore, it is necessary to develop AI-based road perception models to perceive unstructured road information.

[0050] The AI-based road surface perception model can identify the physical characteristics of various road surfaces, such as wetness, soil type, slope, and surface smoothness. It can also accurately identify unpaved surfaces such as sand and snow. Based on this road surface perception, the intelligent driving system can dynamically adjust the vehicle's driving strategy and control parameters. For example, after identifying different types of unpaved roads, the intelligent driving system can adjust acceleration, deceleration, steering, and braking algorithms to adapt to different grip and stability requirements, ensuring safe driving for the driver.

[0051] In related technologies, there are two main road perception models:

[0052] Solution 1: Develop a road surface perception model based on geometric and depth information. The developed road surface perception model primarily estimates traffic flow based on road surface shape. This solution ignores road surface semantic information, resulting in an underestimation of physical properties such as adhesion and traction coefficients. This approach is unsuitable for off-road environments and unpaved roads, where physical properties significantly limit vehicle trafficability. For example, a snowy road surface may be geometrically smooth but have a low adhesion coefficient. Excessive acceleration during driving can cause side slip, significantly impacting the vehicle's trafficability.

[0053] Solution 2: Develop a road perception model based on deep learning. The developed road perception model directly inputs data such as images or point clouds, learns the geometric and semantic information of the road surface, and automatically infers the risks or consequences of the vehicle driving on the road surface. This solution involves model training, and there are usually two ways to perform model training. One is supervised learning based on manual annotation, and the other is self-supervised learning through the interaction between the chassis and different terrains. Supervised learning faces problems such as the difficulty in formulating annotation rules and high annotation costs due to the unstructured nature of unpaved roads. Self-supervised learning requires a large amount of data for training because it learns both the physical model of the ground and the vehicle chassis dynamics model. In addition, the perception results generated by the trained road perception model will produce unnecessary swings in the path generated after passing through the regulation and control module, and the robustness is poor.

[0054] In view of this, embodiments of the present application provide a data processing method, storage medium, program product, device, system, and vehicle. These methods not only address the generalization and training data deficiencies associated with supervised learning, which require extensive and complex manual annotation and are difficult to label, but also improve the robustness and interpretability deficiencies of self-supervised learning. The embodiments of the present application effectively construct a method for estimating road surface parameters based on a chassis dynamics model. This method combines an existing, design-verified chassis dynamics model with actual road conditions to systematically analyze and annotate the physical properties of various road surfaces. For example, using data such as wheel slip, tire pressure, and vehicle acceleration response, the traction coefficient under specific road conditions can be derived. The embodiments of the present application can effectively infer the vehicle's driving performance on different road surfaces, enhancing the interpretability of the overall model. Furthermore, the embodiments of the present application can create a rich training dataset at a low development cost, providing a foundation for training a road perception model. The trained road perception model can more accurately identify and classify different road types, thereby improving the decision-making ability and safety of the autonomous driving system in complex environments. It also avoids the oscillation problem that occurs in the trajectory generated by the trained model, enhancing the robustness of the trained model.

[0055] According to the first aspect of the present application, an embodiment of the present application provides a data processing method.

[0056] See also Figure 1 , Figure 1 This is a flow chart of a data processing method provided by an embodiment of the present application. Figure 1 As shown, the data processing method may include the following steps:

[0057] Step S100: estimating the parameter values ​​of the road surface parameters of the road where the vehicle is located based on the vehicle's driving state data;

[0058] Step S200: Associating the parameter values ​​of the road surface parameters with the driving environment data of the vehicle to construct a training data set for the road surface perception model.

[0059] Driving state data indicates the vehicle's state during driving. The embodiments of the present application do not limit the specific content of the driving state data. In some embodiments, the driving state data includes at least one of the following: motion data and control data. Motion data indicates the vehicle's motion during driving. In some embodiments, the motion data includes at least one of the following: speed, acceleration, wheel speed, and driving trajectory. Control data indicates the vehicle's control state during driving. In some embodiments, the control data includes at least one of the following: throttle data, steering wheel angle data, and front wheel angle data. By acquiring driving state data from multiple dimensions, the embodiments of the present application can enrich and improve the driving state data, more vividly depicting the vehicle's state during driving and helping to improve the accuracy of subsequent estimations. Furthermore, when acquiring driving state data, the embodiments of the present application can also simultaneously acquire timestamps for the driving state data. This allows for alignment of the timestamps across multiple driving state data, ensuring temporal consistency and improving the accuracy of the driving state data. For a detailed description of how to acquire driving state data, please refer to the following embodiments and will not be elaborated upon here.

[0060] In step S100, the parameter values ​​of the road surface parameters of the road on which the vehicle is located can be estimated based on the vehicle's driving state data. The embodiments of the present application do not limit the specific content of the road surface parameters to be estimated. In actual applications, one or more road surface parameters can be flexibly set based on actual needs. In some embodiments, the road surface parameters include at least one of the following: traction coefficient, adhesion coefficient, slip ratio, and friction coefficient. By estimating multiple road surface parameters, the physical properties of the road surface on which the vehicle is located can be more accurately indicated. The embodiments of the present application can associate the vehicle's driving state data with the road surface parameters of the road on which the vehicle is located based on a vehicle dynamics model to establish a road surface parameter estimation problem to achieve parameter value estimation of the road surface parameters. For a detailed description of the parameter value estimation process of the road surface parameters, please refer to the following embodiments and will not be elaborated here. Through the processing of step S100, the parameter values ​​of the road surface parameters at each node during the vehicle's driving process can be obtained. The driving state data can be accompanied by a timestamp, and thus the parameter values ​​of the road surface parameters estimated based on the driving state data can also be accompanied by a corresponding timestamp to facilitate subsequent association with other data.

[0061] The driving environment data is used to indicate the state of the surrounding environment of the vehicle during driving. The driving environment data can be image data or point cloud data, etc., which is not limited in the embodiments of the present application. For an introduction to the method of obtaining the driving environment data, please refer to the following embodiments, which will not be elaborated here. When obtaining the driving environment data, the timestamp of the driving environment data can be obtained at the same time to facilitate association with other data. In step S200, the parameter values ​​of the road surface parameters and the driving environment data of the vehicle are associated. For example, the parameter values ​​of the road surface parameters and the driving environment data of the vehicle can be associated through the timestamp. Based on the associated driving state data and the parameter values ​​of the road surface parameters, a training data set can be constructed, and the training data set can be used to train a road surface perception model. Each training sample in the training data set can include a driving state data and the parameter value of the road surface parameter corresponding to the driving state data, so that the road surface perception model trained based on the training data set can predict the parameter value of the road surface parameter corresponding to the driving environment data based on the driving environment data obtained in real time, thereby realizing real-time and accurate perception of the physical properties of the road surface.

[0062] In summary, the technical solution provided by the embodiment of the present application estimates the parameter values ​​of the road surface parameters of the road where the vehicle is located based on the vehicle's driving state data; then the parameter values ​​of the road surface parameters are associated with the vehicle's driving environment data to construct a training data set. Among them, the training data set can be used to train the road surface perception model so that the road surface perception model predicts the parameter values ​​of the road surface parameters corresponding to the driving environment data based on the driving environment data acquired in real time, thereby realizing real-time and accurate perception of the physical characteristics of the road surface. The embodiment of the present application separately models the driving state data based on the chassis model and the driving environment data based on the visual features, establishes a road surface parameter estimation problem based on the driving state data to estimate the parameter values ​​of the road surface parameters; then performs data annotation based on the parameter estimation results, establishes an association between the driving environment data and the estimated parameter values ​​of the road surface parameters to construct a training data set. The data annotation method provided by the embodiment of the present application is simple and accurate, and can efficiently construct a training data set, which helps to improve the training efficiency and accuracy of the road surface perception model, reduces the development difficulty of the road surface perception model, shortens the development cycle of the road surface perception model, and enhances the training interpretability of the road surface perception model, which helps to efficiently deploy the road surface perception model. In addition, the road perception model trained according to the training data set constructed according to the embodiment of the present application can predict the parameter values ​​of the road parameters in real time based on the input driving environment data. The input content required for model inference and prediction is simple, which avoids the encoding of driving status data and the fusion of driving status data and driving environment data when the road perception model is used, and helps to reduce the computing resource consumption of model inference and prediction.

[0063] In some embodiments, the above step S100 may include: solving the optimization objective function according to the driving state data of the vehicle to obtain parameter values ​​of the road surface parameters of the road where the vehicle is located.

[0064] To achieve pavement parameter estimation, an optimization-based estimation problem can be established to improve the accuracy of pavement parameter estimation. The core of this method is to solve the optimization objective function. The optimization objective function is a mathematical expression used to measure the quality of the solution in an optimization problem. It can be used to quantify the difference between the current solution (including the estimated values ​​of the pavement parameters) and the ideal solution. By minimizing the optimization objective function, the optimal solution can be obtained, thereby estimating the parameter values ​​of the pavement parameters.

[0065] The embodiments of the present application can construct an optimization objective function based on the vehicle's dynamic model. The vehicle's dynamic model refers to a mathematical model used to describe the vehicle's motion, including but not limited to the vehicle's longitudinal dynamics (such as changes in speed and acceleration) and lateral dynamics (such as steering, lateral acceleration, etc.) equations, as well as the vehicle's posture dynamics (such as pitch, yaw, etc.) equations. The vehicle's dynamic model can be used to establish an observation model, which can associate the vehicle's state (the vehicle's actual motion state) with the kinematic quantities (such as speed, displacement, etc.) observed by the sensor, and estimate the vehicle's state through the kinematic quantities observed by the sensor. Further, based on the dynamic model and the observation model, an optimization objective function can be constructed. To improve the accuracy of road surface parameter estimation, in some embodiments, the optimization objective function includes minimizing at least one of the following: the difference between the estimated value of the vehicle's state at the current moment and the estimated value at the previous moment; the difference between the estimated value of the road surface parameter at the current moment and the estimated value at the previous moment; the difference between the estimated value of the vehicle's state in the current time period and the observed value. Among them, the difference between the estimated value of the vehicle state at the current moment and the estimated value at the previous moment is the initial state consistency term in the optimization objective function, which is used to ensure the continuity and stability of the estimation; the difference between the estimated value of the road surface parameter at the current moment and the estimated value at the previous moment is the road surface parameter smoothness term in the optimization objective function, which is used to ensure the smoothness of the estimation; the difference between the estimated value and the observed value of the vehicle state in the current time period is the state observation consistency term in the optimization objective function, which is used to ensure the consistency of the estimation.

[0066] For example, the optimization objective function may be as shown in the following formula 1.

[0067]

[0068] in, is the initial state consistency term, x k is the estimated value of the vehicle's state at the current moment, is the estimated value of the vehicle's state at the previous moment, Px is the weight matrix; is the road surface parameter smoothness term, p r is the estimated value of the road surface parameters at the current moment, is the estimated value of the road surface parameters at the previous moment, P m is the weight matrix; is the state observation consistency term, x i is the estimated value of the vehicle's state at the current time in the current time period, h(z i ) is the observed value of the vehicle's state at the current time of the current time period, P w is a weight matrix. The vehicle state includes trajectory, posture, speed, etc. In complex dynamic models, it can also include tire speed (i.e., wheel speed), etc. The trajectory includes the position in the local coordinate system.

[0069] To enhance the stability and continuity of the estimation results, constraints may be set. In some embodiments, solving the optimization objective function based on the vehicle's driving state data to obtain parameter values ​​for the road surface parameters of the vehicle's road may include: solving the optimization objective function based on the vehicle's driving state data to obtain parameter values ​​for the road surface parameters of the vehicle's road, while satisfying the constraints. The constraints may be constructed based on a vehicle dynamics model. In some embodiments, the data processing method may further include: inputting an estimated value of the vehicle's state at a current moment into the vehicle's dynamics model to obtain a predicted value of the vehicle's state at a next moment; and constructing a constraint based on the predicted and estimated values ​​of the vehicle's state at the next moment. For example, the constraint may include requiring that the predicted and estimated values ​​of the vehicle's state at the next moment be equal. Of course, other constraints may also be set in practical applications, such as setting the vehicle's physical limits (maximum speed, maximum acceleration, etc.) and a reasonable range of road surface parameters.

[0070] For example, the constraint condition may be as shown in the following formula 2.

[0071] Formula 2: x k+1 =f(x k ,u k )

[0072] Among them, x k is the estimated value of the vehicle's state at the current moment; x k+1 is the predicted value of the vehicle's state at the next moment; u k is the control input of the vehicle at the current moment, such as the throttle, steering wheel angle, etc.; f(·) is the dynamic model of the vehicle.

[0073] In summary, the embodiments of the present application establish an optimization-based estimation problem to achieve road surface parameter estimation. The constructed optimization objective function comprehensively considers the consistency of vehicle state, the smoothness of road surface parameters, and the consistency of state observations, effectively establishing a mathematical connection between driving state data and road surface parameters. By minimizing the optimization objective function, the optimal road surface parameter estimates can be obtained while satisfying vehicle dynamics constraints. The embodiments of the present application not only improve the accuracy of road surface parameter estimation, but also ensure the stability and reliability of the estimation results.

[0074] In some embodiments, the above step S200 may include: marking parameter values ​​of road parameters in the vehicle's driving environment data to construct a training data set for the road perception model.

[0075] By marking the parameter values ​​of the road surface parameters in the driving environment data of the vehicle, the relationship between the driving environment data and the parameter values ​​of the road surface parameters can be intuitively displayed.

[0076] To improve the prediction accuracy and pertinence of the road surface perception model, the perspective type of the training samples in the training dataset can match the perspective type of the road surface perception model. Of course, in actual applications, to enable the road surface perception model to predict road surface parameters based on driving environment data from various perspectives, the training dataset can include training samples from multiple perspective types, but this embodiment of the application is not limited to this.

[0077] In some embodiments, the above-mentioned step of labeling the parameter values ​​of the road surface parameters in the vehicle driving environment data to construct a training data set for the road surface perception model may include the following steps:

[0078] Step S210: marking parameter values ​​of road parameters in the vehicle's driving environment data to obtain a first marked sample;

[0079] Step S220: determining a training sample based on the first labeled sample and the perspective type of the road perception model;

[0080] Step S230: constructing a training data set for the road surface perception model based on the training samples.

[0081] To streamline the training samples in the training data set, multiple parameter values ​​of the road surface parameters can be annotated in a set of driving environment data. For example, the parameter values ​​of the road surface parameters along a section of a driving trajectory can be annotated in the set of driving environment data. Based on this, in some embodiments, the above-mentioned step S210 may include: in the vehicle's driving environment data, aligning the parameter values ​​of the road surface parameters with the vehicle's driving trajectory according to the timestamp to obtain a first annotated sample. In particular, when obtaining the vehicle's driving trajectory, the timestamps of the coordinates of each position in the driving trajectory can be obtained, so that the parameter values ​​of the road surface parameters can be aligned with the vehicle's driving trajectory according to the timestamps, thereby annotating the parameter values ​​of the road surface parameters in the driving environment data.

[0082] Based on the first annotated sample obtained by the initial annotation, it can be determined whether a perspective conversion process is required for the first annotated sample according to the perspective type of the road surface perception model to obtain a training sample. The perspective type of the road surface perception model can be matched with the first annotated sample, and whether a perspective conversion process is required can be determined based on the matching result.

[0083] In an embodiment of the present application, the perspective type of the road surface perception model matches the first annotated sample, which may mean that the perspective type of the road surface perception model is consistent with the perspective type of the driving environment data in the first annotated sample, for example, both are first-person perspectives (FPV) or both are bird's-eye perspectives (BEV); accordingly, the perspective type of the road surface perception model does not match the first annotated sample, which may mean that the perspective type of the road surface perception model is inconsistent with the perspective type of the driving environment data in the first annotated sample, for example, one is a first-person perspective and the other is a bird's-eye perspective. In some embodiments, when the perspective type of the road surface perception model is a first-person perspective and the first annotated sample is based on a bird's-eye perspective, the perspective type of the road surface perception model does not match the first annotated sample. In addition, when the perspective type of the road surface perception model is a bird's-eye perspective and the first annotated sample is based on a first-person perspective, it can also be considered that the perspective type of the road surface perception model does not match the first annotated sample.

[0084] In some embodiments, step S220 may include: if the perspective type of the road surface perception model does not match the first annotated sample, performing perspective conversion on the first annotated sample to obtain a training sample. It should be understood that if the perspective type of the road surface perception model matches the first annotated sample, the first annotated sample may be directly used as the training sample.

[0085] During the perspective conversion process, for the conversion from a bird's-eye view to a first-person view, the future driving area may be projected into the current image space. Since the depth information is compressed, part of the future trajectory may be blocked by obstacles in the current foreground. For example, when a vehicle is driving in a wood, the future trajectory will be blocked by trees in the foreground. Therefore, during the data annotation process, it is necessary to remove the part of the trajectory blocked by foreground obstacles to improve the accuracy of data annotation. In some embodiments, taking the perspective type of the road perception model as the first-person perspective and the first annotated sample based on the bird's-eye view as an example, the perspective conversion process of the first annotated sample to obtain a training sample can include the following steps:

[0086] Step S221: performing coordinate system conversion processing on the first labeled sample to obtain a second labeled sample;

[0087] Step S222: performing foreground filtering on the second labeled sample to obtain a training sample.

[0088] The first annotated sample is based on a bird's-eye view, that is, a global coordinate system; the road perception model is based on a first-person perspective, that is, an image coordinate system. A global coordinate system, such as an odometer, is established based on the coordinate axis directions of the vehicle body coordinate system, with the initial position as the origin. The image coordinate system is the sensor's own coordinate system. It can be understood that the global coordinate system is typically a three-dimensional coordinate system, while the image coordinate system is a two-dimensional coordinate system. Therefore, the coordinate system conversion process involves converting from the global coordinate system to the image coordinate system.

[0089] Due to the occlusion of foreground obstacles, after the coordinate system is converted to obtain the second labeled sample, the second labeled sample needs to be subjected to foreground filtering to remove the track portion blocked by the foreground obstacle, thereby improving the rationality and accuracy of the data annotation. In some embodiments, the above step S222 may include: if the driving track in the first labeled sample has foreground occlusion in the second labeled sample, then from the second labeled sample, the track portion with foreground occlusion and the parameter values ​​of the road surface parameters associated with the track portion are removed to obtain a training sample. It should be understood that if the driving track in the first labeled sample does not have foreground occlusion in the second labeled sample, the second labeled sample can be directly used as the training sample.

[0090] In summary, the technical solution provided by the embodiment of the present application labels the parameter values ​​of the road parameters in the vehicle's driving environment data to intuitively show the relationship between the driving environment data and the parameter values ​​of the road parameters. In addition, during the data labeling process, the embodiment of the present application takes into account the perspective type of the road perception model to ensure that the training samples match the perspective type of the road perception model through perspective conversion processing, thereby improving the accuracy of the training data set and thus improving the training accuracy of the road perception model. In addition, during the perspective conversion processing, the embodiment of the present application takes into account the situation where the future trajectory is blocked by the current foreground obstacle due to depth information compression, and performs foreground filtering processing on the labeled samples to further improve the rationality and accuracy of the training samples.

[0091] In some embodiments, the data processing method may further include the following steps:

[0092] Step S300: training a road surface perception model based on a training data set.

[0093] The road surface perception model can be trained according to the training data set to obtain a fully trained road surface perception model, and the trained road surface perception model can be used to achieve real-time perception of the physical characteristics of the road surface. The embodiment of the present application does not limit the specific type of the road surface perception model. In actual applications, the road surface perception model can be flexibly configured in combination with demand. In some embodiments, the road surface perception model can be implemented as a model based on deep learning, for example, a convolutional neural network (CNN) model, a semantic segmentation model, RoadBEV-mono and RoadBEV-stereo perception models.

[0094] To implement road surface perception model training, in some embodiments, step S300 may include the following steps:

[0095] Step S310: Determine the predicted loss value of the road surface perception model based on the training data set;

[0096] Step S320: Adjust the parameters of the road surface perception model according to the predicted loss value.

[0097] By processing the training data set through the road surface perception model, the predicted loss value of the road surface perception model can be obtained. In some embodiments, the above step S310 may include the following steps:

[0098] Step S311: inputting the driving environment data in the training data set into the road surface perception model so that the road surface perception model outputs the predicted value of the road surface parameter;

[0099] Step S312: Determine the predicted loss value of the road surface perception model based on the predicted value of the road surface parameter and the parameter value of the road surface parameter in the training data set.

[0100] The embodiment of the present application can construct a loss function based on the absolute value difference. Thus, step S312 can determine the predicted loss value of the road surface perception model based on the absolute value difference between the predicted value of the road surface parameter and the marked parameter value.

[0101] Due to driving safety issues, vehicles drive more frequently on roads with greater adhesion and smoother driving than on roads with lower adhesion and easy to slip, which objectively causes an imbalance in the training samples in the training data set. In order to enhance the influence of some uncommon roads during overall training and to enhance the generalization ability of the model, the training data set can be balanced. Based on this, in some embodiments, the above-mentioned step S312 may include: for each training sample in the training data set, according to the predicted value and parameter value of the road surface parameter under the training sample, determining the predicted loss value of the training sample; according to the predicted loss values ​​and weight values ​​of multiple training samples, determining the predicted loss value of the road surface perception model. The data balancing process includes targeted setting of the weight value of the training sample.

[0102] The embodiment of the present application can perform data balancing based on the frequency of occurrence of the parameter values ​​of the road surface parameters. Based on this, data balancing includes giving higher weight values ​​to negative samples to increase the model's attention to low-adhesion road surface samples; for high-adhesion road surface samples, when there is a lot of data, methods such as clustering undersampling can be used to reduce the model's bias towards high-adhesion road surface samples. Based on this, in some embodiments, the above-mentioned data processing method can also include: determining multiple value intervals of the road surface parameters based on the parameter values ​​of the road surface parameters in the training data set; for each training sample, determining the value interval corresponding to the training sample based on the parameter value of the road surface parameter under the training sample; determining the weight value of the training sample based on the number of multiple value intervals and the probability that the parameter value of the road surface parameter is in the value interval corresponding to the training sample.

[0103] For example, the weight value of the training sample can be shown as the following formula 3.

[0104] Formula 3: w i =(1-P i ) / (n b -1)

[0105] Among them, n b is the number of value intervals of the divided road surface parameters, and multiple value intervals do not overlap with each other; P i is the probability that the parameter value of the road surface parameter is in the value interval corresponding to the training sample; w i is the weight value of the training sample.

[0106] For example, the predicted loss value of the road surface perception model may be expressed as follows:

[0107] Formula 4:

[0108] Among them, label i is the parameter value of the road surface parameter under the training sample; output i is the predicted value of the road parameters output by the road perception model; m i is the mask correlation coefficient. If the training sample is the area where the vehicle actually travels, then m i If the training sample is an area where the vehicle has not traveled, then m i For areas that have not been traveled, the embodiment of the present application sets a mask to shield the influence of the area, so that this part of the training samples does not participate in the back propagation training of the road perception model.

[0109] To accelerate the training of the road perception model, a loss function related to depth estimation can be specifically incorporated. Based on this, in some embodiments, step S320 may include: determining a target loss value for the road perception model based on the predicted loss value and the road perception model's depth loss value; and adjusting the parameters of the road perception model based on the target loss value. The depth loss value is used to measure the difference between the predicted depth map and the true depth map. The depth loss value can be constructed based on the cross-entropy loss between the predicted depth and the true depth.

[0110] For example, the depth loss value may be expressed as shown in the following formula 5.

[0111] Formula 5:

[0112] Where D(u, v) is the true depth of the pixel; is the predicted depth of the pixel; is the cross entropy loss function; λ is the weight value, which can be implemented as a hyperparameter.

[0113] In summary, the technical solution provided by the embodiments of this application utilizes label distribution smoothing technology to increase the weight of uncommon road label data when constructing the prediction loss function of the road perception model during model training. This achieves data balance and helps improve the training accuracy and generalization ability of the road perception model. Furthermore, the embodiments of this application incorporate loss functions related to depth estimation during model training, accelerating the training speed and accuracy of the road perception model.

[0114] The data processing method provided in the embodiments of the present application is introduced and explained below with several examples.

[0115] The data processing method provided in the embodiments of the present application can be applied to the annotation and perception of the physical characteristics of unpaved roads used in off-road driving, including but not limited to desert roads, muddy roads, snowy roads, etc.

[0116] For example, the driving environment data includes an environment image based on a bird's-eye view, and the road surface parameters include a traction coefficient. Figure 2 As shown in Figure 1, a trained road perception model can predict the traction coefficient parameter value based on the input environment image. The road parameters predicted by the road perception model can be combined with the vehicle's chassis kinematic or dynamic model to further estimate the vehicle's negative motions such as slip, yaw, and vibration after passing over the road, thereby further predicting and planning the driving path.

[0117] Take the driving environment data including the environment image based on the bird's eye view as an example, Figure 3 As shown, the parameter values ​​of the road parameters and the vehicle's driving trajectory are aligned according to the timestamp in the environment image. If the perspective type of the road perception model is a bird's-eye view, it can be directly based on Figure 3 The training samples shown are used to train the road perception model. If the road perception model uses the first-person perspective, Figure 3 The training samples shown are processed with perspective conversion, and foreground filtering is taken into account during the perspective conversion process.

[0118] like Figure 4 As shown, the data processing method provided in the embodiment of the present application may include the following steps S401 to S409.

[0119] Step S401: Acquire vehicle driving status data.

[0120] Vehicle driving status data includes motion data such as speed, acceleration, wheel speed, and driving trajectory, as well as control data such as throttle data, steering wheel angle data, and front wheel angle data. When acquiring vehicle driving status data, a timestamp can be obtained simultaneously to ensure time consistency.

[0121] Step S402: Acquire vehicle driving environment data.

[0122] The driving environment data may include point clouds or images, etc. When obtaining the driving environment data, a timestamp may also be obtained to facilitate subsequent association with other data. In this example, the driving environment data may be based on a bird's-eye view.

[0123] Step S403: Estimate the parameter values ​​of the road surface parameters of the road where the vehicle is located based on the driving state data.

[0124] Step S404: Correlate the parameter values ​​of the road surface parameters with the driving environment data to obtain a first labeled sample.

[0125] The parameter values ​​of the road surface parameters and the driving trajectory of the vehicle may be aligned according to timestamps in the driving environment data to obtain a first labeled sample.

[0126] Step S405: Determine whether the perspective type of the road perception model is a bird's-eye view.

[0127] If it is a bird's-eye view, execute the following step S406; otherwise, execute the following step S407.

[0128] Step S406: Use the first labeled sample as a training sample to construct a training data set.

[0129] If the perspective type of the road surface perception model is a bird's-eye view, the perspective type of the road surface perception model matches the first labeled sample, and the first labeled sample can be directly used as a training sample to construct a training data set.

[0130] Step S407: performing coordinate system conversion processing on the first labeled sample to obtain a second labeled sample.

[0131] The coordinate system conversion process may convert the first labeled sample from the global coordinate system to the image coordinate system.

[0132] Step S408: performing foreground filtering on the second labeled samples to obtain training samples to construct a training data set.

[0133] During the foreground filtering process, the trajectory portion with foreground occlusion and the parameter values ​​of the associated road surface parameters may be removed from the second labeled samples to obtain training samples.

[0134] Step S409: training a road surface perception model according to the training data set.

[0135] The road perception model processes the training dataset to construct a training loss value, which may include the aforementioned prediction loss value and depth loss value. The road perception model parameters are adjusted based on the training loss value, for example, by adjusting the road perception model parameters to converge the training loss value, thereby obtaining a fully trained road perception model.

[0136] Among them, Figure 5 As shown above Figure 4 The illustrated step S403 may include the following steps S501 to S505 .

[0137] Step S501: establishing a vehicle dynamics model based on driving state data.

[0138] For example, the slip rate-adhesion coefficient curve provides the friction characteristics of the tire at different slip rates, which can be used to establish a more accurate tire model and estimate the power generated by the tire and the ground. The vehicle's seven-degree-of-freedom model can be used to analyze the impact of the power generated by the tire on the movement of the entire vehicle. It can also be quickly labeled through a simplified dynamic model. For example, by defining the ratio of the actual speed to the desired control speed (i.e., the traction coefficient) as a label, a simplified dynamic model can be established for faster labeling.

[0139] Step S502: Establishing an observation equation based on the dynamics model.

[0140] An observation model can be established through the vehicle's dynamic model. The observation model can associate the vehicle's state quantity (the vehicle's actual motion state) with the kinematic quantities observed by the sensor (such as speed, displacement, etc.), and estimate the vehicle's state quantity through the kinematic quantities observed by the sensor.

[0141] Step S503: construct an optimization objective function based on the dynamic model and the observation equation.

[0142] Among them, the optimization objective function includes minimizing the following: the difference between the estimated value of the vehicle state at the current moment and the estimated value at the previous moment; the difference between the estimated value of the road surface parameters at the current moment and the estimated value at the previous moment; the difference between the estimated value of the vehicle state in the current time period and the observed value.

[0143] Step S504: Constructing constraint conditions according to the dynamics model.

[0144] By inputting the estimated value of the vehicle's state at the current moment into the vehicle's dynamics model, we can obtain the predicted value of the vehicle's state at the next moment. Based on the predicted and estimated values ​​of the vehicle's state at the next moment, we can construct constraints. For example, a constraint could include requiring that the predicted and estimated values ​​of the vehicle's state at the next moment be equal.

[0145] Step S505: When the constraint conditions are met, the optimization objective function is solved according to the vehicle's driving state data to obtain parameter values ​​of the road surface parameters of the road where the vehicle is located.

[0146] The optimization objective function can be solved by some general tools (such as CasADi). The estimated value of the road surface parameter under the minimized optimization objective function is used as the parameter value of the road surface parameter.

[0147] exist Figure 5 After obtaining the parameter values ​​of the road surface parameters in the embodiment shown, the following steps can be continued: Figure 4 Steps S404 to S408 in the embodiment are to construct a training data set.

[0148] In an embodiment of the present application, driving status data, driving environment data, etc. can be uploaded to a server, and the server can establish an optimization-based road parameter estimation problem to perform data annotation to obtain a training data set. Alternatively, if the vehicle's body control unit itself has developed an estimation algorithm for road parameters, the vehicle itself can also perform data annotation to obtain a training data set. In addition, the training samples in the training data set can be manually corrected. For example, more obvious obstacles (such as large rocks, potholes, etc.) in the training samples can be manually identified, and the road adhesion coefficients and other factors in these areas can be manually set to zero in the training samples.

[0149] In summary, this example can enhance safety in off-road driving and has at least the following beneficial effects: it enables the road perception model to avoid encoding chassis control signals and fusing chassis control signals with external image signals during inference and prediction, thereby reducing computing resource consumption during model inference and prediction; it avoids modeling chassis dynamics and terrain using deep learning methods at the same time, and avoids the simultaneous learning of traversability and motion / dynamic models, which may require tuning of multiple parameters. Improper parameter settings may cause the model's learning process to be unstable, thereby causing unnecessary swings in the path output by the regulation and control module; it reduces the number of parameters for model training, reduces the amount of data required for the training data set, and reduces the time required for model training; the principles of labeling and training are simpler than other self-supervised learning models, and the experience accumulated from existing chassis tuning and modeling can be reused. The system has strong robustness, low development difficulty, and short development cycle; data labeling and model training can be developed independently, and the results of data labeling can be manually verified, and the system has strong robustness.

[0150] According to a second aspect of the present application, embodiments of the present application further provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described data processing method. This non-transitory computer-readable storage medium has all the beneficial effects of the above-described data processing method, and this application will not further elaborate on them.

[0151] According to the third aspect of the present application, an embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the above-mentioned data processing method and has all the beneficial effects of the above-mentioned data processing method. This application will not go into details here.

[0152] According to a fourth aspect of the present application, an embodiment of the present application further provides an electronic device comprising: a memory and a processor, wherein the memory stores a computer program; and the processor is configured to execute the computer program in the memory to implement the steps of the above-described data processing method. The electronic device has all the beneficial effects of the above-described data processing method, which are not further described in this application.

[0153] The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof, and this application does not specifically limit this. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0154] In some embodiments of the present application, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0155] The computer-readable storage medium may be included in the electronic device or may exist independently without being incorporated into the electronic device. The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0156] estimating, based on the vehicle's driving state data, parameter values ​​of road surface parameters of the road on which the vehicle is located;

[0157] The parameter values ​​of the road surface parameters are associated with the driving environment data of the vehicle to construct a training data set for a road surface perception model.

[0158] Computer program code for performing the operations of some embodiments of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or can be connected to an external computer (for example, using an Internet service provider to connect via the Internet).

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function.

[0160] It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures.

[0161] For example, two blocks shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flow charts, and combinations of blocks in the block diagrams and / or flow charts, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or may be implemented using a combination of dedicated hardware and computer instructions.

[0162] The units described in some embodiments of the present application may be implemented in software or hardware, and may also be provided in a processor.

[0163] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and the like.

[0164] According to the fifth aspect of the present application, an embodiment of the present application also provides a data processing system.

[0165] See also Figure 6 , Figure 6 Schematic diagram of a data processing system provided in an embodiment of the present application. The data processing system can be used to execute the above-mentioned data processing method. Figure 6 As shown, the data processing system includes a data annotation module 100 .

[0166] Among them, the data annotation module 100 is used to: estimate the parameter values ​​of the road surface parameters of the road where the vehicle is located based on the vehicle's driving status data; and associate the parameter values ​​of the road surface parameters with the vehicle's driving environment data to construct a training data set for the road surface perception model.

[0167] like Figure 6 As shown, in some embodiments, the data processing system further includes a first acquisition module 200. The first acquisition module 200 is configured to send driving status data to the data annotation module 100. The first acquisition module 200 can be connected to the data annotation module 100 via a hardwire or a network, including but not limited to a CAN (Controller Area Network), Ethernet, or a wireless network.

[0168] The first acquisition module 200 can be implemented as an endogenous sensor. For example, the first acquisition module 200 includes at least one of the following: an inertial measurement unit (IMU), a positioning unit, a wheel speed meter, and a sensor. The inertial measurement unit usually obtains displacement in a short period of time through the pre-integration of the IMU. The initial posture is set as the coordinate origin to obtain the positioning of the vehicle. Since the use of integral positioning will produce accumulated errors, other sensor positioning solutions are required for posture correction. The positioning solutions for correction include real-time kinematic (RTK) navigation, lidar inertial odometry (LIO), visual inertial odometry (Visual Inertial Odometry), etc. The positioning unit includes a global navigation satellite system (GNSS). Sensors include pressure sensors, angle sensors, etc.

[0169] like Figure 6 As shown, in some embodiments, the data processing system further includes a second acquisition module 300. The second acquisition module 300 is configured to send driving environment data to the data annotation module 100. The second acquisition module 300 can be connected to the data annotation module 100 via a hardwire or a network, including but not limited to CAN, Ethernet, or wireless networks.

[0170] The second acquisition module 300 can be implemented as an exogenous sensor. For example, the second acquisition module 300 includes at least one of the following: a camera, a radar. Among them, the radar can include a laser radar, a millimeter wave radar, etc. If the perspective type of the road perception model is a first-person perspective, the driving environment data required to be obtained during the data annotation process can be based on data obtained by a single sensor, such as an image obtained by a single camera. If the perspective type of the road perception model is a bird's-eye view, the driving environment data required to be obtained during the data annotation process can be based on data fusion obtained by multiple sensors, such as the fusion of an image obtained by a camera and a point cloud obtained by a radar.

[0171] like Figure 6 As shown, in some embodiments, the data processing system further includes a model training module 400. Model training module 400 is configured to train a road surface perception model based on a training dataset. Model training module 400 and data annotation module 100 can be connected via a hardwire or network, including but not limited to CAN, Ethernet, or wireless networks.

[0172] It should be understood that the data annotation module 100 and the model training module 400 can be implemented as the same device, such as a controller, which can include the data annotation module 100 and the model training module 400; alternatively, the data annotation module 100 and the model training module 400 can also be implemented as different devices, such as different controllers. In the case where the data annotation module 100 and the model training module 400 are implemented as different devices, the data annotation module 100 and the model training module 400 can be deployed on different entities. For example, the controller where the data annotation module 100 is located is deployed in a vehicle, while the controller where the model training module 400 is located is deployed in a server.

[0173] For an introduction to the steps performed by each module in the data processing system and its beneficial effects, please refer to the above-mentioned embodiment of the data processing method, which will not be elaborated here.

[0174] According to the sixth aspect of this application, Figure 7 As shown, the embodiment of the present application further provides a vehicle 10, which includes the above-mentioned electronic device or the above-mentioned data processing system. The vehicle has all the beneficial effects of the above-mentioned electronic device or data processing system, etc., and this application will not repeat them here.

[0175] The vehicle may be a fuel vehicle, a plug-in hybrid vehicle or a new energy vehicle, etc., and this application does not make any specific restrictions on this.

[0176] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0177] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0178] The embodiments, implementation methods and related technical features of the present application can be combined and replaced with each other without conflict.

[0179] The above are merely preferred embodiments of the present application and do not constitute any form of limitation to the present application. Although the descriptions of each embodiment in the embodiments of the present application have different focuses, for parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: estimating, based on the vehicle's driving state data, parameter values ​​of road surface parameters of the road on which the vehicle is located; The parameter values ​​of the road surface parameters are associated with the driving environment data of the vehicle to construct a training data set for a road surface perception model.

2. The method according to claim 1, characterized in that The driving status data includes at least one of the following: motion data and control data.

3. The method according to claim 2, characterized in that The motion data includes at least one of the following: speed, acceleration, wheel speed, and driving trajectory.

4. The method according to claim 2, characterized in that The control data includes at least one of the following: throttle data, steering wheel angle data, and front wheel angle data.

5. The method according to claim 1, wherein The road surface parameters include at least one of the following: traction coefficient, adhesion coefficient, slip rate, and friction coefficient.

6. The method according to claim 1, characterized in that The estimating, based on the vehicle's driving state data, parameter values ​​of road surface parameters of the road on which the vehicle is located, includes: An optimization objective function is solved based on the vehicle's driving state data to obtain parameter values ​​of road surface parameters of the road where the vehicle is located.

7. The method according to claim 6, characterized in that The optimization objective function includes minimizing at least one of the following: The difference between the estimated value of the vehicle state at the current moment and the estimated value at the previous moment; The difference between the estimated value of the road surface parameter at the current moment and the estimated value at the previous moment; The difference between the estimated value and the observed value of the vehicle's state at the current time period.

8. The method according to claim 6, characterized in that Solving the optimization objective function based on the vehicle's driving state data to obtain parameter values ​​of road surface parameters of the road where the vehicle is located includes: Under the condition that the constraint conditions are met, the optimization objective function is solved according to the driving state data of the vehicle to obtain the parameter values ​​of the road surface parameters of the road where the vehicle is located.

9. The method according to claim 8, characterized in that The method further comprises: Inputting an estimated value of the vehicle state at a current moment into a dynamic model of the vehicle to obtain a predicted value of the vehicle state at a next moment; The constraint condition is constructed according to the predicted value and the estimated value of the vehicle state at the next moment.

10. The method according to claim 1, characterized in that The associating the parameter values ​​of the road surface parameters with the driving environment data of the vehicle to construct a training data set for the road surface perception model includes: The parameter values ​​of the road surface parameters are marked in the driving environment data of the vehicle to construct a training data set for the road surface perception model.

11. The method according to claim 10, characterized in that The step of marking the parameter values ​​of the road surface parameters in the driving environment data of the vehicle to construct a training data set for the road surface perception model includes: marking the parameter value of the road surface parameter in the driving environment data of the vehicle to obtain a first marked sample; Determining a training sample based on the first labeled sample and the perspective type of the road surface perception model; A training data set for the road surface perception model is constructed based on the training samples.

12. The method according to claim 11, characterized in that The step of marking the parameter value of the road surface parameter in the driving environment data of the vehicle to obtain a first marked sample includes: In the driving environment data of the vehicle, the parameter values ​​of the road surface parameters and the driving trajectory of the vehicle are aligned according to timestamps to obtain a first labeled sample.

13. The method according to claim 11, characterized in that The determining of a training sample according to the first labeled sample and the viewing angle type of the road surface perception model includes: If the perspective type of the road surface perception model does not match the first labeled sample, the first labeled sample is subjected to perspective conversion processing to obtain a training sample.

14. The method according to claim 13, characterized in that When the perspective type of the road surface perception model is a first-person perspective and the first labeled sample is based on a bird's-eye view, the perspective type of the road surface perception model does not match the first labeled sample.

15. The method according to claim 14, characterized in that The performing perspective conversion processing on the first labeled sample to obtain a training sample includes: Performing coordinate system conversion processing on the first labeled sample to obtain a second labeled sample; Perform foreground filtering on the second labeled samples to obtain training samples.

16. The method according to claim 15, characterized in that The performing foreground filtering on the second labeled sample to obtain a training sample includes: If the driving trajectory in the first labeled sample has foreground occlusion in the second labeled sample, the trajectory portion with the foreground occlusion and the parameter value of the road surface parameter associated with the trajectory portion are removed from the second labeled sample to obtain a training sample.

17. The method according to claim 1, wherein The method further comprises: The road surface perception model is trained according to the training data set.

18. The method according to claim 17, characterized in that The step of training the road surface perception model according to the training data set includes: Determining a predicted loss value of the road surface perception model based on the training data set; Adjust parameters of the road surface perception model according to the predicted loss value.

19. The method according to claim 18, characterized in that Determining the predicted loss value of the road surface perception model based on the training data set includes: Inputting the driving environment data in the training data set into the road surface perception model so that the road surface perception model outputs a predicted value of the road surface parameter; A predicted loss value of the road surface perception model is determined according to the predicted value of the road surface parameter and the parameter value of the road surface parameter in the training data set.

20. The method according to claim 19, characterized in that Determining the predicted loss value of the road surface perception model according to the predicted value of the road surface parameter and the parameter value of the road surface parameter in the training data set includes: For each training sample in the training data set, determining a predicted loss value of the training sample according to a predicted value and a parameter value of the road surface parameter under the training sample; The predicted loss value of the road surface perception model is determined according to the predicted loss values ​​and weight values ​​of the plurality of training samples.

21. The method according to claim 20, characterized in that The method further comprises: Determining multiple value intervals of the road surface parameter according to the parameter value of the road surface parameter in the training data set; For each of the training samples, determining the value interval corresponding to the training sample according to the parameter value of the road surface parameter in the training sample; The weight value of the training sample is determined according to the number of the plurality of value intervals and the probability that the parameter value of the road surface parameter is located in the value interval corresponding to the training sample.

22. The method according to claim 18, wherein The adjusting the parameters of the road surface perception model according to the predicted loss value includes: determining a target loss value of the road surface perception model according to the predicted loss value and the depth loss value of the road surface perception model; According to the target loss value, the parameters of the road surface perception model are adjusted.

23. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 22 is implemented.

24. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 22 is implemented.

25. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the data processing method according to any one of claims 1 to 22.

26. A data processing system, characterized in that: The data processing system includes a data annotation module (100); wherein, The data annotation module (100) is used to: estimate the parameter values ​​of the road surface parameters of the road where the vehicle is located based on the vehicle's driving state data; and associate the parameter values ​​of the road surface parameters with the vehicle's driving environment data to construct a training data set for a road surface perception model.

27. The system according to claim 26, wherein: The data processing system further comprises a first acquisition module (200); wherein, The first acquisition module (200) is used to send the driving status data to the data annotation module (100).

28. The system according to claim 27, wherein: The first acquisition module (200) includes at least one of the following: an inertial measurement unit, a positioning unit, a wheel speed meter, and a sensor.

29. The system according to claim 26, wherein: The data processing system further comprises a second acquisition module (300); wherein, The second acquisition module (300) is used to send the driving environment data to the data annotation module (100).

30. The system according to claim 29, wherein: The second acquisition module (300) includes at least one of the following: a camera, a radar.

31. The system according to claim 26, wherein: The data processing system further includes a model training module (400); wherein, The model training module (400) is used to train the road surface perception model based on the training data set.

32. A vehicle, characterized in that: The device comprises the electronic device according to claim 25, or the data processing system according to any one of claims 26 to 31.

Images