PINN-based automatic driving prediction method, device and system, and storage medium

Abstract

The embodiment of the invention provides a PINN-based automatic driving prediction method, device and system, and a storage medium, and relates to the technical field of automatic driving. The method comprises the following steps: preprocessing acquired real vehicle operation data to obtain a training sample, and dividing supervision data, initial moment data and unlabeled physical consistency sampling points in the training sample; inputting the training sample into a physical information neural network, outputting a vehicle kinematics state prediction result, and calculating a physical residual error of the prediction result according to a vehicle kinematics differential equation; generating a multi-objective loss function by using the physical residual error, a data error corresponding to the supervision data and an initial condition error corresponding to the initial moment data through adaptive dynamic weight; according to the multi-objective loss function, performing end-to-end training on the physical information neural network until the multi-objective loss function converges to obtain a target network; and the target network is deployed in the vehicle for vehicle motion control, so that the reliability of predicting the motion state by the automatic driving system is improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically to an autonomous driving prediction method, device, system, and storage medium based on PINN (Physics-Informed Neural Networks). Background Technology

[0002] Currently, in the development of autonomous driving technology, accurate prediction of vehicle motion state is a key prerequisite for achieving precise trajectory planning and stable motion control. In real-world driving scenarios, vehicles face complex conditions such as high-speed driving, large-curvature steering, and dynamic load changes. How to reliably predict the vehicle's position, speed, heading angle, and other motion states under these complex conditions has become a crucial problem that urgently needs to be solved.

[0003] Related technologies primarily rely on prediction methods based on traditional physical networks. These methods establish vehicle kinematic or dynamic equations, describing vehicle motion patterns based on physical principles to theoretically predict motion states under different operating conditions. Alternatively, they employ purely data-driven prediction methods, such as using neural networks like LSTM and GRU. These methods learn from large amounts of real-world vehicle operating data to fit the nonlinear characteristics of vehicle motion, improving prediction performance under complex conditions through data fitting. The former provides interpretable predictions when parameters are known, while the latter possesses strong pattern recognition capabilities when training data is abundant.

[0004] In the process of implementing the embodiments of this application, at least the following problems were found in the related technology:

[0005] Employing relevant technologies has improved the accuracy of prediction results to some extent. However, in practical applications, using physical networks requires simplified assumptions about tire side slip characteristics, center of gravity position, and suspension characteristics, making it difficult to accurately capture the real vehicle motion state under highly nonlinear dynamic scenarios, thus limiting prediction accuracy. Purely data-driven networks lack physical constraints, may output results that violate the common sense of vehicle dynamics, have weak generalization ability to conditions outside the training data distribution, and poor physical interpretability, failing to meet the reliability requirements of autonomous driving prediction results. Summary of the Invention

[0006] This application provides an autonomous driving prediction method, device, system, and storage medium based on PINN.

[0007] The first aspect of this application provides an autonomous driving prediction method based on PINN, comprising:

[0008] The acquired real vehicle operation data is preprocessed to obtain training samples, and the training samples are divided into supervision data, initial time data, and unlabeled physical consistency sampling points.

[0009] The training samples are input into the physical information neural network, which outputs the vehicle kinematic state prediction results. The physical residuals are then calculated from the prediction results based on the vehicle kinematic differential equations.

[0010] The physical residuals, the data errors corresponding to the supervised data, and the initial condition errors corresponding to the initial time data are used to generate a multi-objective loss function through adaptive dynamic weights.

[0011] Based on the multi-objective loss function, the physical information neural network is trained end-to-end until the multi-objective loss function converges to obtain the target network.

[0012] The target network is deployed on the vehicle for vehicle motion control.

[0013] In an optional embodiment of this application, the acquired real-vehicle operation data is preprocessed to obtain training samples, including:

[0014] Obtain real-world vehicle operation data;

[0015] Timestamp alignment is performed on the actual vehicle operation data to eliminate sensor sampling time deviations and obtain the first data;

[0016] The sliding window statistical method is used to remove outliers from the first data to obtain the second data.

[0017] The second data is normalized by using minimum to maximum scaling, mapping the values ​​of each dimension to a unified interval to obtain training samples.

[0018] In an optional embodiment of this application, dividing the training samples into supervised data, initial time-time data, and unlabeled physically consistent sampling points includes:

[0019] Use continuous time-series data with real observation labels as supervisory data;

[0020] Use the state data of the first frame of each training sequence as the initial time data;

[0021] Latin hypercube sampling is used to sample within the boundary of the multidimensional feasible region, generating uniformly distributed unlabeled samples as physical consistency sampling points.

[0022] In an optional embodiment of this application, sampling is performed within the boundary of a multidimensional feasible region using Latin hypercube sampling to generate uniformly distributed unlabeled samples, including:

[0023] Determine the feasible boundaries of state variables, control variables, and time to obtain the multidimensional feasible domain boundary;

[0024] Divide the region into equal probability intervals within the boundary of the multidimensional feasible region, and randomly sample within each interval to obtain interval samples;

[0025] The interval samples of each dimension are combined into multidimensional unlabeled samples.

[0026] In an optional embodiment of this application, training samples are input into a physical information neural network, and the output is a vehicle kinematic state prediction result, including:

[0027] Construct a physical information neural network; wherein the hidden layer of the physical information neural network adopts the hyperbolic tangent activation function, and the output layer adopts a linear mapping.

[0028] The monitoring data, initial time data, and physical consistency sampling points are input into the physical information neural network in batches, and the predicted values ​​of vehicle position, speed, heading angle and / or heading angular velocity are output to obtain the vehicle kinematic state prediction results.

[0029] In an optional embodiment of this application, the physical residuals are obtained from the prediction results based on the vehicle's kinematic differential equations, including:

[0030] The prediction results are automatically differentiated to obtain the derivative of the state variable with respect to time, and the derivative result is obtained.

[0031] Substitute the derivative result into the vehicle kinematics differential equation, calculate the equation residual, and obtain the physical residual.

[0032] In an optional embodiment of this application, a multi-objective loss function is generated by adaptive dynamic weighting of the physical residual, the data error corresponding to the supervised data, and the initial condition error corresponding to the initial time data, including:

[0033] The mean square error between the supervised data and its corresponding predicted value is calculated to obtain the data error.

[0034] The initial condition error is obtained by calculating the mean square error between the initial time data and its corresponding predicted value.

[0035] The multi-objective loss function is obtained by adaptively weighting and summing the data error, initial condition error, and physical residual.

[0036] A second aspect of the embodiments of this application provides a PINN-based autonomous driving prediction device, including a processor and a memory storing program instructions. The processor is configured to execute the PINN-based autonomous driving prediction method as described in the first aspect of the embodiments of this application when running the program instructions.

[0037] A third aspect of this application provides a system comprising:

[0038] The system itself; and,

[0039] The PINN-based autonomous driving prediction device, as described in the second aspect of this application, is installed on the system body.

[0040] A fourth aspect of the embodiments of this application provides a computer-readable storage medium storing program instructions that, when executed, cause a computer to perform the PINN-based autonomous driving prediction method as described in the first aspect of the embodiments of this application.

[0041] The autonomous driving prediction method, apparatus, system, and storage medium based on PINN provided in the embodiments of this application have the following beneficial effects:

[0042] This embodiment first preprocesses the real vehicle operation data and divides it into supervised data, initial time data, and unlabeled physical consistency sampling points to provide structured input for the network. Then, the training samples are input into the physical information neural network to output prediction results, and physical residuals are calculated based on the vehicle kinematic differential equations, forcing the network to learn dynamic characteristics that conform to physical laws. Furthermore, through adaptive dynamic weights, the physical residuals, supervised data errors, and initial condition errors are fused into a multi-objective loss function, dynamically balancing the optimization direction of data fitting and physical constraints. Finally, the target network is obtained through end-to-end training and deployed on the vehicle. This method enables the network to simultaneously possess data-driven nonlinear fitting capabilities and strong generalization under physical constraints. The physical residual term ensures that the prediction results strictly follow the principles of vehicle kinematics, avoiding pure data network outputs results that violate physical common sense. The adaptive weighting mechanism solves the multi-objective gradient imbalance problem, improving training stability. The introduction of unlabeled physical points reduces the dependence on fully labeled data and enhances the network's robustness under non-training conditions such as high-speed, high-curvature steering and sudden load changes, thereby providing more reliable motion state prediction for autonomous driving systems. Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0044] Figure 1 This is a schematic diagram of an autonomous driving prediction method based on PINN provided in an embodiment of this application;

[0045] Figure 2 This is a schematic diagram of another PINN-based autonomous driving prediction method provided in an embodiment of this application;

[0046] Figure 3This is a schematic diagram of another PINN-based autonomous driving prediction method provided in an embodiment of this application;

[0047] Figure 4 This is a schematic diagram of an autonomous driving prediction device based on PINN provided in an embodiment of this application.

[0048] Figure label:

[0049] 800: PINN-based autonomous driving prediction device; 801: Processor; 802: Memory; 803: Communication interface; 804: Bus. Detailed Implementation

[0050] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0051] Figure 1 This is a schematic diagram of the PINN-based autonomous driving prediction method provided in the embodiments of this application. Any of the following methods can be executed in the system or in a server or terminal device that is connected to the system.

[0052] like Figure 1 As shown, this application provides an autonomous driving prediction method based on PINN, including:

[0053] S01, preprocess the acquired real vehicle operation data to obtain training samples, and divide the training samples into supervision data, initial time data and unlabeled physical consistency sampling points.

[0054] S02, input the training samples into the physical information neural network, output the vehicle kinematic state prediction result, and calculate the physical residual from the prediction result based on the vehicle kinematic differential equation.

[0055] S03 uses adaptive dynamic weights to generate a multi-objective loss function by combining the physical residuals, the data errors corresponding to the supervised data, and the initial condition errors corresponding to the initial time data.

[0056] S04. Based on the multi-objective loss function, perform end-to-end training on the physical information neural network until the multi-objective loss function converges to obtain the target network.

[0057] S05, deploy the target network on the vehicle for vehicle motion control.

[0058] In this embodiment of the application, the supervision data, physical consistency sampling points and initial time data are mixed according to a set ratio and then divided into training / validation subsets, and further divided into small batches according to the batch length to ensure that each batch contains both data errors and physical residuals.

[0059] The PINN-based autonomous driving prediction method provided in this application first preprocesses the real vehicle operation data and divides it into supervised data, initial time data, and unlabeled physical consistency sampling points to provide structured input to the network. Then, the training samples are input into the physical information neural network to output prediction results, and physical residuals are calculated based on the vehicle kinematic differential equations, forcing the network to learn dynamic characteristics that conform to physical laws. Furthermore, through adaptive dynamic weights, the physical residuals, supervised data errors, and initial condition errors are fused into a multi-objective loss function, dynamically balancing the optimization direction of data fitting and physical constraints. Finally, the target network is obtained through end-to-end training and deployed in the vehicle. This method enables the network to simultaneously possess data-driven nonlinear fitting capabilities and strong generalization under physical constraints. The physical residual term ensures that the prediction results strictly follow the principles of vehicle kinematics, avoiding pure data network outputs results that violate physical common sense. The adaptive weighting mechanism solves the multi-objective gradient imbalance problem and improves training stability. The introduction of unlabeled physical points reduces the dependence on fully labeled data and enhances the robustness of the network under non-training conditions such as high-speed, high-curvature steering and sudden load changes, thereby providing more reliable motion state prediction for autonomous driving systems.

[0060] like Figure 2 As shown in the embodiments of this application, another autonomous driving prediction method based on PINN is provided, including:

[0061] S21, obtain real vehicle operation data.

[0062] S22, perform timestamp alignment on the actual vehicle operation data to eliminate sensor sampling time deviation and obtain the first data.

[0063] S23, using a sliding window statistical method to remove outliers from the first data to obtain the second data.

[0064] S24. The second data is normalized by minimum to maximum scaling, mapping the values ​​of each dimension to a unified interval to obtain training samples.

[0065] S25, divide the training samples into supervised data, initial time data, and unlabeled physical consistency sampling points.

[0066] S02, input the training samples into the physical information neural network, output the vehicle kinematic state prediction result, and calculate the physical residual from the prediction result based on the vehicle kinematic differential equation.

[0067] S03 uses adaptive dynamic weights to generate a multi-objective loss function by combining the physical residuals, the data errors corresponding to the supervised data, and the initial condition errors corresponding to the initial time data.

[0068] S04. Based on the multi-objective loss function, perform end-to-end training on the physical information neural network until the multi-objective loss function converges to obtain the target network.

[0069] S05, deploy the target network on the vehicle for vehicle motion control.

[0070] The PINN-based autonomous driving prediction method provided in this application performs timestamp alignment on the real vehicle operation data. This eliminates the temporal misalignment problem caused by sampling time deviations of different sensors, ensuring the consistency of multi-dimensional data such as position, speed, and heading angle in the time dimension. The resulting first data has an accurate temporal correspondence, avoiding training errors introduced by data asynchrony. Outliers in the first data are removed using a sliding window statistical method, effectively identifying and removing outliers caused by momentary sensor failures or external interference. The resulting second data more closely reflects the actual vehicle operating state, reducing noise interference to the network's learning of real motion patterns. Finally, the second data is normalized using minimum-to-maximum scaling, mapping the values ​​of each dimension to a unified range. This avoids neural network parameter optimization bias caused by excessive differences in the numerical scales of different physical quantities such as position, speed, and angle, enabling the network to learn features of each dimension more evenly, improving training stability and efficiency. Through the above preprocessing steps, the final training samples are temporally consistent, have low noise, and are of uniform scale, providing high-quality input for subsequent training of the physical information neural network.

[0071] In an optional embodiment of this application, dividing the training samples into supervision data, initial time data, and unlabeled physical consistency sampling points includes: using continuous time-series data with real observation labels as supervision data; using the first frame state data of each training sequence as initial time data; and using Latin hypercube sampling to sample within the boundary of the multidimensional feasible region to generate uniformly distributed unlabeled samples as physical consistency sampling points.

[0072] In this way, the supervision data consists of continuous time-series data with real observation labels, ensuring that the network can learn the true mapping relationship of the vehicle's kinematic state based on labeled samples. The initial time-series data selects the first frame of state data from each training sequence, providing the network with a starting anchor point for the time series and avoiding error accumulation due to state drift during prediction. Physically consistent sampling points generate uniformly distributed unlabeled samples within the multidimensional feasible region boundary through Latin hypercube sampling, covering the joint distribution range of state variables, control variables, and time. This ensures that the network still satisfies the physical equation constraints on unlabeled data, thereby improving its adaptability to non-training conditions. Through the above partitioning method, the supervision data and initial time-series data jointly guarantee the network's accuracy within the known data range, while the physically consistent sampling points, by covering the constraint samples of the global feasible region, allow the network output to vary within the framework of the physical dynamic equations. This alleviates the non-physical prediction problems that may occur in purely data-driven networks due to the lack of physical constraints, laying a data foundation for subsequent physical residual calculation and multi-objective loss function construction, thus enhancing the network's physical interpretability while improving its generalization ability.

[0073] In an optional embodiment of this application, Latin hypercube sampling is used to sample within the boundary of a multidimensional feasible region to generate uniformly distributed unlabeled samples, including: determining the feasible boundaries of state variables, control variables, and time to obtain the boundary of the multidimensional feasible region; dividing the boundary of the multidimensional feasible region into equal probability intervals and randomly sampling within each interval to obtain interval samples; and combining the interval samples of each dimension into multidimensional unlabeled samples.

[0074] In this embodiment of the application, the multidimensional unlabeled sample is:

[0075]

[0076] Where C represents the set of physically consistent sampling points, i.e., multidimensional unlabeled samples, used to constrain the network output to conform to physical laws; x i For vehicle state quantities, such as position, speed, heading angle, etc., u i To control quantities such as steering wheel angle, accelerator / brake input, etc., t i For time variables, such as sampling time, p i represents vehicle physical parameters such as wheelbase, track width, mass, and moment of inertia; i is the counting marker for the sampling process.

[0077] This approach defines the feasible boundaries of state variables, control variables, and time. By clearly defining the physical constraints of each dimension (such as speed limits and steering angle change rates), it ensures that subsequent sampling covers extreme or typical conditions that may occur in actual driving scenarios. Within the feasible region boundaries, equally probable intervals are divided and random sampling is performed to ensure statistical uniformity in the sample distribution of each dimension, preventing the omission of certain key areas due to sampling bias. The interval samples from each dimension are combined into multidimensional unlabeled samples. Through joint distribution, the interaction relationships across all dimensions of state, control, and time are covered, ensuring that physically consistent sampling points are not only uniformly distributed in a single dimension but also form a gridded layout in multidimensional space. Using this method, physical points can be uniformly distributed in the unlabeled high-dimensional space. This allows the network to simultaneously satisfy the vehicle dynamics equations in each dimension when calculating physical residuals, thus constraining the network output to conform to physical laws even in non-training conditions. This compensates for the limitations of supervised data in terms of coverage, reduces dependence on fully labeled data, and provides a global constraint sample basis for the subsequent construction of physical residuals and multi-objective loss functions, effectively improving the network's prediction robustness and physical interpretability under complex conditions.

[0078] like Figure 3 As shown in the embodiments of this application, another autonomous driving prediction method based on PINN is provided, including:

[0079] S01, preprocess the acquired real vehicle operation data to obtain training samples, and divide the training samples into supervision data, initial time data and unlabeled physical consistency sampling points.

[0080] S31, construct a physical information neural network; wherein the hidden layer of the physical information neural network adopts the hyperbolic tangent activation function, and the output layer adopts a linear mapping.

[0081] S32 inputs the supervision data, initial time data and physical consistency sampling points into the physical information neural network in batches, and outputs the predicted values ​​of vehicle position, speed, heading angle and / or heading angular velocity to obtain the vehicle kinematic state prediction results.

[0082] S33, based on the vehicle kinematic differential equation, calculate the physical residuals from the prediction results.

[0083] S03 uses adaptive dynamic weights to generate a multi-objective loss function by combining the physical residuals, the data errors corresponding to the supervised data, and the initial condition errors corresponding to the initial time data.

[0084] S04. Based on the multi-objective loss function, perform end-to-end training on the physical information neural network until the multi-objective loss function converges to obtain the target network.

[0085] S05, deploy the target network on the vehicle for vehicle motion control.

[0086] The PINN-based autonomous driving prediction method provided in this application employs a hyperbolic tangent activation function in the hidden layer, which enhances the fitting ability of the physical information neural network to nonlinear vehicle dynamics relationships. Its output range (-1, 1) alleviates the gradient vanishing problem and improves the network's sensitivity to changes in vehicle state (such as high-speed steering or sudden load changes) under complex conditions. The output layer uses a linear mapping to avoid the additional constraints imposed by the nonlinear activation function on the predicted values, allowing state variables such as vehicle position, speed, and heading angle to be output directly in their original numerical form, ensuring consistency between the network output and physical reality. By inputting supervised data, initial time-series data, and physically consistent sampling points in batches, the network learns simultaneously during training: labeled real observations (supervised data), the starting anchor points of the time series (initial time-series data), and unlabeled physically constrained samples covering the multidimensional feasible domain (physically consistent sampling points). This ensures the network's accuracy within the known data range and forces the network output to satisfy the vehicle's kinematic differential equations through physical points. Thus, even in non-training conditions, the prediction results can still be constrained to conform to physical laws. This allows the network to achieve a balance between data-driven nonlinear fitting and strong physical constraints, reducing its dependence on fully labeled data.

[0087] In an optional embodiment of this application, the physical residual is obtained from the prediction result based on the vehicle kinematic differential equation, including: automatically differentiating the prediction result to obtain the derivative of the state quantity with respect to time, and obtaining the derivative result; substituting the derivative result into the vehicle kinematic differential equation, calculating the equation residual, and obtaining the physical residual.

[0088] In this embodiment of the application, the prediction result is automatically differentiated, specifically using the following formula:

[0089]

[0090] Among them, the physical information neural network is based on (x) i ,u i ,t i ,p i The input and output are the prediction results; For the i-th sampling point, the prediction result of the physical information neural network The derivative of θ is the rate of change of the predicted state at the i-th sampling point; θ is the trainable parameter of the physical information neural network, which is updated iteratively through gradient descent during training.

[0091] Substitute the derivative result into the vehicle kinematics differential equation and calculate the equation residuals, as follows:

[0092]

[0093] in, Let x be the state variablei u i p i The theoretical physical rate of change obtained by substituting into the vehicle's kinematic differential equation; L phys The physical loss term, or physical residual, quantifies the overall degree to which the physical information neural network violates the vehicle kinematics at all sampling points and is used for weighting in the multi-objective loss function; N is the total number of physically consistent sampling points, referring to the number of unlabeled samples generated by Latin hypercube sampling, covering the multi-dimensional feasible domain of vehicle state variables, control variables, and time.

[0094] In this way, automatic differentiation of the prediction results can directly and accurately obtain the derivative of the state variables with respect to time, avoiding the approximation errors that may be introduced by traditional numerical differentiation, ensuring the accuracy of derivative calculation, and providing a reliable foundation for the subsequent calculation of physical residuals. Substituting the derivative results into the vehicle kinematics differential equations, the calculated equation residuals, i.e., physical residuals, can intuitively reflect the deviation between the prediction results and the vehicle kinematic laws based on physical principles. Physical residuals can serve as an important indicator for measuring whether the prediction results conform to physical common sense, introducing explicit physical constraints to physical information neural networks. This ensures that the network, during the learning process, not only needs to fit the data but also needs to follow the inherent physical laws of vehicle motion, reducing the possibility of prediction results that violate physical common sense in purely data-driven networks.

[0095] In an optional embodiment of this application, a multi-objective loss function is generated by adaptive dynamic weighting of the physical residual, the data error corresponding to the supervision data, and the initial condition error corresponding to the initial time data. This includes: calculating the mean square error between the supervision data and its corresponding predicted value to obtain the data error; calculating the mean square error between the initial time data and its corresponding predicted value to obtain the initial condition error; and adaptively weighting and summing the data error, the initial condition error, and the physical residual to obtain the multi-objective loss function.

[0096] In this embodiment of the application, the multi-objective loss function L is as follows:

[0097]

[0098] Among them, w data w ic w phys These are the dynamic weighting coefficients for data error, initial condition error, and physical residual, respectively. These are data error, initial condition error, and physical residual.

[0099] In this way, the mean square error between the supervised data and its corresponding predicted value is calculated to obtain the data error, which can quantify the network's fitting deviation to the real-labeled data and ensure that the network learns the observed laws of the vehicle's actual motion. The mean square error between the initial time data and its corresponding predicted value is calculated to obtain the initial condition error, which can measure the deviation between the predicted starting point and the actual initial state, ensuring the accuracy of the initial prediction sequence and laying a reliable foundation for subsequent predictions. The data error, initial condition error, and physical residual are adaptively weighted and summed to obtain the multi-objective loss function. The adaptive dynamic weights can dynamically balance the influence of the three types of errors in training, avoiding the dominance of the training process by a certain type of error due to excessive differences in numerical scale. This allows the network to learn the data laws while also following the physical constraints of vehicle kinematics, taking into account the accuracy of the initial state. Thus, the prediction results of the physical information neural network are both close to the real observation data and in accordance with physical laws, improving the reliability of the prediction and its adaptability to different operating conditions.

[0100] In an optional embodiment of this application, the data error, initial condition error, and physical residual are adaptively weighted and summed to obtain a multi-objective loss function. This includes: calculating the gradient magnitudes of the data error, initial condition error, and physical residual with respect to the physical information neural network parameters in real time to obtain gradient statistics; obtaining weight coefficients by smoothing the gradient statistics using an exponential moving average based on the reciprocal of the gradient statistics; and applying the weight coefficients to the data error, initial condition error, and physical residual respectively, and then summing them up to obtain the multi-objective loss function.

[0101] In this way, the gradient magnitudes of data error, initial condition error, and physical residuals on the parameters of the physical information neural network are calculated in real time to obtain gradient statistics. This reflects the influence of the three types of error terms on network parameter optimization in real time, providing a basis for weight adjustment. Based on the reciprocal of the gradient statistics, weight coefficients are obtained after exponential moving average smoothing. This allows error terms with small gradient magnitudes (i.e., high optimization difficulty) to receive larger weights, while error terms with large gradient magnitudes (i.e., relatively easy optimization) receive smaller weights. Simultaneously, the smoothing process reduces weight fluctuations and ensures training stability. Applying these weight coefficients to the three types of errors and summing them weighted to obtain a multi-objective loss function dynamically balances the optimization priorities between data fitting, initial condition matching, and physical constraint satisfaction. This prevents one type of error from excessively dominating training due to gradient scale advantage, enabling the physical information neural network to consider data accuracy, initial state rationality, and physical consistency during the learning process, thereby improving training stability and the reliability of the final prediction results.

[0102] In an optional embodiment of this application, the physical information neural network is trained end-to-end according to the multi-objective loss function until the multi-objective loss function converges to obtain the target network. This includes: using the ADAM optimizer to perform coarse-grained updates on the network parameters to obtain a preliminary converged network; switching the ADAM optimizer to the LBFGS optimizer to perform fine convergence on the preliminary converged network until the convergence condition is met to obtain the target network.

[0103] In this embodiment, the physical information neural network is trained end-to-end according to the multi-objective loss function until the multi-objective loss function converges to obtain the target network. This includes: in each epoch, firstly, forward inference is performed, i.e., using the current network parameters and input data (including supervision data, initial time data, and unlabeled physical consistency sampling points) to calculate the vehicle kinematic state prediction result; then, backpropagation is performed, i.e., based on the vehicle kinematic differential equation, the physical residual is obtained from the vehicle kinematic state prediction result, and the physical residual, the data error corresponding to the supervision data, and the initial condition error corresponding to the initial time data are combined into the multi-objective loss function; the partial derivative of the multi-objective loss function with respect to each parameter is calculated, i.e., the gradient operator, to determine the rate of change of the loss function relative to the network parameters; then, weight updates are performed, and based on the calculated gradient, a combined optimization strategy of ADAM followed by LBFGS is used to update the network parameters until the multi-objective loss function converges; periodically, physical consistency sampling points (Collocation) and initial condition points (IC) are resampled to maintain physical coverage. This step ensures that the network continuously considers the physical constraints of the vehicle kinematic equation during training, thereby improving the physical consistency and generalization ability of the model. Throughout the training process, the learning rate, loss components, and gradient spectrum are continuously recorded to monitor training progress, evaluate model performance, and provide a basis for subsequent network optimization. Specifically, the partial derivatives of the multi-objective loss function with respect to each parameter are calculated, as follows:

[0104]

[0105] in, For the gradient operator, calculate the partial derivative of the loss with respect to each learnable parameter θ; For a single sample, the multi-objective loss function includes data error, initial condition error, and physical residual. Let b be the batch loss function for a single sample in the e-th epoch and batch b; Let b be the mini-batch sample set of the bth batch in the eth epoch; For any training sample in the b-th batch of the e-th epoch.

[0106] In this embodiment of the application, after the convergence condition is met and / or before the target network is obtained, the method further includes: evaluating the performance of the trained physical information neural network on a validation set, saving the optimal network parameters, and generating the target network. Evaluating the performance of the trained physical information neural network on a validation set and saving the optimal network parameters to generate the target network includes: calculating the value of a multi-objective loss function on the validation set to obtain the validation loss; when the validation loss no longer decreases, saving the current network parameters as the optimal network parameters, and generating the target network.

[0107] Specifically, the multi-objective loss and the SMSE (Simplified Mean Squared Error) of the complete predicted trajectory can be evaluated on a validation set at fixed frequencies to monitor the network's performance on unseen data and ensure good generalization ability. Calculating the multi-objective loss assesses the network's fit to data errors, initial condition errors, and physical residuals. Simultaneously, calculating the SMSE measures the relative magnitude of the deviation between the physical information neural network's prediction and the true state vector along the entire validation trajectory; a smaller SMSE indicates a lower total prediction error and higher prediction accuracy for the physical information neural network. Based on the evaluation results on the validation set, the learning rate is dynamically adjusted to optimize the training process. For example, if the validation loss does not decrease significantly over several consecutive epochs, the learning rate can be reduced to refine the process of searching for optimal network parameters. During training, network checkpoints, including network parameters and optimizer states, are periodically saved for resuming training or model evaluation when needed. Simultaneously, learning curves, including changes in training and validation losses with epochs and gradient information, are recorded to monitor the stability and convergence of training. After training, based on the performance on the validation set, the network parameters corresponding to the epoch with the minimum SMSE are selected as the optimal parameters. This ensures that the network guarantees both data fitting accuracy and that the model strictly adheres to vehicle kinematic constraints, thereby achieving high-reliability physics-to-information fusion prediction. The SMSE is calculated using the following formula:

[0108]

[0109] in, This represents the actual state vector, derived from real vehicle data. This represents the mean state of the validation set in the validation sequence N. v Calculate the average over the length; This represents the predicted state vector at time k. PINN provides the vehicle kinematics prediction at time step k based on the previous state, control input, and vehicle parameters. Typical components include (x, y) position, longitudinal velocity v, heading angle ψ, and heading angular velocity. The SMSE calculation formula can be used as a normalized error metric to evaluate the prediction accuracy of the prediction model on the validation set. This metric can be, but is not limited to, normalized mean squared error, and can also include other normalized error metrics, such as normalized mean absolute error, to adapt to different types of prediction tasks and data characteristics.

[0110] Thus, by employing the ADAM optimizer to perform coarse-grained updates to the network parameters, and leveraging its first-order adaptive learning rate characteristic, the network can quickly traverse the high curvature region of the loss function in the early stages of training, achieving rapid construction of the initial convergent network and avoiding getting trapped in local minima. By switching the ADAM optimizer to the LBFGS optimizer, fine-tuning is achieved based on the quasi-second derivative information of the initially converged network, using a more accurate Hessian matrix approximation to achieve superlinear convergence speed. Finally, continuous optimization is performed until the convergence condition is met, ensuring that the multi-objective loss function sufficiently decreases to a stable state. This optimizer combination strategy balances training efficiency and accuracy. ADAM's strong robustness in the early stages prevents oscillatory divergence under complex loss surfaces, while LBFGS's fine-search capability in the later stages significantly improves the model's convergence quality, jointly ensuring that the target network achieves the optimal balance between physical laws and data fitting within a limited number of iterations.

[0111] In an optional embodiment of this application, fine convergence of the initially converged network is performed, including: setting a maximum number of iterations under the LBFGS optimizer; when the change in the multi-objective loss function is less than the set tolerance in multiple consecutive iterations, convergence is determined and training is stopped to obtain the target network.

[0112] In this way, by setting a maximum number of iterations under the LBFGS optimizer, the computational resource consumption during the fine-tuning convergence phase is explicitly limited, avoiding indefinite training due to local oscillations. Secondly, by monitoring the change in the multi-objective loss function across multiple iterations, convergence is determined when the change falls below a set tolerance. This mechanism, based on the stability of loss changes rather than an absolute threshold, more reliably identifies the true convergence state. Finally, by stopping training when a condition is met, iterations are terminated promptly while ensuring the network is fully optimized. The maximum number of iterations provides computational resource protection, and the loss change threshold enables dynamic identification of the convergence state. Both work together to ensure the target network outputs immediately upon reaching full optimization, avoiding underfitting risks caused by premature termination and preventing overfitting tendencies caused by invalid iterations, thus improving the stability and efficiency of the training process.

[0113] In an optional embodiment of this application, the target network is deployed on the vehicle for vehicle motion control, including: solidifying the network parameters of the target network to obtain a solidified model; porting the solidified model to the vehicle graphics processor; collecting vehicle kinematic state variables, control variables and vehicle parameters in real time, inputting them into the solidified model to obtain real-time prediction results; and sending the real-time prediction results to the trajectory planning module to generate control commands and send them to the vehicle actuators.

[0114] In this way, by solidifying the network parameters of the target network into an immutable solidified model, the random fluctuations during the training phase are eliminated, ensuring the stability of the deployed model's inference. Secondly, by porting the solidified model to an onboard graphics processor, its parallel computing capabilities are utilized to meet the stringent real-time requirements of autonomous driving systems. Next, vehicle kinematic state variables, control variables, and vehicle parameters are collected in real time and input into the solidified model, enabling the model to output prediction results based on the current actual vehicle state, ensuring the timeliness and scenario relevance of the prediction input. Finally, by sending the real-time prediction results to the trajectory planning module, control commands are generated and distributed to the vehicle actuators, forming a closed-loop control chain. This deployment process, through the coordinated efforts of model solidification, hardware adaptation, real-time data closure, and system integration, ensures that the trained target network is efficiently transformed into an onboard executable module, providing low-latency, highly reliable state prediction support for vehicle motion control.

[0115] Combination Figure 4 As shown, this application provides a PINN-based autonomous driving prediction device 800, including a processor 801 and a memory 802. Optionally, the device may further include a communication interface 803 and a bus 804. The processor 801, communication interface 803, and memory 802 can communicate with each other via the bus 804. The communication interface 803 can be used for information transmission. The processor 801 can call logical instructions in the memory 802 to execute the PINN-based autonomous driving prediction method of the above embodiment.

[0116] Furthermore, the logic instructions in the aforementioned memory 802 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0117] The memory 802, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 801 executes functional applications and data processing by running the program instructions / modules stored in the memory 802, thereby implementing the PINN-based autonomous driving prediction method in the above embodiments.

[0118] The memory 802 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 802 may include high-speed random access memory and may also include non-volatile memory.

[0119] This application provides a system comprising: a system body and the aforementioned PINN-based autonomous driving prediction device 800. The PINN-based autonomous driving prediction device 800 is installed within the system body. The installation relationship described herein is not limited to placement within the system, but also includes installation connections with other components of the system, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the PINN-based autonomous driving prediction device 800 can be adapted to feasible system bodies to achieve other feasible embodiments.

[0120] This application provides a computer-readable storage medium storing computer-executable instructions configured to execute the aforementioned PINN-based autonomous driving prediction method.

[0121] The technical solutions of this application embodiment can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this application embodiment. The aforementioned storage medium can be a non-transitory storage medium, including: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, and other media capable of storing program code.

[0122] The technical solutions of this application embodiment can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this application embodiment. The aforementioned storage medium can be a non-transitory storage medium, including: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, and other media capable of storing program code.

[0123] The foregoing description and accompanying drawings fully illustrate embodiments of this application to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or,” as used herein, means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., of the embodiments claimed, if they correspond to the method section of the embodiments claimed, then the relevant parts can be referred to the description of the method section.

[0124] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments claimed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0125] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description; sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for predicting autonomous driving based on PINN, characterized in that, include: The acquired real vehicle operation data is preprocessed to obtain training samples, and the training samples are divided into supervision data, initial time data, and unlabeled physical consistency sampling points. The training samples are input into the physical information neural network, which outputs the vehicle kinematic state prediction results. The physical residuals are then calculated from the prediction results based on the vehicle kinematic differential equations. The physical residuals, the data errors corresponding to the supervised data, and the initial condition errors corresponding to the initial time data are used to generate a multi-objective loss function through adaptive dynamic weights. Based on the multi-objective loss function, the physical information neural network is trained end-to-end until the multi-objective loss function converges to obtain the target network. The target network is deployed on the vehicle for vehicle motion control.

2. The method according to claim 1, characterized in that, The acquired real-vehicle operation data is preprocessed to obtain training samples, including: Obtain real-world vehicle operation data; Timestamp alignment is performed on the actual vehicle operation data to eliminate sensor sampling time deviations and obtain the first data; The sliding window statistical method is used to remove outliers from the first data to obtain the second data. The second data is normalized by using minimum to maximum scaling, mapping the values ​​of each dimension to a unified interval to obtain training samples.

3. The method according to claim 1, characterized in that, The training samples are divided into supervised data, initial time-step data, and unlabeled physically consistent sampling points, including: Use continuous time-series data with real observation labels as supervisory data; Use the state data of the first frame of each training sequence as the initial time data; Latin hypercube sampling is used to sample within the boundary of the multidimensional feasible region, generating uniformly distributed unlabeled samples as physical consistency sampling points.

4. The method according to claim 3, characterized in that, Latin hypercube sampling is used to sample within the boundary of a multidimensional feasible region, generating uniformly distributed unlabeled samples, including: Determine the feasible boundaries of state variables, control variables, and time to obtain the multidimensional feasible domain boundary; Divide the region into equal probability intervals within the boundary of the multidimensional feasible region, and randomly sample within each interval to obtain interval samples; The interval samples of each dimension are combined into multidimensional unlabeled samples.

5. The method according to claim 1, characterized in that, The training samples are input into the physical information neural network, which outputs the predicted vehicle kinematic state, including: Construct a physical information neural network; wherein the hidden layer of the physical information neural network adopts the hyperbolic tangent activation function, and the output layer adopts a linear mapping. The monitoring data, initial time data, and physical consistency sampling points are input into the physical information neural network in batches, and the predicted values ​​of vehicle position, speed, heading angle and / or heading angular velocity are output to obtain the vehicle kinematic state prediction results.

6. The method according to claim 1, characterized in that, Based on the vehicle's kinematic differential equations, the physical residuals of the prediction results are calculated, including: The prediction results are automatically differentiated to obtain the derivative of the state variable with respect to time, and the derivative result is obtained. Substitute the derivative result into the vehicle kinematics differential equation, calculate the equation residual, and obtain the physical residual.

7. The method according to any one of claims 1 to 6, characterized in that, The physical residuals, data errors corresponding to the supervised data, and initial condition errors corresponding to the initial time data are used to generate a multi-objective loss function through adaptive dynamic weights, including: The mean square error between the supervised data and its corresponding predicted value is calculated to obtain the data error. The initial condition error is obtained by calculating the mean square error between the initial time data and its corresponding predicted value. The multi-objective loss function is obtained by adaptively weighting and summing the data error, initial condition error, and physical residual.

8. A PINN-based autonomous driving prediction device, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the PINN-based autonomous driving prediction method as described in any one of claims 1 to 7 when running the program instructions.

9. A system, characterized in that, include: System body; as well as, The PINN-based autonomous driving prediction device as described in claim 8 is installed on the system body.

10. A computer-readable storage medium storing program instructions, characterized in that, When the program instructions are executed, they cause the computer to perform the PINN-based autonomous driving prediction method as described in any one of claims 1 to 7.

Images