Vehicle control method and device, electronic equipment and storage medium

Abstract

The invention provides a vehicle control method and device, electronic equipment and a storage medium, and the method comprises the steps: inputting third driving scene data collected in a vehicle driving process into a first model, and generating decision data of a vehicle; wherein the first model is obtained by performing pre-training based on first driving scene data and second driving scene data, and the second driving scene data is obtained by performing data expansion on the first driving scene data; and controlling the running track of the vehicle according to the decision data. According to the method, expandability of high-quality scene data can be achieved, the first model is trained through cooperation of the first driving scene data and the second driving scene data, the adaptability of the model to a complex driving scene is improved, a vehicle can make a reasonable decision autonomously in the driving process, and the driving efficiency is improved. Therefore, the safety and the stability of driving in different scenes are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a vehicle control method, device, electronic device, and storage medium. Background Technology

[0002] In today's era of rapid development in intelligent transportation, the safety and reliability of autonomous driving technology have become key concerns. Vehicles face a variety of complex and ever-changing driving scenarios during operation, and enabling them to make accurate and safe decisions in these scenarios is a crucial problem that urgently needs to be solved in the field of autonomous driving.

[0003] Traditional vehicle decision-making methods typically rely on limited real-world driving scenario data for model training. However, real-world driving scenario data often has limitations. On one hand, the data coverage is limited, making it difficult to encompass all target driving scenarios that may pose safety risks. For example, driving scenarios under certain extreme weather conditions (such as heavy rain, blizzards, and dense fog), or some special road conditions (such as road construction or traffic chaos caused by sudden traffic accidents), may be rarely encountered in actual data collection, yet these scenarios pose significant threats to safe vehicle operation. On the other hand, the actual collected data may contain data biases, with some scenarios occurring too frequently or too infrequently. This leads to uneven adaptability of models trained based on this data to different scenarios, making them unable to effectively cope with various complex real-world situations.

[0004] Furthermore, training solely based on real-world driving scenario data cannot fully uncover potential risk scenarios and corresponding optimal decisions. Due to the unpredictability and limitations of real-world scenarios, the model may fail to learn the best response strategies for various dangerous situations, potentially leading to incorrect decisions when faced with unseen hazardous scenarios, endangering the safety of the vehicle and passengers. Summary of the Invention

[0005] This application aims to at least partially address one of the technical problems in the related art.

[0006] Therefore, this application proposes a method, apparatus, electronic device, and storage medium.

[0007] One embodiment of this application proposes a vehicle control method, including: The third driving scenario data collected during vehicle operation is input into the first model to generate vehicle decision data; wherein, the first model is pre-trained based on the first driving scenario data and the second driving scenario data, and the second driving scenario data is obtained by data expansion of the first driving scenario data; The vehicle's trajectory is controlled based on the decision data.

[0008] By using a first model pre-trained on a rich variety of driving scenario data, vehicles can better adapt to various complex real-world driving scenarios. Whether in common traffic situations or rare, dangerous ones, the vehicle can generate reasonable decision data based on the model, thereby improving the robustness and generalization of the first model and enhancing safety and stability in different scenarios. The first model, pre-trained on a large amount of data, possesses strong intelligent decision-making capabilities. It can quickly and accurately generate decision data based on real-time collected third-party driving scenario data, enabling the vehicle to make reasonable decisions autonomously during driving, reducing reliance on human intervention and improving the intelligence level of autonomous driving.

[0009] Optionally, the training steps of the first model include: The first driving scenario data is augmented to generate second driving scenario data under the target driving scenario; wherein, the target driving scenario is the training scenario. Based on the first driving scenario data and the second driving scenario data, corresponding target decision data is generated; the target decision data represents the actions performed by the vehicle in the scenario corresponding to the driving scenario data. The first model is trained based on the first driving scenario data, the second driving scenario data, and the target decision data.

[0010] The second driving scenario data generated through data augmentation greatly enriches the diversity of training data, enabling the model to learn coping strategies for various dangerous scenarios. This helps improve the model's decision-making ability in complex and dangerous driving environments, avoiding decision-making errors caused by limited training data.

[0011] The target decision data generated by combining preset constraint rules provides accurate learning targets for model training. During training, the model continuously adjusts itself to approximate these target decision data, thereby improving the accuracy of decision data generated based on driving scenario data and enabling vehicles to make more reasonable and safer decisions in actual driving.

[0012] Training the first model with training data encompassing multiple scenarios enhances its generalization ability. The model is no longer limited to learning from specific scenarios but can abstract universally applicable decision-making patterns from a large amount of data across diverse scenarios. This allows it to make reasonable decisions even when faced with new and unseen driving scenarios, improving the model's reliability and practicality.

[0013] Optionally, the step of augmenting the first driving scenario data to generate second driving scenario data under the target driving scenario includes: A first driving scenario model is constructed based on the first driving scenario data, wherein the first driving scenario model includes multiple scenario elements; The scene elements in the first driving scene model are adjusted to generate a second driving scene model; The second driving scenario data is generated based on the second driving scenario model.

[0014] By specifically adjusting the scene elements in the first driving scenario model, various target driving scenarios with safety risks can be accurately simulated, making the generated second driving scenario data closer to actual dangerous situations. This helps the first model learn coping strategies in real dangerous scenarios and improves the model's decision-making ability in complex and dangerous environments. Because the generated second driving scenario data covers various changes in scene elements, the first model can learn the impact of different scene element changes on decision-making during training, thereby enhancing the model's adaptability to various scene changes. When the vehicle encounters similar scene changes in actual driving, the model can make decisions more quickly and accurately.

[0015] Optionally, adjusting the scene elements in the first driving scene model to generate the second driving scene model includes: Based on the target driving scenario, determine the target scenario element to be adjusted and the corresponding adjustment parameters for the target scenario element; The corresponding target scene elements to be adjusted are adjusted according to the adjustment parameters to obtain the second driving scene model.

[0016] By accurately identifying the target scenario elements and adjustment parameters, the actual situation of the target driving scenario can be realistically simulated, enabling the generated second driving scenario model to more accurately reflect the characteristics of dangerous scenarios. This helps the first model learn the correct decisions in real dangerous scenarios, improving the vehicle's ability to cope with dangerous situations in actual driving.

[0017] The second driving scenario model, generated based on accurate simulations of dangerous situations, provides more effective data for training the first model. During training, the model learns decision-making patterns closely related to actual dangerous scenarios, thereby improving training effectiveness and enabling the model to make more reasonable and accurate decisions when facing real dangerous situations.

[0018] Optionally, the scene elements include a first vehicle element, and adjusting the scene elements in the first driving scene model to generate a second driving scene model further includes: Generate the driving trajectory of at least one of the first vehicle elements to obtain the second driving scene model; wherein, the first vehicle element is the scene element corresponding to the current vehicle.

[0019] By generating driving trajectories for various first vehicle elements, the diversity of the second driving scenario model is greatly increased. This allows the training data to cover more diverse driving situations, such as sudden lane changes and emergency obstacle avoidance, which helps the first model learn decision-making methods in various complex driving situations and improves the model's generalization ability.

[0020] The driving trajectory generated by considering vehicle dynamics characteristics more closely resembles the changes in real driving, making the generated second driving scenario model more realistic. During training, the model can learn the decision-making patterns of the vehicle in response to different driving situations in actual driving, improving the accuracy and rationality of the vehicle's decisions when facing similar situations in real driving.

[0021] Optionally, the scene elements may also include other scene elements, and the method may further include: Based on the interaction relationship between other scene elements in the first driving scenario model and the driving trajectory of the first vehicle element, the state of the other scene elements is adjusted.

[0022] By learning decision-making patterns in various interaction scenarios, the first model can better adapt to the complex and ever-changing situations in real-world driving. When the vehicle encounters similar interaction scenarios in actual driving, the model can make reasonable decisions based on training experience, improving the vehicle's adaptability and safety in different driving environments.

[0023] Adjusting the states of other scene elements based on realistic interaction relationships makes the various situations in the generated second driving scenario model more consistent with actual logic. The decision-making patterns learned by the first model during training are also more reasonable, avoiding unreasonable decisions caused by ignoring interaction relationships and improving the model's decision-making quality in practical applications.

[0024] Optionally, the other scene elements include: a second vehicle element and a pedestrian element, and adjusting the state of the other scene elements includes at least one of the following: Adjust the driving trajectory of the second vehicle element based on the driving trajectory of the first vehicle element in the first driving scenario model. The movement trajectory of the pedestrian element is adjusted based on the driving trajectory of the first vehicle element in the first driving scenario model.

[0025] By employing specific adjustments for the second vehicle element and the pedestrian element, the interaction between them and the first vehicle element in real-world driving scenarios can be accurately simulated. This makes the generated second driving scenario model closer to reality, providing the first model with more accurate training data reflecting actual driving interactions. This helps the model learn decision-making methods under various real-world interaction scenarios, improving the accuracy of its decisions.

[0026] By accurately simulating interactions between vehicles and between vehicles and pedestrians, the realism of the second driving scenario model is enhanced. This realistic scenario model allows the first model to better understand the complexities of actual driving situations, enabling it to make more realistic decisions and improve vehicle safety when faced with real-world driving scenarios.

[0027] After learning from training data based on accurately simulated interaction scenarios, the first model is better able to handle various real-world driving situations. Whether in urban roads, highways, or rural roads, the model can make reasonable decisions based on training experience, enhancing its generalization ability and improving its applicability in different driving environments.

[0028] Optionally, generating the driving trajectory of at least one of the first vehicle elements to obtain the second driving scene model includes: Sampling is performed on the ground plane in the first driving scenario to obtain at least one sampling location; Generate the driving trajectory from the position of the first vehicle element to each of the sampling positions to obtain the second driving scene model.

[0029] By sampling on the ground plane and generating driving trajectories based on the sampling locations, a variety of different driving trajectories can be generated, covering situations where the vehicle changes direction and position. This increases the diversity of the second driving scenario model, provides richer training data for the first model, helps the model learn decision-making methods under various driving conditions, and improves the model's generalization ability.

[0030] Optionally, generating the second driving scenario data based on the second driving scenario model includes: The second driving scene data is rendered to obtain video data from at least one perspective, which is used as the second driving scene data.

[0031] By manipulating factors such as lighting and texture during the rendering process, the details of driving scenes can be enhanced. The model can learn richer scene details, such as road surface conditions and vehicle appearance features. This detailed information is crucial for the model's decision-making in real-world driving scenarios, improving the accuracy of its decisions.

[0032] Optionally, rendering the second driving scene data to obtain video data from at least one perspective includes: Determine the camera parameters corresponding to the preset viewpoint; Based on the camera parameters, the second driving scene data is transformed to obtain the video data corresponding to the viewpoint.

[0033] By accurately determining camera parameters and performing coordinate transformations, the generated video data corresponding to the viewpoint can precisely present the actual situation of the second driving scenario model. During training, the model can acquire accurate scene information, including the position, shape, and relative relationships of objects, which helps improve the model's understanding of the driving scenario and the accuracy of its decisions.

[0034] Accurate video data that reflects the actual conditions of a scene enables vehicle decision-making models to more effectively learn scene characteristics and decision-making basis from different perspectives. The model can learn from precise video data the interaction between the vehicle and its surrounding environment from different perspectives, and how to make reasonable decisions based on this information, thereby enhancing the model's decision-making ability in real-world driving scenarios.

[0035] Optionally, generating corresponding target decision data based on the first driving scenario data and the second driving scenario data includes: Based on the first driving scenario model corresponding to the first driving scenario data, or the second driving scenario model corresponding to the second driving scenario data, and in combination with preset constraint rules, at least one target trajectory data is generated. Based on the target trajectory data, action commands for the vehicle are determined to obtain the target decision data.

[0036] By combining multiple constraint rules to generate target trajectory data, the generated target decision data fully considers various factors in actual driving, ensuring that the vehicle adheres to traffic rules while also guaranteeing safety, meeting vehicle performance and comfort requirements during the decision-making process. This helps the trained first model make reasonable decisions in real-world driving scenarios, avoiding safety issues or uncomfortable experiences caused by unreasonable decisions.

[0037] By using target decision data generated from various types of constraint rules, the first-level model can learn decision-making methods in a variety of complex driving scenarios. Whether on busy urban roads or on mountainous roads with complex road conditions, the model can learn from this target decision data how to make appropriate decisions based on different scenarios and constraints, thereby improving the model's adaptability to different driving scenarios.

[0038] Optionally, the step of generating at least one target trajectory data by combining preset constraint rules includes at least one of the following: Based on each constraint rule, candidate trajectory data corresponding to that constraint rule is generated respectively; Candidate trajectory data that satisfy each of the aforementioned constraint rules are used as the target trajectory data.

[0039] By processing different constraint rules separately and filtering out trajectory data that satisfies all constraints, it is ensured that the generated target trajectory data fully meets various requirements in actual driving, thus improving the accuracy of the trajectory data. This makes the target decision data determined based on the target trajectory data more reasonable and reliable, helping the vehicle decision model learn accurate decision patterns and improving the model's decision-making accuracy in real-world driving scenarios.

[0040] The systematic processing and filtering of trajectory data corresponding to constraint rules enhances the reliability of the process for generating target decision data. Through a rigorous filtering process, unreasonable decisions caused by unmet constraint rules are avoided, ensuring that the vehicle can drive in a realistic manner under various conditions, thus improving the reliability and stability of the entire vehicle decision-making system.

[0041] Optionally, generating candidate trajectory data corresponding to each constraint rule according to each constraint rule includes at least one of the following: Based on preset traffic rule constraints, first candidate trajectory data for vehicles is generated; wherein, the traffic rule constraints are used to restrict the first vehicle element to comply with traffic rules; Based on preset collision safety constraints, second candidate trajectory data for the vehicle is generated; wherein, the collision safety constraints are used to limit the distance between the first vehicle element and other scene elements; Based on preset dynamic feasibility constraints, third candidate trajectory data for the vehicle is generated; wherein the dynamic feasibility constraints are based on limiting the driving trajectory of the first vehicle element to conform to vehicle performance. Based on preset comfort constraints, fourth candidate trajectory data for the vehicle is generated; wherein, the comfort constraints are used to restrict the driving trajectory of the first vehicle element to meet comfort requirements.

[0042] By employing specific generation methods for different constraint rules, the needs of various aspects in actual driving can be accurately met. The generated candidate trajectory data respectively meet the requirements of traffic rules, collision safety, dynamic feasibility, and comfort, making the final generated target decision data more comprehensive and reasonable. This helps the vehicle decision model learn the correct decision-making methods under various actual conditions, thereby improving the model's decision quality.

[0043] Optionally, training the first model based on the first driving scenario data, the second driving scenario data, and the target decision data includes: The first driving scenario data and the second driving scenario data are input into the first model for inference to obtain training decision data; Calculate the loss function based on the training decision data and the target decision data; The first model is trained based on the loss function as the objective.

[0044] The model was trained using first-stage driving scenario data and expanded second-stage driving scenario data, covering a variety of common and dangerous driving scenarios. In learning how to reduce the loss function, the model was able to extract universally applicable decision-making patterns from this diverse data, thereby enhancing its generalization ability. Even when faced with completely new driving scenarios, the model is more likely to make reasonable decisions based on the learned patterns, improving its adaptability to different driving environments.

[0045] Optionally, controlling the vehicle's trajectory based on the decision data includes: Based on at least one control instruction from the decision data, the operating state of the vehicle's components is controlled so that the vehicle runs along the operating trajectory corresponding to the decision data.

[0046] By precisely controlling the operating status of vehicle components based on control commands from decision data, high-precision control of the vehicle's trajectory can be achieved. The vehicle can accurately follow the trajectory planned by the decision data, avoiding deviations and improving the accuracy and stability of vehicle operation. This is especially important in complex driving scenarios, such as driving on narrow roads or changing lanes at high speeds, effectively preventing collisions and ensuring the safety of the vehicle and its passengers.

[0047] By rationally controlling the operating states of vehicle components, vehicles can achieve optimal performance under various driving conditions. For example, during acceleration and deceleration, precise power and braking control can make the acceleration and deceleration process smoother, reducing passenger discomfort; during steering, precise steering control can improve vehicle handling, enabling the vehicle to flexibly handle various curves and steering needs. Overall, this improves the vehicle's driving performance, providing passengers with a more comfortable and safer driving experience.

[0048] Another embodiment of this application provides a vehicle control device, including: The decision module is used to input the third driving scenario data collected during the vehicle's driving process into the first model to generate the vehicle's decision data; wherein, the first model is obtained by pre-training based on the first driving scenario data and the second driving scenario data, and the second driving scenario data is obtained by data expansion of the first driving scenario data; The control module is used to control the vehicle's operating trajectory based on the decision data.

[0049] Optionally, the device further includes: An expansion module is used to expand the first driving scenario data to generate second driving scenario data under the target driving scenario; wherein, the target driving scenario is the training scenario; The decision generation module is used to generate corresponding target decision data based on the first driving scenario data and the second driving scenario data; the target decision data represents the actions performed by the vehicle in the scenario corresponding to the driving scenario data; The training module is used to train the first model based on the first driving scenario data, the second driving scenario data, and the target decision data.

[0050] Optionally, the expansion module includes: The model generation submodule is used to construct a corresponding first driving scenario model based on the first driving scenario data, wherein the first driving scenario model includes multiple scenario elements; The model adjustment submodule is used to adjust the scene elements in the first driving scene model to generate a second driving scene model; The scene data generation submodule is used to generate the second driving scene data based on the second driving scene model.

[0051] Another embodiment of this application proposes a vehicle including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method described in the foregoing aspect.

[0052] Another embodiment of this application proposes a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the foregoing aspect.

[0053] Another embodiment of this application proposes a chip including processing circuitry configured to perform the method described in one aspect above.

[0054] Another embodiment of this application proposes a computer program product that, when executed by a processor, implements the method described in the foregoing aspect.

[0055] The vehicle control method, device, electronic equipment, chip, and storage medium proposed in this application can achieve the following beneficial effects: Improved decision-making adaptability: By using a first model pre-trained with second driving scenario data augmented from the first driving scenario data, the vehicle can better adapt to various complex real-world driving scenarios. Whether it's a common traffic scenario or a rare, dangerous one, the vehicle can generate reasonable decision data based on the model, thereby improving the safety and stability of driving in different scenarios.

[0056] Enhancing Intelligent Decision-Making Capabilities: The first model, pre-trained on a large amount of data, possesses strong intelligent decision-making capabilities. It can quickly and accurately generate decision data based on real-time collected data from third-party driving scenarios, enabling the vehicle to make reasonable decisions autonomously during driving, reducing reliance on human intervention and improving the intelligence level of autonomous driving.

[0057] Optimized driving trajectory control: Precise control of the vehicle's trajectory based on decision data helps avoid accidents such as collisions. Control commands in the decision data can adjust the vehicle's driving status in real time according to the actual driving scenario, ensuring that the vehicle always stays on a safe and reasonable driving trajectory, thus improving the safety and efficiency of vehicle operation.

[0058] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0059] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a schematic flowchart of a vehicle control method provided in an embodiment of this application; Figure 2 A flowchart illustrating a first model training method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a vehicle control device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a chip proposed in an embodiment of this application. Detailed Implementation

[0060] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0061] The vehicle control method, apparatus, electronic device, chip, and storage medium of this application are described below with reference to the accompanying drawings.

[0062] Figure 1 This is a schematic flowchart of a vehicle control method provided in an embodiment of this application.

[0063] In one implementation, the vehicle control method of this application embodiment can be configured in a vehicle control device, which can be applied to any electronic device so that the electronic device can perform vehicle control functions.

[0064] Among them, electronic devices can be any device with computing capabilities, such as mobile terminals, which can be hardware devices with various operating systems, touch screens and / or displays, such as mobile phones, tablets, personal digital assistants, wearable devices, etc.

[0065] As another implementation, the vehicle control method of this application embodiment can also be executed by a chip with processing capabilities. The chip includes an image signal processing chip (ISP), a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a system on a chip (SOC), a reduced instruction set computer (RISC), etc., which will not be listed here.

[0066] It should be noted that all data collection operations related to users in this application are conducted with the user's authorization and in strict compliance with relevant laws and regulations such as privacy and security.

[0067] like Figure 1 As shown, the method may include the following steps: Step 101: Input the third driving scenario data collected during vehicle driving into the first model to generate vehicle decision data; wherein, the first model is obtained by pre-training based on the first driving scenario data and the second driving scenario data, and the second driving scenario data is obtained by data expansion of the first driving scenario data; Step 102: Control the vehicle's trajectory based on the decision data.

[0068] In this embodiment, the first model is pre-trained based on first driving scenario data and second driving scenario data derived from it. The first driving scenario data typically originates from real-world data collected during actual driving, covering various common driving scenarios such as urban road driving and highway driving. The second driving scenario data is generated by augmenting the first driving scenario data. This augmented data simulates various target driving scenarios with safety risks, as well as long-tail driving scenarios, such as driving in extreme weather conditions and scenarios involving sudden road accidents. By pre-training on these two diverse types of driving scenario data, the first model can learn the appropriate decision-making patterns that the vehicle should adopt in different scenarios.

[0069] It should be noted that the first driving scenario data and the second driving scenario data are two relatively independent sets of data. The second driving scenario data is an expanded set of data for more driving scenarios obtained by simulating the first driving scenario data. By expanding the scenario data, the processing capability of the first model is greatly improved.

[0070] During vehicle operation, real-time data from a third driving scenario is input into a pre-trained first model. Based on the knowledge and patterns learned during pre-training, the first model analyzes and infers from the third driving scenario data to generate decision data for the vehicle. This decision data includes various decision-making information that the vehicle should take in the current scenario, such as acceleration, deceleration, and steering commands, to ensure the vehicle can drive safely and efficiently in complex real-world driving environments.

[0071] Based on the generated decision data, the vehicle's control system adjusts the operating status of various vehicle components accordingly, thereby precisely controlling the vehicle's trajectory. For example, if the decision data includes a left turn instruction, the vehicle's steering system will execute the corresponding action, causing the vehicle to turn left and travel along the trajectory determined by the decision data.

[0072] Analysis of beneficial effects: Improved decision-making adaptability: By using a first model pre-trained on rich and diverse driving scenario data, vehicles can better adapt to various complex real-world driving scenarios. Whether it's a common traffic scenario or a rare, dangerous one, the vehicle can generate reasonable decision data based on the model, thereby improving the safety and stability of driving in different scenarios.

[0073] Enhancing Intelligent Decision-Making Capabilities: The first model, pre-trained on a large amount of data, possesses strong intelligent decision-making capabilities. It can quickly and accurately generate decision data based on real-time collected data from third-party driving scenarios, enabling the vehicle to make reasonable decisions autonomously during driving, reducing reliance on human intervention and improving the intelligence level of autonomous driving.

[0074] Optimized driving trajectory control: Precise control of the vehicle's trajectory based on decision data helps avoid accidents such as collisions. Control commands in the decision data can adjust the vehicle's driving status in real time according to the actual driving scenario, ensuring that the vehicle always stays on a safe and reasonable driving trajectory, thus improving the safety and efficiency of vehicle operation.

[0075] Optionally, the training steps of the first model include: The first driving scenario data is augmented to generate second driving scenario data under the target driving scenario; wherein, the target driving scenario is a scenario that needs to be trained; wherein, the scenario to be trained includes driving scenarios with safety risks, and / or, out-of-distribution (OOD) long-tail scenarios, and / or specified target scenarios, etc.

[0076] Based on the first driving scenario data and the second driving scenario data, corresponding target decision data is generated; the target decision data represents the actions performed by the vehicle in the scenario corresponding to the driving scenario data. The first model is trained based on the first driving scenario data, the second driving scenario data, and the target decision data.

[0077] In this embodiment, obtaining comprehensive and targeted training data is crucial when training the first model for vehicle decision-making. Traditional model training methods often rely on limited real-world driving scenario data, which may not cover all potential hazards, resulting in poor model performance in complex and ever-changing real-world driving environments. This embodiment augments the first driving scenario data to generate second driving scenario data under target driving scenarios with safety risks, and generates target decision data accordingly. This provides richer and more targeted training data for the first model, helping to improve model performance and decision accuracy.

[0078] First, the primary driving scenario data is analyzed in depth to extract various key information and generate a corresponding primary driving scenario model. This model includes multiple scenario elements, such as vehicles, pedestrians, and road environment. Then, based on the characteristics of the target driving scenario, the target scenario elements that need adjustment and their corresponding adjustment parameters are determined. For example, in simulating a dangerous scenario where a vehicle ahead suddenly brakes, the speed parameter of the vehicle ahead (as a target scenario element) is adjusted to a sharp decrease. Based on these adjustment parameters, the target scenario elements are adjusted to generate a secondary driving scenario model. Finally, secondary driving scenario data is generated based on the secondary driving scenario model. This data simulates driving conditions under various dangerous scenarios, providing rich material for model training.

[0079] Optionally, the training scenarios include driving scenarios with safety risks; for example, encountering complex road conditions and road structures while driving. Scenario details include: roads without or with unclear markings (rural roads, construction sections); non-standard intersections (T-junctions, five-way intersections, roundabouts without guide lines); road sections with numerous damaged or potholed surfaces; and temporary traffic control (road construction, accident scene detours). Risk points include: significant differences between the road topology and the standard roads in the training data, leading to failure of lane detection, intersection recognition, and path planning modules; and the scarcity of temporary obstacle (construction cones, warning lines) samples causing model misjudgment.

[0080] Optionally, the training scenario includes out-of-distribution (OOD) long-tail scenarios; out-of-distribution refers to samples whose distributions differ significantly from the training data. When a model encounters OOD samples, it typically produces high-confidence erroneous predictions, which can lead to serious consequences in safety-critical fields such as autonomous driving.

[0081] Long-tail distribution refers to a distribution pattern in a dataset where a few categories ("head") have a large number of samples, while the vast majority of categories ("tail") have very few samples. Head categories have a small percentage but a large number of samples; tail categories have a large number of categories but very few individual samples. Long-tail distributions follow a power-law distribution, meaning frequency is inversely proportional to rank. OOD long-tail scenarios refer to the complex situation where, when training data has a long-tail distribution, the model faces the dual challenges of OOD detection and long-tail classification.

[0082] Examples of long-tail scenarios outside of distribution: Unexpected road abnormality (extremely high risk) Scene details: Bridges / roads suddenly collapse; cargo suddenly falls from vehicles ahead (such as steel or containers on trucks); falling objects from the air (billboards, tree branches, building materials); natural disasters such as earthquakes / mudslides cause road closures.

[0083] Risk point: This type of scenario belongs to an extreme long-tail event (probability of occurrence < 10). -6 The training data contains almost no samples, and the model is completely unable to recognize or respond to it (pure OOD scenario).

[0084] Extreme behavior by other road users (extremely high risk) Scene details: Other vehicles maliciously cut in front of other vehicles, run red lights or rush through checkpoints; pedestrians suddenly climb over guardrails and cross the highway; non-motorized vehicles travel against the flow of traffic on the highway (such as electric vehicles traveling on the highway); large vehicles (trucks, buses) suddenly overturn / go out of control, blocking all lanes.

[0085] Risk points: Malicious behavior / extreme violations fall under the OOD interaction scenario, and the model cannot predict its trajectory; large vehicle rollovers fall under the long-tail target occlusion scenario, and the model cannot quickly identify complete road blockage ahead.

[0086] Based on the first driving scenario model corresponding to the first driving scenario data and the second driving scenario model corresponding to the expanded second driving scenario data, and in conjunction with various preset constraint rules, at least one target trajectory data is generated. These constraint rules include traffic rules, safety rules, vehicle dynamics rules, etc., ensuring that the generated target trajectory data meets safety requirements and is feasible in actual operation. For example, traffic rule constraints ensure that the vehicle obeys traffic lights and lane rules when driving on the generated target trajectory; collision safety constraints ensure that the vehicle maintains a safe distance from other objects. Then, based on the generated target trajectory data, action commands for the vehicle are determined, such as acceleration, deceleration, and steering. These action commands constitute the target decision data, clarifying the actions the vehicle should perform in different scenarios.

[0087] The first model is trained using first driving scenario data, second driving scenario data, and the generated target decision data as the training set. During training, the first and second driving scenario data are input into the first model for inference, and the model outputs training decision data. By comparing the training decision data and the target decision data, a loss function is calculated, which reflects the difference between the model's current output and the expected output. With the goal of reducing the loss function, the parameters of the first model are continuously adjusted, enabling the model to gradually learn how to generate accurate decision data based on different driving scenario data, thereby improving the model's performance.

[0088] Analysis of beneficial effects: Enhancing the diversity of training data: The second driving scenario data generated through data augmentation greatly enriches the diversity of training data, enabling the model to learn coping strategies in various dangerous scenarios. This helps improve the model's decision-making ability in complex and dangerous driving environments, avoiding decision-making errors caused by limited training data.

[0089] Improving decision-making accuracy: The target decision data generated by combining preset constraint rules provides accurate learning targets for model training. During training, the model continuously adjusts to approximate these target decision data, thereby improving the accuracy of decision data generated based on driving scenario data and enabling vehicles to make more reasonable and safer decisions in actual driving.

[0090] Enhanced model generalization ability: Training the first model with training data encompassing multiple scenarios enhances its generalization ability. The model is no longer limited to learning from specific scenarios but can abstract universally applicable decision-making patterns from a large amount of data across diverse scenarios. This allows it to make reasonable decisions even when facing new and unseen driving scenarios, improving the model's reliability and practicality.

[0091] Optionally, the step of augmenting the first driving scenario data to generate second driving scenario data under the target driving scenario includes: A first driving scenario model is constructed based on the first driving scenario data, wherein the first driving scenario model includes multiple scenario elements; The scene elements in the first driving scene model are adjusted to generate a second driving scene model; The second driving scenario data is generated based on the second driving scenario model.

[0092] In this embodiment, a key step in generating second driving scenario data for training the vehicle decision-making model is how to reasonably transform the first driving scenario data. Traditional data generation methods may lack systematicity and specificity, failing to accurately simulate various target driving scenarios with safety risks. This embodiment constructs a first driving scenario model and makes targeted adjustments to its scenario elements to generate a second driving scenario model, thereby obtaining second driving scenario data. This provides training data that more closely reflects actual dangerous scenarios, helping to improve the model's ability to cope with complex and dangerous situations.

[0093] A detailed analysis and modeling of the first driving scenario data was performed, extracting multiple scene elements that comprehensively describe various aspects of the driving scenario. These elements include information such as the position, speed, and direction of travel of the first vehicle element (i.e., the current vehicle), relevant parameters of the second vehicle elements (other surrounding vehicles), the position and direction of movement of pedestrian elements, and environmental elements (such as road type and weather conditions). By integrating these scene elements, a first driving scenario model that accurately reflects the characteristics of the first driving scenario was constructed, providing a basic framework for subsequent scenario adjustments.

[0094] Based on the characteristics of the target driving scenario, the target scenario elements that need adjustment and their corresponding adjustment parameters are clearly defined. For example, if the target driving scenario is a vehicle skidding due to water accumulation on the road ahead during heavy rain, then the target scenario elements might include environmental elements (adjusting the weather conditions to heavy rain) and first vehicle elements (adjusting the vehicle's friction parameters to simulate skidding). After determining the adjustment parameters, the target scenario elements in the first driving scenario model are precisely adjusted to generate a second driving scenario model that can simulate the dangerous situation. This targeted adjustment based on the target scenario makes the generated second driving scenario model closer to the dangerous situations that might actually be encountered.

[0095] Based on the generated second driving scenario model, relevant data generation techniques are used to transform various information from the model into specific second driving scenario data. For example, computer graphics techniques can be used to render the second driving scenario model, generating video data from different perspectives, or generating structured data containing the motion trajectories of elements such as vehicles and pedestrians. This second driving scenario data can vividly present the situation of the target driving scenario, providing intuitive and effective training materials for the training of the first model.

[0096] Analysis of beneficial effects: Accurate Simulation of Hazardous Scenarios: By specifically adjusting the scenario elements in the first driving scenario model, various target driving scenarios with safety risks can be accurately simulated, making the generated second driving scenario data closer to actual dangerous situations. This helps the first model learn coping strategies in real dangerous scenarios and improves the model's decision-making ability in complex and dangerous environments.

[0097] Enhancing the model's adaptability to scene changes: Because the generated second driving scenario data covers various changes in scene elements, the first model can learn the impact of different scene element changes on decision-making during training, thereby enhancing the model's adaptability to various scene changes. When the vehicle encounters similar scene changes in actual driving, the model can make decisions more quickly and accurately.

[0098] Optionally, adjusting the scene elements in the first driving scene model to generate the second driving scene model includes: Based on the target driving scenario, determine the target scenario element to be adjusted and the corresponding adjustment parameters for the target scenario element; The corresponding target scene elements to be adjusted are adjusted according to the adjustment parameters to obtain the second driving scene model.

[0099] In this embodiment, accurately determining the scene elements to be adjusted and the corresponding adjustment parameters is crucial for achieving precise simulation of dangerous scenarios during the generation of the second driving scenario model from the first driving scenario model. Traditional methods may lack in-depth analysis of the target driving scenario, leading to unreasonable adjustments of scene elements and the generation of a second driving scenario model that fails to accurately reflect the characteristics of dangerous scenarios. By accurately determining the target scene elements and adjustment parameters based on the target driving scenario, a second driving scenario model that better matches actual dangerous situations can be generated, providing more effective data support for the training of vehicle decision-making models.

[0100] A thorough study of the characteristics and potential risk factors of the target driving scenario is conducted. For example, in the target driving scenario of a vehicle ahead suddenly experiencing a tire blowout on a highway, risk analysis of this scenario identifies the vehicle ahead (i.e., the second vehicle element) as the target scenario element to be adjusted. Further analysis of the potential impact of the blowout determines adjustment parameters, such as the sudden drop in speed and the angle of loss of steering control after the blowout. These adjustment parameters are determined based on the physical principles of the target driving scenario, practical experience, and relevant research data to ensure accurate simulation of the actual changes in the vehicle's state under a blowout scenario.

[0101] After determining the target scene elements and adjustment parameters to be adjusted, the target scene elements in the first driving scenario model are adjusted accordingly. Taking a scenario of a vehicle experiencing a tire blowout on a highway as an example, the elements representing the vehicle ahead in the first driving scenario model are adjusted according to set adjustment parameters such as a sudden drop in speed and changes in the angle of loss of directional control, thereby obtaining a second driving scenario model that can accurately simulate this dangerous scenario. This adjustment of scene elements based on accurate adjustment parameters makes the generated second driving scenario model more realistic and reliable.

[0102] Analysis of beneficial effects: Realistic simulation of hazardous scenarios: By accurately identifying the target scenario elements and adjustment parameters, the actual situation of the target driving scenario can be realistically simulated, enabling the generated second driving scenario model to more accurately reflect the characteristics of hazardous scenarios. This helps the first model learn the correct decisions in real hazardous scenarios, improving the vehicle's ability to cope with dangerous situations in actual driving.

[0103] Improving the effectiveness of model training: The second driving scenario model, generated based on accurately simulated dangerous scenarios, provides more effective data for training the first model. During training, the model can learn decision-making patterns closely related to actual dangerous scenarios, thereby improving training effectiveness and enabling the model to make more reasonable and accurate decisions when facing real dangerous scenarios.

[0104] Optionally, the scene elements include a first vehicle element, and adjusting the scene elements in the first driving scene model to generate a second driving scene model further includes: Generate the driving trajectory of at least one of the first vehicle elements to obtain the second driving scene model; wherein, the first vehicle element is the scene element corresponding to the current vehicle.

[0105] In this embodiment, appropriately adjusting the current vehicle's (first vehicle element's) trajectory when generating the second driving scenario model is a crucial means of simulating different driving scenarios and increasing the diversity of training data. Traditional methods may lack flexibility and diversity in generating vehicle trajectories, resulting in scenario models that cannot fully reflect all possible driving situations. This embodiment obtains the second driving scenario model by generating the trajectory of at least one first vehicle element, which can simulate more diverse driving scenarios, provide richer material for training the vehicle decision-making model, and improve the model's ability to cope with complex driving situations.

[0106] Based on the initial driving scenario model, at least one driving trajectory is generated for the first vehicle element (the current vehicle) to simulate different driving conditions. For example, considering that a vehicle may change its direction or speed for various reasons during actual driving, diverse driving trajectories are generated using specific algorithms or rules. One approach is to sample the ground plane in the initial driving scenario, obtaining multiple sampling locations. These sampling locations can be set according to actual needs, such as in different lanes, near the roadside, or near obstacles. Then, the driving trajectory of the first vehicle element from its current position to each sampling location is generated. The generation of the driving trajectory can consider the vehicle's dynamic characteristics, such as acceleration, deceleration, and steering limitations, to ensure that the generated trajectory is feasible in actual driving. In this way, multiple different driving trajectories can be generated, thereby simulating the vehicle's driving path under different conditions.

[0107] The driving trajectory of the first vehicle element is integrated into the first driving scene model to obtain the second driving scene model. In this process, not only does the driving trajectory of the first vehicle element change, but the entire driving scene model is also adjusted accordingly to adapt to the new trajectory of the first vehicle element. For example, the relative positions and interactions between other scene elements (such as the second vehicle element, pedestrian elements, etc.) and the first vehicle element also change due to the change in the first vehicle element's driving trajectory, resulting in the second driving scene model presenting a completely new driving scenario.

[0108] Analysis of beneficial effects: Increased scenario diversity: By generating driving trajectories for various first vehicle elements, the diversity of the second driving scenario model is greatly increased. This allows the training data to cover more diverse driving situations, such as sudden lane changes and emergency obstacle avoidance, which helps the first model learn decision-making methods in various complex driving situations and improves the model's generalization ability.

[0109] Simulating Real-World Driving Changes: The generated driving trajectories, taking into account vehicle dynamics, more closely resemble real-world driving conditions, making the generated second driving scenario model more realistic. During training, the model learns the decision-making patterns of vehicles in response to different driving situations, improving the accuracy and rationality of decisions made by the vehicle when facing similar situations in actual driving.

[0110] Enhancing the model's ability to handle complex situations: Diverse driving trajectories simulate complex and ever-changing driving scenarios. By learning decision-making patterns in these scenarios, the first model can improve its ability to handle complex driving situations. When the vehicle encounters sudden changes in driving conditions during actual driving, the model can quickly make reasonable decisions based on training experience, ensuring the safety of vehicle operation.

[0111] Optionally, the scene elements may also include other scene elements, and the method may further include: Based on the interaction relationship between other scene elements in the first driving scenario model and the driving trajectory of the first vehicle element, the state of the other scene elements is adjusted.

[0112] In this embodiment, when constructing the second driving scenario model for training the vehicle decision-making model, it is insufficient to consider only the changes in the first vehicle element; the interaction relationships between other scene elements and the first vehicle element must also be comprehensively considered. Traditional methods often neglect these interactions, resulting in the generated scenario model failing to realistically reflect the complexities of actual driving, leading to inaccurate decision-making in practical applications. This embodiment, by adjusting the states of other scene elements based on the driving trajectory of the first vehicle element, can construct a more realistic second driving scenario model that better matches actual driving conditions. This provides more effective training data for the vehicle decision-making model, improving the model's decision-making accuracy and adaptability.

[0113] In the first driving scenario model, we conduct an in-depth analysis of the various interaction relationships between the first vehicle element and other scenario elements (such as second vehicle elements, pedestrian elements, etc.). For example, in traffic flow, the acceleration or deceleration of the first vehicle element may cause surrounding second vehicle elements to adjust their speed accordingly to maintain a safe distance; the turning action of the first vehicle element may affect the walking path of pedestrian elements to avoid collisions. These interaction relationships are determined based on traffic rules, physical laws, and actual driving experience, and are an important basis for constructing realistic driving scenarios.

[0114] When the trajectory of the first vehicle element changes, the states of other scene elements are adjusted accordingly based on the established interaction relationships. For example, if the first vehicle element suddenly changes lanes to the left, the trajectory of the second vehicle element is adjusted according to traffic rules and safety distance requirements, causing it to slow down or change direction to avoid a collision. Simultaneously, for nearby pedestrian elements, their movement trajectories are adjusted based on pedestrian reaction habits and safety considerations, such as stopping walking or changing direction to avoid vehicles. In this way, the relationships between various scene elements in the second driving scenario model are ensured to conform to the interaction logic in actual driving, constructing a more realistic driving scenario.

[0115] Analysis of beneficial effects: Constructing realistic and complex scenarios: By considering and adjusting the interaction relationships between the first vehicle element and other scene elements, a more realistic and complex second driving scenario model can be constructed. This model is closer to the complex situations in actual driving, providing more effective training data for the first model, enabling the model to learn decision-making methods in complex interaction scenarios and improve the accuracy of decision-making.

[0116] Enhancing Model Adaptability: By learning decision-making patterns under various interaction scenarios, the first model can better adapt to the complex and ever-changing situations in real-world driving. When the vehicle encounters similar interaction scenarios in actual driving, the model can make reasonable decisions based on training experience, improving the vehicle's adaptability and safety in different driving environments.

[0117] Improving the rationality of model decisions: Adjusting the states of other scene elements based on real-world interaction relationships makes the various situations in the generated second driving scenario model more consistent with actual logic. The decision patterns learned by the first model during training are also more rational, avoiding unreasonable decisions caused by ignoring interaction relationships, thus improving the model's decision-making quality in practical applications.

[0118] Optionally, the other scene elements include: a second vehicle element and a pedestrian element, and adjusting the state of the other scene elements includes at least one of the following: Adjust the driving trajectory of the second vehicle element based on the driving trajectory of the first vehicle element in the first driving scenario model. The movement trajectory of the pedestrian element is adjusted based on the driving trajectory of the first vehicle element in the first driving scenario model.

[0119] In this embodiment, when adjusting the states of other scene elements in the second driving scenario model, specific adjustment methods are adopted for different types of scene elements, which is crucial for accurately simulating real-world driving scenarios. Traditional methods may employ general but imprecise adjustment strategies, failing to fully consider the unique interaction relationships between different scene elements (such as the second vehicle element and pedestrian element) and the first vehicle element, resulting in a scene model that does not realistically reflect actual driving conditions. This embodiment proposes specific adjustment methods based on the driving trajectory of the first vehicle element, taking into account the characteristics of the second vehicle element and pedestrian element. This enables the construction of a more realistic second driving scenario model, providing higher-quality training data for the vehicle decision-making model and improving the model's decision-making capabilities.

[0120] When the trajectory of the first vehicle element changes, the trajectory of the second vehicle element is adjusted according to traffic rules and safe driving principles. For example, if the first vehicle element suddenly brakes and decelerates on the road, the second vehicle element needs to decelerate accordingly to avoid a rear-end collision. The specific deceleration magnitude and timing can be determined based on factors such as the distance between the two vehicles, their speeds, and traffic flow. By accurately calculating these factors, the trajectory of the second vehicle element is adjusted to adapt to the changes in the first vehicle element's trajectory, simulating the cooperative and avoidance relationships between vehicles in a real traffic scenario.

[0121] Considering the behavioral characteristics and safety needs of pedestrians on the road, the trajectory of the pedestrian element is adjusted when the trajectory of the first vehicle element changes. For example, if the first vehicle element turns towards the pedestrian's direction, the pedestrian will typically choose to stop walking, back up, or change direction to avoid the vehicle. Based on this behavioral pattern of pedestrians, and combined with factors such as the turning angle, speed, and distance of the first vehicle element from the pedestrian, the trajectory of the pedestrian element is adjusted to conform to the avoidance behavior of pedestrians in real-world scenarios, thereby more realistically simulating the interaction between vehicles and pedestrians on the road.

[0122] Analysis of beneficial effects: Accurate simulation of real-world interactions: By employing specific adjustments for the second vehicle element and pedestrian element, the interaction between them and the first vehicle element in actual driving scenarios can be accurately simulated. This makes the generated second driving scenario model closer to reality, providing the first model with more accurate training data reflecting actual driving interactions. This helps the model learn decision-making methods under various real-world interaction scenarios, improving the accuracy of decision-making.

[0123] Enhancing scenario realism: By accurately simulating interactions between vehicles and between vehicles and pedestrians, the realism of the second driving scenario model is improved. This realistic scenario model allows the first model to better understand the complexities of actual driving situations, enabling it to make more realistic decisions and improve vehicle driving safety when faced with real-world driving scenarios.

[0124] Enhancing Model Generalization Ability: After learning from training data based on accurately simulated interaction scenarios, the first model is better able to handle various real-world driving situations. Whether in urban roads, highways, or rural roads, the model can make reasonable decisions based on training experience, enhancing its generalization ability and improving its applicability in different driving environments.

[0125] Optionally, generating the driving trajectory of at least one of the first vehicle elements to obtain the second driving scene model includes: Sampling is performed on the ground plane in the first driving scenario to obtain at least one sampling location; Generate the driving trajectory from the position of the first vehicle element to each of the sampling positions to obtain the second driving scene model.

[0126] In this embodiment, when generating the driving trajectory of the first vehicle element to construct the second driving scenario model, employing reasonable sampling and trajectory generation methods is crucial to ensuring the diversity and realism of the generated trajectories. Traditional methods may have shortcomings in sampling methods or trajectory generation logic, resulting in generated driving trajectories that cannot fully reflect various possible situations in actual driving, or that the generated trajectories do not conform to the physical laws and behavioral habits of actual driving. This embodiment, by sampling the ground plane of the first driving scenario and generating the driving trajectory of the first vehicle element based on the sampling location, can generate diverse and realistic driving trajectories, providing strong support for constructing a more realistic second driving scenario model and improving the quality of training data for the vehicle decision model.

[0127] Ground plane sampling: Sampling is performed on the ground plane in the first driving scenario to obtain multiple sampling locations. The sampling process can be designed according to actual needs and scenario characteristics. For example, uniform sampling can be used, sampling at certain intervals across the entire ground plane to cover different locations; non-uniform sampling can also be used, increasing the density of sampling points in some key areas (such as intersections and near obstacles) to simulate the possible driving situations of the vehicle in these areas in more detail. Through a reasonable sampling strategy, a series of representative sampling locations are obtained, which will serve as target points for generating the driving trajectory.

[0128] Starting from the current position of the first vehicle element, driving trajectories to various sampling positions are generated. When generating these trajectories, the vehicle's dynamic characteristics are fully considered, such as its maximum acceleration, deceleration, and turning radius. For example, if a sampling position is slightly to the left of the vehicle's current position, a reasonable driving trajectory from the current position to that sampling position is calculated based on the vehicle's turning radius and acceleration limitations, including acceleration, deceleration, and turning processes. Simultaneously, traffic rules and actual driving habits can also be considered, such as following traffic light signals and lane rules at intersections. In this way, the generated driving trajectories conform to both the vehicle's physical characteristics and actual driving rules and habits, resulting in a high degree of realism.

[0129] Analysis of beneficial effects: Generate diverse trajectories: By sampling on the ground plane and generating driving trajectories based on the sampling locations, a variety of different driving trajectories can be generated, covering situations where the vehicle changes direction and position. This increases the diversity of the second driving scenario model, provides richer training data for the first model, helps the model learn decision-making methods under various driving conditions, and improves the model's generalization ability.

[0130] Ensuring trajectory realism: The generated driving trajectory, considering vehicle dynamics and actual driving rules, better aligns with the physical laws and behavioral habits of real-world driving. This makes the generated second driving scenario model more realistic, allowing the first model to learn accurate decision-making patterns during training, thus improving the model's decision-making accuracy and rationality in real-world driving scenarios.

[0131] Improving the accuracy of scenario simulation: The second driving scenario model, based on reasonable sampling and real-world trajectory generation, can more accurately simulate real-world driving scenarios. The model can better learn the interaction relationships between the vehicle and other scene elements under different driving trajectories, as well as the decisions to be made in various real-world scenarios, thereby improving the model's simulation accuracy of real-world driving scenarios and providing more reliable decision support for vehicles in actual driving.

[0132] Optionally, generating the second driving scenario data based on the second driving scenario model includes: The second driving scene data is rendered to obtain video data from at least one perspective, which is used as the second driving scene data.

[0133] In this embodiment, when the second driving scenario model is transformed into specific second driving scenario data, the application of rendering technology can intuitively present various information of the driving scenario, providing more intuitive and expressive data for the training of the vehicle decision-making model. Traditional data presentation methods may be relatively abstract and difficult to fully display the details and dynamic changes of the driving scenario, which is not conducive to the model learning key information in the actual driving scenario. This embodiment generates video data from at least one perspective as the second driving scenario data by rendering the second driving scenario model, which can vividly display the whole picture of the driving scenario, improve the quality and effectiveness of training data, and help improve the training effect of the vehicle decision-making model.

[0134] Before rendering the second driving scene model, some rendering parameters need to be determined, such as lighting conditions and material properties, to create a realistic driving environment. Simultaneously, the viewpoint to be acquired needs to be determined based on actual requirements. Different viewpoints can showcase different aspects of the driving scene; for example, a first-person view can simulate the driver's field of vision, while a third-person view can more comprehensively show the relationship between the vehicle and its surrounding environment. After determining the viewpoint, the corresponding camera parameters, including camera position, orientation, and focal length, need to be determined.

[0135] Based on the determined camera parameters, a coordinate transformation is performed on the second driving scene model, converting the 3D scene information in the model into 2D image information. During the rendering process, computer graphics techniques are used to draw various elements in the scene (such as vehicles, pedestrians, roads, buildings, etc.), taking into account factors such as lighting, shadows, and textures to make the generated images more realistic. By continuously rendering a series of images and playing them at a certain frame rate, video data from at least one perspective is generated. This video data vividly demonstrates various dynamic changes in the second driving scene model, such as vehicle movement, pedestrian movement, and environmental changes, providing intuitive visual information for the training of the first model.

[0136] Analysis of beneficial effects: Enhanced Scene Detail: By processing factors such as lighting and texture during the rendering process, the detail of the driving scene can be enhanced. The model can learn richer scene details, such as road surface conditions and vehicle appearance features. This detailed information is crucial for the model's decision-making in real-world driving scenarios, improving the accuracy of the model's decisions.

[0137] Simulating Real-World Driving Perspectives: By generating video data based on different preset perspectives, it is possible to simulate various perspectives in real-world driving, such as the driver's view and the bystander's view. This helps the model learn information from different angles in driving scenarios, better understand the vehicle's decision-making needs under different perspectives, and improve the model's adaptability and decision-making ability in actual driving.

[0138] Optionally, rendering the second driving scene data to obtain video data from at least one perspective includes: Determine the camera parameters corresponding to the preset viewpoint; Based on the camera parameters, the second driving scene data is transformed to obtain the video data corresponding to the viewpoint.

[0139] In this embodiment, accurate coordinate transformation is a crucial step in converting the 3D information of the second driving scene model into 2D image information during the rendering process of generating video data corresponding to the viewpoint. Traditional coordinate transformation methods may lack precision, resulting in generated video data that fails to accurately reflect the actual situation of the second driving scene model, thus affecting the vehicle decision-making model's learning of scene information. This embodiment, by determining the camera parameters corresponding to a preset viewpoint and performing coordinate transformation based on these parameters, can accurately convert the second driving scene data into video data corresponding to the viewpoint, improving the quality of the rendered video data and providing more accurate training data for the vehicle decision-making model.

[0140] Based on the preset viewpoint, the camera parameters are determined in detail. For example, for a first-person viewpoint (simulating the driver's perspective), the camera should be placed inside the vehicle's cab, close to the driver's eyes, facing the same direction as the vehicle's movement to simulate the driver's field of vision. The focal length is set according to the visual characteristics of the human eye and actual needs, generally set to a focal length value that can clearly display the road and objects within a certain distance ahead. For different viewpoints, such as a third-person overhead viewpoint, the camera position will be set at a certain height above the scene, facing vertically downwards or at a certain tilt angle, and the focal length will also be adjusted according to the required scene coverage. The accurate determination of these camera parameters is the foundation for subsequent coordinate transformations.

[0141] Based on the determined camera parameters, coordinate transformation is performed on the second driving scene data. In three-dimensional space, each point in the second driving scene model has its specific three-dimensional coordinates. Coordinate transformation converts these three-dimensional coordinates into coordinates on a two-dimensional image plane. This process involves a series of mathematical operations, including rotation, translation, and projection. For example, rotation and translation operations are performed based on the camera's position and orientation to transform points in the three-dimensional scene into the camera coordinate system. Then, a projection transformation projects the three-dimensional points in the camera coordinate system onto the two-dimensional image plane, obtaining the corresponding two-dimensional coordinates. Through these coordinate transformations, the three-dimensional scene information in the second driving scene model is accurately mapped onto the two-dimensional image, thereby generating video data corresponding to that viewpoint and ensuring that the video data accurately reflects the actual situation of the second driving scene model.

[0142] Analysis of beneficial effects: Accurate Scene Representation: Through precise camera parameter determination and coordinate transformation, the generated video data corresponding to the viewpoint accurately represents the actual situation of the second driving scene model. During training, the model can acquire accurate scene information, including the position, shape, and relative relationships of objects, which helps improve the model's understanding of driving scenes and the accuracy of its decisions.

[0143] Enhancing Model Learning Performance: Accurate video data reflecting real-world scenarios enables vehicle decision-making models to more effectively learn scene characteristics and decision-making criteria from different perspectives. Models can learn from precise video data the interaction between the vehicle and its surrounding environment from different viewpoints, and how to make reasonable decisions based on this information, thereby enhancing the model's decision-making capabilities in real-world driving scenarios.

[0144] Optionally, generating corresponding target decision data based on the first driving scenario data and the second driving scenario data includes: Based on the first driving scenario model corresponding to the first driving scenario data, or the second driving scenario model corresponding to the second driving scenario data, and in combination with preset constraint rules, at least one target trajectory data is generated. Based on the target trajectory data, action commands for the vehicle are determined to obtain the target decision data.

[0145] In this embodiment, to train a vehicle decision-making model capable of making reasonable decisions in various driving scenarios, accurate and practical target decision data needs to be generated. Traditional methods may lack comprehensiveness and accuracy in generating target decision data, failing to fully consider the complexity of driving scenarios and various constraints, leading to unreasonable decisions by the trained model. This embodiment generates target trajectory data by combining a first or second driving scenario model with preset constraint rules, and determines action commands accordingly to obtain target decision data. This generates more reasonable target decision data that better reflects actual driving conditions, ensuring effective training of the vehicle decision-making model.

[0146] Generate target trajectory data: Based on the driving scenario described by the first driving scenario model corresponding to the first driving scenario data, or the second driving scenario model corresponding to the second driving scenario data, and in conjunction with preset constraint rules, generate at least one target trajectory data. These constraint rules cover multiple aspects, such as traffic rule constraints to ensure that the vehicle complies with traffic lights, lane rules, etc. during driving; collision safety constraints to ensure that the vehicle maintains a safe distance from other scene elements (such as other vehicles, pedestrians, obstacles, etc.) to avoid collisions; dynamic feasibility constraints to ensure that the generated vehicle trajectory conforms to the vehicle's own dynamic performance, such as maximum acceleration, deceleration, turning radius, etc.; and comfort constraints to consider the comfort requirements during vehicle driving and avoid overly abrupt acceleration, deceleration, or steering operations. By comprehensively considering these constraint rules, target trajectory data that meets various requirements is generated. For example, in a crossroads scenario, by combining traffic rule constraints (stop at red lights, go at green lights), collision safety constraints (maintain a safe distance from other vehicles), and dynamic feasibility constraints (vehicle braking performance), target trajectory data of the vehicle from its current position to the next road segment at the crossroads is generated, ensuring that the vehicle drives under the premise of complying with traffic rules, ensuring safety, and conforming to its own performance.

[0147] Determining Action Commands to Obtain Target Decision Data: Based on the generated target trajectory data, action commands for the vehicle are determined. The target trajectory data describes the vehicle's position and driving state at different points in time. Based on this information, the action commands that the vehicle needs to execute, such as acceleration, deceleration, and steering, are calculated. For example, if the target trajectory data shows that the vehicle needs to accelerate from its current speed to a certain speed and turn left within a certain time period, then the corresponding action commands are to perform acceleration and left turn operations within that time period. These action commands constitute the target decision data, clarifying the specific decisions the vehicle should make in a specific driving scenario, and providing accurate guidance for the training of the first model.

[0148] Analysis of beneficial effects: Ensuring the rationality of decision-making: By combining multiple constraint rules to generate target trajectory data, the generated target decision data fully considers various factors in actual driving, ensuring that the vehicle adheres to traffic rules while also guaranteeing safety and meeting vehicle performance and comfort requirements during the decision-making process. This helps the trained first model make reasonable decisions in real-world driving scenarios, avoiding safety issues or uncomfortable experiences caused by unreasonable decisions.

[0149] Improving model adaptability: By using target decision data generated from different types of constraint rules, the first-line model can learn decision-making methods in various complex driving scenarios. Whether on busy urban roads or on mountainous roads with complex road conditions, the model can learn how to make appropriate decisions based on different scenarios and constraints, thereby improving the model's adaptability to different driving scenarios.

[0150] Optionally, the step of generating at least one target trajectory data by combining preset constraint rules includes at least one of the following: Based on each constraint rule, candidate trajectory data corresponding to that constraint rule is generated respectively; Candidate trajectory data that satisfy each of the aforementioned constraint rules are used as the target trajectory data.

[0151] In this embodiment, when generating target trajectory data by combining preset constraint rules, the key to generating accurate target decision data lies in how to systematically process different constraint rules and select suitable trajectory data. Traditional methods may not handle different constraint rules clearly and reasonably enough, resulting in the generated target trajectory data failing to simultaneously satisfy multiple constraint conditions, or the selection process lacking scientific rigor, thus affecting the quality of the target decision data. This embodiment provides a systematic and scientific method for generating target trajectory data by generating candidate trajectory data corresponding to each constraint rule separately and selecting candidate trajectory data that satisfies each constraint rule as target trajectory data. This helps improve the accuracy and reliability of target decision data, thereby enhancing the training effect of the vehicle decision model.

[0152] Based on various preset constraint rules, candidate trajectory data is generated for each constraint rule. For example, for preset traffic rule constraints, first candidate trajectory data for the vehicle that conforms to traffic light rules, lane driving rules, etc., is generated. This means that when generating the trajectory, it is ensured that the vehicle stops at red lights, proceeds at green lights, and always travels within the designated lane. For preset collision safety constraints, considering the distance limitations between the vehicle and other scene elements (such as other vehicles, pedestrians, obstacles, etc.), second candidate trajectory data for the vehicle is generated to ensure that the vehicle maintains a safe distance from surrounding objects during driving and avoids collisions. For preset dynamic feasibility constraints, based on the vehicle's dynamic performance parameters, such as maximum acceleration, deceleration, and turning radius, third candidate trajectory data for the vehicle is generated, ensuring that the generated trajectory is within the actual range of motion that the vehicle can achieve. For preset comfort constraints, considering the comfort requirements during vehicle driving, such as avoiding excessively abrupt acceleration, deceleration, or steering operations, fourth candidate trajectory data for the vehicle is generated. By generating these candidate trajectory data separately, the impact of different constraint rules on the vehicle's driving trajectory can be comprehensively considered.

[0153] Candidate trajectory data that meets all the constraints are used as target trajectory data. This requires a comprehensive evaluation and screening of the generated candidate trajectory data. For example, a candidate trajectory might meet traffic rule requirements but have collision safety issues, i.e., it is too close to other objects. By checking all candidate trajectory data one by one, it is ensured that the final selected target trajectory data simultaneously meets various constraints such as traffic rule constraints, collision safety constraints, dynamic feasibility constraints, and comfort constraints. The target trajectory data selected in this way not only meets various requirements in actual driving but also provides a reliable basis for determining the target decision data.

[0154] Analysis of beneficial effects: Improving the accuracy of trajectory data: By processing different constraint rules separately and filtering out trajectory data that meets all constraints, it is ensured that the generated target trajectory data fully meets various requirements in actual driving, thus improving the accuracy of the trajectory data. This makes the target decision data determined based on the target trajectory data more reasonable and reliable, helping the vehicle decision model learn accurate decision patterns and improving the model's decision-making accuracy in real-world driving scenarios.

[0155] Enhancing system reliability: The systematic processing and filtering of trajectory data corresponding to constraint rules improves the reliability of the process for generating target decision data. Through a rigorous filtering process, unreasonable decisions caused by unmet constraint rules are avoided, ensuring that the vehicle can drive in a realistic manner under various conditions, thus improving the reliability and stability of the entire vehicle decision-making system.

[0156] Optionally, generating candidate trajectory data corresponding to each constraint rule according to each constraint rule includes at least one of the following: Based on preset traffic rule constraints, first candidate trajectory data for vehicles is generated; wherein, the traffic rule constraints are used to restrict the first vehicle element to comply with traffic rules; Based on preset collision safety constraints, second candidate trajectory data for the vehicle is generated; wherein, the collision safety constraints are used to limit the distance between the first vehicle element and other scene elements; Based on preset dynamic feasibility constraints, third candidate trajectory data for the vehicle is generated; wherein the dynamic feasibility constraints are based on limiting the driving trajectory of the first vehicle element to conform to vehicle performance. Based on preset comfort constraints, fourth candidate trajectory data for the vehicle is generated; wherein, the comfort constraints are used to restrict the driving trajectory of the first vehicle element to meet comfort requirements.

[0157] In this embodiment, when generating candidate trajectory data that conforms to various constraint rules, it is crucial to adopt specific and targeted generation methods for different types of constraint rules. Traditional methods may handle different constraint rules in a general way, failing to fully consider the unique requirements of each constraint rule, resulting in candidate trajectory data that cannot accurately meet the various needs of actual driving. This embodiment proposes specific candidate trajectory data generation methods for traffic rule constraints, collision safety constraints, dynamic feasibility constraints, and comfort constraints, respectively. These methods can generate candidate trajectory data that better meets the requirements of actual driving, laying a solid foundation for generating accurate target decision data and thus improving the training quality of the vehicle decision model.

[0158] Pre-defined traffic rule constraints are used to ensure that the first vehicle element complies with traffic rules. When generating candidate trajectory data, various traffic rules are strictly followed. For example, at intersections with traffic lights, the vehicle's driving or stopping trajectory is generated based on the light's status. When the light is green, a trajectory is generated showing the vehicle accelerating through the intersection from its current position; when the light is red, a trajectory is generated showing the vehicle slowing down and stopping before the stop line. Simultaneously, lane rules are considered to ensure that vehicles always travel within their designated lanes. For example, on multi-lane roads, the vehicle's trajectory is generated according to the lane's direction of travel, preventing vehicles from illegally changing lanes or occupying oncoming lanes. In this way, first candidate trajectory data that complies with traffic rule constraints is generated, ensuring that vehicles abide by traffic regulations during operation.

[0159] Pre-defined collision safety constraints limit the distance between the first vehicle element and other scene elements. When generating candidate trajectory data, the positional relationships between the vehicle and surrounding scene elements such as other vehicles, pedestrians, and obstacles are considered in real time. For example, when a vehicle is detected ahead, a safe following distance is calculated based on the speed, distance, and relative direction of movement of the two vehicles, and a trajectory is generated that maintains this safe following distance. If there are pedestrians on the road, their walking trajectory is predicted based on their walking speed and direction, and a trajectory is generated that allows the vehicle to avoid pedestrians while maintaining a safe distance. Through precise calculation and planning, second candidate trajectory data that meets the collision safety constraints is generated, ensuring that the vehicle avoids collisions with other objects during its journey.

[0160] Pre-defined dynamic feasibility constraints, based on the vehicle's dynamic performance, are used to limit the trajectory of the first vehicle element to conform to its performance characteristics. When generating candidate trajectory data, dynamic parameters such as the vehicle's maximum acceleration, deceleration, and turning radius are fully considered. For example, when the vehicle needs to accelerate, the generated trajectory acceleration does not exceed the vehicle's maximum acceleration; when the vehicle brakes, the deceleration is within the vehicle's tolerable range. When the vehicle performs a steering operation, the turning radius meets the vehicle's design requirements, avoiding loss of control or uneven driving due to an excessively small or large turning radius. By generating a trajectory based on the vehicle's dynamic performance, third candidate trajectory data that conforms to the dynamic feasibility constraints is obtained, ensuring that the vehicle's trajectory is within its practically achievable range of motion.

[0161] Pre-defined comfort constraints are used to ensure that the driving trajectory of the first vehicle element meets comfort requirements. When generating candidate trajectory data, overly abrupt acceleration, deceleration, or steering maneuvers are avoided as much as possible. For example, during vehicle acceleration, a smooth acceleration curve is used to prevent sudden acceleration from causing discomfort such as forward leaning of passengers; similarly, a smooth deceleration curve is used during deceleration to prevent sudden braking from causing significant impact on passengers. For steering maneuvers, changes in steering angle and speed are controlled to ensure smooth steering and prevent sharp turns from causing lateral tilting of passengers. By considering these comfort factors, fourth candidate trajectory data that meets comfort constraints is generated, improving the passenger comfort experience during vehicle operation.

[0162] Analysis of beneficial effects: Precisely meeting diverse needs: Specific generation methods are adopted for different constraint rules, accurately meeting various needs in actual driving. The generated candidate trajectory data respectively meet traffic rules, collision safety, dynamic feasibility, and comfort requirements, making the final target decision data more comprehensive and reasonable. This helps the vehicle decision model learn the correct decision-making methods under various actual conditions, improving the model's decision quality.

[0163] Improving trajectory generation quality: Targeted generation based on the characteristics of each constraint rule enhances the quality of candidate trajectory data. Considering the stringency of traffic rules, the importance of collision safety, limitations in dynamic performance, and comfort requirements, the generated trajectory data is more scientific and reasonable, ensuring the selection of high-quality target trajectory data and thus improving the performance of the entire vehicle decision-making system.

[0164] Enhancing Model Learning Capabilities: Accurate candidate trajectory data reflecting different constraint rules provides rich and accurate learning materials for the primary model. During training, the model can better understand the decision-making methods a vehicle should take under different constraints, enhancing its learning capabilities. This allows the model to make reasonable decisions by comprehensively considering various factors when facing complex and ever-changing driving scenarios, improving its adaptability and reliability.

[0165] Optionally, training the first model based on the first driving scenario data, the second driving scenario data, and the target decision data includes: The first driving scenario data and the second driving scenario data are input into the first model for inference to obtain training decision data; Calculate the loss function based on the training decision data and the target decision data; The first model is trained based on the loss function as the objective.

[0166] In this embodiment, effective training of the first model is crucial for its ability to accurately generate vehicle decision data based on driving scenario data. Traditional model training methods may lack systematicity and specificity, resulting in poor training performance and an inability to make accurate decisions in complex and ever-changing driving scenarios. This embodiment provides a scientific and systematic first model training method by inputting first and second driving scenario data into the first model for inference, calculating the loss function, and training with the goal of reducing the loss function. This helps improve the model's performance and decision accuracy, enabling the model to better adapt to the needs of real-world driving scenarios.

[0167] The first driving scenario data and the second driving scenario data obtained through data augmentation are input into the first model. Based on its internal neural network structure and parameters, the first model performs feature extraction, analysis, and inference on the input driving scenario data. For example, the model identifies various elements in the driving scenario, such as vehicles, pedestrians, and road conditions, and attempts to generate corresponding training decision data based on the relationships between these elements and patterns learned during training. This training decision data represents the model's current prediction of the decisions the vehicle should make in a given driving scenario.

[0168] The training decision data generated by the model is compared with the target decision data generated based on the first and second driving scenario data combined with preset constraint rules. A specific loss function is used to measure the difference between the two. The larger the loss function value, the greater the gap between the model's prediction and the ideal decision; conversely, the smaller the loss function value, the closer the model's prediction is to the ideal decision.

[0169] With the goal of reducing the loss function, the parameters of the first model are adjusted through an optimization algorithm. The model parameters are updated based on the loss function. After each parameter update, the first and second driving scenario data are input into the model again for inference, and the loss function is recalculated. This process is repeated continuously, causing the value of the loss function to gradually decrease, meaning the model's output becomes increasingly closer to the target decision data. As training progresses, the first model gradually learns how to accurately generate decision data that meets actual needs based on different driving scenario data, thereby improving the model's performance.

[0170] Analysis of beneficial effects: Enhancing Model Generalization: The model is trained using first-stage driving scenario data and expanded second-stage driving scenario data, covering a variety of common and dangerous driving scenarios. As the model learns how to reduce the loss function, it extracts universally applicable decision-making patterns from this diverse data, thereby enhancing its generalization ability. Even when faced with completely new driving scenarios, the model is more likely to make reasonable decisions based on the learned patterns, improving its adaptability to different driving environments.

[0171] Optionally, controlling the vehicle's trajectory based on the decision data includes: Based on at least one control instruction from the decision data, the operating state of the vehicle's components is controlled so that the vehicle runs along the operating trajectory corresponding to the decision data.

[0172] In this embodiment, accurately controlling the vehicle's trajectory based on decision data is crucial for achieving safe and efficient vehicle operation during vehicle control. Traditional vehicle control methods may fail to accurately translate decision data into the vehicle's actual trajectory, leading to unstable driving or failure to follow the intended path. This embodiment provides a precise and effective vehicle trajectory control method by controlling the operating state of vehicle components based on control commands within the decision data, thereby enabling the vehicle to operate along the trajectory corresponding to the decision data. This helps improve vehicle performance and safety.

[0173] Decision data contains a series of control commands for vehicle movement, such as acceleration, deceleration, steering, and gear shifting commands. The decision data is first analyzed to identify each control command and its corresponding parameters. For example, an acceleration command might include the desired speed or acceleration magnitude; a steering command might include parameters such as the steering angle and direction. By accurately analyzing the decision data, the specific actions and target states that the vehicle needs to perform are clearly defined.

[0174] Based on the control commands obtained from the analysis, the operating status of corresponding vehicle components is controlled. For acceleration commands, the vehicle's power system (such as the engine and electric motor) increases power output to accelerate the vehicle. Specifically, if the decision data requires the vehicle to accelerate from the current 50 km / h to 80 km / h within 5 seconds, the engine control system will adjust the throttle opening or the electric motor's current output to provide sufficient power to achieve this acceleration. For steering commands, the vehicle's steering system (such as the steering gear and power steering system) controls the steering action of the wheels according to the steering angle and direction in the command. For example, when the decision data issues a command to turn 30 degrees to the left, the steering gear drives the wheels to turn 30 degrees to the left through the hydraulic or electric power steering system, changing the vehicle's direction of travel. For braking commands, the braking system (such as brake discs and brake pads) applies braking force to decelerate the vehicle. For example, if the decision data requires the vehicle to reduce its speed from 80 km / h to 0 km / h within 100 meters, the braking system will precisely control the braking force based on factors such as the vehicle's speed, mass, and road friction to achieve smooth deceleration.

[0175] By coordinating the operation of various vehicle components, the vehicle travels along the trajectory corresponding to the decision data. Multiple components, such as the powertrain, steering system, and braking system, work together. For example, during a turn, the powertrain appropriately reduces power output to accommodate the centrifugal force, the steering system precisely controls the wheel steering angle, and the braking system may apply slight braking when necessary to maintain vehicle stability, thus ensuring the vehicle travels safely and smoothly along the curve trajectory determined by the decision data. Throughout the entire driving process, the vehicle continuously adjusts the operation of each component based on the decision data, consistently maintaining its position on the desired trajectory.

[0176] Analysis of beneficial effects: Precise trajectory control: Based on control commands from decision data, this system precisely controls the operating status of vehicle components, enabling high-precision control of the vehicle's trajectory. The vehicle can accurately follow the trajectory planned by the decision data, avoiding deviations and improving the accuracy and stability of vehicle operation. This is particularly important in complex driving scenarios, such as driving on narrow roads or changing lanes at high speeds, effectively preventing collisions and ensuring the safety of the vehicle and its passengers.

[0177] Enhancing driving performance: By rationally controlling the operating states of vehicle components, the vehicle can achieve optimal performance under various driving conditions. For example, during acceleration and deceleration, precise power and braking control can make the acceleration and deceleration process smoother, reducing passenger discomfort; during steering, precise steering control can improve vehicle handling, enabling the vehicle to flexibly handle various curves and steering needs. Overall, this improves the vehicle's driving performance, providing passengers with a more comfortable and safer driving experience.

[0178] Enhanced system reliability: The clear transformation process from decision data to vehicle component control enhances the reliability of the entire vehicle control system. Each control command has a clearly defined execution object and objective, reducing the risk of system failures caused by unclear control logic. Simultaneously, the coordinated operation between vehicle components, based on unified coordination of decision data, makes the entire system operate more stably, improving the vehicle's reliability and durability in various complex environments.

[0179] Figure 2 This is a flowchart illustrating a first model training method provided in an embodiment of this application. Figure 2 As shown in this embodiment, the end-to-end (E2E) method is an important development direction in the field of autonomous driving. It aims to use a unified deep learning model to directly output driving control commands or planned paths from perception inputs (such as images and radar point clouds), thereby simplifying the complex process that traditionally consists of multiple modules such as perception, prediction, and planning.

[0180] Drawing on advancements in natural language processing and computer vision, many models employ the Transformer architecture to handle spatiotemporal information and multimodal inputs, achieving superior scene understanding and long-range planning. Unified BEV (Bird's-Eye View) representation: Fusing data from different sensors into a unified BEV space for decision-making and planning has become a mainstream paradigm, significantly improving the model's spatial reasoning capabilities. Data-Driven: End-to-End (E2E) models heavily rely on large-scale, high-quality training data. Therefore, efficiently and cost-effectively acquiring, labeling, and expanding data, especially data from corner cases, has become a key bottleneck for technological breakthroughs.

[0181] Simulation systems have become the infrastructure for autonomous driving research and development. Due to the high cost and sparsity of real-world data, the industry is shifting from "testing only in the real world" to "training and testing in simulation".

[0182] In related technologies, high-fidelity rendering: Traditional game engines still lag behind in realism. In recent years, new technologies, such as neural rendering, especially 3D Gaussian Splatting (3DGS), can quickly generate extremely high-fidelity simulation scenes based on real data, narrowing the gap between simulation and reality. Scalable scene generation: Research focuses on how to systematically and automatically generate tens of thousands of diverse, safety-critical test and training scenarios from limited real-world driving logs. This includes perturbations of weather, lighting, traffic density, and vehicle trajectory. Closed-loop interaction and data circulation: Excellent simulation is no longer just static scene rendering, but possesses reactive multi-agent interaction capabilities, enabling closed-loop testing and automatically generating new simulation training data by combining failure cases found in real-world testing, forming a data flywheel.

[0183] Neural rendering techniques, exemplified by 3D Gaussian Splash (3DGS), can generate highly visually faithful simulation images, significantly narrowing the visual gap between simulation and reality. Simultaneously, world models attempt to learn dynamic models of the environment to plan and predict within the model itself. However, a core limitation of these techniques lies in the fact that while they can generate out-of-distribution (OOD) states (S), neither neural rendering nor world models reliably generate corresponding high-quality, feasible expert action trajectories (action supervision / A) for these new states. World model predictions often exhibit drift and distortion in long-term and complex OOD scenarios, failing to provide stable, usable expert supervision for imitation learning.

[0184] Traditional closed-loop simulation platforms or simple data augmentation methods (such as trajectory noise injection) attempt to increase data diversity through perturbation. However, these methods have a dual drawback: on the one hand, game engine-based simulations fall short of the real world in terms of visual fidelity; on the other hand, motion supervision generated through simple rules or reuse of original actions is often unsafe, infeasible, or suboptimal for complex OOD states, which makes the "robustness" learned by the planner lack quality assurance.

[0185] The method provided in this embodiment uses real-world data as a basis and high-fidelity neural rendering technology to reconstruct real scenes into interactive simulation environments.

[0186] The system then moves on to the Scalable Simulation Data generation phase. By perturbing the vehicle's trajectory, the system automatically generates millions of unprecedented Out-of-Depth (OOD) states (such as extremely close interactions, sudden lane departures, etc.). In these states, the system activates pseudo-experts to calculate and output safe, feasible, and optimal driving trajectories for the vehicle in a reactive multi-agent environment.

[0187] Finally, the generated simulation data is mixed with the original real-world data for co-training any end-to-end planner. In this way, the planner learns how to correctly handle sparse and dangerous scenarios in simulation before being deployed to actual roads, significantly improving its robustness and generalization ability on real-world benchmarks, thereby ensuring the safety of the autonomous driving system.

[0188] First, based on existing real-world driving logs D real (That is, first-hand driving scenario data) and a high-fidelity neural rendering simulation engine (such as a 3DGS-based engine) are used to extract and reconstruct an interactive simulation environment (i.e., a first-hand driving scenario model) from the real scene. Subsequently, the system generates potentially risky Out-of-Depth (OOD) states on a large scale by systematically or randomly perturbing the vehicle's trajectory. S pert Next, S pert When injected into a reactive simulation environment, other agents in the vicinity will react in real time and reasonably to the disturbances of the vehicle based on physical and behavioral models, thus obtaining a second driving scenario model.

[0189] The key feature lies in pseudo-expert-supervised generation: for the generated OOD state S pert This invention enables a planner-based pseudo-expert. This expert has access to environmental truth (privileged information) and can calculate a safe, optimized, and feasible recovery or avoidance trajectory. τ pseudo (That is, the aforementioned target decision data) serves as high-quality action supervision. Finally, the OOD state S is rendered with high fidelity. pert Its corresponding τ pseudo Encapsulated into a scalable simulation training dataset D sim .

[0190] Sim-Real Co-training aims to utilize the generated high-quality simulation data. D sim Enhanced target end-to-end planning model π θ (That is, the robustness of the first model mentioned above). This training process affects the target model. π θ The architecture is versatile (it can be based on regression, diffusion models, or vocabulary scoring). During training iterations, a simulation-reality co-sampling strategy is employed to sample data from a hybrid dataset. D mix = D real ∪ D sim ( D real and D sim The model randomly selects batches of samples from the union of the sets (i.e., the set of elements in the model). π θ Receive the perceived state S from the mixed data, predict the planned trajectory (The predicted value of τ). The feedback mechanism employs imitation learning supervised loss: by minimizing the trajectory predicted by the model. Corresponding expert supervision trajectory τ exp (Right now D real Human trajectory in τ real or D sim The trajectory of pseudo-experts in τ pseudo The model parameters θ are updated using the distance between the two parameters. Through this co-training, the model... π θ I have learned D real The distribution of normal human driving behavior in the data, and through D sim The planner-style pseudo-expert provides safe and exploratory supervision, which significantly improves its robustness and generalization ability in sparse and safety-critical OOD scenarios.

[0191] To achieve the above embodiments, this application also proposes a vehicle control device.

[0192] Figure 3 This is a schematic diagram of the structure of a vehicle control device provided in an embodiment of this application.

[0193] like Figure 3As shown, the device may include: The decision module 310 is used to input the third driving scenario data collected during the vehicle's driving process into the first model to generate the vehicle's decision data; wherein, the first model is obtained by pre-training based on the first driving scenario data and the second driving scenario data, and the second driving scenario data is obtained by data expansion of the first driving scenario data; The control module 320 is used to control the running trajectory of the vehicle based on the decision data.

[0194] Optionally, the device further includes: An expansion module is used to expand the first driving scenario data to generate second driving scenario data under the target driving scenario; wherein, the target driving scenario is the training scenario; The decision generation module is used to generate corresponding target decision data based on the first driving scenario data and the second driving scenario data; the target decision data represents the actions performed by the vehicle in the scenario corresponding to the driving scenario data; The training module is used to train the first model based on the first driving scenario data, the second driving scenario data, and the target decision data.

[0195] Optionally, the expansion module includes: The model generation submodule is used to construct a corresponding first driving scenario model based on the first driving scenario data, wherein the first driving scenario model includes multiple scenario elements; The model adjustment submodule is used to adjust the scene elements in the first driving scene model to generate a second driving scene model; The scene data generation submodule is used to generate the second driving scene data based on the second driving scene model.

[0196] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of this embodiment, and will not be repeated here.

[0197] To implement the above embodiments, this application also proposes a vehicle, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method described in the foregoing method embodiments.

[0198] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing method embodiments.

[0199] To implement the above embodiments, this application also proposes a computer program product having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the foregoing method embodiments.

[0200] To implement the above embodiments, this application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method described in the foregoing method embodiments.

[0201] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example, the electronic device 800 may be a mobile phone, a server, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0202] Reference Figure 4 The electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input / output (I / O) interface 812, sensor component 814, and communication component 816.

[0203] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0204] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of such data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0205] Power component 806 provides power to various components of electronic device 800. Power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0206] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0207] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0208] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0209] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0210] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0211] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0212] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0213] To implement the above embodiments, this application also proposes a chip, including: the chip includes a processing circuit configured to perform the methods provided in the foregoing embodiments.

[0214] Figure 5 This is a schematic diagram of the structure of a chip according to an embodiment of this application. See also... Figure 5 The diagram shown is a schematic representation of the structure of chip 1100, but it is not limited to this.

[0215] Chip 1100 includes processing circuitry 1101, which is configured to perform any of the above methods.

[0216] In some embodiments, chip 1100 further includes one or more interface circuits 1102. Optionally, the interface circuit 1102 is connected to memory 1103, and the interface circuit 1102 can be used to receive signals from memory 1103 or other devices, and the interface circuit 1102 can be used to send signals to memory 1103 or other devices. For example, the interface circuit 1102 can read instructions stored in memory 1103 and send the instructions to processing circuit 1101.

[0217] In some embodiments, the interface circuit 1102 performs at least one of the communication steps such as sending and / or receiving in the above method, while the processing circuit 1101 performs other steps.

[0218] In some embodiments, the terms interface circuit, interface, transceiver pin, transceiver, etc., can be used interchangeably.

[0219] In some embodiments, chip 1100 further includes one or more memories 1103 for storing instructions. Optionally, all or part of the memories 1103 may be located outside of chip 1100.

[0220] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0221] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0222] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0223] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0224] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0225] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0226] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0227] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A vehicle control method, characterized in that, include: The third driving scenario data collected during vehicle operation is input into the first model to generate vehicle decision data; wherein, the first model is pre-trained based on the first driving scenario data and the second driving scenario data, and the second driving scenario data is obtained by data expansion of the first driving scenario data; The vehicle's trajectory is controlled based on the decision data.

2. The method according to claim 1, characterized in that, The training steps for the first model include: The first driving scenario data is augmented to generate second driving scenario data under the target driving scenario; wherein, the target driving scenario is the training scenario. Based on the first driving scenario data and the second driving scenario data, corresponding target decision data is generated; the target decision data represents the actions performed by the vehicle in the scenario corresponding to the driving scenario data. The first model is trained based on the first driving scenario data, the second driving scenario data, and the target decision data.

3. The method according to claim 2, characterized in that, The step of augmenting the first driving scenario data to generate second driving scenario data under the target driving scenario includes: A first driving scenario model is constructed based on the first driving scenario data, wherein the first driving scenario model includes multiple scenario elements; The scene elements in the first driving scene model are adjusted to generate a second driving scene model; The second driving scenario data is generated based on the second driving scenario model.

4. The method according to claim 3, characterized in that, The step of adjusting the scene elements in the first driving scene model to generate the second driving scene model includes: Based on the target driving scenario, determine the target scenario element to be adjusted and the corresponding adjustment parameters for the target scenario element; The corresponding target scene elements to be adjusted are adjusted according to the adjustment parameters to obtain the second driving scene model.

5. The method according to claim 3, characterized in that, The scene elements include a first vehicle element, and adjusting the scene elements in the first driving scene model to generate a second driving scene model further includes: Generate the driving trajectory of at least one of the first vehicle elements to obtain the second driving scene model; wherein, the first vehicle element is the scene element corresponding to the current vehicle.

6. The method according to claim 5, characterized in that, The scene elements also include other scene elements, and the method further includes: Based on the interaction relationship between other scene elements in the first driving scenario model and the driving trajectory of the first vehicle element, the state of the other scene elements is adjusted.

7. The method according to claim 6, characterized in that, The other scene elements include: a second vehicle element and a pedestrian element, and adjusting the state of the other scene elements includes at least one of the following: Adjust the driving trajectory of the second vehicle element based on the driving trajectory of the first vehicle element in the first driving scenario model. The movement trajectory of the pedestrian element is adjusted based on the driving trajectory of the first vehicle element in the first driving scenario model.

8. The method according to claim 5, characterized in that, Generating the driving trajectory of at least one of the first vehicle elements to obtain the second driving scene model includes: Sampling is performed on the ground plane in the first driving scenario to obtain at least one sampling location; Generate the driving trajectory from the position of the first vehicle element to each of the sampling positions to obtain the second driving scene model.

9. The method according to claim 3, characterized in that, The step of generating the second driving scenario data based on the second driving scenario model includes: The second driving scene data is rendered to obtain video data from at least one perspective, which is used as the second driving scene data.

10. The method according to claim 9, characterized in that, The rendering of the second driving scene data to obtain video data from at least one perspective includes: Determine the camera parameters corresponding to the preset viewpoint; Based on the camera parameters, the second driving scene data is transformed to obtain the video data corresponding to the viewpoint.

11. The method according to claim 9, characterized in that, The step of generating corresponding target decision data based on the first driving scenario data and the second driving scenario data includes: Based on the first driving scenario model corresponding to the first driving scenario data, or the second driving scenario model corresponding to the second driving scenario data, and in combination with preset constraint rules, at least one target trajectory data is generated. Based on the target trajectory data, action commands for the vehicle are determined to obtain the target decision data.

12. The method according to claim 11, characterized in that, The process of generating at least one target trajectory data by combining preset constraint rules includes at least one of the following: Based on each constraint rule, candidate trajectory data corresponding to that constraint rule is generated respectively; Candidate trajectory data that satisfy each of the aforementioned constraint rules are used as the target trajectory data.

13. The method according to claim 12, characterized in that, The step of generating candidate trajectory data corresponding to each constraint rule, respectively, includes at least one of the following: Based on preset traffic rule constraints, first candidate trajectory data for vehicles is generated; wherein, the traffic rule constraints are used to restrict the first vehicle element to comply with traffic rules; Based on preset collision safety constraints, second candidate trajectory data for the vehicle is generated; wherein, the collision safety constraints are used to limit the distance between the first vehicle element and other scene elements; Based on preset dynamic feasibility constraints, third candidate trajectory data for the vehicle is generated; wherein the dynamic feasibility constraints are based on limiting the driving trajectory of the first vehicle element to conform to vehicle performance. Based on preset comfort constraints, fourth candidate trajectory data for the vehicle is generated; wherein, the comfort constraints are used to restrict the driving trajectory of the first vehicle element to meet comfort requirements.

14. The method according to claim 2, characterized in that, The step of training the first model based on the first driving scenario data, the second driving scenario data, and the target decision data includes: The first driving scenario data and the second driving scenario data are input into the first model for inference to obtain training decision data; Calculate the loss function based on the training decision data and the target decision data; The first model is trained based on the loss function as the objective.

15. The method according to any one of claims 1-14, characterized in that, The step of controlling the vehicle's trajectory based on the decision data includes: Based on at least one control instruction from the decision data, the operating state of the vehicle's components is controlled so that the vehicle runs along the operating trajectory corresponding to the decision data.

16. A vehicle control device, characterized in that, include: The decision module is used to input the third driving scenario data collected during the vehicle's driving process into the first model to generate the vehicle's decision data; wherein, the first model is obtained by pre-training based on the first driving scenario data and the second driving scenario data, and the second driving scenario data is obtained by data expansion of the first driving scenario data; The control module is used to control the vehicle's operating trajectory based on the decision data.

17. The apparatus according to claim 16, characterized in that, The device further includes: An expansion module is used to expand the first driving scenario data to generate second driving scenario data under the target driving scenario; wherein, the target driving scenario is the training scenario; The decision generation module is used to generate corresponding target decision data based on the first driving scenario data and the second driving scenario data; the target decision data represents the actions performed by the vehicle in the scenario corresponding to the driving scenario data; The training module is used to train the first model based on the first driving scenario data, the second driving scenario data, and the target decision data.

18. The apparatus according to claim 17, characterized in that, The expansion module includes: The model generation submodule is used to construct a corresponding first driving scenario model based on the first driving scenario data, wherein the first driving scenario model includes multiple scenario elements; The model adjustment submodule is used to adjust the scene elements in the first driving scene model to generate a second driving scene model; The scene data generation submodule is used to generate the second driving scene data based on the second driving scene model.

19. A vehicle, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method as described in any one of the preceding claims 1-15.

20. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of the preceding claims 1-15.

21. A chip, characterized in that, The chip includes processing circuitry configured to perform the method described in any one of claims 1-15.

22. A computer program product, characterized in that, It includes a computer program, which, when executed by a processor, implements the method as described in any one of claims 1-15.

Images