Virtual scene information generation method of vehicle, vehicle and storage medium

Abstract

The invention discloses a virtual scene information generation method of a vehicle, the vehicle and a storage medium, and the method can comprise the steps: obtaining initial virtual scene information of the vehicle, the initial virtual scene information being used for rendering an initial virtual scene, and the initial virtual scene being used for simulating a real scene of the vehicle in a historical time period; generating a virtual driving track of a virtual vehicle in the initial virtual scene based on the first historical driving data of the vehicle and the second historical driving data of the target reference object; based on the virtual driving track, at least one first target virtual object is generated, and the first target virtual object is used for simulating an object colliding with the virtual vehicle; and updating the initial virtual scene information by using the first target virtual object to obtain target virtual scene information of the vehicle. The technical problem that the generation efficiency of the virtual scene of the vehicle is low is solved.

Description

Technical Field

[0001] This application relates to the field of vehicles, and more specifically, to a method for generating virtual scene information of a vehicle, a vehicle, and a storage medium. Background Technology

[0002] Currently, in the process of generating virtual vehicle scenarios, the simulation scenarios are often manually built on simulation software to test the vehicles, or test scenarios are obtained by recording real vehicle data, thereby conducting re-testing on the vehicles.

[0003] However, the above-mentioned methods for generating scenarios suffer from high costs and long cycles, resulting in low efficiency in generating virtual vehicle scenarios.

[0004] There is currently no effective solution to the technical problem of low efficiency in generating virtual scenes of the aforementioned vehicles. Summary of the Invention

[0005] This application provides a method for generating virtual scene information of a vehicle, a vehicle, and a storage medium, to at least solve the technical problem of low efficiency in generating virtual scenes of vehicles.

[0006] According to one aspect of the embodiments of this application, a method for generating virtual scene information of a vehicle is provided, comprising: acquiring initial virtual scene information of the vehicle, wherein the initial virtual scene information is used to render an initial virtual scene, and the initial virtual scene is used to simulate the real scene in which the vehicle is located during a historical period; generating a virtual driving trajectory of the virtual vehicle in the initial virtual scene based on first historical driving data of the vehicle and second historical driving data of a target reference object, wherein the target reference object is an object in the real scene other than the vehicle, the first historical driving data is used to represent the driving state of the vehicle during the historical period, the second historical driving data is used to represent the driving state of the target reference object during the historical period, and the virtual vehicle is used to simulate the vehicle; generating at least one first target virtual object based on the virtual driving trajectory, wherein the first target virtual object is used to simulate an object that collides with the virtual vehicle; and updating the initial virtual scene information using the first target virtual object to obtain target virtual scene information of the vehicle, wherein the target virtual scene information is used to render a target virtual scene, and the target virtual scene is used to simulate the real scene in which the vehicle is located during a future period.

[0007] Furthermore, based on the vehicle's first historical driving data and the target reference object's second historical driving data, a virtual driving trajectory of the virtual vehicle is generated in the initial virtual scene, including: extracting the vehicle's first real trajectory from the first historical driving data; and extracting the target reference object's second real trajectory from the second historical driving data; and generating a virtual driving trajectory in the initial virtual scene based on the first real trajectory and the second real trajectory.

[0008] Furthermore, based on the first real trajectory and the second real trajectory, a virtual driving trajectory is generated in the initial virtual scene, including: filtering the first real trajectory and filtering the second real trajectory; and generating a virtual driving trajectory in the initial virtual scene based on the filtered first real trajectory and the filtered second real trajectory.

[0009] Further, based on the filtered first real trajectory and the filtered second real trajectory, a virtual driving trajectory is generated in the initial virtual scene, including: inputting the filtered first real trajectory and the filtered second real trajectory into a first prediction model for trajectory prediction to obtain a first prediction result, wherein the first prediction model is constructed based on real trajectory samples and virtual trajectory samples, and the first prediction result includes the predicted trajectory of the virtual vehicle and the predicted trajectory of the second target virtual object, the second target virtual object being used to simulate a target reference object; and generating a virtual driving trajectory in the initial virtual scene according to the first prediction result.

[0010] Further, based on the virtual driving trajectory, generating at least one first target virtual object includes: inputting the virtual driving trajectory into a second prediction model for object prediction to obtain a second prediction result, wherein the second prediction model is constructed based on virtual driving trajectory samples and virtual object samples, and the second prediction result includes at least one initial virtual object, which is an object that has the possibility of colliding with the virtual vehicle; from the second prediction result, the initial virtual object with the highest probability is determined as the first target virtual object.

[0011] Furthermore, the initial virtual scene information is updated using the first target virtual object to obtain the target virtual scene information of the vehicle, including: arranging the first target virtual object, the virtual vehicle, and the second target virtual object in the initial virtual scene to obtain the target virtual scene.

[0012] Furthermore, the method also includes: acquiring different historical driving data samples of the vehicle; slicing the different historical driving data samples; smoothing the sliced ​​different historical driving data samples; performing scene recognition on the smoothed different historical driving data samples to obtain real scene information corresponding to the different historical driving data samples, wherein the real scene information is used to render real scene samples; constructing a real scene information set based on the real scene information corresponding to the different historical driving data samples; acquiring the initial virtual scene information of the vehicle, including: extracting at least one real scene information from the real scene information set; and using the real scene information to reproduce the corresponding real scene sample to obtain the initial virtual scene corresponding to the initial virtual scene information.

[0013] Furthermore, based on the real-scene information corresponding to different historical driving data samples, a real-scene information set is constructed, including: classifying the corresponding real-scene information; and constructing the classified real-scene information into a real-scene information set.

[0014] According to another aspect of the embodiments of this application, a virtual scene information generation device for a vehicle is also provided. The device may include: a first acquisition unit, configured to acquire initial virtual scene information of the vehicle, wherein the initial virtual scene information is used to render an initial virtual scene, and the initial virtual scene is used to simulate the real scene in which the vehicle is located during a historical period; a first generation unit, configured to generate a virtual driving trajectory of a virtual vehicle in the initial virtual scene based on first historical driving data of the vehicle and second historical driving data of a target reference object, wherein the target reference object is an object in the real scene other than the vehicle, the first historical driving data is used to represent the driving state of the vehicle during the historical period, the second historical driving data is used to represent the driving state of the target reference object during the historical period, and the virtual vehicle is used to simulate the vehicle; a second generation unit, configured to generate at least one first target virtual object based on the virtual driving trajectory, wherein the first target virtual object is used to simulate an object that collides with the virtual vehicle; and an update unit, configured to update the initial virtual scene information using the first target virtual object to obtain target virtual scene information of the vehicle, wherein the target virtual scene information is used to render a target virtual scene, and the target virtual scene is used to simulate the real scene in which the vehicle is located during a future period.

[0015] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0016] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0017] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0018] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0019] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0020] In this embodiment, when generating virtual scene information for a vehicle, initial virtual scene information for the vehicle can be obtained. Based on the vehicle's first historical driving data and the target reference object's second historical driving data, a virtual driving trajectory of the virtual vehicle can be generated in the initial virtual scene. Based on the generated virtual driving trajectory, at least one first target virtual object can be generated, and the initial virtual scene information can be updated using the generated first target virtual object to obtain target virtual scene information. Since this embodiment generates at least one first target virtual object based on the virtual driving trajectory generated in the initial virtual scene using the first and second historical driving data, and updates the initial virtual scene information using the first target virtual object to obtain target virtual scene information for rendering the target virtual scene, it achieves the goal of reducing the generation cycle of the vehicle's virtual scene, thereby solving the technical problem of low generation efficiency of the vehicle's virtual scene and achieving the technical effect of improving the generation efficiency of the vehicle's virtual scene. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1(a) is a schematic diagram of an application scenario of a method for generating virtual scene information of a vehicle according to an embodiment of the present invention;

[0023] Figure 1(b) is a flowchart of a method for generating virtual scene information of a vehicle according to an embodiment of the present invention;

[0024] Figure 2 This is a flowchart of a method for generating an AI adversarial scenario model according to an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of data interaction between a vehicle and a server according to an embodiment of the present invention;

[0026] Figure 4 This is a structural block diagram of a virtual scene information generation device for a vehicle according to an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] According to an embodiment of this application, an embodiment of a method for generating virtual scene information of a vehicle is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] As an optional implementation, the above-described method for generating virtual scene information for vehicles can be applied, but is not limited to, the application scenario shown in Figure 1(a). Figure 1(a) is a schematic diagram of an application scenario of a method for generating virtual scene information for vehicles according to an embodiment of the present invention. As shown in Figure 1(a), in the application scenario, the terminal device 10 can communicate with the server 13 via the network 11, but is not limited to. The server 13 can perform operations on the database, such as writing or reading data. The terminal device 10 may include, but is not limited to, a human-computer interaction screen, a processor, and a memory. The human-computer interaction screen can be used, but is not limited to, to display a virtual machine on the mobile terminal 10. The vehicle 12 can be used, but is not limited to, to respond to the above-described human-computer interaction operations, execute corresponding operations, or generate corresponding instructions and send the generated instructions to the server 13.

[0031] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here. Specifically, the method for generating virtual scene information for a vehicle in this application may include: step S102, obtaining initial virtual scene information for the vehicle; step S104, generating a virtual driving trajectory for the virtual vehicle in the initial virtual scene based on first historical driving data of the vehicle and second historical driving data of a target reference object; step S106, generating at least one first target virtual object based on the virtual driving trajectory, wherein the first target virtual object is used to simulate an object colliding with the virtual vehicle; and step S108, updating the initial virtual scene information using the first target virtual object to obtain target virtual scene information for the vehicle.

[0032] It should be noted that the relevant information (including but not limited to initial virtual scene information and target virtual scene information) and data (including but not limited to first historical driving data and second historical driving data) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0033] Figure 1(b) is a flowchart of a method for generating virtual scene information of a vehicle according to an embodiment of the present invention. As shown in Figure 1(b), the method may include the following steps:

[0034] Step S112: Obtain the initial virtual scene information of the vehicle, wherein the initial virtual scene information is used to render the initial virtual scene, and the initial virtual scene is used to simulate the real scene in which the vehicle is located during the historical period.

[0035] In the technical solution provided by step S112 of the present invention, the initial virtual scene information can be used to render an initial virtual scene, which can be used to simulate the real scene in which the vehicle is located during a historical period. For example, if the initial virtual scene is a highway following virtual scene, then the initial virtual scene can be used to simulate the real scene in which the vehicle is located during a highway following virtual scene during a historical period; if the initial virtual scene is a rural street straight-through virtual scene, then the initial virtual scene can be used to simulate the real scene in which the vehicle is located during a rural street straight-through virtual scene during a historical period. This is only an example and is not specifically limited.

[0036] In this embodiment, the vehicle can be a main vehicle in a real scene, and there is at least one background vehicle around the main vehicle.

[0037] In this embodiment, initial virtual scene information of the vehicle is obtained. Optionally, this embodiment can extract at least one real scene information from the real scene information set; based on the extracted real scene information, the initial virtual scene information can be determined.

[0038] Optionally, initial virtual scene information of the vehicle can be obtained. For example, initial virtual scene information can be determined from a set of initial virtual scene information, wherein the set of initial virtual scene information may include different initial virtual scene information.

[0039] Step S114: Based on the first historical driving data of the vehicle and the second historical driving data of the target reference object, a virtual driving trajectory of the virtual vehicle is generated in the initial virtual scene. The target reference object is an object in the real scene other than the vehicle. The first historical driving data is used to represent the driving state of the vehicle in the historical period. The second historical driving data is used to represent the driving state of the target reference object in the historical period. The virtual vehicle is used to simulate the vehicle.

[0040] In the technical solution provided by step S114 of the present invention, the target reference object can be an object in a real scene other than a vehicle. For example, the target reference object may include at least one of the following objects: background vehicles, background pedestrians, and background roadblocks.

[0041] In this embodiment, the aforementioned first historical driving data can be used to represent the driving status of a vehicle within a historical time period. This first historical driving data can be represented in the form of a structure. For example, the structure can be defined as follows: a first structure (e.g., represented by Vehicle_Record0), a second structure (e.g., represented by snapshot0), and a third structure (e.g., represented by vehicle0). The first structure can be used to record the identity information of the main vehicle, the second structure can be used to record the main vehicle appearing within the historical time period, and the third structure can be used to record the driving trajectory of the main vehicle. The driving trajectory can be displayed in the form of trajectory coordinate points.

[0042] In this embodiment, the aforementioned second historical driving data can be used to represent the driving status of the target reference object within a historical time period. This second historical driving data can be represented in the form of a structure. For example, the structure can be defined as follows: a fourth structure (e.g., represented by Vehicle_Record1), a fifth structure (e.g., represented by snapshot1), and a sixth structure (e.g., represented by vehicle1). The fourth structure can be used to record the identity information of background vehicles, the fifth structure can be used to record background vehicles, background pedestrians, and background roadblocks appearing within the historical time period, and the sixth structure can be used to record the driving trajectories of background vehicles and background pedestrians. These driving trajectories can be displayed in the form of trajectory coordinate points.

[0043] In this embodiment, the virtual vehicle can be used to simulate a vehicle, that is, it can be used to simulate the main vehicle in a real scene.

[0044] In this embodiment, after acquiring the initial virtual scene information of the vehicle, a virtual driving trajectory of the virtual vehicle is generated in the initial virtual scene based on the vehicle's first historical driving data and the target reference object's second historical driving data. Optionally, based on the acquired initial virtual scene information, this embodiment can determine the vehicle's first true trajectory based on the first historical driving data, and determine the target reference object's second true trajectory based on the second historical driving data; based on the first and second true trajectories, a virtual driving trajectory of the virtual vehicle can be generated in the initial virtual scene.

[0045] Optionally, based on the first and second real trajectories described above, a virtual driving trajectory of the virtual vehicle can be generated in the initial virtual scene. For example, by predicting the trajectory of the virtual vehicle according to the first real trajectory, a predicted trajectory of the virtual vehicle can be obtained; and by predicting the trajectory of the target reference object according to the second real trajectory, a predicted trajectory of the target reference object can be obtained; based on the predicted trajectory of the virtual vehicle and the predicted trajectory of the target reference object, a virtual driving trajectory of the virtual vehicle can be generated in the initial virtual scene.

[0046] Step S116: Based on the virtual driving trajectory, generate at least one first target virtual object, wherein the first target virtual object is used to simulate an object that collides with the virtual vehicle.

[0047] In the technical solution provided by step S116 of the present invention, the first target virtual object can be used to simulate an object that collides with a virtual vehicle. For example, the first target virtual object may include at least one of the following objects: a virtual collision vehicle that collides with a virtual vehicle, a virtual collision pedestrian that collides with a virtual vehicle, and a virtual collision road barrier that collides with a virtual vehicle, etc.

[0048] In this embodiment, after generating a virtual driving trajectory of the virtual vehicle in the initial virtual scene based on the vehicle's first historical driving data and the target reference object's second historical driving data, at least one first target virtual object is generated based on the virtual driving trajectory. Optionally, based on the generated virtual driving trajectory, this embodiment performs object prediction on the virtual vehicle according to the aforementioned virtual driving trajectory to obtain at least one first target virtual object, thereby achieving the purpose of generating objects that collide with the virtual vehicle.

[0049] Optionally, by performing object prediction on the virtual vehicle according to the aforementioned virtual driving trajectory, at least one first target virtual object can be obtained. For example, by performing object prediction on the virtual vehicle according to the aforementioned virtual driving trajectory, at least one initial virtual object can be obtained; from the aforementioned at least one initial virtual object, the initial virtual object with the highest probability of colliding with the virtual vehicle can be determined, and then the initial virtual object with the highest probability can be determined as the first target virtual object.

[0050] It should be noted that the above method for generating at least one first target virtual object is merely illustrative and is not intended to impose specific limitations. Any process or method that generates at least one first target virtual object based on the virtual driving trajectory of a virtual vehicle is within the protection scope of the embodiments of this application, and will not be described in detail here.

[0051] Step S118: Using the first target virtual object, update the initial virtual scene information to obtain the target virtual scene information of the vehicle. The target virtual scene information is used to render the target virtual scene, and the target virtual scene is used to simulate the real scene in which the vehicle will be located in the future.

[0052] In the technical solution provided by step S118 of the present invention, the aforementioned target virtual scene information can be used to render the target virtual scene, and the target virtual scene can be used to simulate the real scene in which the vehicle is located in a future time period. For example, if the target virtual scene is a virtual scene of a collision occurring on a highway lane change, then the target virtual scene can be used to simulate the virtual scene of a collision occurring on a highway lane change in a future time period; if the target virtual scene is a virtual scene of a collision occurring on a city street left turn, then the target virtual scene can be used to simulate the virtual scene of a collision occurring on a city street left turn in a future time period. This is only an example and is not specifically limited.

[0053] In this embodiment, after generating at least one first target virtual object based on the virtual driving trajectory, the initial virtual scene information is updated using the first target virtual object to obtain the vehicle's target virtual scene information. Optionally, in this embodiment, based on generating at least one first target virtual object, the initial virtual scene information is updated using the first target virtual object to obtain target virtual scene information; the target virtual scene can then be rendered using the target virtual scene information.

[0054] Optionally, the target virtual scene information can be obtained by updating the initial virtual scene information using the aforementioned first target virtual object. For example, the first target virtual object is arranged in the initial virtual scene to obtain the arranged initial virtual scene; the arranged initial virtual scene is then encapsulated into scene information, thereby completing the update of the initial virtual scene information. The scene information obtained by the encapsulation is the target virtual scene information.

[0055] In steps S112 to S118 of this application, when generating virtual scene information of the vehicle, initial virtual scene information of the vehicle can be obtained; based on the vehicle's first historical driving data and the target reference object's second historical driving data, a virtual driving trajectory of the virtual vehicle can be generated in the initial virtual scene; based on the generated virtual driving trajectory, at least one first target virtual object can be generated, and the initial virtual scene information can be updated using the generated first target virtual object to obtain target virtual scene information. Since this embodiment of the application, based on the first and second historical driving data, generates at least one first target virtual object in the initial virtual scene after generating a virtual driving trajectory, and uses the first target virtual object to update the initial virtual scene information to obtain target virtual scene information for rendering the target virtual scene, it achieves the goal of reducing the generation cycle of the vehicle's virtual scene, thereby solving the technical problem of low generation efficiency of the vehicle's virtual scene and achieving the technical effect of improving the generation efficiency of the vehicle's virtual scene.

[0056] The method described in this embodiment will be further described below.

[0057] As an optional embodiment, step S114, based on the vehicle's first historical driving data and the target reference object's second historical driving data, generates a virtual driving trajectory of the virtual vehicle in the initial virtual scene, including: extracting the vehicle's first real trajectory from the first historical driving data; and extracting the target reference object's second real trajectory from the second historical driving data; and generating a virtual driving trajectory in the initial virtual scene based on the first real trajectory and the second real trajectory.

[0058] In this embodiment, after acquiring the initial virtual scene information of the vehicle, a first true trajectory of the vehicle is extracted from the first historical driving data; and a second true trajectory of the target reference object is extracted from the second historical driving data. Optionally, based on the acquired initial virtual scene information, this embodiment can extract the first true trajectory of the vehicle from the first historical driving data according to the structure of the first historical driving data; and can extract the second true trajectory of the target reference object from the second historical driving data according to the structure of the second historical driving data.

[0059] In this embodiment, after extracting the first true trajectory of the vehicle and the second true trajectory of the target reference object, a virtual driving trajectory is generated in the initial virtual scene based on the first and second true trajectories. Optionally, based on the extracted first and second true trajectories, this embodiment performs trajectory prediction on the virtual vehicle according to the first true trajectory to obtain the predicted trajectory of the virtual vehicle; and performs trajectory prediction on the target reference object according to the second true trajectory to obtain the predicted trajectory of the target reference object; based on the predicted trajectory of the virtual vehicle and the predicted trajectory of the target reference object, a virtual driving trajectory of the virtual vehicle can be generated in the initial virtual scene, thereby achieving the goal of generating a virtual driving trajectory in the initial virtual scene, and thus realizing the technical effect of improving the effectiveness of the virtual driving trajectory.

[0060] The method for generating a virtual driving trajectory in an initial virtual scene based on the first and second real trajectories described in this embodiment will be further explained below.

[0061] As an optional implementation method, a virtual driving trajectory is generated in an initial virtual scene based on a first real trajectory and a second real trajectory, including: filtering the first real trajectory and filtering the second real trajectory; and generating a virtual driving trajectory in the initial virtual scene based on the filtered first real trajectory and the filtered second real trajectory.

[0062] In this embodiment, the filtering operation described above can be performed using Kalman filtering.

[0063] In this embodiment, after extracting the first true trajectory of the vehicle and the second true trajectory of the target reference object, the first true trajectory is filtered, and the second true trajectory is also filtered. Based on the filtered first and second true trajectories, a virtual driving trajectory is generated in the initial virtual scene. Optionally, this embodiment, based on the extracted first and second true trajectories, filters both the first and second true trajectories; according to the filtered first true trajectory, trajectory prediction is performed on the virtual vehicle to obtain a predicted trajectory; and according to the filtered second true trajectory, trajectory prediction is performed on the target reference object to obtain a predicted trajectory. Based on the predicted trajectories of the virtual vehicle and the target reference object, a virtual driving trajectory of the virtual vehicle can be generated in the initial virtual scene, thereby achieving the goal of generating a virtual driving trajectory in the initial virtual scene and thus realizing the technical effect of improving the effectiveness of the virtual driving trajectory.

[0064] The method for generating a virtual driving trajectory in an initial virtual scene based on the filtered first real trajectory and the filtered second real trajectory described in this embodiment will be further explained below.

[0065] As an optional implementation, a virtual driving trajectory is generated in an initial virtual scene based on the filtered first real trajectory and the filtered second real trajectory. This includes: inputting the filtered first real trajectory and the filtered second real trajectory into a first prediction model for trajectory prediction to obtain a first prediction result. The first prediction model is constructed based on real trajectory samples and virtual trajectory samples. The first prediction result includes the predicted trajectory of the virtual vehicle and the predicted trajectory of a second target virtual object, which is used to simulate a target reference object. The virtual driving trajectory is then generated in the initial virtual scene according to the first prediction result.

[0066] In this embodiment, the first prediction model can be constructed based on real trajectory samples and virtual trajectory samples. For example, the first prediction model can be a prediction model implemented through simulation environment tools.

[0067] In this embodiment, the first prediction result may include the predicted trajectory of the virtual vehicle and the predicted trajectory of the second target virtual object, which can be used to simulate the target reference object. For example, if the target reference object is a background vehicle, then the second target virtual object is a virtual background vehicle used to simulate the background vehicle; if the target reference object is a background pedestrian, then the second target virtual object is a virtual background pedestrian used to simulate the background pedestrian. This is only an example and is not specifically limited.

[0068] In this embodiment, after filtering the first real trajectory and the second real trajectory, the filtered first real trajectory and the filtered second real trajectory are input into the first prediction model for trajectory prediction to obtain a first prediction result. Optionally, based on obtaining the filtered first real trajectory and the filtered second real trajectory, this embodiment inputs the filtered first real trajectory and the filtered second real trajectory into the prediction layer of the first prediction model for trajectory prediction, and then outputs the first prediction result from the output layer of the first prediction model, thereby achieving the purpose of predicting the predicted trajectory of the virtual vehicle and the predicted trajectory of the second target virtual object.

[0069] In this embodiment, after inputting the filtered first true trajectory and the filtered second true trajectory into the first prediction model for trajectory prediction and obtaining the first prediction result, a virtual driving trajectory is generated in the initial virtual scene according to the first prediction result. Optionally, this embodiment can generate a virtual driving trajectory in the initial virtual scene based on the obtained first prediction result, thereby achieving the purpose of generating a virtual driving trajectory in the initial virtual scene and thus realizing the technical effect of improving the effectiveness of the virtual driving trajectory.

[0070] Optionally, based on the first prediction result described above, a virtual driving trajectory can be generated in the initial virtual scene. For example, by inputting the first prediction result into the trained driver model for trajectory constraint, a constrained first prediction result can be obtained; based on the constrained first prediction result, a virtual driving trajectory can be generated in the initial virtual scene.

[0071] For example, the reward function of the driver model described above can be expressed as follows (1):

[0072] Reward = R c +αR spd +βR onrd +γR simu (1)

[0073] Among them, R c It can be used to represent the collision penalty of the primary vehicle and is an adjustable parameter; R onrd It can be used to represent penalties for vehicles leaving the road, and it is an adjustable parameter; R simu This can be used to represent the simulation duration reward for a vehicle, recorded as steps / duration, where steps is the simulation run step size and duration is the scene execution duration, set to n; R spd It can be used to represent the vehicle's driving speed reward, as shown in the following formula (2):

[0074]

[0075] The method for generating at least one first target virtual object based on a virtual driving trajectory described in this embodiment will be further explained below.

[0076] As an optional embodiment, step S116, generating at least one first target virtual object based on the virtual driving trajectory, includes: inputting the virtual driving trajectory into a second prediction model for object prediction to obtain a second prediction result, wherein the second prediction model is constructed based on virtual driving trajectory samples and virtual object samples, and the second prediction result includes at least one initial virtual object, which is an object that has the possibility of colliding with the virtual vehicle; from the second prediction result, the initial virtual object with the highest probability is determined as the first target virtual object.

[0077] In this embodiment, the second prediction model can be constructed based on virtual driving trajectory samples and virtual object samples. For example, the second prediction model can be an artificial intelligence (AI) adversarial scenario model.

[0078] In this embodiment, the second prediction result may include at least one initial virtual object, which may be an object that has the potential to collide with the virtual vehicle.

[0079] In this embodiment, after generating a virtual driving trajectory of the virtual vehicle in the initial virtual scene based on the vehicle's first historical driving data and the target reference object's second historical driving data, the virtual driving trajectory is input into a second prediction model for object prediction to obtain a second prediction result. Optionally, in this embodiment, based on the generated virtual driving trajectory of the virtual vehicle, the virtual driving trajectory is input into the prediction layer of the second prediction model for object prediction, and then the second prediction result is output from the output layer of the second prediction model.

[0080] For example, if the second prediction model mentioned above is an AI adversarial scenario model, then the natural adversarial reward function of the second prediction model can be expressed as follows (3):

[0081]

[0082] Among them, R adv,t It can be used to represent the natural adversarial reward function; p AV,t0 and p agent,t0 These can be used to represent the position coordinates of the tested AV vehicle and the intelligent agent at time t0 during the initialization of the simulation environment; p AV,t and p agent,t These can be used to represent the position coordinates of the tested AV vehicle and the intelligent agent at time t during the initialization of the simulation environment; r c,t It can be used to represent the collision reward function; r d,t This can be used to represent the distance between an AV vehicle and an intelligent agent; the smaller the distance, the greater the danger, and the larger the reward. d,tIt can be calculated using the following formula (4):

[0083]

[0084] In the simulation environment, since the tested AV vehicle and the intelligent agent are two rigid bodies, their r d,t The distance is always greater than 0. Therefore, to increase the collision probability, r is designed... c,t As the collision reward function, if the agent successfully collides with the tested AV vehicle, a reward is given; if the agent collides with other vehicles, a penalty is imposed; the collision reward function can be expressed as follows (5):

[0085]

[0086] In this embodiment, after inputting the virtual driving trajectory into the second prediction model for object prediction and obtaining the second prediction result, the initial virtual object with the highest probability from the second prediction result is determined as the first target virtual object. Optionally, based on the second prediction result, this embodiment extracts at least one initial virtual object from the second prediction result; from the at least one initial virtual object, the initial virtual object with the highest probability of colliding with the virtual vehicle can be determined, and then the initial virtual object with the highest probability is determined as the first target virtual object, thereby achieving the purpose of generating the first target virtual object, and thus realizing the technical effect of improving the accuracy of the first target virtual object.

[0087] The method described below for updating the initial virtual scene information using a first target virtual object to obtain the target virtual scene information of the vehicle in this embodiment will be further explained.

[0088] As an optional embodiment, step S118 involves updating the initial virtual scene information using the first target virtual object to obtain the target virtual scene information of the vehicle, including: arranging the first target virtual object, the virtual vehicle, and the second target virtual object in the initial virtual scene to obtain the target virtual scene.

[0089] In this embodiment, after generating at least one first target virtual object based on a virtual driving trajectory, the first target virtual object, the virtual vehicle, and the second target virtual object are arranged in an initial virtual scene to obtain a target virtual scene. Optionally, this embodiment, based on generating at least one first target virtual object, arranges the first target virtual object, the virtual vehicle, and the second target virtual object in the initial virtual scene to obtain an arranged initial virtual scene; the arranged initial virtual scene is then encapsulated as scene information, thereby updating the initial virtual scene information. The encapsulated scene information is the target virtual scene information, thus achieving the goal of reducing the generation cycle of the vehicle's virtual scene and improving the technical effect of generating the vehicle's virtual scene.

[0090] The method for obtaining the initial virtual scene information of the vehicle described in this embodiment will be further explained below.

[0091] As an optional embodiment, the method further includes: acquiring different historical driving data samples of the vehicle; slicing the different historical driving data samples; smoothing the sliced ​​different historical driving data samples; performing scene recognition on the smoothed different historical driving data samples to obtain real scene information corresponding to the different historical driving data samples, wherein the real scene information is used to render real scene samples; constructing a real scene information set based on the real scene information corresponding to the different historical driving data samples; step S112, acquiring the initial virtual scene information of the vehicle, including: extracting at least one real scene information from the real scene information set; using the real scene information to reproduce the corresponding real scene sample to obtain the initial virtual scene corresponding to the initial virtual scene information.

[0092] In this embodiment, the above slicing operation can be achieved by the following operations: retaining trajectory segments with a duration of more than T seconds, trajectories with a travel distance of more than S meters, and removing trajectories with an instantaneous acceleration of more than a meters per second^2.

[0093] In this embodiment, the smoothing operation described above can be a data smoothing operation.

[0094] In this embodiment, the aforementioned real-world scene information can be used to render real-world scene samples.

[0095] In this embodiment, based on the acquisition of different historical driving data samples of the vehicle, the different historical driving data samples are sliced, the sliced ​​different historical driving data samples are smoothed, and scene recognition is performed on the smoothed different historical driving data samples to obtain the real scene information corresponding to the different historical driving data samples, and a real scene information set is constructed based on the real scene information corresponding to the different historical driving data samples.

[0096] In this embodiment, after constructing a real scene information set based on real scene information corresponding to different historical driving data samples, the real scene information is used to reproduce the corresponding real scene samples to obtain the initial virtual scene corresponding to the initial virtual scene information. Optionally, based on the constructed real scene information set, at least one real scene information can be extracted from the real scene information set; using the extracted real scene information to reproduce the corresponding real scene samples, the initial virtual scene corresponding to the initial virtual scene information can be obtained. This achieves the goal of obtaining the initial virtual scene information of the vehicle, thereby improving the technical effect of rendering the initial virtual scene.

[0097] The following section further explains the method for constructing a real-scene information set based on real-scene information corresponding to different historical driving data samples in this embodiment.

[0098] As an optional implementation method, a real scene information set is constructed based on the real scene information corresponding to different historical driving data samples, including: classifying the corresponding real scene information; and constructing the classified real scene information into a real scene information set.

[0099] In this embodiment, the above classification can also be clustering.

[0100] In this embodiment, the corresponding real-world scene information is classified; the classified real-world scene information is then used to construct a real-world scene information set. Optionally, based on the classification of the corresponding real-world scene information, this embodiment can further divide the real-world scene information corresponding to different historical driving data samples into their respective categories, and then construct a real-world scene information set from the real-world scene information divided into its respective categories.

[0101] In this embodiment of the invention, when generating virtual scene information for a vehicle, initial virtual scene information for the vehicle can be obtained. Based on the vehicle's first historical driving data and the target reference object's second historical driving data, a virtual driving trajectory of the virtual vehicle can be generated in the initial virtual scene. Based on the generated virtual driving trajectory, at least one first target virtual object can be generated, and the initial virtual scene information can be updated using the generated first target virtual object to obtain target virtual scene information. Since this embodiment of the application, based on the first and second historical driving data, generates at least one first target virtual object in the initial virtual scene by generating a virtual driving trajectory, and uses the first target virtual object to update the initial virtual scene information to obtain target virtual scene information for rendering the target virtual scene, it achieves the goal of reducing the generation cycle of the vehicle's virtual scene, thereby solving the technical problem of low generation efficiency of the vehicle's virtual scene and achieving the technical effect of improving the generation efficiency of the vehicle's virtual scene.

[0102] The technical solutions of the embodiments of the present invention will be illustrated below with reference to preferred embodiments.

[0103] Currently, in the process of generating virtual vehicle scenarios, the simulation scenarios are often manually built on simulation software to test the vehicles, or test scenarios are obtained by recording real vehicle data, thereby conducting re-testing on the vehicles.

[0104] However, the above-mentioned methods for generating scenarios suffer from high costs and long cycles, resulting in low efficiency in generating virtual vehicle scenarios.

[0105] However, this invention proposes a method for generating virtual scene information for vehicles. Based on first historical driving data and second historical driving data, a virtual driving trajectory is generated in an initial virtual scene. At least one first target virtual object can be generated, and the initial virtual scene information can be updated using the first target virtual object to obtain target virtual scene information for rendering the target virtual scene. This achieves the goal of reducing the generation cycle of the vehicle's virtual scene, thereby solving the technical problem of low generation efficiency of the vehicle's virtual scene and achieving the technical effect of improving the generation efficiency of the vehicle's virtual scene.

[0106] In this embodiment, by performing as follows Figure 2 The method for generating AI adversarial scenario models shown can be trained to obtain AI adversarial scenario models for use in generating AI adversarial scenarios. For example, Figure 2 This is a flowchart of a method for generating an AI adversarial scenario model according to an embodiment of the present invention. The method may include the following steps:

[0107] Step S201: Preprocess the traffic flow data and build a scenario library.

[0108] In the technical solution provided by step S201 of the present invention, the preprocessing of traffic flow data and the construction of the scene library can be accomplished by the following steps: step S2011, preprocessing traffic flow data; step S2012, performing cluster analysis on the scenes; and step S2013, constructing the scene library.

[0109] In this embodiment, a publicly available scene dataset is acquired. This dataset can include the driving conditions of vehicles and pedestrians over a certain period of time on various roads (e.g., roundabouts, intersections with traffic lights, lane merging / changing, etc.). Then, direction and steering information is constructed from the aforementioned scene dataset. All acquired driving trajectory data is preprocessed, for example, by slicing the data (e.g., retaining trajectory segments with a duration exceeding T seconds, trajectories with a distance exceeding S meters, and removing trajectories with instantaneous acceleration exceeding a meters per second², etc.). The sliced ​​trajectories are then smoothed. From the smoothed trajectories, scenarios of following other vehicles, changing lanes, going straight, turning left, and turning right are selected. Finally, clustering features for each typical scenario are extracted, and the optimal clustering result is found, thus forming a time-series sample set, which serves as the foundational data for training and validating the driver behavior prediction and decision-making model.

[0110] After preprocessing the traffic flow data and building the scenario library, step S202 is taken to reproduce the natural traffic flow environment.

[0111] In the technical solution provided by step S202 of the present invention, the natural traffic flow environment is reproduced to obtain a natural traffic flow simulation environment, which can be accomplished by the following steps: step S2021, reproducing the road network structure; and step S2022, reproducing the vehicle trajectory.

[0112] In this embodiment, a traffic flow simulation environment is modeled. For road network reconstruction, simulation environment tools (e.g., Highway-env simulation tool) are used to define and implement the road network segment lengths, number of lanes, and lane widths. For vehicle trajectory reconstruction, three structures are first defined: Vehicle_Record, which can be used to record the identification (ID), vehicle ID, and frame ID of any vehicle at any time; snapshot, which can be used to record all vehicles appearing at any time; and vehicle, which can be used to record all information of each vehicle, including: vehicle ID, a list of Vehicle_Records for the vehicle, and the vehicle's trajectory coordinates. Based on the defined structures, the trajectory of each vehicle is extracted. Since the original trajectory contains noise, it needs to be preprocessed using Kalman filtering.

[0113] After reproducing the natural traffic flow environment, proceed to step S203 to generate a traffic flow driver model.

[0114] In the technical solution provided by step S203 of the present invention, the generation of a traffic flow driver model can be accomplished through the following steps: step S2031, extracting a natural traffic flow scene; step S2032, designing an objective function; step S2033, training the driver model; and step S2034, generating the driver model.

[0115] In this embodiment, the Proximal Policy Optimization (PPO) algorithm is used to train the driver model. The PPO algorithm mainly consists of two parts: an actor network based on policy gradients and a criterion network based on value evaluation. The actor network generates continuous actions based on environmental state observations, while the criterion network evaluates the actions output by the actor network, influencing the actor's subsequent action choices. For example, using the PPO algorithm, a traffic flow simulation environment is run to obtain a model composed of (s... t ,a t ,r t+1 ,s t+1 Different trajectory data composed of S are used to form a trajectory database. t It can be used to represent the state quantity at the current moment, a t It can be used to represent the output motion space quantity, r t+1 It can be used to represent the immediate reward received by an agent after transitioning to the next state, S t+1It can be used to represent the state variables of the next time step; then, the PPO algorithm is used to perform mini-batch sampling from the trajectory database to optimize and train the Actor network and Critic network. When the vehicles in the simulation environment are initialized, a vehicle is randomly selected as the master vehicle. The master vehicle drives according to the actions output by the PPO policy network, while other background vehicles drive according to the pre-processed trajectory and do not respond to the behavior of the master vehicle, thereby ensuring the effectiveness of the agent's strategy to simulate human driving behavior.

[0116] Optionally, for the input and output of the PPO algorithm, the state can consist of 56-dimensional observation features. These features may include the vehicle's length and width, lateral displacement from the lane centerline, lateral and longitudinal velocities and yaw angle, and n*5 features representing the relative lateral distance, relative longitudinal distance, relative lateral velocity, relative longitudinal velocity, and relative yaw angle to the nearest n vehicles within a range of s meters. For example, if the number of vehicles closest to the main vehicle is less than n, it is padded with 0. The action output by the PPO policy network can be a two-dimensional vector including acceleration and yaw angle, where the acceleration ranges from (-am / s², am / s²) and the yaw angle ranges from (-θrad, θrad).

[0117] Optionally, the reward function in the driver model can be designed from four aspects: collision, driving speed, simulation duration, and whether the vehicle is on the road. If the main vehicle collides or goes off the road, a penalty is imposed; within a certain speed range, the higher the driving speed, the greater the reward; if the speed range is exceeded, no reward is given, thereby ensuring driving efficiency and preventing driving too slowly or speeding. It should be noted that the longer the simulation duration, the more likely that each step of the simulation is performed without violation (e.g., no collision or going off the road), and therefore the greater the reward will be. For example, the above reward function can be as shown in (1) above.

[0118] After generating the traffic flow driver model, proceed to step S204 to generate the AI ​​adversarial scenario model.

[0119] In the technical solution provided by step S204 of the present invention, the generation of the AI ​​adversarial scenario model can be accomplished through the following steps: step S2041, extracting the natural traffic flow scenario; step S2042, designing the objective function; step S2043, training the natural adversarial scenario model; and step S2044, generating the natural adversarial scenario model.

[0120] In this embodiment, the agent is trained using the PPO reinforcement learning algorithm and a natural adversarial reward function as shown in equation (3) above. A pre-trained driver model is used to supervise the agent's naturalness. The natural adversarial reward function is designed to enable the trained agent to generate disruptive actions (e.g., emergency braking and sudden lane changes) against the tested automated vehicle (AV) to create more dangerous and extreme scenarios.

[0121] If we only consider the adversarial nature of the agent design while ignoring its real-world probability, some unreasonable and invalid scenarios will be generated. Therefore, a pre-trained driver model is used to supervise the naturalness of the agent's behavior, ensuring that more effective and reasonable scenarios are generated. Based on the Actor-Critic PPO reinforcement learning algorithm, the Generative Adversarial Imitation Learning (GAIL) framework is used to model human driving behavior strategies to further supervise the natural training of the natural adversarial model.

[0122] Alternatively, the GAIL structure is similar to that of a Generative Adversarial Network (GAN). The GAIL structure can consist of a generator... and discriminator D ξ It consists of two parts. The generator can be used to model the driving behavior strategies of human experts, based on the input state s. t Output action a t The discriminator can be used to accept generator state-action pairs. and expert trajectory state-action pairs (s E,t ,a E,t The generator takes a state-action pair as input and outputs a real number between 0 and 1 to determine whether the input state-action pair comes from the generator policy or the expert policy. Finally, through continuous adversarial training between the generator and the discriminator, the data distribution generated by the generator policy will become closer and closer to the real expert data distribution, further realizing the modeling of human expert driving behavior policies.

[0123] Optionally, relative entropy and minimum mean squared error (MSE) can be used to measure naturalness. Relative entropy can be expressed as Kullback-Leibler divergence, or KL divergence for short. The KL divergence metric is explained as follows: If the adversarial traffic participant's vehicle policy and the driver model's GAIL policy output actions both belong to a multivariate Gaussian distribution, then KL divergence can be used to measure the difference between these two distributions. The naturalness activation function can be expressed as:

[0124]

[0125] in, It can be used to represent a generator for a driver model, based on state s t Output action distribution, π θ (·|s t KL divergence can be used to represent the action distribution output by an agent's PPO policy model; KL divergence is the default. The goal is to fit the distribution π to the actual distribution of human driving behavior strategies. θ (·|s t It is closer to the true distribution, thus ensuring the naturalness of the agent's output actions; M can be used to represent the KL divergence measure during the initial training of the agent. It is generally large and obtained from experience. For example, in the experiment, M = 25 can be taken. The value here is only for illustrative purposes and is not specifically limited.

[0126] Figure 3 This is a schematic diagram of data interaction between a vehicle and a server according to an embodiment of the present invention, such as... Figure 3 As shown, vehicle 300 can upload first historical driving data and second historical driving data to server 301. Server 301 can generate a virtual driving trajectory of the virtual vehicle in the initial virtual scene based on the first historical driving data of the vehicle and the second historical driving data of the target reference object; generate at least one first target virtual object based on the virtual driving trajectory; and update the initial virtual scene information using the first target virtual object to obtain the target virtual scene information of the vehicle, and send the target virtual scene information to vehicle 300.

[0127] In this embodiment, when generating virtual scene information for a vehicle, initial virtual scene information for the vehicle can be obtained. Based on the vehicle's first historical driving data and the target reference object's second historical driving data, a virtual driving trajectory of the virtual vehicle can be generated in the initial virtual scene. Based on the generated virtual driving trajectory, at least one first target virtual object can be generated, and the initial virtual scene information can be updated using the generated first target virtual object to obtain target virtual scene information. Since this embodiment generates at least one first target virtual object based on the virtual driving trajectory generated in the initial virtual scene using the first and second historical driving data, and updates the initial virtual scene information using the first target virtual object to obtain target virtual scene information for rendering the target virtual scene, it achieves the goal of reducing the generation cycle of the vehicle's virtual scene, thereby solving the technical problem of low generation efficiency of the vehicle's virtual scene and achieving the technical effect of improving the generation efficiency of the vehicle's virtual scene.

[0128] According to another aspect of the present invention, corresponding to the above-described method for generating virtual scene information of a vehicle, the present invention also provides a device for generating virtual scene information of a vehicle. Figure 4 This is a structural block diagram of a virtual scene information generation device for a vehicle according to an embodiment of the present invention, such as... Figure 4 As shown, the virtual scene information generation device 400 for the vehicle may include: a first acquisition unit 402, a first generation unit 404, a second generation unit 406, and an update unit 408.

[0129] The first acquisition unit 402 is used to acquire the initial virtual scene information of the vehicle, wherein the initial virtual scene information is used to render the initial virtual scene, and the initial virtual scene is used to simulate the real scene in which the vehicle is located during a historical period.

[0130] The first generation unit 404 is used to generate a virtual driving trajectory of a virtual vehicle in an initial virtual scene based on the vehicle's first historical driving data and the target reference object's second historical driving data. The target reference object is an object in the real scene other than the vehicle. The first historical driving data is used to represent the vehicle's driving state in a historical period, the second historical driving data is used to represent the target reference object's driving state in a historical period, and the virtual vehicle is used to simulate the vehicle.

[0131] The second generation unit 406 is used to generate at least one first target virtual object based on the virtual driving trajectory, wherein the first target virtual object is used to simulate an object that collides with the virtual vehicle.

[0132] The update unit 408 is used to update the initial virtual scene information using the first target virtual object to obtain the target virtual scene information of the vehicle. The target virtual scene information is used to render the target virtual scene, and the target virtual scene is used to simulate the real scene in which the vehicle will be located in the future.

[0133] Optionally, the first generation unit 404 may include: a first extraction module, used to extract the first real trajectory of the vehicle from the first historical driving data; and extract the second real trajectory of the target reference object from the second historical driving data; and a generation module, used to generate a virtual driving trajectory in the initial virtual scene based on the first real trajectory and the second real trajectory.

[0134] Optionally, the generation module may include: a filtering submodule for filtering the first real trajectory and the second real trajectory; and a generation submodule for generating a virtual driving trajectory in the initial virtual scene based on the filtered first real trajectory and the filtered second real trajectory.

[0135] Optionally, the generation submodule can generate a virtual driving trajectory in the initial virtual scene by performing the following steps: inputting the filtered first real trajectory and the filtered second real trajectory into a first prediction model for trajectory prediction to obtain a first prediction result, wherein the first prediction model is constructed based on real trajectory samples and virtual trajectory samples, and the first prediction result includes the predicted trajectory of the virtual vehicle and the predicted trajectory of the second target virtual object, which is used to simulate the target reference object; and generating a virtual driving trajectory in the initial virtual scene according to the first prediction result.

[0136] Optionally, the second generation unit 406 may include: a prediction module, used to input the virtual driving trajectory into the second prediction model to perform object prediction and obtain a second prediction result, wherein the second prediction model is constructed based on virtual driving trajectory samples and virtual object samples, and the second prediction result includes at least one initial virtual object, which is an object that has the possibility of colliding with the virtual vehicle; and a determination module, used to determine the initial virtual object with the highest probability from the second prediction result as the first target virtual object.

[0137] Optionally, the update unit 408 may include an arrangement module, used to arrange the first target virtual object, the virtual vehicle and the second target virtual object in the initial virtual scene to obtain the target virtual scene.

[0138] Optionally, the virtual scene information generation device 400 for the vehicle may include: a second acquisition unit for acquiring different historical driving data samples of the vehicle; a slicing unit for slicing the different historical driving data samples; a smoothing unit for smoothing the sliced ​​different historical driving data samples; a recognition unit for performing scene recognition on the smoothed different historical driving data samples to obtain real scene information corresponding to the different historical driving data samples, wherein the real scene information is used to render real scene samples; and a construction unit for constructing a real scene information set based on the real scene information corresponding to the different historical driving data samples. The first acquisition unit 402 may include: a second extraction module for extracting at least one real scene information from the real scene information set; and a reproduction module for reproducing the corresponding real scene sample using the real scene information to obtain the initial virtual scene corresponding to the initial virtual scene information.

[0139] Optionally, the construction unit may include: a classification module for classifying the corresponding real-world scene information; and a construction module for constructing the classified real-world scene information into a real-world scene information set.

[0140] In this embodiment, the vehicle virtual scene information generation device includes the following units: a first acquisition unit, used to acquire initial virtual scene information of the vehicle, wherein the initial virtual scene information is used to render an initial virtual scene, and the initial virtual scene is used to simulate the real scene in which the vehicle is located during a historical period; and a first generation unit, used to generate a virtual driving trajectory of the virtual vehicle in the initial virtual scene based on the vehicle's first historical driving data and the target reference object's second historical driving data, wherein the target reference object is an object in the real scene other than the vehicle, the first historical driving data is used to represent the vehicle's driving state during the historical period, and the second historical driving data is used to represent the target reference object's driving state during the historical period. The system comprises a virtual vehicle for simulating a vehicle; a second generation unit for generating at least one first target virtual object based on a virtual driving trajectory, wherein the first target virtual object is used to simulate an object colliding with the virtual vehicle; and an update unit for updating the initial virtual scene information using the first target virtual object to obtain target virtual scene information for the vehicle, wherein the target virtual scene information is used to render the target virtual scene, and the target virtual scene is used to simulate the real scene in which the vehicle will be located in a future time period. This achieves the goal of reducing the generation cycle of the vehicle's virtual scene, thereby solving the technical problem of low generation efficiency of the vehicle's virtual scene, and thus achieving the technical effect of improving the generation efficiency of the vehicle's virtual scene.

[0141] Embodiments of this application also provide a vehicle, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods described in various embodiments of this application when it runs.

[0142] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0143] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0144] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0145] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this application.

[0146] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0148] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0150] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0151] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method of generating virtual scene information of a vehicle, characterized by, include: Obtain initial virtual scene information of the vehicle, wherein the initial virtual scene information is used to render an initial virtual scene, and the initial virtual scene is used to simulate the real scene in which the vehicle is located during a historical period; Based on the first historical driving data of the vehicle and the second historical driving data of the target reference object, a virtual driving trajectory of the virtual vehicle is generated in the initial virtual scene. The target reference object is an object in the real scene other than the vehicle. The first historical driving data is used to represent the driving state of the vehicle in the historical period. The second historical driving data is used to represent the driving state of the target reference object in the historical period. The virtual vehicle is used to simulate the vehicle. Based on the virtual driving trajectory, at least one first target virtual object is generated, wherein the first target virtual object is used to simulate an object that collides with the virtual vehicle; Using the first target virtual object, the initial virtual scene information is updated to obtain the target virtual scene information of the vehicle, wherein the target virtual scene information is used to render the target virtual scene, and the target virtual scene is used to simulate the real scene in which the vehicle will be located in a future time period.

2. The method of claim 1, wherein, Based on the vehicle's first historical driving data and the target reference object's second historical driving data, a virtual driving trajectory of the virtual vehicle is generated in the initial virtual scene corresponding to the initial virtual scene information, including: The first true trajectory of the vehicle is extracted from the first historical driving data; and the second true trajectory of the target reference object is extracted from the second historical driving data. Based on the first real trajectory and the second real trajectory, the virtual driving trajectory is generated in the initial virtual scene.

3. The method of claim 2, wherein, Based on the first real trajectory and the second real trajectory, the virtual driving trajectory is generated in the initial virtual scene, including: Filter the first real trajectory and filter the second real trajectory; Based on the filtered first real trajectory and the filtered second real trajectory, the virtual driving trajectory is generated in the initial virtual scene.

4. The method of claim 3, wherein, Based on the filtered first real trajectory and the filtered second real trajectory, the virtual driving trajectory is generated in the initial virtual scene, including: The filtered first real trajectory and the filtered second real trajectory are input into the first prediction model for trajectory prediction to obtain the first prediction result. The first prediction model is constructed based on real trajectory samples and virtual trajectory samples. The first prediction result includes the predicted trajectory of the virtual vehicle and the predicted trajectory of the second target virtual object, which is used to simulate the target reference object. Based on the first prediction result, the virtual driving trajectory is generated in the initial virtual scene.

5. The method of claim 4, wherein, Based on the virtual driving trajectory, at least one first target virtual object is generated, including: inputting the virtual driving track into a second prediction model to perform object prediction and obtain a second prediction result, wherein the second prediction model is constructed based on virtual driving track samples and virtual object samples, the second prediction result comprises at least one initial virtual object, and the initial virtual object is an object that has a possibility of colliding with the virtual vehicle; determining the initial virtual object with the highest possibility from the second prediction result as the first target virtual object.

6. The method of claim 4, wherein, updating the initial virtual scene information by using the first target virtual object to obtain target virtual scene information of the vehicle, comprising: arranging the first target virtual object, the virtual vehicle and the second target virtual object in the initial virtual scene to obtain the target virtual scene.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: obtaining different historical driving data samples of the vehicle; slicing the different historical driving data samples; smoothing the sliced different historical driving data samples; performing scene recognition on the smoothed different historical driving data samples to obtain real scene information corresponding to the different historical driving data samples, wherein the real scene information is used to render real scene samples; constructing a real scene information set based on the real scene information corresponding to the different historical driving data samples; obtaining initial virtual scene information of the vehicle, comprising: extracting at least one real scene information from the real scene information set; and reproducing the corresponding real scene sample by using the real scene information to obtain the initial virtual scene information corresponding to the initial virtual scene.

8. The method of claim 7, wherein, constructing a real scene information set based on the real scene information corresponding to the different historical driving data samples, comprising: classifying the corresponding real scene information; constructing the classified real scene information into the real scene information set.

9. A vehicle characterized by comprising: comprising: a memory storing an executable program; a processor configured to run the program, wherein the program performs the virtual scene information generation method of the vehicle according to any one of claims 1 to 8 when running.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium comprises a stored executable program, wherein the executable program controls the device where the storage medium is located to perform the virtual scene information generation method of the vehicle according to any one of claims 1 to 8 when running.

Images