Scenario data generation system, method, and program

Abstract

The present invention makes it possible to create desired scenario data in a simulation of autonomous driving, for example, with reduced labor.　This scenario data generation system includes a storage device and a processor. The storage device stores scenario data that includes information about operations of one or more objects at each sample time in a scene in which the one or more objects operate, and prompt data that instructs a generative artificial intelligence to perform a desired operation of each object. The processor starts a step-by-step simulation at each sample time based on the scenario data, causes, in each step, the generative artificial intelligence to generate a next state that is a state of the object at the next sample time due to the desired operation based on the state of the object in the scenario data, updates the state of the object at the next sample time in the scenario data to the next state, and outputs the scenario data when the simulation is completed.

Description

Scenario data generation system, method, and program

[0001] The present disclosure relates to a technology for generating scenario data.

[0002] In the development of an autonomous driving system or a driving support system, the operation of the system is verified by simulating the scenes around the vehicle (see Patent Document 1). Since it is necessary to verify various scenes encountered during driving of the vehicle, it is required to prepare a large amount of scenario data.

[0003] Japanese Unexamined Patent Application Publication No. 2019-43157

[0004] The work of creating scenario data for simulating vehicle driving is not easy, and a great deal of labor is required to create a large amount of scenario data. Also, in the development of an autonomous driving system or a driving support system, scenario data for scenes with few chances of encounter such as dangerous scenes is desired, but when creating scenario data based on images captured by a drive recorder or the like, images of the desired scenes are not necessarily obtained by the drive recorder or the like.

[0005] One object included in the present disclosure is to provide a technology that enables creation of desired scenario data with less labor.

[0006] A scenario data generation system according to one aspect of the present disclosure comprises a storage device and a processor, the storage device stores scenario data containing information about the operation of one or more objects at each sample time in a scene in which the objects are operating, and prompt data instructing a generative artificial intelligence to perform a desired operation of the objects, the processor starts a step-by-step simulation at each sample time based on the scenario data, instructs the generative artificial intelligence to generate a next state which is the state of the object at the next sample time based on the desired operation based on the state of the object in the scenario data, updates the state of the object at the next sample time in the scenario data to the next state, and outputs the scenario data when the simulation is completed.

[0007] A scenario data generation method according to one aspect of the present disclosure is a scenario data generation method using a device having a storage device and a processor, wherein the storage device stores scenario data including information on the operation of one or more objects at each sample time in a scene in which the objects are operating, and prompt data instructing a generative artificial intelligence to perform a desired operation of the objects, the processor starts a step-by-step simulation at each sample time based on the scenario data, in each step causes the generative artificial intelligence to generate a next state which is the state of the object at the next sample time based on the desired operation based on the state of the object in the scenario data, updates the state of the object at the next sample time in the scenario data to the next state, and outputs the scenario data when the simulation is completed.

[0008] A scenario data generation program according to one aspect of the present disclosure is a scenario data generation program that causes a device having a storage device and a processor to generate scenario data, wherein the storage device stores scenario data including information on the operation of one or more objects at each sample time in a scene in which the objects are operating, and prompt data instructing a generative artificial intelligence to perform a desired operation of the objects, and by executing the scenario data generation program, the processor starts a step-by-step simulation at each sample time based on the scenario data, in each step, causes the generative artificial intelligence to generate a next state which is the state of the object at the next sample time based on the desired operation based on the state of the object in the scenario data, updates the state of the object at the next sample time in the scenario data to the next state, and outputs the scenario data when the simulation is completed.

[0009] According to one aspect of this disclosure, it is possible to create desired scenario data with minimal effort.

[0010] This is a conceptual diagram showing an example of the configuration of a computer included in the scenario data generation system according to the embodiment of this disclosure. This is a conceptual diagram showing an example of the processing of the scenario data generation system according to the embodiment of this disclosure. This is a diagram illustrating the data structure of a configuration file according to the embodiment of this disclosure. This is a conceptual diagram showing a road defined by the configuration file according to the embodiment of this disclosure. This is a diagram illustrating the data structure of a configuration file according to the embodiment of this disclosure. This is a conceptual diagram showing a camera defined by the configuration file according to the embodiment of this disclosure. This is a diagram illustrating the image from a camera according to the embodiment of this disclosure. This is a conceptual diagram showing the road conditions in a scene of a driving simulation according to the embodiment of this disclosure. This is a diagram illustrating a vehicle driving scenario file according to the embodiment of this disclosure. This is a conceptual diagram showing the road conditions in a scene of a driving simulation according to the embodiment of this disclosure. This is a diagram illustrating a generated AI vehicle driving scenario file according to the embodiment of this disclosure. This is a diagram showing an example of the configuration of a vehicle control prompt according to the embodiment of this disclosure. This is a diagram showing an example of the configuration of a vehicle detection prompt according to the embodiment of this disclosure. This is a flowchart illustrating the execution process of a simulation including the generated AI according to the embodiment of this disclosure. This is a flowchart illustrating the execution process of a simulation including the generated AI according to the embodiment of this disclosure.

[0011] Embodiments of the present invention will be described below with reference to the drawings.

[0012] Figure 1 is a conceptual diagram showing an example of the configuration of a computer included in the scenario data generation system according to the embodiment of this disclosure.

[0013] The computer 10 includes an input unit 11, a display unit 12, a communication unit 13, a CPU (Central Processing Unit) 14, a memory 15, and a storage device 16.

[0014] The input unit 11 receives information input from an external device such as a keyboard or touch panel (not shown) connected to the computer 10. The display unit 12 displays the information processed by the CPU 14. The communication unit 13 communicates with external devices via a network 30, such as the Internet. The CPU 14 reads and executes programs or information stored in the memory 15 or storage device 16. Temporary data, for example, is stored in the memory 15.

[0015] The storage device 16 stores the vehicle driving scenario file 161, the configuration file 162, the generated AI vehicle driving scenario file 163, and the prompt file 164. The storage device 16 also stores the program data for the generated AI scenario extension system 5.

[0016] The storage device 16 stores scenario data containing information about the actions of one or more objects at each sample time in a scene in which they are operating, and prompt data that instructs the generative artificial intelligence to perform the desired actions of the objects. The scenario data is, for example, information showing the appearance of the moving object in a scene in which the object is moving. The initial scenario data may be created from video footage of a vehicle in motion taken by a dashcam. The scenario data is stored as a generative AI vehicle driving scenario file 163. Objects will be described later. The prompt data is stored as a prompt file 164.

[0017] The AI ​​scenario generation system 5 includes a simulator 51. The simulator 51 includes a simulation execution unit 511, an AI generation execution unit 512, and a communication processing unit 513. The CPU 14 reads and executes the software program of the AI ​​scenario generation system 5, thereby enabling the simulation execution unit 511, the AI ​​generation execution unit 512, and the communication processing unit 513 to function.

[0018] The simulation execution unit 511 has the function of executing a simulation.

[0019] The generative AI execution unit 512 has the function of executing or causing the generative artificial intelligence to execute processing based on a prompt. In this specification, generative artificial intelligence may be abbreviated as generative AI. The generative artificial intelligence referred to here is a Large Language Model (LLM), which is an artificial intelligence that generates information corresponding to a prompt by taking a prompt written in text or the like as input, such as chatGPT (registered trademark).

[0020] The communication processing unit 513 has the function of communicating with external devices.

[0021] The CPU 14 reads and executes the software program of the generation AI scenario extension system 5 stored in the storage device 16, and performs the following processing: The simulation execution unit 511 starts a step-by-step simulation for each sample time based on the scenario data. At each step for each sample time, the simulation execution unit 511 causes the generation artificial intelligence to generate the next state, which is the state of the object at the next sample time, based on a desired action based on the state of the object in the scenario data, updates the state of the object at the next sample time in the scenario data to the next state, and outputs the scenario data when the simulation is completed.

[0022] An object may be a virtual object in a virtual space. For example, an object may be a virtual moving object. While an object is typically a virtual vehicle, it may also be other objects such as a virtual building or a virtual pedestrian.

[0023] If the object is a moving object, the prompt data includes text instructing the generative artificial intelligence to perform a desired movement of the moving object. The scenario data includes information on the moving object's position, which represents the position of the moving object in the scene in which it moves, and the moving object's state, which represents the state of the moving object. In this case, at each step of the sample time, the CPU 14 causes the generative artificial intelligence to generate the next moving object state, which is the state of the moving object at the next sample time, based on the desired movement according to the moving object's position and state in the scenario data. At each step of the sample time, the CPU 14 calculates the next moving object position, which is the position of the moving object at the next sample time, based on the moving object's position and the next moving object state. At each step of the sample time, the CPU 14 updates the next moving object position and state in the scenario data to reflect the next moving object position and state at the next sample time.

[0024] If the moving object is a vehicle, more specifically a virtual vehicle virtually defined within the scenario data, the scenario data includes, for example, vehicle position information representing the location of the vehicle at each sample time in a scenario where one or more vehicles are traveling on a road, and vehicle state information representing the state of the vehicle. The prompt data includes text instructing the generative artificial intelligence to simulate a vehicle driver and give instructions for the desired driving.

[0025] The vehicle's moving state may include the vehicle's speed and yaw angle. The prompt data may include an instruction to output the angular velocity at which the vehicle's steering wheel is turned.

[0026] The desired driving action may be a lane change operation in which a vehicle traveling in the second lane adjacent to the first lane moves from the second lane to the first lane. In this case, the prompt data includes an instruction to output the angular velocity resulting from steering maneuvers to move from the second lane to the first lane, simulating the actions of the vehicle's driver.

[0027] The scenario data includes video footage from an onboard camera installed in the vehicle. If the vehicle is a virtually defined virtual vehicle, then the onboard camera is also a virtually defined camera. The desired driving action may be to change lanes from the second lane to the first lane when a vehicle traveling in the first lane behind appears in the video footage.

[0028] Figure 2 is a conceptual diagram showing an example of processing of a scenario data generation system according to the present disclosure.

[0029] Computer 10 reads the prompt file 164 (St101). Computer 10 reads the configuration file 162 (St102). Computer 10 reads the vehicle driving data from the vehicle driving scenario file 161 (St103). The simulation execution unit 511 of computer 10 executes the simulation (St104).

[0030] The generation AI processing server 20 generates instructions (St201). The generation AI processing server 20 sends the generated instructions to the computer 10 (St202). The computer 10 receives the instructions (St105).

[0031] The AI ​​generation execution unit 512 of the computer 10 executes a prompt based on the received instruction (St106). The computer 10 outputs the vehicle driving data as the AI ​​vehicle driving scenario file 163 (St107).

[0032] Figure 3 is a diagram illustrating the data structure of a configuration file according to the embodiment of this disclosure.

[0033] Figure 3 shows road data saved as a configuration file 162. The road data includes the road ID, the coordinates of the road center, the number of lanes, and the lane width as information items. The road data may include other information items. For a straight road, the coordinates of the road center may be expressed as a pair of three-dimensional coordinates of the start and end points that define the road, such as [0,0,0;200,0,0]. The information defining the road may be defined by other representation methods. Figure 4 is a conceptual diagram showing a road defined by the configuration file shown in Figure 3, according to an embodiment of this disclosure. Referring to Figure 4, a two-lane straight road with the road center being the straight line connecting [0,0,0] and [200,0,0] extends for 200 meters in the x direction.

[0034] Figure 5 is a diagram illustrating the data structure of a configuration file according to the embodiment of this disclosure.

[0035] Figure 5 shows the camera data saved as configuration file 162. The camera is an on-board camera installed in the vehicle. The vehicle referred to here may be, for example, another vehicle driving around the user's vehicle during a driving simulation. In this example, the intention is to control other objects, such as other vehicles around the user's vehicle, using generated AI.

[0036] The camera data includes the following information items: camera height, view position, Yaw, Pitch, and Roll. The view position is two-dimensional coordinate data indicating the relative position from the vehicle. Yaw, Pitch, and Roll are data indicating the camera's rotation angle. Figure 6 is a conceptual diagram showing a camera defined by the configuration file shown in Figure 5, according to an embodiment of this disclosure. The camera is installed at the relative position [0,0] with the vehicle's center as the origin. The camera is oriented in a direction rotated 140 degrees along the Yaw axis. In the figure, the x direction is Roll and the y direction is Pitch. If the vertical direction perpendicular to the x and y directions is the z direction, then the z direction is Yaw.

[0037] Figure 7 is a diagram illustrating the image from a camera according to an embodiment of the present disclosure.

[0038] Figure 7 shows the images captured by the in-vehicle cameras shown in Figures 5 and 6. Figure 7 also shows two examples: camera footage without a vehicle and camera footage with a vehicle. The vehicle shown in the camera footage is, for example, the vehicle used in a driving simulation.

[0039] Figure 8 is a conceptual diagram showing the road conditions in a scene of a driving simulation according to an embodiment of this disclosure.

[0040] In step St104 of FIG. 2, the simulation execution unit 511 of the computer 10 is executing a simulation. In this driving simulation, the vehicle and other vehicles are driving on the road with Road ID = 1 shown in FIGS. 3 and 4. The Vehicle ID of the host vehicle is 1. The Vehicle ID of the other vehicle is 2. Regarding the in-vehicle camera of the other vehicle, it is as described above with reference to FIGS. 5 to 7. The host vehicle is traveling at a speed of 60 km / h. The other vehicle is traveling at a speed of 40 km / h.

[0041] FIG. 9 is a diagram illustrating a vehicle driving scenario file according to an embodiment of the present disclosure. The vehicle driving scenario file 161 is a file of a scenario that serves as a source for generating a desired vehicle driving scenario.

[0042] FIG. 9 shows the data structure of the vehicle driving scenario file 161 shown in FIG. 1. The vehicle driving scenario file 161 has, as information items, Time (s), Vehicle ID, Position, Speed, and Yaw. Time (s) is the elapsed time from a certain reference time in the simulation. Vehicle ID is an ID for uniquely identifying a vehicle. Position is three-dimensional coordinate data indicating the relative position of the vehicle with respect to a predetermined reference position. Speed is the speed of the vehicle. Yaw indicates the direction of the vehicle on the Yaw axis, rather than the in-vehicle camera. Speed and Yaw indicate the state of the vehicle.

[0043] FIG. 10 is a conceptual diagram showing the road conditions in a scene of a driving simulation according to an embodiment of the present disclosure.

[0044] FIG. 11 is a diagram illustrating a generated AI vehicle driving scenario file according to an embodiment of the present disclosure. The generated AI vehicle driving scenario file 163 is a file of a vehicle driving scenario generated based on the vehicle driving scenario file 161.

[0045] As shown in Figure 2, the AI ​​generation execution unit 512 of the computer 10 executes the instructions received from the AI ​​generation processing server 20 in the driving simulation started in step St104 (St106). The computer 10 outputs the vehicle driving data when the instructions are executed as the AI ​​generation vehicle driving scenario file 163 (St107). Figure 11 shows the state of the AI ​​generation vehicle driving scenario file 163 output in this way.

[0046] As shown in Figure 10, the AI ​​generation server 20 generates a dangerous scene for the own vehicle. In this example, a dangerous scene is generated in which the own vehicle is traveling in the left lane at 60 km / h when another vehicle traveling in the right lane at 40 km / h suddenly changes lanes to the left lane. The sudden lane change by the other vehicle is represented as shown in Figure 11, when the other vehicle with VehicleID=2 changes Yaw from 0 (turns the steering wheel left or right) at a timing of Time(s) of 11.6 seconds.

[0047] Figure 12 shows an example of the configuration of a vehicle control prompt according to an embodiment of the present disclosure. The vehicle control prompt is recorded in a prompt file 164. The prompt data stored in the prompt file 164 includes text that instructs the generative artificial intelligence to simulate the driver of a vehicle and give instructions for a desired driving action. The generative AI execution unit 512 reads the prompt from the prompt file 164 (St 101) and executes the prompt based on the instructions received from the generative AI processing server 20 (St 106). Figure 12 shows an example of a prompt that instructs the generative artificial intelligence to simulate the driver of another vehicle and instruct another vehicle to change lanes. When the generative AI execution unit 512 executes the prompt, the yaw angular velocity is output. In step St 107, the computer 10 outputs a generative AI vehicle driving scenario file 163 (see Figure 11) in which the output yaw angular velocity is described in the information item Yaw.

[0048] FIG. 13 is a diagram showing a configuration example of a vehicle detection prompt according to an embodiment of the present disclosure. The vehicle detection prompt is recorded in the prompt file 164. The generation AI execution unit 512 reads the prompt from the prompt file 164 (St101) and executes the prompt based on an instruction received from the generation AI processing server 20 (St106). FIG. 13 shows an example of a prompt for detecting whether a vehicle is reflected in an image captured by an in-vehicle camera mounted on another vehicle. When the generation AI execution unit 512 executes the prompt, 0 is output when no vehicle is detected, and 1 is output when a vehicle is detected.

[0049] FIG. 14 is a flowchart illustrating an execution process of a simulation including a generation AI according to an embodiment of the present disclosure. FIG. 15 is a flowchart illustrating an execution process of a simulation including a generation AI according to an embodiment of the present disclosure.

[0050] FIGS. 14 and 15 show an example in which after detecting a vehicle with the generation AI, another vehicle is further caused to change lanes by the generation AI.

[0051] The computer 10 reads the vehicle control prompt and the vehicle detection prompt included in the prompt file 164 and transmits them to the generation AI processing server 20 (St301).

[0052] The computer 10 reads data from the setting file 162 and sets it in the simulator 51 (St302). The computer 10 reads data from the vehicle driving scenario file 161 and sets it in the simulator 51 (St303).

[0053] The computer 10 sets initial values for dist and yaw, and sets an initial value of 0 for detect (St304). The computer 10 starts the execution of the simulation (St305). dist is the position in the y direction from the center of the left lane. yaw is the current yaw angle. Detect is the detection result of the presence or absence of a vehicle in the image.

[0054] Computer 10 determines whether it is time to finish the simulation (St306). If it is time to finish (St306: Yes), the processes shown in Figures 14 and 15 are terminated. If it is not time to finish (St306: No), the process proceeds to step St307.

[0055] In step St307, the computer 10 determines whether the value of detect is 1 or not. If the value of detect is 1 (St307: Yes), the process proceeds to step St314. If the value of detect is not 1 (St307: No), the process proceeds to step St308.

[0056] In step St308, the computer 10 acquires the camera image as an image file. The computer 10 sends the acquired image file to the generation AI processing server 20 along with the string "Is there a car in the image?" (St309).

[0057] Computer 10 receives the data {"detect":"XXX"} from the generation AI processing server and sets the value of XXX to detect (St310).

[0058] Computer 10 calculates the position of VehicleID:2 (another vehicle) after one sample time from the vehicle speed and yaw, and sets it in the simulation (St311). Computer 10 runs the simulation for one sample time (St312). Computer 10 writes the data to the generated AI vehicle driving scenario file 163 (St313). Then the process returns to step St306.

[0059] In step St314, the computer 10 obtains the location of VehicleID:2 (another vehicle).

[0060] Computer 10 calculates the distance (YYY) from the position of VehicleID:2 (another vehicle) to the center of the left lane, and sets the calculated value to dist (St315).

[0061] Computer 10 sends "dist=YYY,yaw=ZZZ" as a string to the generation AI processing server 20 (St316).

[0062] Computer 10 receives the data {"yawrate":"WWW"} from the generation AI processing server and sets the value of WWW to yawrate (St317). Computer 10 adds the acquired value of yawrate to yaw (St318).

[0063] Computer 10 obtains the location of VehicleID:2 (another vehicle) (St319). Then the process transitions to step St311.

[0064] The embodiments described above are illustrative for explaining the present invention and are not intended to limit the scope of the invention to those embodiments only. Those skilled in the art can implement the present invention in various other forms without departing from its scope. Furthermore, these embodiments include the following; however, the contents of these embodiments are not limited to those described below.

[0065] (Item 1) As described above, the scenario data generation system comprises a memory device and a processor, the memory device stores scenario data containing information on the operation of one or more objects at each sample time in a scene in which the objects are operating, and prompt data instructing the generative artificial intelligence to perform a desired operation of the objects, the processor starts a step-by-step simulation at each sample time based on the scenario data, instructs the generative artificial intelligence to generate a next state which is the state of the object at the next sample time based on the desired operation according to the state of the object in the scenario data, updates the state of the object at the next sample time in the scenario data to the next state, and outputs the scenario data when the simulation is completed. This makes it possible to create desired scenario data with little effort. As the generative artificial intelligence calculates and updates the state of the object while the simulation progresses step by step, and calculates the next state based on the updated state, the desired scenario can be created by intuitive instructions even if the prompts do not necessarily have mathematically strict accuracy.

[0066] (Item 2) In the scenario data generation system described in Item 1, the object is a moving body, the prompt data includes text instructing the generative artificial intelligence to perform a desired movement of the moving body, the scenario data includes information on the moving body position representing the position of the moving body in a scene in which the moving body is moving, and information on the moving body state representing the state of the moving body, and the processor, in the step, causes the generative artificial intelligence to generate a next moving body state, which is the state of the moving body at the next sample time, based on the desired movement based on the moving body position and the moving body state in the scenario data, calculates the next moving body position, which is the position of the moving body at the next sample time, based on the moving body position and the next moving body state, and updates the moving body position and the moving body state at the next sample time in the scenario data to the next moving body position and the next moving body state. This makes it possible to create scenario data for a desired moving body with little effort.

[0067] (Item 3) In the scenario data generation system described in Item 2, the moving object is a vehicle, the scenario data includes vehicle position information representing the position of the vehicle at each sample time in a scene in which one or more vehicles are driving, and vehicle state information representing the state of the vehicle, and the prompt data includes text instructing the generation system artificial intelligence to simulate a vehicle driver and give instructions for the desired driving. This makes it possible to create scenario data for desired vehicle driving with little effort.

[0068] (Item 4) In the scenario data generation system described in Item 3, the vehicle is a virtual vehicle virtually defined within the scenario data. By having the virtual vehicle defined within the scenario data driven moment by moment through step-by-step decisions by the generating artificial intelligence, it is possible to easily create scenario data for driving that is natural and as intended, as if a human were actually driving the virtual vehicle.

[0069] (Item 5) In the scenario data generation system described in Item 3, the vehicle's moving state includes the vehicle's speed and its yaw angle, and the prompt data includes an instruction to output the angular velocity at which the vehicle's steering wheel is turned. This allows the generated artificial intelligence to control a vehicle virtually defined in the scenario data in a manner that the generated artificial intelligence steers, based on step-by-step decisions made by the generated artificial intelligence.

[0070] (Item 6) In the scenario data generation system described in Item 5, the desired driving is a lane change operation in which the vehicle traveling in the second lane adjacent to the first lane moves from the second lane to the first lane, and the prompt data includes an instruction to output the angular velocity resulting from steering operations to move from the second lane to the first lane, simulating the driver of the vehicle. This makes it possible to have the generating artificial intelligence perform lane change operations for a vehicle virtually defined in the scenario data, based on step-by-step decisions by the generating artificial intelligence.

[0071] (Item 7) In the scenario data generation system described in Item 6, the scenario data includes video footage from an on-board camera mounted on the vehicle, and the desired driving is a lane change from the second lane to the first lane when a vehicle traveling in the first lane behind is captured in the video footage. This makes it easy to create a scenario in which a vehicle changes lanes from an adjacent lane just before the intended lane.

[0072] (Item 8) A method for generating scenario data using a device having a memory device and a processor, wherein the memory device stores scenario data containing information on the operation of one or more objects at each sample time in a scene in which the objects are operating, and prompt data instructing a generative artificial intelligence to perform a desired operation of the objects; the processor starts a step-by-step simulation at each sample time based on the scenario data; in each step, the generative artificial intelligence generates a next state, which is the state of the object at the next sample time, based on the desired operation according to the state of the object in the scenario data; updates the state of the object at the next sample time in the scenario data to the next state; and outputs the scenario data when the simulation is completed. This makes it possible to create desired scenario data with little effort. As the generative artificial intelligence calculates and updates the state of the object while the simulation progresses step by step, and calculates the next state based on the updated state, the desired scenario can be created by intuitive instructions even if the prompts do not necessarily have mathematically strict accuracy.

[0073] (Item 9) A scenario data generation program that causes a device having a memory device and a processor to generate scenario data, wherein the memory device stores scenario data including information on the operation of one or more objects at each sample time in a scene in which the objects are operating, and prompt data that instructs a generative artificial intelligence to perform a desired operation of the objects, and by executing the scenario data generation program, the processor starts a step-by-step simulation at each sample time based on the scenario data, and at each step, the generative artificial intelligence generates a next state which is the state of the object at the next sample time based on the desired operation based on the state of the object in the scenario data, updates the state of the object at the next sample time in the scenario data to the next state, and outputs the scenario data when the simulation is completed. This makes it possible to create desired scenario data with little effort. As the generative artificial intelligence calculates and updates the state of the object while the simulation progresses step by step, and calculates the next state based on the updated state, the desired scenario can be created by intuitive instructions even if the prompts do not necessarily have mathematically strict accuracy.

[0074] 10...Computer, 11...Input unit, 12...Display unit, 13...Communication unit, 14...CPU, 15...Memory, 16...Storage device, 161...Vehicle driving scenario file, 162...Configuration file, 163...Generated AI vehicle driving scenario file, 164...Prompt file, 20...Generated AI processing server, 30...Network, 5...Generated AI scenario expansion system, 51...Simulator, 511...Simulation execution unit, 512...Generated AI execution unit, 513...Communication processing unit

Claims

1. A scenario data generation system comprising a memory device and a processor, wherein the memory device stores scenario data containing information about the operation of one or more objects at each sample time in a scene in which the objects are operating, and prompt data instructing a generative artificial intelligence to perform a desired operation of the objects; the processor starts a step-by-step simulation at each sample time based on the scenario data; in each step, causes the generative artificial intelligence to generate a next state, which is the state of the object at the next sample time, based on the desired operation of the object in the scenario data; updates the state of the object at the next sample time in the scenario data to the next state; and outputs the scenario data when the simulation is completed.

2. The scenario data generation system according to claim 1, wherein the object is a moving body, the prompt data includes text instructing a generative artificial intelligence to perform a desired movement of the moving body, the scenario data includes information on a moving body position representing the position of the moving body in a scene in which the moving body is moving, and a moving body state representing the state of the moving body, and the processor, in the step, causes the generative artificial intelligence to generate a next moving body state, which is the state of the moving body at the next sample time, based on the desired movement of the moving body position and the moving body state in the scenario data, calculates a next moving body position, which is the position of the moving body at the next sample time, based on the moving body position and the next moving body state, and updates the moving body position and moving body state at the next sample time in the scenario data to the next moving body position and the next moving body state.

3. The scenario data generation system according to claim 2, wherein the moving object is a vehicle, the scenario data includes vehicle position information representing the position of the vehicle at each sample time in a scene in which one or more vehicles are driving, and vehicle state information representing the state of the vehicle, and the prompt data includes text instructing the generative artificial intelligence to simulate a vehicle driver and give instructions for a desired driving.

4. The scenario data generation system according to claim 3, wherein the vehicle is a virtual vehicle virtually defined within the scenario data.

5. The scenario data generation system according to claim 3, wherein the moving state of the vehicle includes the speed of the vehicle and the yaw angle of the vehicle, and the prompt data includes an instruction to output the angular velocity of turning the steering wheel of the vehicle.

6. The scenario data generation system according to claim 5, wherein the desired driving is a lane change operation in which the vehicle traveling in a second lane adjacent to the first lane moves from the second lane to the first lane, and the prompt data includes an instruction to output the angular velocity due to steering operations that simulate the driver of the vehicle moving from the second lane to the first lane.

7. The scenario data generation system according to claim 6, wherein the scenario data includes video footage from an on-board camera mounted on the vehicle, and the desired driving is a driving maneuver in which the vehicle changes lanes from the second lane to the first lane when a vehicle traveling in the first lane behind is captured in the video footage.

8. A method for generating scenario data using a device having a memory device and a processor, wherein the memory device stores scenario data including information on the operation of one or more objects at each sample time in a scene in which the objects are operating, and prompt data instructing a generative artificial intelligence to perform a desired operation of the objects; the processor starts a step-by-step simulation at each sample time based on the scenario data; in each step, causes the generative artificial intelligence to generate a next state, which is the state of the object at the next sample time, based on the desired operation of the object in the scenario data; updates the state of the object at the next sample time in the scenario data to the next state; and outputs the scenario data when the simulation is completed.

9. A scenario data generation program that causes a device having a memory device and a processor to generate scenario data, wherein the memory device stores scenario data including information on the operation of one or more objects at each sample time in a scene in which the objects are operating, and prompt data that instructs a generative artificial intelligence to perform a desired operation of the objects, and by executing the scenario data generation program, the processor starts a step-by-step simulation at each sample time based on the scenario data, and in each step, causes the generative artificial intelligence to generate a next state which is the state of the object at the next sample time based on the desired operation based on the state of the object in the scenario data, updates the state of the object at the next sample time in the scenario data to the next state, and outputs the scenario data when the simulation is completed.

Images