Method for simulating an autonomous vehicle in a virtual environment

Abstract

The present application relates to a virtual environment automatic driving vehicle simulation method, as a mobile body simulation method, comprising: a simulation preprocessing step, a first server generates environment information within a specified time, the environment information includes at least one of simulation object road information, surrounding vehicle information, surrounding pedestrian information, surrounding obstacle information, surrounding traffic signal information, surrounding signboard information, event information; and a simulation step, a second server calculates the sensor output data of the simulation object vehicle generated based on the driving of the simulation object vehicle using the environment information within the specified time.

Description

TECHNICAL FIELD

[0001] The present application relates to a simulation method for an autonomous vehicle in a virtual environment, and more particularly, to a simulation method for maintaining resource optimization through efficient data processing in a simulator for an autonomous vehicle, which is a virtual environment for verifying a control algorithm of an autonomous vehicle. BACKGROUND

[0002] An autonomous vehicle is a car that can travel without a driver directly driving the vehicle. Such an autonomous vehicle generally follows a travel path based on two-dimensional / three-dimensional map information, determines a surrounding object in real time during travel, and thus changes the travel path if a variable occurs.

[0003] Also, the autonomous vehicle is automatically driven according to data identified from a surrounding terrain and a travel control algorithm that processes the same, and in the case of the identification, a sensor such as a radar, a lidar, a global positioning system (GPS), a camera, etc. mounted on the vehicle identifies a travel environment such as a vehicle, a pedestrian, and an obstacle existing in a road. Meanwhile, the travel control algorithm calculates / infer a driving situation based on output data calculated in the identification step, and can generate a control signal for a vehicle component based on the calculated / inferred value.

[0004] In this case, as disclosed in Korean Patent Laid-Open Publication No. 10-2018-0086632 ("Behavior determination device and method of autonomous vehicle", published on August 1, 2018) or Korean Patent Laid-Open Publication No. 10-2018-0104947 ("Control system and method of autonomous vehicle", published on September 27, 2018), the autonomous vehicle is developed in a direction of more detailed recognition of more information and control thereof. For this reason, in the past, the development has been made in a direction of progress of the travel control algorithm so that an accurate judgment can be made in a plurality of variables.

[0005] Also, the developed travel control algorithm is accompanied by a process of verifying the same before being applied to an actual autonomous vehicle, which is disclosed in Korean Patent Registration Publication No. 10-1984762 ("Autonomous vehicle simulator applying a network platform", published on June 3, 2019) or Korean Patent Registration Publication No. 10-1996230 ("Test information providing method and apparatus for simulating an autonomous vehicle", published on July 4, 2019), etc.

[0006] In the simulator, virtual sensors simulating real sensors acquire data in a virtual environment formed by a physical engine based on real-time calculation of physical laws, and process the data according to a driving control algorithm of each autonomous vehicle. As described above, the physical engine of the simulator and the plurality of virtual sensors and the driving control algorithm need to input and output and calculate a large amount of data. However, in the case of processing data exceeding the performance of the processor, it results in exceeding the level of the entire system that can be processed, and there is a problem that resources are not allocated to a part and a delay phenomenon or an interruption phenomenon occurs in the entire virtual environment. SUMMARY

[0007] Technical problem to be solved

[0008] The present application is proposed to solve the problems as described above, and the object of the present application is to provide an autonomous vehicle simulation method in a virtual environment in which, in a simulator as a virtual environment, a case requiring more detailed driving control algorithm verification or requiring accurate data for a virtual sensor is separately managed, a data set is constructed, and more various verifications are performed on the constructed data set, thereby generating more accurate sensor output data while improving the interruption phenomenon of the simulator.

[0009] Technical solution

[0010] The mobile body simulation method of the present application for achieving the object as described above can include: step (a) in which a first server performs simulation preprocessing of generating environment information including at least one of simulation target road information, surrounding vehicle information, surrounding pedestrian information, surrounding obstacle information, surrounding traffic signal information, surrounding signboard information, and event information for a predetermined time; and step (b) in which a second server performs simulation of calculating sensor output data of a simulation target vehicle generated by driving of the simulation target vehicle using the environment information for the predetermined time.

[0011] Also, according to the present application, step (b) is performed after step (a) is performed, and step (a) can further include a step of storing the environment information, and in step (b), the second server can reduce a computational resource burden for simulation using the environment information stored in step (a).

[0012] Also, the first server and the second server can be constituted by the same server.

[0013] Also, step (a) and step (b) can be simultaneously performed, and the second server can reduce a computational resource burden of the second server for simulation by receiving the environment information used in step (b) from the first server in real time.

[0014] Also, the first server and the second server can be constituted by different servers.

[0015] Also, the environment information can include simulation time information to synchronize the simulation time of the step (a) and the step (b).

[0016] Also, the simulation object vehicle can be an autonomous vehicle.

[0017] Also, the sensor part can include at least one of a laser radar, a radar, a global positioning system, and a camera.

[0018] Also, the present application can further include a step of receiving simulation setting information including at least one of simulation object vehicle information, sensor part information of the simulation object vehicle, and vehicle travel control algorithm information from a user before or after the step (a).

[0019] Also, the simulation object vehicle information can include at least one of a vehicle type, a shape, a size, a wheel base, a height, and a center of gravity.

[0020] Also, the sensor part information of the simulation object vehicle can include at least one of a number, a type, a specification, and an installation position of a sensor included in the sensor part.

[0021] Also, the vehicle travel control algorithm information can be an algorithm for calculating a travel instruction of a vehicle based on sensor output data of the simulation object vehicle.

[0022] Also, the travel instruction of the vehicle can include at least one of acceleration, deceleration, stop, a travel direction, a turn signal, and vehicle lighting of the vehicle.

[0023] Also, in the step (b), the travel of the simulation object vehicle can be controlled based on the calculated sensor output data of the vehicle.

[0024] Also, the present application can be constituted by a mobile body simulation system including a first server and a second server for executing the mobile body simulation method as described above.

[0025] Also, the present application can be constituted by a recording medium in which an instruction for executing the mobile body simulation method as described above is stored.

[0026] Advantages

[0027] The autonomous vehicle simulation method in a virtual environment according to the structure of the present application as described above has the advantage that the first server and the second server are mutually divided to calculate and process, thereby effectively solving the interrupt phenomenon of the simulator due to the processing of a plurality of calculation data.

[0028] Also, in the present application, information in a prescribed time in which more accurate sensor output data is required among the entire operation information is divided into data sets to generate environment information, thereby having an effect of suppressing occurrence of errors in output data caused by interruption or delay of the simulator.

[0029] Also, in the present application, the simulator is continuously operated in the first server, and in the second server, a plurality of information in a prescribed time can be received to reproduce only a portion requiring verification, thereby providing an environment in which a plurality of visitors can access the first server to receive required environment information and verify their own algorithms. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A diagram of a system for performing an autonomous vehicle simulation method of the present application.

[0031] Figure 2 A system configuration diagram of an embodiment of an autonomous vehicle simulation method.

[0032] Figure 3 And Figure 4 A control flow diagram of an embodiment of an autonomous vehicle simulation method.

[0033] Figure 5 A system configuration diagram of another embodiment of an autonomous vehicle simulation method.

[0034] Figure 6 And Figure 7 A control flow diagram of another embodiment of an autonomous vehicle simulation method. DETAILED DESCRIPTION

[0035] Hereinafter, an autonomous vehicle simulation method in a virtual environment according to various embodiments of the present application will be described in detail with reference to the accompanying drawings. The accompanying drawings are provided to give a general understanding of the present application and thus, the present application is not limited thereto. It should be understood that the present application can be carried out in other ways than those specifically disclosed herein without departing from the spirit and essential characteristics of the present application. In order to clearly describe the present application, the accompanying drawings will be described below.

[0036] In this case, unless otherwise defined, technical and scientific terms used herein have the same meaning as those commonly understood by one of ordinary skill in the art to which the present application pertains. It will be further understood that the terms of a given drawing, are used for the purpose of illustrative convenience only and are not intended to limit the present application.

[0037] Figure 1 Referring to FIG. 1, a system for performing an autonomous vehicle simulation method of the present application, Figure 1 A diagram of a system is shown. Referring to Figure 1The autonomous vehicle simulation system of the present application can communicate data with a plurality of external devices, and can include a first server 100 and a second server 200 capable of verifying an algorithm for an autonomous vehicle due to the built-in physical engine-based simulator. In this case, data communication can transmit or receive data with other servers or a plurality of devices through wired connection with a plurality of peripheral devices, or can also transmit or receive data with a remote server or a plurality of devices through wireless communication.

[0038] Also, data communication can be performed between the first server 100 and the second server 200, each of which includes an internal memory device to store a plurality of information for operating the simulator, and can also be connected to a separate memory device 10 disposed outside to input or output data.

[0039] The first server 100 and the second server 200 described above can verify the control algorithm of the autonomous vehicle. Also, data of a topographic feature, a road type, a pedestrian, an autonomous vehicle, weather, etc. can be embodied on a virtual environment, and can also be provided in the form of event information data that generates a rockfall or a traffic accident, etc. At the same time, a plurality of vehicle data can be implemented in the virtual environment, a part of the vehicles can be provided in such a manner that a travel path is changed according to a peripheral environment according to an input travel algorithm, and the other vehicles can also be configured in such a manner that a travel algorithm is verified according to a predetermined travel path. Also, the first server 100 and the second server 200 described above can further include an output device connected to the simulator, in which a plurality of data as described above can be visualized to be provided so that a user can confirm through an image.

[0040] The second server 200 described above can acquire sensor output data about a peripheral environment centered on a selected object autonomous vehicle or a vehicle configuration in a virtual environment based on a sensor simulated in the virtual environment. Among them, the sensor can include at least one of a laser radar, a radar, a global positioning system, a camera of the autonomous vehicle simulated in the virtual environment. Also, the sensor simulated in the virtual environment can generate measurement data through a processor calculation of other visualized data associated with the selected autonomous vehicle. This will be described in more detail through various embodiments of the present application described later.

[0041] Embodiment 1

[0042] Figures 2 to 4 Referring to an embodiment of the autonomous vehicle simulation method of the present application, Figure 2 showing a system structure diagram, Figure 3 and Figure 4 showing a control flowchart of the autonomous vehicle simulation method, respectively.

[0043] First, referring to Figure 2The first server and the second server of the present application can be constituted by the same server, and in a corresponding embodiment, the same first server and second server are collectively referred to as a server. In this case, the server can include a memory device, a simulator including a processor that receives data from the memory device to perform control / computation, and an input / output device that receives data from the outside or outputs data to the outside.

[0044] The server provides the processor with environment information, a physics engine, a plurality of information of virtual sensors that simulate actual sensors (hereinafter, referred to as "sensing part"), a control algorithm that controls driving (hereinafter, referred to as "driving algorithm") from the memory device, performs control / computation based on the information provided in the processor, visualizes and displays the output value on the output device, and stores the output value again in the memory device. Among them, the plurality of information provided from the memory device is stored separately in a plurality of disks, not in one disk, and a part thereof can also be received from the outside.

[0045] The environment information stored in the memory device can be divided into first environment information and second environment information, and the second environment information can be a data set composed of a part of the total data of the first environment information. Among them, the first environment information and the second environment information can include at least one of simulated object road information, surrounding vehicle information, surrounding pedestrian information, surrounding obstacle information, surrounding traffic signal information, surrounding signboard information, and event information. Also, the second environment information can include at least one of simulated object road information, surrounding vehicle information, surrounding pedestrian information, surrounding obstacle information, surrounding traffic signal information, and surrounding signboard information, which are divided according to a reference input to the first environment information, and the reference input here can be a predetermined time reference. In this case, with respect to the predetermined time reference, in the case where the simulator is driven to output the computed information by the first environment information, the driving time of the simulator can become a reference, and can also be determined as a region between a predetermined time (T1-T2). At the same time, the simulation object can be a vehicle for verifying the driving algorithm, and the object's surroundings can be a terrain and a facility configured within a predetermined range on a virtual field with the corresponding vehicle as a reference.

[0046] Next, referring to Figure 3The automatic driving vehicle simulation method in the case where the environment data is stored in the internal memory device of the server will be explained sequentially. First, the simulator of the server can transmit simulation information based on the first environment information to the output device, whereby visualization can be performed through an image. Also, if the communication device of the above server receives an environment information registration request from the user terminal, a second environment information generation request message can be transmitted to the memory device of the server. In this case, the second environment information can be divided from the first environment information and constituted by a data set, and the server of the present application can manage it through a processor or through a separate database management system (DBMS) or the like. At this time, the above environment information registration request can be information about data related to an image between a simulation image reference output in real time and a predetermined time (T1~T2).

[0047] If the second environment information is generated in the memory device of the server, the simulator driven by the first environment information is interrupted, and the simulator can be driven based on the second environment information. Also, the server provides the sensor part data stored in the memory device to the above simulator, and the simulator can calculate sensor data as sensor part output data through the second environment information and the sensor part data. Also, the above sensor data can be stored again in the above memory device and utilized. In this case, after or before the above steps, sensor part parameter setting information can also be input from the user terminal, and the sensor part parameter setting information can be which sensor among all sensor parts is reflected in the control signal of the simulator. For example, in the case where the total amount of the stored sensor parts is 6, it is possible to control in a manner that is set to reflect only 4 sensor parts in the simulator or set to reflect all sensor parts, etc. Also, in the present application, the second environment information and the sensor part information can also be input from the above memory device to the above simulator together. At this time, the sensor part information can include at least one of the number, kind, specification, and installation position in the vehicle of the sensor, and each sensor can have sensor part data of a calculation algorithm for converting data into a unique output value.

[0048] Next, the present application can apply the driving algorithm transmitted from the user terminal or the driving algorithm stored on the memory device to the above simulator that calculates the sensor data to perform verification. At this time, the simulation information of the received algorithm can be displayed in real time through the output device.

[0049] Reference Figure 4An autonomous vehicle simulation method is described in the case of storing second environment information on an external memory device. The mobile body simulation method of the present invention is roughly composed of a preprocessing step and a simulation step. Also, the preprocessing step can divide and generate second environment information from the first environment information of the memory device within a predetermined time based on a signal input through an input device on the server or a signal transmitted from the user terminal, and the second environment information includes at least one of simulation target road information, surrounding vehicle information, surrounding pedestrian information, surrounding obstacle information, surrounding traffic signal information, surrounding signboard information, and event information. Also, the second environment information can be transmitted to the external memory device through a communication device or an output device to be stored in the external memory device.

[0050] When the second environment information is stored as a data set on the external memory device, the server can receive the environment data from the external memory device through an input device or a communication device. The second environment information thus divided can also be provided to multiple servers through the external memory device.

[0051] When simulated in real time in the server, the environment data in the case of not using the autonomous vehicle in a normal manner or requiring more detailed algorithm verification in the real-time output image can be classified into a data set in advance. Thus, the case requiring detailed verification is repeated in the server, and multiple driving algorithms can be input and verified. At the same time, the present invention can generate more detailed data sets such as third environment information, Nth environment information, etc. by including the second environment information divided from the first environment information.

[0052] Embodiment 2

[0053] Figures 5 to 7 Another embodiment related to the autonomous vehicle simulation method of the present invention, Figure 5 shows a system structure diagram, Figure 6 and Figure 7 respectively shows a control flowchart of the autonomous vehicle simulation method.

[0054] First, referring to Figure 6The present application can be composed of a first server including a memory device including first environment information, a simulator, an input / output device, and a communication device, and a second server including a memory device including second environment information, a simulator, an input / output device, and a communication device. The second environment information of the second server can be a part of the first environment information divided by time from the first environment information of the first server. Also, the memory devices of the first and second servers can further include a physical engine for driving the simulator based on physical laws, and the second server can further include a sensing part as a virtual sensor simulating an actual sensor and a driving algorithm for calculating a driving instruction of a vehicle based on sensor data.

[0055] The first server can convert simulation information according to the first environment information into image data in real time and display it through the output device or the communication device. Also, in the second server, the simulator can be driven based on the second environment information in which only a part having a high degree of importance is divided from all data. Also, the second server can convert simulation information according to the second environment information into image data and display it through the output device or the communication device of the corresponding server.

[0056] Also, the first and second environment information can include vehicle configuration, terrain and object, event information, pedestrian information, road information, etc. Also, the second environment information can include at least one of selected simulation object reference road information, surrounding vehicle information, surrounding pedestrian information, surrounding obstacle information, surrounding traffic signal information, surrounding signboard information, and event information. The output of the second environment information can be divided into a data set by, for example, a case where there is a problem in the driving algorithm of the vehicle driving while a user confirms the image output from the first server in real time, or a case where more detailed verification is required for an event such as a pedestrian walking on a road or a sudden rockfall. In this case, the divided reference can be divided based on the time displayed by the simulator driving time of the first server or RTC, etc.

[0057] Hereinafter, the control between the respective structures will be described in more detail. Figure 6 The first server can provide the first environment information stored in the memory device to the first simulator as a processor, and the first simulator can transmit simulation information calculated in real time to the output device. In this case, the simulation information transmitted from the first simulator can be information converted into image data, and can be transmitted to other servers including the second server through the communication device.

[0058] Thereafter, the first server can receive an environment information registration request message that is a generation request message for second environment information including at least one of simulated object road information, surrounding vehicle information, surrounding pedestrian information, surrounding obstacle information, surrounding traffic signal information, surrounding sign information, and event information within a predetermined time from an input device, another server including a second server, or a user terminal. The first server can separately manage fixed data that does not change during simulation, such as topographical features or road information, and variable data that can change during simulation, such as vehicle configurations or pedestrians. In this case, the second environment information can include fixed data within a predetermined range of the selected autonomous vehicle and variable data within a set time based on the autonomous vehicle in the user's confirmation in the simulation image of the first server. As described above, the second environment information divided from the first environment information can be transmitted to the communication device of the second server through the communication device of the first server. Also, the second environment information received from the communication device of the second server can be stored in the memory device or provided to the second simulator. The simulation preprocessing step of storing the divided second environment information on the first server or transmitting and storing the second environment information to the second server can be performed through the above-described process.

[0059] Also, the sensor information including sensor data and the second environment information stored in the memory device of the second server can be input to the second simulator. Also, the second simulator can perform simulation based on the above-described information and output the simulation as an image. The present application can receive sensor parameter setting information from the user terminal, and the sensor parameter setting information can be a message regarding a request to select a sensor to which the second simulator is applied among a plurality of sensors. Also, the simulation step of performing simulation based on the driving of the simulated vehicle through the second environment information and the sensor data can be performed through the above-described process. Also, the sensor information includes sensor data that is virtual sensor data simulating an actual sensor, and can further include at least one of the number, kind, specification, and installation position in the vehicle of the sensor included in the sensor.

[0060] The second simulator of the second server can simultaneously simulate the received second environment information and the sensor part data, and sensor data outputted as a result of the simulation can be stored in the memory device of the second server. Also, in the present application, a driving algorithm provided from the user terminal or stored in the second server is inputted to the second simulator, and the driving algorithm is verified by inputting the sensor data to the driving algorithm. Thus, there are advantages in that a user can repeatedly verify his or her algorithm in a relatively short setting time without using a large amount of data, and more efficient verification can be performed by inputting various driving algorithms. At the same time, a plurality of users can access the first server to receive only the environment information required by each user, and can verify their respective algorithms.

[0061] Thereafter, the second server can simulate the second environment information and the sensor part data, and the sensor data outputted as a result of the simulation can be stored in the memory device of the second server. Figure 7 In the present application, the sensor part can be composed of a plurality of sensor parts, and each of the plurality of sensor part data can be provided in a manner of being sequentially calculated in the second simulator. In more detail, the second server can change the number and kind of the sensor parts, etc. based on the same second environment information from the user terminal, and thus the second simulator can be driven a plurality of times. Also, the first sensor data, the second sensor data, etc. as a result of each simulation can be stored in the memory device and can be synchronized with each other. Also, the second server inputs the synchronized sensor data and the second environment information to the second simulator, and applies the driving algorithm inputted from the user terminal or stored in the second server to the simulator to verify the algorithm.

[0062] Also, the present application can include a step of receiving simulation setting information in each of the embodiments as described above. The step of receiving the simulation setting information can be performed before or after the preprocessing step, and the simulation setting information can include at least one of simulation object vehicle information from the user terminal, sensor part information of the simulation object vehicle, and vehicle driving algorithm information. The simulation object vehicle information can include at least one of a kind, a shape, a size, a wheel base, a height, and a center of gravity of the vehicle, and the sensor part information of the simulation object vehicle can include at least one of the number, the kind, the specification, and the installation position in the vehicle of the sensor included in the sensor part. Also, the vehicle driving control algorithm information can be composed of an algorithm for calculating a driving instruction of the vehicle based on the sensor part output data of the simulation object vehicle. In this case, the driving instruction of the vehicle can include at least one of acceleration, deceleration, stop, driving direction, and a turn signal, and vehicle lighting of the vehicle. The sensor part information of the simulation object vehicle can be received before the sensor part output data is calculated by the second environment information and the sensor part data from the server, and the vehicle driving control algorithm information can be received before verification is performed in the second simulator, but a plurality of steps can be repeated by changing a plurality of variables in the simulation, and thus it is not limited thereto.

[0063] As described above, in the present application, specific matters such as specific structural elements and the like and the illustrated embodiments are described, but this is provided only to help understand the present application as a whole, and the present application is not limited to the embodiment described above, and various modifications and alterations can be made from such description by those of ordinary skill in the art to which the present application pertains.

[0064] Therefore, the idea of the present application is not limited to the illustrated embodiments, and modifications and alterations equivalent or equivalent to the scope of the present application including the scope of the claims are within the scope of the idea of the present application.

Claims

1. A moving body simulation method characterized by, The method comprises: Step (a), a first server performs simulation preprocessing of environment information including at least one of simulation object road information, surrounding vehicle information, surrounding pedestrian information, surrounding obstacle information, surrounding traffic signal information, surrounding signboard information, and event information within a specified time; and Step (b), a second server performs simulation of sensor output data of a simulation object vehicle generated based on driving of the simulation object vehicle using the environment information within the specified time, wherein in the step (a), the first server generates the environment information based on the specified time in response to receiving an environment information login request from a user terminal, in the step (b), the second server receives the environment information from the first server and calculates the sensor output data based on the environment information and the selected sensor in response to receiving sensor parameter setting information including a request to select a sensor to be applied from a plurality of sensors from the user terminal.

2. The mobile body simulation method according to claim 1, wherein the step (b) is performed after the step (a) is performed, the step (a) further includes a step of storing the environment information, in the step (b), the second server can reduce the computational resource burden for simulation using the environment information stored in the step (a).

3. The mobile body simulation method according to claim 1, wherein the step (a) and the step (b) are performed simultaneously, the second server can reduce the computational resource burden of the second server for simulation by receiving the environment information used in the step (b) from the first server in real time.

4. The moving body simulation method according to claim 3, characterized by, The first server and the second server are different servers.

5. The mobile body simulation method according to claim 1, characterized by, The environment information includes simulation time information to synchronize the simulation time of the step (a) and the step (b).

6. The mobile body simulation method according to claim 1, characterized by, The simulation object vehicle is an autonomous vehicle.

7. The mobile body simulation method according to claim 1, characterized by, The sensor includes at least one of a laser radar, a radar, a global positioning system, and a camera.

8. The mobile body simulation method according to claim 1, characterized by, The method further includes a step of receiving simulation setting information including at least one of simulation object vehicle information, sensor information of the simulation object vehicle, and vehicle driving control algorithm information from a user before or after the step (a).

9. The moving body simulation method according to claim 8, characterized by, The simulation object vehicle information includes at least one of vehicle type, shape, size, wheelbase, height, and center of gravity.

10. The mobile body simulation method according to claim 8, characterized by, The sensor information of the simulation object vehicle includes at least one of the number, type, specification, and installation position within the vehicle of the sensor included in the sensor.

11. The mobile body simulation method according to claim 8, characterized by, The vehicle driving control algorithm information is an algorithm for calculating a driving instruction of the vehicle based on the sensor output data of the simulation object vehicle.

12. The mobile body simulation method according to claim 11, characterized by, The driving instruction of the vehicle includes at least one of acceleration, deceleration, stop, driving direction, and turn signal, and vehicle lighting of the vehicle.

13. The mobile body simulation method according to claim 1, characterized by, In the step (b), the driving of the simulation object vehicle is controlled based on the calculated sensor output data of the simulation object vehicle.

14. A mobile body simulation system characterized by comprising: A first server and a second server for executing a moving body simulation method having one feature of claims 1 to 13.

15. A recording medium, characterized by comprising: An instruction for executing a moving body simulation method having one feature of claims 1 to 13 is stored.

Images