Autonomous driving verification system based on fusion of real and virtual information

Abstract

To provide an autonomous driving verification system based on fusion of real and virtual information, which enables simulation of actual roads and situations by integrating reality and virtual space, and realizes various situations as if they were real.SOLUTION: The autonomous driving verification system includes: a vehicle drive unit comprising a plurality of devices that performs functions of steering, driving, decelerating and braking; a simulation unit that generates virtual information including an environment of an actual urban road; and an autonomous driving unit that constructs a virtual environment by fusing actual surrounding environment information acquired from a sensor with the virtual information of the simulation unit, and controls the vehicle drive unit to perform realistic driving within the virtual environment in order to verify autonomous driving. The virtual information of the simulation unit may include signal indication information received from infrastructure, which is installed at an intersection of an urban road and includes at least one of a signal controller, an unexpected situation detector, a pedestrian detector and CCTV.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an autonomous driving verification system that uses real-virtual information fusion to design and test an environment that is very similar to a real environment in a virtual environment so that autonomous driving performance can be verified based on a real vehicle for unexpected scenarios that are difficult to test in reality, and to perform reliable autonomous driving verification based on a real vehicle for unexpected scenarios that are difficult to test in reality. [Background technology]

[0002] An autonomous vehicle is a vehicle that can reach its destination by itself without the driver having to operate the steering wheel, accelerator pedal, brake, etc. As autonomous driving technology has recently been recognized as a core technology for smart cars, not only major automakers but also software companies are actively challenging themselves to develop autonomous driving technology.

[0003] A variety of technologies are required to realize autonomous driving, such as highway driving assistance (HDA) systems that automatically maintain a safe distance from other vehicles, lane departure warning systems (LDWS), lane keeping assist systems (LKAS), rear and side warning systems (BSD), advanced smart cruise control (ASCC), and automatic emergency braking systems (AEB).

[0004] The completion of an autonomous vehicle will take a long time and require a lot of trial and error, as various technologies must all reach a certain level.

[0005] Therefore, along with the development technology for autonomous vehicles, test methods are also being developed to check the driving performance of autonomous vehicles and whether there are any malfunctions in their various systems.

[0006] However, conventional testing of autonomous vehicles is not easy because there is a high possibility of accidents when testing on actual roads. Therefore, conventionally, a system for testing autonomous vehicles has been proposed that can provide an environment similar to actual road conditions, but not on actual roads.

[0007] However, with such conventional autonomous driving test systems, it was difficult to create specific areas that matched the conditions of the actual road environment (e.g., roads, traffic signal systems, vehicles, pedestrians, various physical structures, weather, lighting, etc.), and it was not possible to create a test environment that combined the various corner cases and dilemmas of urban driving environments, where unexpected situations are common, with road infrastructure such as control systems and road peripheral devices.

[0008] In other words, conventional autonomous vehicle testing requires testing in a variety of situations that could actually occur, but this is difficult to do on real roads due to issues such as the risk of accidents. Simulated environments also have limitations in realizing real environments and situations, which limits the ability to verify the performance of autonomous vehicles. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Korean Patent Registration No. 10-1938064 Summary of the Invention [Problem to be solved by the invention]

[0010] Therefore, the present invention has been devised to solve the above-mentioned conventional problems, and aims to provide an autonomous driving verification system that combines reality and virtual space to simulate actual roads and situations, thereby realizing various real-life situations. [Means for solving the problem]

[0011] In order to achieve the above object, the present invention can include the following embodiments.

[0012] An embodiment of the present invention includes a vehicle driving unit consisting of multiple devices that perform steering, driving, deceleration, and braking functions; a simulation unit that generates virtual information including the environment of an actual urban road; and an autonomous driving unit that creates a virtual environment that combines actual surrounding environment information obtained from sensors with the virtual information of the simulation unit, controls the vehicle driving unit to actually drive in the virtual environment, and verifies autonomous driving, where the virtual information of the simulation unit can include signal phase information received from infrastructure that includes at least one of an intersection signal controller, an unexpected situation detector, a pedestrian detector, and CCTV installed on an urban road.

[0013] In the above embodiment, the autonomous driving unit may include a sensor information acquisition module equipped with a plurality of sensors to acquire information about the surroundings of the autonomous driving vehicle; a vehicle information acquisition module that outputs vehicle status information including at least one of driving speed and its change amount, battery charge level, equipment operating status, steering status, brake operation or not, and lighting status; a driving judgment module that controls the vehicle driving unit according to the real information output from the vehicle information acquisition module and the sensor information acquisition module and values ​​set in the virtual information of the simulation unit; and a virtual environment construction module that realizes a virtual environment in which the virtual information including the specific urban road environment and traffic light indication information of the simulation unit is integrated with actual roads.

[0014] In the above embodiment, the simulation unit can set setting values ​​including the route and speed for the autonomous vehicle's travel, control values ​​for steering, travel, and braking, and scenarios for avoiding unexpected situations, and output them to the autonomous driving unit.

[0015] In the above embodiment, the autonomous driving unit may further include a driving judgment module that transmits control values ​​required for autonomous driving to the vehicle driving unit based on the setting values ​​received from the simulation unit so that the vehicle driving unit is controlled according to the set scenario when an unexpected situation is detected in the virtual environment. [Effects of the Invention]

[0016] Therefore, the present invention can combine real and virtual information to create various test environments, thereby enabling the performance of autonomous vehicles to be verified and improving the safety and performance reliability of autonomous vehicles. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a diagram illustrating an overview of the present invention. [Figure 2] 1 is a block diagram illustrating an autonomous driving verification system using real-virtual information fusion according to the present invention. [Figure 3] FIG. 2 is a block diagram illustrating an autonomous driving unit. [Figure 4] FIG. 2 is a block diagram illustrating a simulation unit. [Figure 5] FIG. 1 is a block diagram illustrating a V2X communication unit. [Figure 6] 1 is a flowchart illustrating a method for verifying autonomous driving by integrating real and virtual information according to the present invention. [Figure 7] 1 is a diagram illustrating a travel path of an actual autonomous vehicle according to an embodiment of the present invention; [Figure 8] FIG. 8 illustrates a virtual environment implemented in the autonomous vehicle of FIG. 7. DETAILED DESCRIPTION OF THE INVENTION

[0018] Although the present invention can be modified in various ways and can have various embodiments, a specific embodiment will be illustrated in the drawings and described in detail. This is not intended to limit the present invention to the specific embodiment, but should be understood to apply to any one of all modifications, equivalents, or alternatives within the spirit and technical scope of the present invention for connecting and / or fixing structures extending in different directions.

[0019] The terms used in this specification are merely used to describe specific embodiments and are not intended to limit the present invention. The singular expressions include the plural expressions unless the context clearly indicates otherwise.

[0020] In this specification, the terms "comprise" or "have" and the like are intended to specify the presence of any feature, number, step, operation, component, part, or combination thereof stated in the specification, but should be understood as not precluding the possible presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0021] Hereinafter, a preferred embodiment of an autonomous driving verification system based on the fusion of real and virtual information according to the present invention will be described with reference to the accompanying drawings.

[0022] FIG. 1 is a diagram illustrating an overview of the present invention.

[0023] Referring to FIG. 1, in the present invention, an autonomous vehicle traveling on a road or test road receives virtual information from, for example, an external control center via a V2X communication device, and combines actual information sensed by the autonomous vehicle's own sensor function with the virtual information to create a virtual road environment.

[0024] Here, the received virtual information is virtual information linked to precise infrastructure for a specific area that matches the domestic road environment conditions (road signal system, vehicles, pedestrians, various object structures, lighting, weather, etc.).

[0025] Here, the infrastructure may include a traffic light controller, an emergency situation detector, a pedestrian detector, a CCTV, etc. in an intersection.

[0026] The virtual environment can be configured to suit the required scenario, including surrounding areas such as intersections and shadowed areas caused by buildings, similar to the urban road environment.

[0027] In addition, the present invention can configure various objects (vehicles, pedestrians, etc.) and traffic conditions (smooth traffic, delays, congestion, presence of traffic accidents ahead) around the autonomous vehicle, and in the case of traffic, can realize actual traffic signal information in conjunction with infrastructure systems within a specific area.

[0028] Furthermore, in order to conduct tests based on an actual vehicle, the present invention allows for precise simulation of autonomous driving SW by simulating a vehicle model in the same way as the actual vehicle environment and receiving information from the actual vehicle.

[0029] For example, as shown in Figure 1, the actual autonomous vehicle is in an environment with no obstacles around it, as shown on the left, while the virtual environment is an urban road environment, as shown on the right.

[0030] That is, the autonomous vehicle verification system according to the present invention can be linked with infrastructure information of actual city roads to create a virtual environment like the one shown in the drawing.

[0031] FIG. 2 is a block diagram illustrating an autonomous driving verification system based on the fusion of real and virtual information according to the present invention, FIG. 3 is a block diagram illustrating an autonomous driving unit, FIG. 4 is a block diagram illustrating a simulation unit, and FIG. 5 is a block diagram illustrating a V2X communication unit.

[0032] 2 to 5, the present invention can include an autonomous driving unit 100, a simulation unit 200, a V2X communication unit 300, and a vehicle driving unit 400.

[0033] The autonomous driving unit 100 controls the autonomous driving vehicle by combining real-world information, including information about actual roads and road surroundings, with virtual information to create a virtual environment in which the autonomous driving vehicle can be verified.

[0034] The simulation unit 200 simulates the actual urban road environment of a specific area and generates virtual information including signal phase information linked to the infrastructure of the area.

[0035] The V2X communication unit 300 receives infrastructure information and outputs it to the simulation unit 200 and the autonomous driving unit 100, and receives and stores data acquired during the actual driving of the autonomous vehicle. The data acquired during autonomous driving can be used as reference data for verification during autonomous driving.

[0036] Of these, the autonomous driving unit 100 can include a sensor information acquisition module 110 , a vehicle information acquisition module 120 , a driving determination module 130 , a data transmission module 140 , and a virtual environment construction module 150 .

[0037] The sensor information acquisition module 110 collects information from sensors such as a camera, a lidar, a radar, a GPS / IMU, and an ultrasonic sensor.

[0038] The vehicle information acquisition module 120 collects driving-related information, such as the speed and its change rate, the battery charge level, the operating status of the devices, the steering status, whether the brakes are operating, the lighting status, and other status information of the autonomous vehicle before and during driving.

[0039] The driving determination module 130 controls the vehicle driving unit 400. For example, the driving determination module 130 determines elements (speed, steering value, etc.) required for autonomous driving based on setting values ​​(route, maximum speed, etc.) received from the simulation unit 200, and transmits control values ​​based on the determination to the vehicle driving unit 400.

[0040] In addition, the driving determination module 130 may include the information collected by the sensor information acquisition module 110 in the determination factors (for example, whether an object actually exists ahead or whether the road being driven is a dead end).

[0041] That is, the driving determination module 130 controls the vehicle driving unit 400 according to the values ​​set in the real information and virtual information.

[0042] The data transmission module 140 transmits the collected information to infrastructure (for example, a control center and / or roadside devices) via the V2X communication unit 300.

[0043] The virtual environment construction module 150 constructs a virtual environment in which actual roads and virtual information are integrated. Here, the signal control of vehicles, pedestrians, traffic lights, pedestrian detectors, and sudden collision detectors in the constructed virtual environment reflects the received infrastructure information.

[0044] That is, the virtual environment construction module 150 constructs a virtual environment in which the real information acquired from the sensor information acquisition module 110 and the virtual information of the simulation unit 200 are combined.

[0045] The simulation unit 200 virtually realizes an environment 210, an object 220, a traffic 230, a scenario 240, and a model (sensor, vehicle, traffic light, pedestrian) 250. The environment 210, the object 220, the traffic 230, the scenario 240, the model 250, etc. can be linked to the infrastructure of a specific area.

[0046] For example, the simulation unit 200 generates virtual information including urban roads in area A according to information received from infrastructure in area A via the V2X communication unit 300. Here, the virtual information may include traffic light aspect information linked to infrastructure in area A (signal controllers in intersections, emergency situation detectors, pedestrian detectors, CCTV). That is, the simulation unit 200 may realize objects, environments, traffic, scenarios, and models in the urban road environment of area A according to the road and surrounding conditions of the urban road from the infrastructure in area A.

[0047] The V2X communication unit 300 includes a vehicle data receiving module 310 that receives vehicle data, a data storage module 320 that stores collected data, and an infrastructure data transmission module 330 that transmits and receives infrastructure information.

[0048] The vehicle data receiving module 310 receives vehicle and surrounding information (actual information) from the sensor information acquiring module 110 and the vehicle driving unit 400. The actual information can be stored in the data storage module 320.

[0049] The infrastructure data transmission module 330 receives linked information from the infrastructure and outputs it to the simulation unit 200. Furthermore, the infrastructure data transmission module 330 can receive data in real time via the linked infrastructure, or store the received data in the data storage module 320 and transmit selected data in response to a request from the simulation unit 200.

[0050] The infrastructure data transmission module can receive real-time traffic signal information from the infrastructure of a specific area and output it to the simulation unit in real time, so that the simulation unit can realize traffic signal information in the virtual environment in real time according to the actual urban road environment.

[0051] That is, the present invention can realize the urban road environment of a specific area in real time.

[0052] The present invention includes the above-described configuration, and the following describes an autonomous driving verification method based on the fusion of real and virtual information achieved by the above-described configuration.

[0053] FIG. 6 is a flowchart illustrating a method for verifying autonomous driving by combining real and virtual information according to the present invention, FIG. 7 is a diagram illustrating the actual route of an autonomous vehicle according to an embodiment of the present invention, and FIG. 8 is a diagram illustrating a virtual environment realized by the autonomous vehicle of FIG. 7.

[0054] Referring to Figures 6 to 8, the present invention includes step S110 of receiving infrastructure data, step S120 of generating a virtual environment, step S130 of setting a scenario, step S140 of transmitting driving judgment and control values, step S150 of performing autonomous driving, and step S160 of verifying the autonomous driving.

[0055] Step S110 is a step in which the verification system receives infrastructure data. The V2X communication unit 300 receives data from the infrastructure that is linked and outputs the data to the simulation unit 200.

[0056] Step S120 is a step of generating a virtual environment in the autonomous driving verification system. The simulation unit 200 generates and / or sets main objects of the virtual environment, such as objects and models included in the received infrastructure data and scenarios, and outputs them to the autonomous driving unit 100.

[0057] Here, the infrastructure data is data received from infrastructure linked to a specific area, such as traffic lights, pedestrian detectors, and sudden incident detectors.

[0058] Therefore, the autonomous driving unit 100 realizes a virtual environment including a plurality of objects included in an urban environment of a specific area realized by the simulation unit 200.

[0059] Step S130 is a scenario setting step. The simulation unit 200 sets an autonomous driving scenario based on an emergency situation or an urban situation in the virtual environment. For example, the simulation unit 200 sets a scenario in which the autonomous vehicle detects and avoids a rear-end collision situation ahead of the vehicle after passing through an intersection in the virtual environment of FIG. 8, and outputs the scenario to the autonomous driving unit 100. The autonomous driving unit 100 registers the scenario in the simulation unit 200.

[0060] Step S140 is a step of transmitting driving judgment and control values ​​in the autonomous driving verification system. Driving judgment criteria and control values ​​are set by the autonomous driving unit 100. The control values ​​may include steering values ​​(steering range, steering order), driving speed, whether or not turn signals are used, etc., and the driving judgment criteria may be any one of the range of sensing distance, signal lights, driving speed, and whether or not there are objects in the lane ahead and adjacent lanes.

[0061] Step S150 is a step of autonomous driving. The autonomous driving unit 100 executes the virtual environment and then controls the vehicle driving unit 400 to drive the actual autonomous vehicle.

[0062] Step S160 is a step for verifying autonomous driving. Here, the autonomous driving unit 100 controls the vehicle driving unit 400 according to a scenario that, when the traffic light at an intersection changes to a driving-allowed signal, the autonomous driving unit 100 drives straight ahead and, if an object ahead is detected, checks whether there are other objects in the adjacent lane, moves to the adjacent lane, overtakes the object ahead, and then returns to the original lane.

[0063] Here, the actual autonomous vehicle is activated in accordance with the control of the vehicle drive unit 400 as described above, according to a scenario that deals with unexpected situations in the virtual environment, such as moving straight ahead, then moving to the next lane, and then returning to the direction of the first lane, as shown in Figure 7.

[0064] Here, the V2X communication unit 300 can receive sensor information and status information of the vehicle driving unit 400, store and / or transmit the data acquired during verification to a configured external terminal, or store it in a data storage unit.

[0065] In other words, the present invention virtually realizes situations that are difficult to realize on actual roads, such as unexpected situations for the verification of an autonomous vehicle, and the autonomous vehicle can safely perform difficult verifications on actual roads by actually starting up to avoid the unexpected situations realized in the virtual world.

[0066] Furthermore, the present invention can precisely simulate the conditions and situations of a specific area by linking with the infrastructure of the specific area.

[0067] Although the present invention has been described above using limited embodiments and drawings, it goes without saying that the present invention is not limited thereto, and various modifications and variations can be made by those having ordinary skill in the art to which the present invention pertains.

[0068] Those skilled in the art will understand that the present invention may be embodied in various modified forms without departing from the essential characteristics described above. Therefore, the disclosed method should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the claims, not the above description, and all variations within the scope of the claims should be construed as being within the scope of the present invention. [Explanation of symbols]

[0069] 100 Autonomous Driving Unit 110 Sensor information acquisition module 120 Vehicle information acquisition module 130 Driving Judgment Module 140 Data Transmission Module 150 Virtual Environment Construction Module 200 Simulation Department 300 V2X communication unit 400 Vehicle drive unit

Claims

1. a vehicle drive unit consisting of a plurality of devices that perform the functions of steering, running, deceleration, and braking; a simulation unit that generates virtual information including the actual urban road environment; an autonomous driving unit that constructs a virtual environment that combines actual surrounding environment information acquired from a sensor with virtual information from a simulation unit, controls a vehicle driving unit so that the vehicle actually drives in the virtual environment, and verifies autonomous driving; This is an autonomous driving verification system that combines real and virtual information, characterized in that the virtual information of the simulation unit includes signal phase information received from infrastructure including at least one of traffic light controllers, sudden situation detectors, pedestrian detectors, and CCTVs installed at intersections on urban roads.

2. The autonomous driving part is a sensor information acquisition module that includes a plurality of sensors and acquires information about the surroundings of the autonomous vehicle; a vehicle information acquisition module that outputs vehicle state information including at least one of a traveling speed and a change therein, a battery charge amount, an operating state of an appliance, a steering state, whether or not the brake is operating, and a lighting state; a driving determination module that controls a vehicle drive unit according to real information output from the vehicle information acquisition module and the sensor information acquisition module and values ​​set in the virtual information of the simulation unit; The autonomous driving verification system by integrating real and virtual information as described in claim 1, characterized in that it includes virtual information including specific urban road environment and traffic light indication information of the simulation unit, and a virtual environment construction module that realizes a virtual environment that integrates actual roads.

3. The simulation section is The autonomous driving verification system using the fusion of real and virtual information according to claim 2, characterized in that it sets setting values ​​including the route and speed for the autonomous vehicle's driving, control values ​​for steering, driving and braking, and scenarios for avoiding unexpected situations, and outputs them to the autonomous driving unit.

4. The autonomous driving part is The autonomous driving verification system based on the fusion of real and virtual information according to claim 3, further comprising a driving judgment module that transmits control values ​​required for autonomous driving to the vehicle driving unit based on the setting values ​​received from the simulation unit so that the vehicle driving unit is controlled according to the set scenario when an unexpected situation is detected in the virtual environment.

Images