Method for Judging Road Conditions through Sharing of Sensor Data and Object Recognition Results in V2X Communication Environment

Abstract

To provide a determination method of a road state through sharing of sensor data and an object recognition result under V2X environment.SOLUTION: Sensor data is acquired, the acquired sensor data is analyzed, an object is recognized, sensor data and an object recognition result are received from an external terminal, reliability to the received object recognition result and the sensor data is determined, and a road state is determined on the basis of the acquired sensor data, the object recognition result, the object recognition result and the sensor data to which the reliability is given. Thus, performance of autonomous travel is improved by accurately determining the road state through sharing of the sensor data and the object recognition result under V2X environment, and also accuracy of road state determination can be improved by determining whether or not to utilize the sensor data and the object recognition result to be received from the outside through the sharing for determination of the road state on the basis of determination of the reliability.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to autonomous driving control based on V2X, and more particularly to a method for controlling vehicle driving by judging road conditions through information sharing in a V2X communication environment.

Background Art

[0002] Conventional autonomous driving involves each vehicle using its own vehicle sensors and algorithms to recognize and judge objects, and analyzing object information and object conditions. In communication environments such as WAVE and LTE, there are limitations in high-capacity and low-latency data sharing, so there are limitations in recognizing objects and judging situations in real time using information transmitted from other vehicles and surrounding infrastructure. As a result, since each vehicle recognizes and judges objects, the recognition / judgment results vary from vehicle to vehicle, and in the case of a specific vehicle, there may be a case of incorrect recognition / judgment. For example, when a traffic signal changes to red, the vehicle in front recognizes / judges the information of the traffic signal and stops. However, in the case of a following vehicle, the information of the traffic signal is blocked by the vehicle in front and becomes difficult to see, and since the vehicle in front has stopped, the following vehicle recognizes / judges the part where the inter-vehicle distance with the vehicle in front narrows and slows down.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide a method for judging road conditions through sharing of sensor data and object recognition results in a V2X environment.

Means for Solving the Problems

[0004] A method for determining road conditions according to an embodiment of the present invention for achieving the above object includes: a step of acquiring sensor data; a step of analyzing the acquired sensor data to recognize an object; a step of receiving the sensor data and the object recognition result from an external terminal; a first determination step of determining the reliability of the received object recognition result; a second determination step of determining the reliability of the received sensor data; and a third determination step of determining the road conditions based on the sensor data acquired from the acquisition step, the object recognition result acquired from the recognition step, the object recognition result given reliability from the first determination step, and the sensor data given reliability from the second determination step. The external terminal may include terminals of surrounding vehicles, infrastructure, and pedestrian terminals. The receiving step may receive the sensor data and the object recognition result from the surrounding terminals via V2X communication. The first determination step may include a step of measuring the reception delay time of the object recognition result, and if the measured delay time is less than or equal to a threshold value, a step of giving reliability to the object recognition result. The step of giving reliability may include a step of confirming the object recognition rate of the external terminal that transmitted the object recognition result if the measured delay time is less than or equal to the threshold value, and if the confirmed object recognition rate is greater than or equal to the threshold value, a step of giving reliability to the object recognition result. The step of giving reliability may not give reliability to the object recognition result if the confirmed object recognition rate is less than the threshold value. The first determination step may not give reliability to the object recognition result if the measured delay time exceeds the threshold value. The second determination step may include a step of measuring the reception delay time of the sensor data, and if the measured delay time is less than or equal to a threshold value, a step of giving reliability to the sensor data. The step of giving reliability may not give reliability to the sensor data if the measured delay time exceeds the threshold value. A road condition determination system according to another embodiment of the present invention includes an acquisition unit that acquires sensor data, a recognition unit that analyzes the acquired sensor data and recognizes an object, a reception unit that receives the sensor data and the object recognition result from an external terminal, a first determination unit that determines the reliability of the received object recognition result, a second determination unit that determines the reliability of the received sensor data, and a third determination unit that determines the road condition based on the sensor data acquired from the acquisition unit, the object recognition result acquired from the recognition unit, the object recognition result given reliability from the first determination unit, and the sensor data given reliability from the second determination unit. A road condition determination method according to still another embodiment of the present invention includes a step of receiving sensor data and an object recognition result from an external terminal, a step of determining the reliability of the received sensor data and the object recognition result, a step of determining the road condition based on the internally acquired sensor data, the internally acquired object recognition result, and the sensor data and the object recognition result given reliability from the determination step, and a step of controlling vehicle travel based on the determined road condition. A road condition determination system according to still another embodiment of the present invention includes a reception unit that receives sensor data and an object recognition result from an external terminal, a first determination unit that determines the reliability of the received sensor data and the object recognition result, a second determination unit that determines the road condition based on the internally acquired sensor data, the internally acquired object recognition result, and the sensor data and the object recognition result given reliability from the first determination unit, and a control unit that controls vehicle travel based on the determined road condition.

Advantages of the Invention

[0005] As described above, according to the present invention, by accurately determining the road condition through sharing of sensor data and object recognition results in a V2X environment, the performance of autonomous driving can be improved. According to the embodiments of the present invention, by determining whether to utilize the sensor data received from the outside and the object recognition result for road condition determination based on the reliability judgment through sharing, the accuracy of road condition judgment can be further improved.

Brief Description of the Drawings

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0007] Hereinafter, the present invention will be described in more detail with reference to the drawings. In the embodiments of the present invention, a method for determining road conditions through sharing of sensor data and object recognition results in a V2X communication environment is presented. In the embodiments of the present invention, V2X communication includes V2V (Vehicle to Vehicle) communication, V2I (Vehicle to Infra) communication, V2P (Vehicle To Pedestrian) communication, and V2C (Vehicle to Cloud) communication. Note that, as a matter of course, communication methods such as WAVE, LTE, 5G-Uu, 5G-NR-V2X, and 6G, as well as other communication methods, can also be applied to the V2X communication method.

[0008] FIG. 1 is a diagram showing a V2X communication environment to which an embodiment of the present invention is applicable. As shown in the figure, the V2X communication environment to which the embodiment of the present invention is applicable is constructed by interconnecting an autonomous driving vehicle terminal 100, surrounding vehicle terminals 10, infrastructure 20, and pedestrian terminals 30 via V2X communication. The autonomous driving vehicle terminal 100 is a V2X terminal for controlling a vehicle while sharing information with surrounding terminals 10, 20, and 30 in an embodiment of the present invention. The surrounding vehicle terminal 10 is a V2X terminal installed in a vehicle located around the autonomous driving vehicle terminal 100. The surrounding vehicle terminal 10 also has something in common with the autonomous driving vehicle terminal 100 in controlling the vehicle through information sharing. However, in the embodiment of the present invention, the vehicle control through information sharing will be described only for the autonomous driving vehicle terminal 100.

[0009] The infrastructure 20 refers to a signal controller, an RSU (Road Side Unit), etc. that transmits traffic-related information to other terminals 10, 30, and 100 at the roadside, intersections, etc. The pedestrian terminal 30 has an application for V2X communication installed on the terminal carried by the pedestrian. On the other hand, in FIG. 1, many of the surrounding vehicle terminals 10, infrastructure 20, and pedestrian terminals 30 are for illustration purposes, and in reality, there are generally more of them than shown.

[0010] FIG. 2 is a diagram showing the details of the configuration of the autonomous driving vehicle terminal 100 shown in FIG. 1. As shown in the figure, the autonomous driving vehicle terminal 100 includes a sensor input unit 110, an object recognition unit 120, a storage unit 130, a communication unit 140, a determination unit 150, and a vehicle control unit 160. The sensor input unit 110 is a configuration for inputting sensor data generated by sensors mounted on the vehicle 100 sensing the surrounding environment. The sensors may include a camera, LiDAR, Radar, etc.

[0011] The object recognition unit 120 analyzes the sensor data input via the sensor input unit 110 to detect an object, and recognizes and classifies the detected object. The objects to be recognized include surrounding vehicles, people, animals, buildings, traffic lights, pedestrians, etc. The storage unit 130 stores the sensor data input via the sensor input unit 110 and the object recognition results by the object recognition unit 120. The communication unit 140 transmits the sensor data and the object recognition results stored in the storage unit 130 to the surrounding vehicle terminal 10, the infrastructure 20, and the pedestrian terminal 30.

[0012] Note that the communication unit 140 may receive the sensor data and the object recognition results from the surrounding vehicle terminal 10, the infrastructure 20, and the pedestrian terminal 30. The received sensor data and object recognition results are also stored in the storage unit 130. As shown in the figure, the determination unit 150 includes a reliability determination unit 151 for the object recognition result, a preprocessing unit 152 for the sensor data, a reliability determination unit 153 for the sensor data, and a road condition determination unit 154.

[0013] The reliability determination unit 151 for the object recognition result determines the reliability of the object recognition results received from the external (surrounding vehicle) terminal 10, the infrastructure 20, and the pedestrian terminal 30 stored in the storage unit 130. The preprocessing unit 152 for the sensor data performs preprocessing on the sensor data stored in the storage unit 130. The reliability determination unit 153 for the sensor data determines the reliability of the sensor data received from the outside among the sensor data preprocessed by the preprocessing unit 152 for the sensor data. The road condition determination unit 154 determines the road condition based on: 1) the object recognition results generated from inside the vehicle where the autonomous driving vehicle terminal 100 is installed, 2) the object recognition results received from the outside that are determined to be reliable by the reliability determination unit 151 for the object recognition results, 3) the sensor data generated from inside the vehicle, and 4) the sensor data received from the outside that are determined to be reliable by the reliability determination unit 153 for the sensor data. The vehicle control unit 160 controls the running of the vehicle based on the road conditions determined by the road condition determination unit 154.

[0014] Hereinafter, a method for determining the reliability of the object recognition result received from the outside will be described in detail with reference to FIG. 3. As shown in the figure, first, when an object recognition result is received from the outside (S210-Y), the reliability determination unit 151 of the object recognition result performs time synchronization between the autonomous driving vehicle terminal 100 and the external terminals 10, 20, and 30 that transmitted the object recognition result (S220), and measures the reception delay time (t L ) (S230). The time synchronization in step S220 may be performed based on the PPS value of the GPS receiver. The delay time (t L ) measured in step S230 exceeds the threshold value (t th ), for example, 3 ms (S240-N), the reliability determination unit 151 of the object recognition result does not give reliability to the object recognition result received from the external terminals 10, 20, and 30. As a result, the object recognition results received from the external terminals 10, 20, and 30 are not utilized.

[0015] On the other hand, when the delay time (t L ) measured in step S230 is equal to or less than the threshold value (t th ) (S240-Y), the reliability determination unit 151 of the object recognition result checks the object recognition rate (Pr) of the external terminals 10, 20, and 30 (S250). The object recognition rate (Pr) may be received and checked from the external terminals 10, 20, and 30. The better the performance of the sensor and the object recognition model, the higher the object recognition rate of the external terminals 10, 20, and 30.

[0016] When the object recognition rate (Pr) confirmed in step S250 is the threshold value (P th) For example, when it is less than 90% (S260-N), the reliability determination unit 151 of the object recognition result does not give reliability to the object recognition result received from the external terminals 10, 20, and 30. As a result, the object recognition result received from the external terminals 10, 20, and 30 is not utilized.

[0017] On the other hand, when the object recognition rate (Pr) confirmed in step S250 is equal to or higher than the threshold value (P th ) (S260-Y), the reliability determination unit 151 of the object recognition result gives reliability to the object recognition result received from the external terminals 10, 20, and 30 (S270). The object recognition result received from the external terminals 10, 20, and 30 with reliability given is utilized for the road condition determination by the road condition determination unit 154. If there are a plurality of external terminals 10, 20, and 30 with an object recognition rate (Pr) equal to or higher than the threshold value, only the object recognition result of the external terminal 10, 20, or 30 with the highest object recognition rate (Pr) may be utilized.

[0018] Furthermore, when the object recognition rates for the plurality of external terminals 10, 20, and 30 are the same or at an equal level, the object recognition result for grasping an accident utilizes the object recognition result received from the external terminals 10, 20, and 30 that are close to the autonomous driving vehicle terminal 100. The distance between the autonomous driving vehicle terminal 100 and the external terminals 10, 20, and 30 can be calculated by utilizing the position information received from the external terminals 10, 20, and 30.

[0019] Hereinafter, a method for determining the reliability of sensor data received from the outside will be described in detail with reference to FIG. 4. As shown in the figure, first, when sensor data is received from the outside (S310 - Y), the pre - processing unit 152 of the sensor data performs pre - processing such as data Cleaning, Integration, Transformation, Reduction, Discretization, and Descriptive Charateristics Mining on the received sensor data (S320).

[0020] Next, the reliability judgment unit 153 of the sensor data measures the reception delay time (t L ) of the sensor data (S330). When the delay time (t L ) measured in step S330 exceeds the threshold value (t th ), for example, 3 ms (S340 - N), the reliability judgment unit 153 of the sensor data does not give reliability to the sensor data received from the external terminals 10, 20, 30. Accordingly, the sensor data received from the external terminals 10, 20, 30 is not utilized.

[0021] On the other hand, when the delay time (t L ) measured in step S330 is less than or equal to the threshold value (t th ) (S340 - Y), the reliability judgment unit 153 of the sensor data gives reliability to the sensor data received from the external terminals 10, 20, 30 (S350). The sensor data received from the external terminals 10, 20, 30 with reliability given is utilized for the road condition judgment by the road condition judgment unit 154. If there are a plurality of external terminals 10, 20, 30 whose reception delay time (t L ) is less than or equal to the threshold value, only the sensor data of the external terminal 10, 20, 30 with the shortest reception delay time (t L ) can be utilized.

[0022] Furthermore, when the reception delay times (t L ) for a plurality of external terminals 10, 20, 30 are the same or at an equal level, the sensor data received from the external terminals 10, 20, 30 that are close to the autonomous driving vehicle terminal 100 is utilized. An application example of controlling an autonomous vehicle according to the above embodiment is shown in FIG. 5. The application example shown in FIG. 5 is a situation where the vehicle ahead utilizes the traffic signal recognition result and the traffic signal information received from the traffic signal controller via V2I, stops in front of the intersection, and the following vehicle's view is blocked by the vehicle ahead and cannot recognize the traffic signal. However, it recognizes the decrease in the inter-vehicle distance from the vehicle ahead and stops by recognizing the traffic signal through the traffic signal information received from the vehicle ahead via V2V.

[0023] The above application example is also applicable when the vehicle starts. For example, after waiting for a signal, when the vehicle starts, the following vehicle excluding the vehicle ahead may not be able to see the traffic signal. In such a situation, the signal information is provided from the vehicle ahead via V2V and recognized to start. In this case, when the following vehicles start one after another after waiting for the signal, as many vehicles as possible can pass before the signal changes, enabling both smooth traffic and energy savings, and enabling systematic autonomous driving. In the above embodiment, since the threshold of the reception delay time used for reliability assignment can be increased or decreased by the time between t1 and t2 in FIG. 5, an appropriate value must be applied according to the situation of the application service. By controlling the autonomous vehicle according to the above embodiment, it is possible to increase the utilization rate of the road. The conventional V2X method mainly judges based on the information obtained from its own vehicle. In the above embodiment, information such as surrounding vehicles and infrastructure is received and utilized, and the judgment result of the reliability of the information is reflected.

[0024] For example, as shown in FIG. 6, in the case of d1, reliability is given to the vehicle ahead, and reliable data reception at a level close to or equal to the result of the object judged by its own vehicle is continuously performed. In such a situation, since it is a situation of trusting the vehicle ahead, autonomous driving is possible with approaching driving of the vehicle, and it can be expected that the inter-vehicle distance will decrease and the road utilization rate will improve. So far, a preferred embodiment has been described in detail regarding a method for judging road conditions through sharing of sensor data and object recognition results in a V2X communication environment. In the prior art, each vehicle uses its own vehicle sensors and algorithms to recognize and judge objects, analyze object information and the situation of the objects, and perform autonomous driving. Therefore, the recognition / judgment results recognized for each vehicle are different when each vehicle recognizes and judges objects. In the case of a specific vehicle, incorrect recognition / judgment has occurred. For example, when the traffic signal changes to red, etc., the vehicle in front recognizes / judges the information of the traffic signal and steps on the brake to stop. However, when the following vehicle cannot see the information of the traffic signal blocked by the vehicle in front, since the vehicle in front has stopped, a situation occurs where the following vehicle recognizes / judges the part where the distance to the vehicle in front narrows and reduces its speed and steps on the brake.

[0025] In an embodiment of the present invention, in order to solve the corresponding situation, in a similar situation, the vehicle in front transmits the information on the change in the situation of the recognized / judged object (change in the traffic signal) to the following vehicle via V2X communication. Thereby, when the following vehicle receives the situation information of the object transmitted from the vehicle in front, it synthesizes the situation / recognition information of the object judged by itself, the situation information of the object in front that could not be judged because it was blocked by the vehicle in front, etc., and compares and utilizes the surrounding situation with the information recognized only by its own sensors in the past, so that it can recognize the surrounding situation more accurately than before and step on the brake earlier than before.

[0026] On the other hand, the technical idea of the present invention can also be applied to a computer-readable recording medium incorporating a computer program that causes a computer to perform the functions of the apparatus and method according to this embodiment. Note that the technical idea according to various embodiments of the present invention may be realized in a computer-readable code format recorded on a computer-readable recording medium. A computer-readable recording medium can be any data storage device that can be read by a computer and can store data. For example, a computer-readable recording medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. Note that a computer-readable code or program stored on a computer-readable recording medium may be transmitted via a network connected between computers.

[0027] As described above, the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. It is obvious that those having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modification examples or correction examples within the scope of the technical gist described in the claims, and it is naturally understood that these also belong to the technical scope of the present invention.

Claims

A method for determining road conditions using a terminal device provided in a vehicle, comprising: acquiring sensor data from a sensor provided in the vehicle; analyzing the acquired sensor data to recognize an object; receiving the sensor data and the object recognition result from an external terminal; a first determination step of determining the reliability of the received object recognition result; a second determination step of determining the reliability of the received sensor data; a third determination step of determining the road conditions based on the sensor data acquired from the acquisition step, the object recognition result acquired from the recognition step, the object recognition result given reliability from the first determination step, and the sensor data given reliability from the second determination step; causing the terminal device to execute the above; A method for determining road conditions, characterized by the above. **Claim 2** The external terminal includes the terminals of surrounding vehicles, infrastructure, and pedestrian terminals. The method for determining road conditions according to claim 1, characterized by the above. **Claim 3** The receiving step receives the sensor data and the object recognition result from surrounding terminals via V2X communication. The method for determining road conditions according to claim 2, characterized by the above. **Claim 4** The first determination step includes a step of measuring the reception delay time of the object recognition result; and a step of giving reliability to the object recognition result if the measured delay time is equal to or less than a threshold value. The method for determining road conditions according to claim 1, characterized by including the above. **Claim 5** The step of giving reliability includes a step of checking the object recognition rate of the external terminal that transmitted the object recognition result if the measured delay time is equal to or less than a threshold value; and a step of giving reliability to the object recognition result if the confirmed object recognition rate is equal to or more than a threshold value. The method for determining road conditions according to claim 4, characterized by including the above. **Claim 6** The step of giving reliability does not give reliability to the object recognition result if the confirmed object recognition rate is less than a threshold value. The method for determining road conditions according to claim 5, characterized by the above. **Claim 7** The first determination step does not give reliability to the object recognition result if the measured delay time exceeds the threshold value. The method for determining road conditions according to claim 4, characterized by the above. **Claim 8** The second determination step includes a step of measuring the reception delay time of the sensor data; If the measured delay time is less than or equal to a threshold value, a step of giving reliability to the sensor data The method for determining road conditions according to claim 1, characterized by including this.

9. The step of giving reliability is When the measured delay time exceeds the threshold value, reliability is not given to the sensor data The method for determining road conditions according to claim 8, characterized by this.

10. A road condition determination system including a terminal device provided in a vehicle, The terminal device is An acquisition unit that acquires sensor data from a sensor provided in the vehicle, A recognition unit that analyzes the acquired sensor data and recognizes an object, A reception unit that receives sensor data and an object recognition result from an external terminal, A first determination unit that determines the reliability of the received object recognition result, A second determination unit that determines the reliability of the received sensor data, Based on the sensor data acquired from the acquisition unit, the object recognition result acquired from the recognition unit, the object recognition result given reliability from the first determination unit, and the sensor data given reliability from the second determination unit, a third determination unit that determines the road conditions The road condition determination system characterized by including this.

11. A vehicle control method using a terminal device provided in a vehicle, A step of receiving sensor data and an object recognition result from an external terminal, A step of determining the reliability of the received sensor data and object recognition result, Based on the sensor data acquired from the sensor provided in the vehicle, the object recognition result based on the sensor data acquired from the sensor provided in the vehicle, and the sensor data and object recognition result given reliability from the determination step, a step of determining the road conditions, A step of controlling vehicle travel based on the determined road conditions To be executed by the terminal device, The vehicle control method characterized by this.

Images