Method for improving object recognition performance of autonomous vehicle according to environmental changes

Abstract

The present invention relates to a method for improving the object recognition performance of an autonomous vehicle according to environmental changes. According to the present invention, environmental changes such as fog, rainfall, or snowfall are recognized in real time, and object recognition criteria are reset according to environmental changes, and thus object recognition accuracy is improved to ensure safe driving.

Description

A method for improving object recognition performance of autonomous vehicles according to environmental changes.

[0001] The present invention relates to a method for improving object recognition performance of an autonomous vehicle according to environmental changes, and more specifically, to a technology for improving object recognition accuracy by resetting object recognition criteria according to changes in the driving environment, such as weather.

[0002] In the fields of smart and autonomous vehicles, object recognition is a fundamental yet essential technological element for safety. It recognizes objects around the vehicle, such as other vehicles, pedestrians, and obstacles, and applies longitudinal control to accelerate or decelerate, or lateral control to avoid collisions or change lanes.

[0003] There are two ways to recognize objects: analyzing images captured by a camera and using environmental sensors such as radar or lidar.

[0004] Among these, lidar is a device that can precisely measure the distance, direction, shape, etc. to a target object by emitting a high-power pulse laser with strong straightness and receiving the light that is reflected back from the surrounding target object.

[0005] Figure 1 is a conceptual diagram illustrating an example of object recognition using LiDAR. LiDAR scans its surroundings and recognizes objects through signals reflected from them. By acquiring high-precision data in the form of a point cloud (Point Cloud Data, PCD), it can extract three-dimensional shape data that reflects not only the distance to the object but also its width and height.

[0006] However, due to the nature of LiDAR, even when scanning the same object from the same direction, as can be seen in the graph shown in Fig. 2, the farther the distance to the object, the smaller the point cloud data (PCD) of the object. As the distance gets closer, the point cloud data of the object increases. Therefore, a standard is set in advance so that an object is recognized as such only when the PCD of the scanned object reaches a certain number.

[0007] For example, if the object recognition standard PCD is set to 100, if a specific object is very far from the vehicle (10), the PCD for the object may be less than 100 even if it is scanned by lidar. In this case, it is not recognized as an object because it does not directly affect autonomous driving. On the other hand, if the distance to the object gets closer during driving, the PCD will also increase, and only when the standard point of 100 is reached, the object is recognized as a meaningful object and tracking becomes possible.

[0008] However, due to the nature of LiDAR, it is inevitably affected by weather. In fog, rain, or snow, the laser emitted from LiDAR can be scattered or reflected by water particles in the air, hindering object recognition. Unlike PCD, which concentrates signals from objects within a specific area, these signals are irregularly distributed. Therefore, while signals reflected from water particles are not recognized as objects, this data is clearly noise, hindering the recognition of objects such as vehicles and pedestrians. Technology to address this issue is urgently needed.

[0009] Meanwhile, technologies related to object detection in autonomous vehicles include Korean Patent Publication No. 10-2022-0150164 (November 10, 2022, 'Object detection device, object detection method, and autonomous vehicle').

[0010] The present invention has been devised to solve the problems of the prior art as described above, and its purpose is to provide a technology that can improve object recognition accuracy and ensure safe driving by recognizing environmental changes such as fog, rain, and snowfall in real time and resetting object recognition criteria according to the environmental changes.

[0011] In order to achieve the above object, the method for improving the object recognition performance of an autonomous vehicle according to the present invention comprises: (a) a step of confirming the current location of the vehicle through a GPS module; (b) a step of extracting precision map data near the current location of the vehicle confirmed in step (a) through a precision map DB; (c) a step of determining an environment detection area in which the existence of an object is not confirmed in the precision map data extracted in step (b); (d) a step of performing environment detection through a sensor unit for the environment detection area determined in step (c); and (e) a step of resetting a standard for whether an object detected by the sensor unit can be recognized as an object based on the environment detection result of step (d).

[0012] Here, the step (c) can determine a predetermined area in a direction greater than a certain height (or a certain angle) as an environment detection area even though point cloud data is not confirmed on the precision map data extracted in the step (b).

[0013] In addition, the step (d) performs a scan through the sensor unit for the environment detection area determined in the step (c) to check the number of data reflected in the environment detection area, and the step (e) increases the number of object recognition reference PCDs in proportion to the number of environment detection area reflection data checked in the step (d), so that when noise occurs due to environmental change, the object can be recognized as an object when it is relatively close.

[0014] In addition, the above step (e) can generate a warning when the number of object recognition criteria PCDs reaches a limit, or output a control signal to increase the proportion of use of an environmental sensor of a different type from the sensor unit.

[0015] According to the method for improving the object recognition performance of an autonomous vehicle according to the present invention, after extracting precise map data around the current location of the vehicle, an empty area above a certain height where no objects are expected to exist is determined as an environment detection area, and the environment detection area is periodically scanned to check for the presence of reflection data, thereby enabling real-time detection of rainfall conditions, etc. Thereafter, by resetting the object recognition standard according to the degree of rainfall, in bad weather, it is possible to accurately detect and track objects when the distance to the object becomes slightly closer, thereby ensuring safe driving.

[0016] Figure 1 is a diagram conceptually illustrating an example of recognizing an object through lidar.

[0017] Figure 2 is a graph that explains how the point cloud data (PCD) of a specific object changes depending on the distance to the object.

[0018] Figure 3 is a block diagram illustrating a system for improving object recognition performance of an autonomous vehicle according to an embodiment of the present invention.

[0019] Figure 4 is a flowchart for explaining a method for improving object recognition performance of an autonomous vehicle according to an embodiment of the present invention.

[0020] Figure 5 is a diagram illustrating a process for determining an environment detection area from precision map data.

[0021] Figure 6 is a graph to explain the change in data reflected in the environment detection area according to environmental changes.

[0022] Figure 7 is a diagram illustrating an example of resetting the object recognition criteria according to changes in data reflected in the environment detection area.

[0023]

[0024] <Explanation of symbols>

[0025] 10: Your own vehicle

[0026] 20: GPS module

[0027] 30: Sensor section

[0028] 40: Precision Map DB

[0029] 50: Object Recognition Performance Improvement System

[0030] 51: Location Information Confirmation Department

[0031] 52: Precision Map Operation Department

[0032] 53: Environmental detection area determination unit

[0033] 54: Environmental Detection Department

[0034] 55: Object recognition criteria setting section

[0035] Hereinafter, preferred embodiments of the present invention will be described with reference to the attached drawings. Some components irrelevant to the gist of the invention will be omitted or compressed. However, these omitted components do not necessarily mean they are unnecessary for the present invention, and those skilled in the art can combine and use them.

[0036] FIG. 3 is a block diagram illustrating an object recognition performance improvement system for an autonomous vehicle according to an embodiment of the present invention (hereinafter referred to as the “object recognition performance improvement system”). As illustrated in FIG. 3, the object recognition performance improvement system (50) according to an embodiment of the present invention includes a location information confirmation unit (51), a precision map operation unit (52), an environment detection area determination unit (53), an environment detection unit (54), and an object recognition criterion setting unit (55).

[0037] The location information confirmation unit (51) is provided to periodically receive GPS coordinate information obtained from the GPS module (20) and confirm the current location of the vehicle (10).

[0038] The precision map operation unit (52) is provided to extract precision map data near the current location of the vehicle (10) in conjunction with the precision map DB (40).

[0039] The environment detection area determination unit (53) is provided to determine an area in which environment detection is to be performed from the precision map data extracted from the precision map operation unit (52) near the vehicle (10). The term "environment detection" used herein refers to the process of determining whether there is fog, rain, or snow, for example. In addition, the "environment detection area" refers to an area in the extracted precision map data where no objects exist at all, i.e., an empty space. A detailed explanation will be provided later.

[0040] The environment detection unit (54) performs environment detection for the environment detection area determined by the environment detection area determination unit (53) in conjunction with the sensor unit (30). Here, the sensor unit (30) may be a lidar sensor, and accordingly, the environment detection unit (54) performs a process of checking whether reflection data exists in the scan results of the environment detection area.

[0041] The object recognition standard setting unit (55) is provided to reset the object recognition standard according to the environment detection result of the environment detection unit (54). That is, if the environment detection unit (54) scans the environment detection area and the result shows that there is little reflection data, the number of object recognition standard PCDs is reduced, and if there is a lot of reflection data, the number of object recognition standard PCDs is increased.

[0042] Below, a method for improving object recognition performance by resetting the object recognition criteria through the object recognition performance improvement system illustrated in FIG. 3 will be described in detail through FIGS. 4 to 7.

[0043] Figure 4 is a flowchart illustrating a method for improving object recognition performance of an autonomous vehicle according to an embodiment of the present invention. First, the location information confirmation unit (51) of the object recognition performance improvement system (50) periodically confirms the current location of the vehicle (10) through the GPS module (20). <s405>Afterwards, the precision map operation department (52) extracts the precision map data corresponding to the current location of the vehicle (10) obtained through the location information confirmation department (51) from the precision map DB (40). <s410>Do it.

[0044] The precision map stored in the precision map DB (40) can be expressed in detail down to the lane level, and further includes detailed information such as the center line of the lane, traffic lights, signs, road markings, road facilities, buildings, etc. The precision video image stored in the precision map DB (40) is in the form of a collection of numerous points (point cloud data, PCD), and each point has absolute coordinates such as latitude and longitude.

[0045] When the precision map operation unit (52) extracts precision map data near the current location of the vehicle (10), the environment detection area determination unit (53) determines an area to perform environment detection from the extracted precision map data. <s415>Do it.

[0046] As previously explained, "environmental detection" in this embodiment refers to the process of determining weather changes, such as fog, rain, or snowfall. This environmental detection is also performed through the lidar scanning process. To achieve this, an area with a very low probability of object presence must be designated as the environmental detection area.

[0047] Figure 5 is a diagram illustrating the process of determining an environment detection area from precision map data. In other words, it conceptually depicts precision map data near the current location of the vehicle (10) extracted through the precision map operation unit (52). As can be seen from Figure 5, the precision map data includes object data for buildings, signs, curbs, lanes, traffic signals, utility poles, etc. Therefore, by checking the precision map data extracted by the environment detection area determination unit (53), it is possible to determine in which direction objects such as buildings or signs exist.

[0048] Conversely, this means that it is possible to determine in which direction and which area there are no objects at all. In other words, the area between buildings or between a building and a sign, based on the current position of the vehicle (10), can be considered as an empty area. In other words, a certain area in the precision map data where point cloud data does not exist can be determined as an environment detection area.

[0049] However, an area without point cloud data in the precision map data should not be unconditionally determined as an environment detection area. In other words, if the precision map data determines that the area is empty due to the absence of point cloud data, fixed objects such as buildings or signs are likely not to be detected in real-time detection using the sensor unit (30). However, in areas below a certain height, objects that do not exist in the precision map data, such as other vehicles or pedestrians, may be detected through real-time detection using the sensor unit (30).

[0050] Therefore, the environment detection area determination unit (53) satisfies the condition that point cloud data does not exist in the extracted precision map data, while at the same time determining an area above a certain height (or above a certain angle) as an environment detection area. For example, in Fig. 5, since point cloud data is not confirmed in the area between the building located at the farthest left and the sign located to its right, this area is designated first, and then, among those areas, a certain area above a certain height (angle) is finally determined as an environment detection area. If the environment detection area determined in this way is scanned by the sensor unit (30), the possibility of unexpected objects such as vehicles or pedestrians being detected will be very low.

[0051] Afterwards, the environmental detection unit (54) performs environmental detection for the environmental detection area determined by the environmental detection area determination unit (53). <s420>That is, if point cloud data is not confirmed on the precision map data and an area above a certain height is determined as an environment detection area, a scanning process is performed through the sensor unit (30) in the direction of the environment detection area.

[0052] The sensor unit (30) may be a Lidar sensor. Lidar is a device that can precisely measure the distance and direction to a target object by emitting a high-power pulse laser with strong straightness instead of radio waves and receiving the light that is reflected from the surrounding target object. Lidar may include an optical unit such as a lens, a laser emitting / receiving unit, a laser driving unit, a processor that processes a laser signal, etc. By using Lidar, high-precision data (Point Cloud Data, PCD) in the form of a point cloud (point cloud) which is a collection of points can be secured, and three-dimensional points that reflect width, distance, and height can be collected to extract shape data of an object. In addition, by analyzing the scan data of Lidar, relative coordinates for a specific scanned point can also be extracted.

[0053] At this time, the environment detection unit (54) does not attempt to detect objects through a typical lidar scan, but rather detects the environment. In other words, an area where nothing is expected to exist in the precision map data is designated as the environment detection area. If this area is scanned with lidar, no reflection data should be confirmed in the scan results. However, if an environmental change, such as weather, occurs, reflection data may be confirmed when scanning the empty environment detection area.

[0054] Figure 6 is a graph illustrating changes in reflected data from the environment detection area depending on environmental changes. Referring to Figure 6, on a clear day with no rain, no point cloud data is visible on the precision map data. When a lidar scans an empty area (the environment detection area) above a certain height, no reflected data is returned.

[0055] However, if there is fog, rain, or snow, when scanning the environment detection area, which is an empty area, data reflected from water particles in the air will be confirmed, and as the amount of rain increases, the data reflected from that area will increase proportionally.

[0056] Of course, signals reflected from water particles are irregularly distributed, unlike PCDs for objects that are concentrated in a certain area, and thus are not recognized as objects. However, data reflected from raindrops is noise. Therefore, in these environments, it can affect the detection of objects below a certain height.

[0057] In other words, if there is noise, it will be difficult to detect distant objects. To this end, the object recognition criteria setting unit (55) resets the criteria for recognizing objects detected by the sensor unit (30) as objects based on the environmental detection results of the environmental detection unit (54). <s425>Do it.

[0058] Referring briefly to Figure 2, when scanning the surroundings with LiDAR, if the distance to the object is far, the point cloud data (PCD) of the object will decrease, and if the distance is close, the point cloud data of the object will increase. Therefore, a standard is set in advance so that an object is recognized as an object only when the scanned object PCD reaches a certain number. Figure 2 illustrates an example where the object recognition standard PCD is set to 100.

[0059] However, if the empty area (environmental detection area) is determined through the environment detection area determination unit (53), and the empty area is detected in conjunction with the sensor unit (30) in the environment detection unit (54), but it is raining (or fog, snow), and if a certain number of environmental detection area reflection data is confirmed, the data reflected by the raindrops will act as noise, and thus will affect the object data detected through the sensor unit (30). Therefore, the object recognition standard setting unit (55) resets the object recognition standard in response to the number of environmental detection area reflection data confirmed through the environment detection unit (54).

[0060] Figure 7 is a diagram illustrating an example of resetting the object recognition criteria according to changes in data reflected in the environment detection area.

[0061] In explaining the example of Fig. 7, if no reflection data is confirmed at all in the environment detection area as a result of the detection of the environment detection unit (54), the object recognition standard setting unit (55) maintains the object recognition standard PCD at the initially set standard of 100. In other words, if 100 object PCDs for a specific object are confirmed, it will be recognized as an object, and since it is a clear day, even objects confirmed from a considerable distance will be recognized as objects.

[0062] On the other hand, if a small amount of rain falls and the reflection data for the environment detection area is confirmed to be 100, the object recognition standard setting unit (55) resets the object recognition standard PCD to 200. Therefore, when the sensor unit (30) LiDAR scans later, an object is recognized only when 200 object PCDs for a specific object are confirmed. In other words, in a rainy situation, the reflection data due to raindrops acts as noise in the LiDAR scan data, so that the object is recognized as an object and tracking is possible only when it gets a little closer. In the same way, if the amount of rain increases and 400 reflection data for the environment detection area are confirmed, the object recognition standard setting unit (55) resets the object recognition standard PCD to 500. In other words, the object is recognized as an object only when it gets closer.

[0063] Meanwhile, in a situation where heavy rain is pouring down to the point where visibility is reduced, the amount of reflection data in the environment detection area may increase significantly. As the amount of rain increases, the number of reflection data confirmed in the environment detection area will also increase, and eventually, the object recognition standard PCD will also reach its limit. For example, in a situation of heavy rain where 900 reflection data are confirmed in the environment detection area, the object recognition standard setting unit (55) sets the object recognition standard PCD to 1000 and simultaneously issues a warning through a separate warning means to inform the driver that object detection is possible only when the object is very close, which is dangerous.

[0064] In other words, because it is a heavy rain situation, manual driving can be induced, or because it is difficult to detect objects using lidar, a control signal can be output to increase the proportion of object detection using environmental sensors other than lidar, such as radar.

[0065] Meanwhile, the graph shown in Fig. 7 is only an example, and the number of object recognition criteria PCDs reset by the object recognition criteria setting unit according to the number of reflection data confirmed in the environment detection area can be precisely set according to what has been learned in advance in a safe situation.

[0066] As described above, in the present invention, after extracting precise map data around the current location of the magnetic vehicle (10), an empty area above a certain height where no object is expected to exist is determined as an environment detection area, and the environment detection area is periodically scanned to check whether reflection data exists, thereby enabling real-time confirmation of rainfall conditions, etc. Thereafter, by resetting the object recognition standard according to the degree of rainfall, in bad weather, it is possible to accurately detect and track an object when the distance to the object becomes slightly closer, thereby ensuring safe driving.

[0067] The above preferred embodiments of the present invention are disclosed for the purpose of illustration, and those skilled in the art with ordinary knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes, and additions should be considered to fall within the scope of the claims of the present invention.

Claims

1. Step (a) of checking the current location of the vehicle through the GPS module; Step (b) of extracting precision map data near the current location of the vehicle confirmed in step (a) through the precision map DB; Step (c) of determining an environmental detection area in which the existence of an object is not confirmed from the precision map data extracted in step (b); Step (d) of performing environmental detection through a sensor unit for the environmental detection area determined in step (c) above: and A method for improving object recognition performance of an autonomous vehicle, characterized by including a step (e) of resetting a criterion for whether an object detected by the sensor unit can be recognized as an object based on the result of the environmental detection of the step (d).

2. In paragraph 1, A method for improving object recognition performance of an autonomous vehicle, characterized in that the step (c) above determines a predetermined area in a direction higher than a certain height (or at a certain angle) while point cloud data is not confirmed on the precision map data extracted in the step (b) above as an environment detection area.

3. In paragraph 1, The above step (d) checks the number of data reflected in the environment detection area determined in the above step (c) by performing a scan through the sensor unit. The above step (e) is a method for improving object recognition performance of an autonomous vehicle, characterized in that it increases the number of object recognition reference PCDs in proportion to the number of environmental detection area reflection data confirmed in the above step (d), thereby processing the object so that it can be recognized as an object when noise occurs due to environmental change and the object becomes relatively close.

4. In paragraph 3, The above step (e) is a method for improving object recognition performance of an autonomous vehicle, characterized in that it generates a warning when the number of object recognition criterion PCDs reaches a limit or outputs a control signal to increase the use of an environmental sensor of a different type from the sensor unit.

Images