Method for improving object recognition performance of infrastructure sensor according to environmental changes

Abstract

The present invention relates to a method for improving object recognition performance of an infrastructure sensor according to environmental changes. According to the present invention, environmental changes such as fog, rainfall, snowfall, and the like are recognized in real time, and object recognition criteria are reset according to the environmental changes, thereby improving object recognition accuracy and supporting safe driving of surrounding vehicles.

Description

Method to improve object recognition performance of infrastructure sensors in response to environmental changes

[0001] The present invention relates to a method for improving the object recognition performance of an infrastructure sensor in response to environmental changes, and more specifically, to a technology that enables the improvement of object recognition accuracy by resetting object recognition criteria according to environmental changes such as weather.

[0002] In the fields of smart cars and autonomous vehicles, object recognition is the most fundamental yet essential technical element for safety. In other words, it involves recognizing objects such as other vehicles, pedestrians, or obstacles around the vehicle to enable longitudinal control for acceleration or deceleration, or lateral control for avoidance or lane changes.

[0003] In this case, object recognition is not performed solely by the vehicle; it can be performed by infrastructure sensors and then communicated to surrounding vehicles via V2I (Vehicle to Infrastructure, wireless communication between vehicles and infrastructure) or made available for verification by the control system.

[0004] Methods for recognizing objects include analyzing video captured by a camera and using environmental sensors such as radar or lidar.

[0005] Among these, LiDAR refers to a device that emits high-power pulsed lasers with strong directional properties and receives the light reflected back from surrounding objects, enabling precise measurement of the distance, direction, shape, and other characteristics of the target object.

[0006] Figure 1 is a diagram conceptually illustrating an example of object recognition using LiDAR. LiDAR scans the surroundings to recognize objects through signals reflected from them, and by obtaining high-precision data in the form of a point cloud (Point Cloud Data), it can extract three-dimensional shape data that reflects not only the distance to the object but also its width and height.

[0007] However, due to the characteristics of LiDAR, even if the same object is scanned from the same direction, as can be seen from the graph shown in Fig. 2, the point cloud data (PCD) of the object decreases as the distance to the object increases, and increases as the distance decreases. Therefore, a standard is set in advance so that an object is recognized only when the scanned object PCD for a single object reaches a specific number.

[0008] For example, if the object recognition standard PCD is set to 100, if a specific object is very far from the infrastructure sensor (10), the PCD for that object may be less than 100 even if it is scanned via LiDAR, and in this case, the object is not recognized as an object because the significance of detection is reduced at the location where the infrastructure sensor (10) is installed. On the other hand, if the distance to the object becomes closer, the PCD will also increase, and once the standard point of 100 is reached, the object is recognized as a meaningful object from that point onward, making tracking possible.

[0009] However, due to the nature of LiDAR, it is inevitably affected by weather conditions. Specifically, in foggy, rainy, or snowy conditions, lasers emitted by the LiDAR may scatter or reflect off water particles in the air, which can interfere with object recognition. Of course, unlike PCDs, which focus on objects concentrated in a specific area, the signals reflected back from water particles are irregularly dispersed. Therefore, although signals reflected from water particles are not recognized as objects, this data is clearly noise and thus hinders the recognition of objects such as vehicles or pedestrians; consequently, technology to resolve this issue is urgently required.

[0010] Meanwhile, technologies related to object detection include Korean Patent Publication No. 10-2022-0150164 (November 10, 2022, 'Object detection device, object detection method and autonomous vehicle').

[0011] The present invention was devised to solve the problems of the prior art as described above, and aims to provide a technology that improves object recognition accuracy by recognizing environmental changes such as fog, rain, and snowfall in real time and resetting object recognition criteria according to environmental changes, thereby enabling notification to surrounding vehicles or control.

[0012] A method for improving the object recognition performance of an infrastructure sensor according to the present invention for achieving the above objective comprises: (a) a step of confirming information regarding an environment detection area where the presence of an object around the infrastructure sensor is not confirmed; (b) a step of performing environment detection through a sensor unit on the environment detection area confirmed in step (a); and (c) a step of resetting a criterion for whether an object detected by the sensor unit can be recognized as an object according to the result of the environment detection in step (b).

[0013] Here, the environment detection area identified in step (a) above may be a predetermined area in a direction greater than a certain height (or certain angle) where no fixed object exists in the surrounding direction where the infrastructure sensor is installed.

[0014] In addition, step (b) performs a scan through the sensor unit on the environment detection area confirmed through step (a) to determine the number of data reflected from the environment detection area, and step (c) increases the number of object recognition criteria PCDs in proportion to the number of reflected data from the environment detection area confirmed in step (b), thereby allowing the object to be recognized as an object when it becomes relatively close when noise is generated due to environmental changes.

[0015] Step (c) above may transmit a warning signal to surrounding vehicles or the control system when the number of object recognition criteria PCDs reaches a limit.

[0016] According to the method for improving the object recognition performance of an infrastructure sensor according to the present invention, an empty area above a certain height where it is presumed that no objects exist around the infrastructure sensor is set as an environment detection area, and by periodically scanning the environment detection area to check for the presence of reflection data, conditions such as rainfall can be checked in real time. Subsequently, by allowing the object recognition criteria to be reset according to the degree of rainfall, it is possible to accurately detect and track objects when the distance to them becomes slightly closer during bad weather, thereby supporting the safe operation of surrounding vehicles.

[0017] FIG. 1 is a diagram conceptually illustrating an example of recognizing an object through LiDAR in an infrastructure sensor.

[0018] FIG. 2 is a graph for explaining how point cloud data (PCD) of a specific object changes according to the distance to the object.

[0019] FIG. 3 is a block diagram illustrating an object recognition performance improvement system for an infrastructure sensor according to an embodiment of the present invention.

[0020] FIG. 4 is a flowchart illustrating a method for improving the object recognition performance of an infrastructure sensor according to an embodiment of the present invention.

[0021] Fig. 5 is a diagram illustrating the environment detection area set in the infrastructure sensor.

[0022] Figure 6 is a graph to explain the change in data reflected from the environment detection area according to environmental changes.

[0023] FIG. 7 is a diagram illustrating an example of resetting object recognition criteria based on changes in data reflected from an environment detection area.

[0024]

[0025] [Explanation of the symbol]

[0026] 10: Infrastructure Sensors

[0027] 30: Sensor section

[0028] 50: Object Recognition Performance Improvement System

[0029] 53 : Environment detection area storage unit

[0030] 54 : Environmental Detection Department

[0031] 55 : Object recognition criteria setting section

[0032]

[0033] Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. However, some components unrelated to the gist of the invention may be omitted or compressed; nevertheless, omitted components are not necessarily unnecessary for the present invention and may be combined and used by those skilled in the art to which the present invention pertains.

[0034]

[0035] FIG. 3 is a block diagram illustrating an object recognition performance improvement system for an infrastructure sensor according to an embodiment of the present invention (hereinafter referred to as the "object recognition performance improvement system"). As shown in FIG. 3, the object recognition performance improvement system (50) according to an embodiment of the present invention is mounted on an infrastructure sensor (10). An infrastructure sensor (10) refers to a device or system that supports autonomous driving by being installed near a road signal, utility pole, bus stop, etc., using various environmental sensors to search the surroundings, processing the searched information, and transmitting it to a control center or transmitting it to surrounding vehicles via V2I (Vehicle to Infrastructure, wireless communication between a vehicle and infrastructure). Basically, such an infrastructure sensor (10) is provided with a sensor unit (30) capable of detecting objects and a communication means (not shown), but in this embodiment, the infrastructure sensor (10) additionally includes an object recognition performance improvement system (50).

[0036] The object recognition performance improvement system (50) includes an environment detection area storage unit (53), an environment detection unit (54), and an object recognition standard setting unit (55).

[0037] The environment detection area storage unit (53) stores information about the area where environment detection is to be performed at the location where the infrastructure sensor (10) is installed. The term "environment detection" here refers to the process of checking, for example, whether there is fog, rain, or snow. Additionally, the "environment detection area" refers to an area where no objects exist at all when viewed from the perspective of the infrastructure sensor (10), that is, an empty space, and a detailed explanation will be provided below.

[0038] The environment detection unit (54) performs environment detection for the environment detection area stored in the environment detection area storage unit (53) in conjunction with the sensor unit (30). Here, the sensor unit (30) may be a LiDAR sensor, and accordingly, the environment detection unit (54) performs the process of checking whether reflection data exists in the scan result of the environment detection area.

[0039] The object recognition standard setting unit (55) is provided to reset the object recognition standard according to the environment detection result of the environment detection unit (54). That is, if the reflection data is low as a result of scanning the environment detection area by the environment detection unit (54), the number of object recognition standard PCDs is lowered, and if the reflection data is high, the number of object recognition standard PCDs is increased.

[0040] Below, a method for improving object recognition performance by resetting object recognition criteria through the object recognition performance improvement system of the infrastructure sensor illustrated in FIG. 3 will be explained in detail through FIGs. 4 to 7.

[0041] FIG. 4 is a flowchart illustrating a method for improving the object recognition performance of an infrastructure sensor according to an embodiment of the present invention. First, the environment detection area stored in the environment detection area storage unit (53) of the object recognition performance improvement system (50) is described as follows. As previously explained, 'environment detection' in this embodiment refers to the process of checking for weather changes such as fog, rain, or snowfall. At this time, environment detection is also performed through the scanning process of the LiDAR, and for this to happen, an area where the probability of an object existing is very low must be designated as the environment detection area.

[0042] FIG. 5 briefly illustrates the situation near the infrastructure sensor. As can be seen from FIG. 5, various fixed objects such as buildings, signs, curbs, lanes, traffic signals, and utility poles will exist near the infrastructure sensor (10). In addition to fixed objects, moving objects such as vehicles and pedestrians will also frequently exist on the ground.

[0043] Therefore, an area where the likelihood of an object existing is very low should be an area free of any signs or buildings, and it is also desirable that it be an area above a certain height (or angle) where the likelihood of vehicles or pedestrians appearing is low.

[0044] Since the infrastructure sensor (10) is fixedly installed at a specific location, after installing the infrastructure sensor (10), the operator determines the environment detection area during the initial setup phase and stores it in the environment detection area storage unit (53). Preferably, the environment detection area can be set to a predetermined area of ​​a certain height (angle) or higher among the empty areas between buildings. In this case, no objects will be detected in that area unless a new building is constructed.

[0045] In this state, where the environment detection area is stored in the environment detection area storage unit (53) through the initial setting process, the environment detection unit (54) checks the environment detection area stored in the environment detection area storage unit (53) at regular intervals. <s405>Perform water environment detection <s410>That is, the environment detection area, which is set to be an area above a certain height where no buildings or signs exist, is scanned through the sensor unit (30).

[0046] The sensor unit (30) may be a LiDAR sensor. LiDAR refers to a device that emits high-power pulsed lasers with strong directional properties instead of radio waves, and receives the light reflected back from surrounding objects to precisely measure the distance and direction to the objects. LiDAR may include an optical unit such as a lens, a laser emitting / receiving unit, a laser driving unit, and a processor that processes laser signals. By using LiDAR, high-precision data in the form of a point cloud (Point Cloud Data, PCD), which is a collection of points, can be obtained, and three-dimensional points reflecting width, distance, and height can be gathered together to extract shape data of objects. In addition, by analyzing the scan data of the LiDAR, relative coordinates for a specific scanned point can also be extracted.

[0047] At this time, the environment detection unit (54) is not intended to detect objects through a general LiDAR scan, but rather to detect the environment. That is, if an area where nothing is expected to exist is scanned with LiDAR, no reflection data should be confirmed in the scan results. However, if environmental changes such as weather occur, reflection data may be confirmed when scanning the environment detection area, which is an empty area.

[0048] Figure 6 is a graph illustrating the change in data reflected from the environmental detection area according to environmental changes. Referring to Figure 6, on a clear day with no rain, when the empty area (environmental detection area) is scanned by LiDAR, there will be no reflected data returning.

[0049] However, in foggy, rainy, or snowy conditions, scanning the empty environmental detection area will reveal data reflected from water particles in the air, and as the amount of rain increases, the amount of data reflected from that area will increase proportionally.

[0050] Of course, unlike PCD for objects that are concentrated in a specific area, signals reflected back from water particles are irregularly dispersed and are therefore not recognized as objects. However, data reflected from raindrops is noise data. Consequently, in such environments, it can also affect the process of detecting objects below a certain height.

[0051] In other words, if there is noise, it will be difficult to detect objects at a distance. To this end, the object recognition standard setting unit (55) resets the standard for recognizing an object as an object for an object detected by the sensor unit (30) based on the environment detection result of the environment detection unit (54).

[0052] Referring briefly to FIG. 2, when scanning the surroundings with a LiDAR, if the distance to an object is far, the point cloud data (PCD) of that object decreases, and if the distance decreases, the point cloud data of the object increases. Therefore, a criterion is set in advance so that an object is recognized only when the scanned object PCD for a single object reaches a specific number. FIG. 2 illustrates an example where the object recognition criterion PCD is set to 100.

[0053] However, if the environment detection unit (54) scans the empty area (environment detection area) set in the environment detection area storage unit (53) in conjunction with the sensor unit (30), but rain (or fog, snow) falls and the number of environment detection area reflection data is confirmed to be more than a certain number, the data reflected by the raindrops will act as noise and thus affect the object data detected through the sensor unit (30). Therefore, the object recognition standard setting unit (55) resets the object recognition standard in correspondence with the number of environment detection area reflection data confirmed through the environment detection unit (54). <s415>does.

[0054] FIG. 7 is a diagram illustrating an example of resetting object recognition criteria based on changes in data reflected from an environment detection area.

[0055] To explain the example in Fig. 7, if no reflection data is detected in the environment detection area as a result of detection by the environment detection unit (54), the object recognition standard setting unit (55) maintains the object recognition standard PCD at 100, which is the initial standard. In other words, it means that an object will be recognized as such only when 100 object PCDs for a specific object are confirmed, and since it is a clear day, an object that is detected from a considerable distance will also be recognized as an object.

[0056] On the other hand, if a small amount of rain falls and the reflection data for the environment detection area is confirmed to be 100, the object recognition standard setting unit (55) resets the object recognition standard PCD to 200. Therefore, when the LiDAR, which is the sensor unit (30), scans later, it is recognized as an object only when the object PCD for a specific object is confirmed to be 200. In other words, in a rainy situation, the reflection data caused by raindrops acts as noise in the LiDAR's scan data, so the object is recognized as an object and tracking is made possible when it gets a little closer. In the same way, if the amount of rain increases and the reflection data for the environment detection area is confirmed to be 400, the object recognition standard setting unit (55) resets the object recognition standard PCD to 500. In other words, it is to recognize the object only when it gets closer.

[0057] Meanwhile, in a situation where heavy rain is pouring down to the point where visibility is poor, the amount of reflected data in the environment detection area may increase significantly. As the amount of rain increases, the number of reflected data confirmed in the environment detection area will also increase, and consequently, the object recognition standard PCD will also reach a limit. For example, in a heavy rain situation where 900 pieces of reflected data are confirmed in the environment detection area, the object recognition standard setting unit (55) sets the object recognition standard PCD to 1000 pieces and simultaneously issues a warning to the control system or nearby vehicles to indicate that object detection is possible only when the object is very close, thus indicating that it is dangerous.

[0058] That is, it is possible to output control signals that encourage manual driving due to heavy rain conditions, enable monitoring by an administrator in the control system, or recommend increasing the proportion of object detection using environmental sensors other than lidar, such as radar, in autonomous vehicles.

[0059] Meanwhile, the graph shown in FIG. 7 is merely an example, and the number of object recognition standard PCDs reset by the object recognition standard setting unit according to the number of reflection data confirmed in the environment detection area can be precisely set based on what has been learned in advance in a safe situation.

[0060]

[0061] As explained above, in the present invention, an empty area above a certain height where it is believed that no objects exist around the infrastructure sensor (10) is set as an environment detection area, and by periodically scanning the environment detection area to check whether reflection data exists, conditions such as rainfall can be checked in real time. Subsequently, by resetting the object recognition criteria according to the degree of rainfall, it is possible to accurately detect and track objects when the distance to them becomes slightly closer during bad weather, thereby supporting surrounding vehicles to drive safely.

[0062]

[0063] The preferred embodiments of the present invention described above are disclosed for illustrative purposes only, and those skilled in the art with ordinary knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes, and additions should be deemed to fall within the scope of the claims of the present invention.

Claims

1. Step (a) of confirming information about an environment detection area where the presence of objects around an infrastructure sensor is not confirmed; Step (b) of performing environmental detection through a sensor unit on the environmental detection area identified in step (a) above; and A method for improving object recognition performance of an infrastructure sensor, characterized by including: (c) a step of resetting the criteria for whether an object detected by the sensor unit can be recognized as an object based on the environmental detection result of the above step (b).

2. In Paragraph 1, A method for improving object recognition performance of an infrastructure sensor, characterized in that the environment detection area identified in step (a) above is a predetermined area in a direction greater than a certain height (or certain angle) where no fixed object exists in the surrounding direction where the infrastructure sensor is installed.

3. In Paragraph 1, The above step (b) involves performing a scan through the sensor unit on the environment detection area identified through the above step (a) to determine the number of data reflected from the environment detection area, and A method for improving the object recognition performance of an infrastructure sensor, characterized in that step (c) increases the number of object recognition criteria PCDs in proportion to the number of environment detection area reflection data confirmed in step (b), so that when noise is generated due to environmental changes, an object is recognized as an object when it is relatively close.

4. In Paragraph 3, The above step (c) is a method for improving the object recognition performance of an infrastructure sensor, characterized by transmitting a warning signal to a surrounding vehicle or control system when the number of object recognition criteria PCDs reaches a limit.

Images