Apparatus and method for identifying a travel lane based on multiple sensors

Abstract

The present application relates to an apparatus and method for identifying a travel lane based on a plurality of sensors. An apparatus for identifying a travel lane based on a plurality of sensors is provided. The apparatus includes a first sensor configured to calculate road information, a second sensor configured to calculate moving obstacle information, a third sensor configured to calculate movement information of a vehicle, and a controller configured to remove the moving obstacle information from the road information to extract only road boundary data, accumulate the road boundary data to calculate a plurality of pieces of candidate position information about the vehicle based on the movement information, and select final candidate position information from the plurality of pieces of candidate position information.

Description

TECHNICAL FIELD

[0001] The disclosure relates to an apparatus and method for identifying a travel lane based on a plurality of sensors. BACKGROUND

[0002] Technologies using an expensive global navigation satellite system (GNSS) / inertial navigation system (INS) are being researched to implement a demonstration automatic vehicle. Specifically, in the case of automatic travel, a side direction position recognition is performed using a camera with an INS and landmarks on a road, such as road markings, lanes, traffic signal lights, and road signs.

[0003] However, in the case of automatic travel, lateral direction position recognition is important. For this, a technology for comprehensively recognizing a travel lane and an in-lane position is required. In addition, in order to recognize a travel lane, road boundary detection is required.

[0004] However, there is an inaccuracy problem in road boundary detection due to weather, obstacle types, and sensor characteristics.

[0005] In addition, it can be undesirable to apply GNSS / INS to an automatic vehicle for mass production. SUMMARY

[0006] The disclosure relates to an apparatus and method for identifying a travel lane based on a plurality of sensors, which can improve performance of accurate positioning on a road by a technology for road boundary detection and travel lane recognition based on fusion of a low-cost global positioning system (GPS), a plurality of environment recognition sensors, and an accurate map.

[0007] Other objects and advantages of the disclosure can be understood by the following description and will be apparent from the description to those skilled in the art to which the disclosure pertains. In addition, it will be obvious to those skilled in the art that the objects and advantages of the disclosure can be achieved by the claimed method and combinations thereof.

[0008] In some forms of the disclosure, an apparatus for identifying a travel lane based on a plurality of sensors is provided, which improves performance of accurate positioning on a road by a technology for road boundary detection and travel lane recognition based on fusion of a low-cost GPS, a plurality of environment recognition sensors, and an accurate map.

[0009] An apparatus for identifying a travel lane includes a first sensor configured to calculate road information, a second sensor configured to calculate moving obstacle information, a third sensor configured to calculate movement information of a vehicle, and a controller configured to remove the moving obstacle information from the road information to extract only road boundary data, accumulate the road boundary data to calculate a plurality of pieces of candidate position information about the vehicle based on the movement information, and select one piece of the plurality of pieces of candidate position information as final candidate position information.

[0010] In this case, the road boundary data can be extracted from a plurality of previous frames using the movement information and accumulated in a current frame.

[0011] Further, the plurality of pieces of candidate position information can be calculated based on current vehicle position information using travel lane information obtained using a fourth sensor or preset accurate map lane information.

[0012] Further, the plurality of pieces of candidate position information can be calculated based on a grid formed by dividing a predetermined region of interest placed on a map road boundary of the accurate map lane information at regular intervals.

[0013] Further, the grid can be formed of a plurality of lateral directions and a plurality of lateral directions, and all of the plurality of lateral directions and the plurality of lateral directions can be set as a plurality of pieces of lateral candidate position information and a plurality of pieces of side candidate position information.

[0014] Further, the grid can be formed of a plurality of heading angles, the plurality of heading angles can be generated by dividing the region of interest at regular intervals based on a center point of the vehicle, and all of the plurality of heading angles can be set as candidate heading angle information.

[0015] Further, the number of the plurality of pieces of candidate position information can be obtained by multiplying the number of side position information, the number of lateral position information, and the number of heading angle position information.

[0016] Further, the plurality of pieces of candidate position information can include only a preset number of pieces of lateral candidate position information based on a lateral direction, considering a position error of the vehicle.

[0017] Further, a position of the vehicle existing within one road boundary can be determined according to a ratio of a side offset calculated between the travel lane and the center of the vehicle.

[0018] Further, a preset number of pieces of lateral candidate position information for each lane can be determined according to the ratio of the side offset, based on a lane position and a lane number identified in a region of interest preset in the accurate map lane information.

[0019] Further, a heading angle of each of the plurality of pieces of lateral candidate position information of the preset number can be calculated using a difference between a first average value, which is an average value of a heading angle of a left lane and a heading angle of a right lane in front of the vehicle identified by the second sensor, and a second average value, which is an average value of a heading angle of a left lane and a heading angle of a right lane of the plurality of pieces of lateral candidate position information of the preset number calculated by the precise map lane information.

[0020] Further, the controller can match the precise map road boundary information searched based on the plurality of pieces of candidate position information with the accumulated road boundary data to assign a matching degree score to each of the plurality of pieces of candidate position information, and select a lane assigned with a highest matching degree score among the final candidate position information as the travel lane.

[0021] Further, the controller can convert the precise map road boundary information and the accumulated road boundary data into a coordinate system formed of a Y-axis coordinate and an X-axis coordinate, and compare only values of the X-axis coordinates based on an index of the Y-axis coordinates to calculate the matching degree score.

[0022] Further, the matching degree score can be calculated using a number of data points located in the vicinity of the precise map road boundary information, the data points can be generated by the first sensor, and the number of data points can be based on a number of left points, a number of right points, and a number of points between road boundaries of the precise map road boundary information.

[0023] Further, when the number of previous frames is greater than a preset value, the controller can remove one most distant previous frame and add one current frame to maintain the accumulated number of frames constant.

[0024] Further, the number of accumulated frames can be variably set according to a vehicle speed or a moving distance of the vehicle.

[0025] Further, the first sensor can be a distance sensor, the second sensor can be a vision sensor, the third sensor can be a motion sensor, and the fourth sensor can be a global positioning system (GPS) sensor or an image sensor.

[0026] Further, the controller can apply a preset invalid determination condition to the final candidate position information to determine whether the final candidate position information is valid.

[0027] Further, the invalidity determination condition can include any one of whether a matching degree score of the final candidate position information is less than a preset first matching degree comparison condition, whether a remaining amount of the distance sensor existing in the travel lane is greater than a preset second matching degree comparison condition in the final candidate position information, and a determination condition that determines whether left and right lane directions in front of the vehicle are parallel to each other according to a difference between the left and right lane directions in the final candidate position information, and determines whether the vehicle deviates from a lane width by comparing a difference between a left lane side offset and a right lane side offset based on a center of the vehicle with a preset road width.

[0028] In some forms of the present disclosure, there is provided a method for identifying a travel lane based on a plurality of sensors, the method including: a first information calculation operation of calculating road information and moving obstacle information by a first sensor and a second sensor; a second information calculation operation of calculating movement information of a vehicle by a third sensor; and a selection operation of removing the moving obstacle information from the road information to extract only road boundary data by a controller, accumulating the road boundary data to calculate a plurality of pieces of candidate position information about the vehicle based on the movement information, and selecting one of the plurality of pieces of candidate position information as final candidate position information.

[0029] Further, the selection operation can include calculating the plurality of pieces of candidate position information by the controller using accurate map lane information that is preset based on at least one of travel lane information obtained using a fourth sensor and current vehicle position information obtained from a position information acquisition section.

[0030] Further, the selection operation can include matching accurate map road boundary information searched based on the plurality of pieces of candidate position information with the accumulated road boundary data by the controller to assign a matching degree score to each of the plurality of pieces of candidate position information, and selecting a lane to which a highest matching degree score is assigned in the final candidate position information as the travel lane.

[0031] Further, the selection operation can include converting the accurate map road boundary information and the accumulated road boundary data into a coordinate system formed of a Y-axis coordinate and an X-axis coordinate by the controller, and comparing only values of the X-axis coordinates based on an index of the Y-axis coordinate to calculate the matching degree score.

[0032] Further, the selection operation can include applying an invalidity determination condition preset for the final candidate position information to the final candidate position information by the controller to determine whether the final candidate position information is valid. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a block diagram illustrating a configuration of an apparatus for identifying a travel lane based on a plurality of sensors in some forms of the present disclosure.

[0034] Figure 2 is a detailed block diagram showing the configuration of the controller. Figure 1

[0035] Figure 3 is a flowchart showing the configuration of a process for identifying a travel lane based on a plurality of sensors in some forms of the present disclosure.

[0036] Figure 4 is a graph for describing the operation of the output of the distance sensor. Figure 3

[0037] Figure 5 is a graph for describing the operation of the output of the distance sensor. Figure 3

[0038] Figure 6 is a graph for describing the operation of the candidate position generation. Figure 3

[0039] Figure 7 is a conceptual diagram showing an example of generating a heading angle candidate within a predetermined range according to Figure 6

[0040] Figure 8 is a conceptual diagram showing another example of generating a candidate position according to Figure 6

[0041] Figure 9 is a conceptual diagram for describing the search of the direction of the identified travel lane according to Figure 8

[0042] Figure 10 is a conceptual diagram for describing the search of the heading angle of the accurate map according to Figure 8

[0043] Figure 11 is a flowchart of calculating the heading angle of each candidate position according to Figure 8 to Figure 10

[0044] Figure 12 is a graph showing one example for describing the operation of selecting the best candidate position shown in Figure 3

[0045] Figure 13 is a graph showing another example for describing the operation of selecting the best candidate position shown in Figure 3

[0046] Figure 14 is a graph showing one example for describing the operation of selecting the best candidate position shown in Figure 3 ​​​​​​​​​​​The diagram shows the score distribution of correctly identified final candidate location information and the score distribution of incorrectly identified final candidate location information, which is used to set the matching degree.

[0047] Figure 15 It is used to describe Figure 3 The diagram illustrates the concept of determining an invalid operation. Detailed Implementation

[0048] This disclosure may be modified and taken in various forms, and therefore, particular forms will be shown and described in detail in the accompanying drawings. However, these forms should not be construed as limiting this disclosure to the specific form, but rather as modifications, equivalents, or substitutions included within the spirit and technical scope of this disclosure.

[0049] In describing each figure, similar reference numerals are assigned to similar components. Although the terms "first" and "second," etc., may be used herein to describe various components, these components should not be limited by these terms. The terms are used only to distinguish one component from another.

[0050] For example, without departing from the scope of this disclosure, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term "and / or" includes a combination of multiple related listed items or any one of multiple related listed items.

[0051] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0052] General terms defined in the dictionary should be interpreted as having a meaning consistent with the context of the prior art and should not be interpreted as having an idealized or overly formal meaning, unless explicitly defined in this disclosure.

[0053] In the following, devices and methods for identifying driving lanes based on multiple sensors, in some forms of this disclosure, will be described in detail with reference to the accompanying drawings.

[0054] Figure 1 This is a block diagram illustrating the construction of a device 100 for identifying driving lanes based on multiple sensors, as shown in some forms of this disclosure. (Refer to...) Figure 1The device 100 for identifying a travel lane can include a controller 110, a position information acquisition part 120 for acquiring current position information of a vehicle, a sensor system 130 for calculating information about a road on which the vehicle travels, information about a moving obstacle moving on the road, and movement information of a vehicle traveling on the road, and a storage 140 for storing a plurality of pieces of information. In addition, the device 100 for identifying a travel lane can further include a power supply part 150 for supplying power to the above components, and an output part 160 for outputting information. In addition, in some forms of the disclosure, Figure 1 The components shown in FIG. 1 are mainly described as components of a device for identifying a travel lane based on a plurality of sensors, except that other components are omitted.

[0055] The controller 110 transmits and receives signals to and from the components 120 to 160, and performs a function of controlling the components 120 to 160. In addition, the controller 110 performs a function of removing moving obstacle information from road information based on movement information to extract only road boundary data, a function of accumulating road boundary data to calculate a plurality of pieces of candidate position information about a vehicle, and a function of selecting one from among the plurality of pieces of candidate position information.

[0056] The position information acquisition part 120 performs a function of acquiring current position information of a vehicle. Accordingly, the position information acquisition part 120 can be a global positioning system (GPS) receiver, but the disclosure is not limited thereto, and the position information acquisition part 120 can be an inertial measurement unit (IMU), a light detection and ranging (LiDAR), or a radio detection and ranging (RADAR). In addition, the IMU can include an accelerometer, a tachometer, etc.

[0057] The sensor system 130 can include a distance sensor 131 for sensing a road on which a vehicle travels and generating distance information about the road, a vision sensor 132 for generating a moving obstacle moving on the same road, a motion sensor 133 for sensing movement of the vehicle and generating movement information, and an image sensor 134 for sensing a lane of the road and generating travel lane information.

[0058] The distance sensor 131 can be an ultrasonic sensor, an infrared sensor, a time of flight (TOF) sensor, a laser sensor, a LiDAR, or a RADAR.

[0059] The vision sensor 132 refers to a sensor for recognizing and evaluating a moving target and a scene.

[0060] The motion sensor 133 refers to a motion recognition sensor for recognizing a motion and a position of an object. Accordingly, the motion sensor 133 can include a sensor for detecting a direction, a sensor for detecting movement, and a sensor for measuring a speed.

[0061] The image sensor 134 performs a function of capturing a lane on a general road. The image sensor 134 can be a charge-coupled device (CCD) sensor, a complementary metal-oxide semiconductor (CMOS) sensor, or the like.

[0062] The storage 140 performs a function of storing a program having an algorithm for recognizing a travel lane based on a plurality of sensors, data, software, or the like. The storage 140 can include at least one type of storage medium: a flash memory, a hard disk type memory, a multimedia card micro memory, a card type memory (e.g., a secure digital (SD) or extreme digital (XD) memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0063] The power supply 150 performs a function of supplying power to components. Accordingly, the power supply 150 can be a battery pack formed of a rechargeable battery cell, a lead-acid battery, or the like.

[0064] The output 160 performs a function of outputting a process of recognizing a travel lane to a screen. Also, the output 160 can output processed data to the controller 110. To this end, the output 160 can include a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic LED (OLED) display, a touch screen, a flexible display, a head-up display (HUD), or the like. The touch screen can be used as an input. Also, a sound system for inputting and / or outputting a voice, a sound, or the like can be configured.

[0065] Figure 2 is a detailed block diagram illustrating a configuration of the controller 110 shown in Figure 1 Figure 1 . Referring to Figure 2 , the controller 110 can include a collection 210 for collecting sensor data generated from the sensor system 130, an accumulator 220 for accumulating road boundary data extracted from a previous frame in a current frame using movement information of a vehicle, a calculation 230 for calculating a candidate position of a vehicle to be placed in each lane using travel lane information obtained based on position information and precise map lane information, and assigning a score to the candidate position to select a lane of a candidate position having a high score as a travel lane, and a determination 240 for performing a function of processing an exception that an abnormal situation that can occur in an actual road.

[0066] ​The collection section 210, the accumulator 220, the calculation section 230, and the determination section 240 can be implemented in software and / or hardware to perform the above-described functions. The hardware can be implemented with an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a processor, a microprocessor, another electronic unit, or a combination thereof, designed to perform the above-described functions. The software implementation can include software configured components (elements), object-oriented software configured components, class configured components, and work configured components, processes, functions, attributes, programs, subroutines, program code segments, drivers, firmware, microcode, data, databases, data structures, tables, arrays, and variables. The software and data can be stored in a memory and executed by a processor. The memory or the processor can employ various components known to those skilled in the art.

[0067] Figure 3 is a flowchart illustrating a configuration of a process for identifying a travel lane based on a plurality of sensors in some forms of the present disclosure. Referring to Figure 3 The collection section 210 of the controller 110 removes surrounding vehicles corresponding to moving obstacles from the output of the distance sensor 131 and extracts only road boundary data (S310). That is, moving obstacle information is removed from road information calculated by the distance sensor 131, and thus only road boundary information is extracted. To generate the moving obstacle information, the vision sensor 132 can be used.

[0068] Thereafter, the accumulator 220 of the controller 110 accumulates the road boundary data extracted from N (greater than 1) previous frames in the current frame using movement information of the vehicle obtained from the output of the motion sensor 133 (S320).

[0069] Then, the calculation section 230 of the controller 110 searches for lateral candidate position information of the host vehicle to be placed in each lane using accurate map lane information obtained based on travel lane information obtained from the image sensor 134 and current vehicle position information obtained from the position information acquisition section 120 (S330).

[0070] The accurate map lane information is stored in the storage 140 in advance. Alternatively, the accurate map lane information can be provided from the outside through communication. That is, since the position in which the vehicle is currently traveling is determined, the latest accurate map information corresponding to the determined position is used to search for the lateral candidate position information. To this end, the accurate map information can be stored in a database. Further, the lane information and the road boundary information can be included in the accurate map information.

[0071] Further, the accurate map lane information can be obtained based on the travel lane information acquired from the image sensor 134 and the current vehicle position information acquired from the position information acquisition section 120, or can be selectively obtained based on only one of the travel lane information and the current vehicle position information.

[0072] Thereafter, the calculation section 230 of the controller 110 matches the accurate map road boundary information obtained based on the candidate positions with the output of the road boundary information accumulated in S320 to give a score to each candidate position. Thus, the lane of the candidate position with the highest score is selected as the travel lane. When matching is performed at the position of the travel lane, the distance sensor data near the position of the accurate map road boundary can be detected as the road boundary.

[0073] Thereafter, the determination section 240 determines whether the selected optimal candidate position information is invalid using a function of processing an anomaly with respect to a specific situation that can occur in the actual travel lane (S350).

[0074] Thereafter, the determination section 240 displays the result of detecting the road boundary and recognizing the travel lane through the output section 160 (S360). The result information can be output as a combination of graphics, text, and voice.

[0075] Figure 4 Concept for describing separation Figure 3 The figure showing the output of the distance sensor (S310) is a concept. That is, Figure 4 is a flowchart of extracting only the road boundary portion of all data detected by the distance sensor 131 using the result of recognition by the vision sensor 132.

[0076] Reference Figure 4 With respect to a moving object, the difference between the distance sensor data and the vision sensor data is measured in advance, modeled, and stored in the storage device 140 as a database.

[0077] Thereafter, the modeling result is reflected to the position and size of the moving obstacle recognized by the vision sensor 132 and converted in the form of an occupancy grid map. In this case, a value 0 can be assigned to the moving obstacle, and a value 1 can be assigned to other areas.

[0078] Meanwhile, the output of the distance sensor 131 is also converted in the form of an occupancy grid map. In this case, a value 1 can be assigned to the position of the output of the sensor and a value 0 can be assigned to other areas.

[0079] In the process of fusing the sensors, the road boundary map 430 is calculated by performing an AND operation on the distance sensor map 410 and the vision sensor map 420 in units of grids. Thus, moving obstacles are removed from the output of the distance sensor, and only stationary objects corresponding to the road boundary are obtained.

[0080] Alternatively, a distance sensor for separating moving obstacles from stationary objects and outputting the separation result can be used. That is, a plurality of distance sensors can be configured, and thus some of the plurality of distance sensors can be used to sense only moving obstacles, and the rest thereof can be used to sense only stationary objects.

[0081] Figure 5 is a graph for describing the operation of accumulating Figure 3 is a graph for describing the operation of accumulating Figure 5 is a graph for describing the operation of accumulating Figure 5 The concept is as follows.

[0082] 1) Set the number of frames to be accumulated. In this case, a fixed number N (greater than 0) can be set.

[0083] 2) Regarding the positions where the N previous frame data 502 are located, position prediction is required in the current frame (i.e., segment) (510). The position prediction can be calculated by applying the vehicle speed and angular velocity information (501) output from the motion sensor 133 to a constant turning rate and speed (CTRV) model. Generally, a frame is formed of a plurality of fields. In this case, the previous frame data is placed in the data field of the current frame.

[0084] 3) Accumulated road boundary data is obtained by accumulating the road boundary data of the current frame and the plurality of road boundary data of the previous frames (520 and 530). At the time of accumulation, when the number of previous frames is greater than the set value N, the farthest previous frame is removed, and the data of the current frame is added to maintain the accumulation number.

[0085] By the above accumulation, the road boundary data is increased.

[0086] Meanwhile, the number of frames to be accumulated can be variably set according to the vehicle speed or the moving distance. Further, in the case where the number of frames is set to the vehicle speed, the accumulation number is set to be smaller at low speed and larger at high speed. Since low speed is mainly due to traffic congestion, it is difficult to sufficiently secure the road boundary data due to surrounding vehicles.

[0087] Figure 6 is a graph for describing the operation of generating Figure 3 is a graph for describing the operation of generating Figure 6is a conceptual diagram illustrating a concept of generating candidate positions on a three-dimensional space consisting of lateral / width positions and heading angle position information based on a grid search. Referring to Figure 6 , a cross 610 indicates a center of the host vehicle, a rectangle 600 indicates the host vehicle, and a square 620 including the host vehicle indicates a region of interest as a range of searching for a position of the host vehicle in a lateral / width direction. Further, the rectangle 600 indicates a position of the host vehicle determined by GPS, a small dot rectangle 611 indicates a position of the host vehicle determined by Emergency Vehicle Priority (EVP), and a small dot triangle 612 indicates a position of the host vehicle determined by RT3002. Here, RT3002 is a model name of an inertial and GPS measurement system.

[0088] Generally, a position of a vehicle is determined by a lateral / width position and a heading angle. As for the lateral / width position, the set region of interest 620 is divided at regular intervals, and all lateral / width positions generated during the division are set as candidate positions. For example, when the candidate positions are indicated as coordinates, the coordinates can be (50, 50), (-50, 50), (-50, 50), and (50, -50).

[0089] Further, in Figure 6 , a travel lane information 602 obtained using the image sensor 134 and a map road boundary 601 are shown.

[0090] Figure 7 is a conceptual diagram illustrating an example of generating candidate positions within a predetermined range according to Figure 6 . Referring to Figure 7 , as for the heading angle position information, the set range (e.g., left, right ±10°) is divided at regular intervals, and all heading angles generated during the division are set as candidate heading angles. The number of generated candidate positions becomes (the number of pieces of lateral position information x the number of pieces of width position information x the number of pieces of heading angle position information).

[0091] The center of the host vehicle is obtained based on the GPS position, and the basic heading angle also becomes the heading angle obtained from the GPS. The range of candidate positions can be set considering a GPS position error, and the search interval can be set according to an experiment considering precision.

[0092] Figure 8 is a conceptual diagram illustrating another example of generating candidate positions according to Figure 6 . Figure 8 is a conceptual diagram illustrating a process of setting only a few width candidate positions using accurate map lane information and travel lane information recognized by the vision sensor 132. Referring to Figure 8 , the setting process is as follows.

[0093] 1) Set the range of the region of interest 800 that will set the candidate position in the lateral direction, considering the GPS position error. The size of the region of interest 800 can be about 20m x 10m. Although it is omitted in Figure 8 , the accurate map lane (not shown) exists in the map road boundary 601.

[0094] 2) Calculate the side offset between the driving lane recognized by the vision sensor 132 and the center of the host vehicle, and determine the position of the host vehicle located within one lane according to the ratio of the side offset.

[0095] 3) Determine the position and number of lanes entering the region of interest at the accurate map boundary obtained based on GPS, and search for a plurality of lateral candidate position information 811, 812, 813, and 814 in the ratio of the side offset of each lane.

[0096] Figure 9 is a conceptual diagram for describing the search of the direction of the driving lane recognized according to Figure 8 . Figure 9 is a conceptual diagram for describing the search of the direction of the driving lane recognized by the vision sensor 132. Referring to Figure 9 , the left and right lanes L and R within a range of 5m to 10m in front are obtained, each heading angle is calculated, the average of the two heading angles is calculated. Thus, the direction of the driving lane recognized by the vision sensor 132 can be determined.

[0097] Figure 10 is a conceptual diagram for describing the search of the heading angle of the accurate map according to Figure 8 . Figure 10 is a conceptual diagram of searching for the lane direction in the accurate map. Referring to Figure 10 , the accurate map is obtained based on the pre-calculated lateral direction value, the GPS side direction value, and the heading angle of each candidate position information, then the heading angles of the left and right lanes L and R are calculated, and the average of the two heading angles is calculated. The azimuth angle of the left and right lanes corresponding to each candidate position is obtained.

[0098] Figure 11 is a flowchart of calculating the azimuth angle of each candidate position according to Figure 8 to Figure 10 . Referring to Figure 11 , when the difference between the two azimuth angles obtained as a result of Figure 9 and Figure 10 is calculated, the GPS azimuth angle error is calculated, and when the GPS azimuth angle error is applied to the GPS azimuth angle, the azimuth angle of each candidate position is obtained.

[0099] The generation of the candidate position is performed by searching a lateral candidate position of each lane and an azimuth angle of each candidate position. The position provided by the GPS can be directly used as a side direction position.

[0100] Figure 12 is a diagram illustrating a concept of an operation for describing selection Figure 3 of the best candidate position. That is, Figure 12 is a flowchart of selecting the best candidate position by searching a position having the highest matching degree by performing matching between the road boundary of the precise map and the road boundary data of the distance sensor 131 based on the distance transform. Referring to Figure 12 , the concept is as follows.

[0101] 1) The precise map road boundary 1201 is brought to the GPS reference position. The data 1210 is displayed in the form of an occupancy grid map.

[0102] 2) When a binary image 1220 about the precise map road boundary is generated, the distance transform is applied to the binary image 1220.

[0103] 3) The road boundary data 1230 of the distance sensor 131 is also brought in the form of an occupancy grid map. The reference points of the two grid maps (the center of the vehicle) are multiplied in units of grids, and a matching result 1240 based on the distance transform is calculated. The matching score is calculated by calculating an average value of the values assigned to the matching grids. The higher the matching score, the higher the matching degree. Specifically, Figure 12 shows that the candidate position is searched based on the distance transform, and the result is obtained by detecting the precise map road boundary 1203 and the distance sensor road boundary 1202 at a position having the highest matching degree.

[0104] 4) In order to search the candidate position, the reference point of the distance transform image is moved to the candidate position set in S330, and the above-described process 3) is performed to calculate the matching score of each candidate position, and a position having the highest matching degree is searched.

[0105] Figure 13 is a diagram illustrating a concept of an operation for describing Figure 3 another example of the concept of the operation for selecting the best candidate position shown in FIG. 10. Figure 13 is a conceptual diagram illustrating that the matching degree between the road boundary of the precise map and the road boundary data of the distance sensor 131 is calculated without the occupancy grid map, and a candidate position having the highest matching degree is searched to select the best candidate position. Referring to Figure 13 , the concept is as follows.

[0106] 1) Transform all data in the form of an array having Y-axis (lateral axis) coordinate values sampled at regular intervals and X-axis (transverse axis) coordinate values corresponding to the Y-axis coordinate values. In addition, the region of interest is preset in front of, behind, to the left of, and to the right of the host vehicle.

[0107] 2) Bring the precise map road boundary data to the candidate position, and also bring the road boundary data of the distance sensor 131 so that the center of the host vehicle becomes the candidate position.

[0108] 3) When comparing only the X-axis coordinate values of the two pieces of data corresponding to the Y-axis coordinate index values sampled so that the data point of the distance sensor 131 is located near the precise map road boundary, calculate the score in the same number as the number of points. For example, show the road boundary points, the input LiDAR points, the LiDAR points within the right road boundary range, and the LiDAR points within the left road boundary range. Thus, the matching degree score is as follows.

[0109] Matching degree score = number of left points + number of right points - | number of left points - number of right points | / 2 - number of points between road boundaries.

[0110] 4) Calculate the matching degree score for each set of candidate positions to search for a candidate position having the highest matching degree.

[0111] 5) Identify the lane in which the candidate position having the high matching degree exists as the travel lane (1310). Give the candidate positions matching degree scores, and compare the candidate positions using the matching degree scores.

[0112] 6) In order to detect the road boundary, select the output of the distance sensor 131 existing near the precise map road boundary at the candidate position having the high matching degree (1320).

[0113] 7) Since the data of the selected distance sensor is data accumulated in S320, multiple points are placed in a very close position. Calculate the average of the adjacent points to remove redundant data.

[0114] 8) Finally, in order to easily use the distance sensor data detecting only the road boundary, group the points having connectivity as an object to be finally output (1330). That is, remove the accumulated points, and detect only the road boundary of the distance sensor data.

[0115] Among the outputs of the distance sensor 131 removing the moving object (i.e., the moving obstacle), not only the road boundary is simply included, but also various noises, such as the moving object, the artificial structure outside the road boundary, the tree, and the shrub, which have not been removed in S310.

[0116] Therefore, in order to remove various noises and to detect only a pure road boundary or to minimize the influence caused by the noises, when matching the distance sensor data with the precise map, distance sensor data near the road boundary of the precise map is selected.

[0117] Further, when the ratio of the number of left points is similar to the ratio of the number of right points, the method for calculating the matching degree score can be designed to assign a higher score. Reducing the score as much as the number of points remaining in the road boundary makes it possible to minimize the influence of a moving object that has not been removed.

[0118] Figure 14 is a conceptual diagram illustrating an operation of setting a matching degree of final candidate position information of correct recognition and final candidate position information of non-correct recognition. Refer to Figure 3 In actual road situations, abnormal situations such as errors in the output of sensors, environmental noise, and lane changes can occur. In such abnormal situations, the technology of some forms of the present disclosure for recognizing a travel lane can not operate normally. Therefore, the detection result or the recognition result in an abnormal situation can be determined as abnormal by invalidity determination processing. Figure 14

[0119] To this end, whether each frame is valid is output based on a predetermined invalidity determination condition. Three types of invalidity determination conditions can be set.

[0120] Invalidity determination condition 1:

[0121] When the matching degree score of the final selected final candidate position information is less than the matching degree, the final candidate position information is processed as invalid. Each of the matching degree scores of the correctly recognized candidate position information generally has a value of zero or more (1410). In contrast, the trend of the matching degree scores of the non-correctly recognized candidate position information, which has a value less than zero, can be confirmed by the distribution map (1420). The matching degree can be set by the data distribution obtained from the results of some of the plurality of pieces of data.

[0122] Invalidity determination condition 2:

[0123] When the remaining amount of a sensor (e.g., LiDAR) existing in the lane in the final selected final candidate position information is greater than the matching degree, the remaining amount is processed as invalid. The greater the remaining amount of the distance sensor 131 means that the final selected candidate is a false recognition. The matching degree can be set in the same manner as in the invalidity determination condition 1.

[0124] Invalidity determination condition 3:

[0125] When the travel lane recognized by the image sensor 134 is determined to be invalid by the validity test, the travel lane is processed as invalid. This will be described below with reference to​Figure 15 The description is made.

[0126] Figure 15 is used to describe Figure 3 The figure shows the concept of the operation of determining invalidity. Specifically, Figure 15 The method using invalidity determination condition 3 is shown. Referring to Figure 15 Since the candidate position of the host vehicle is set using the lane information identified by the image sensor 134, when the result identified by the image sensor 134 is invalid, it is inevitable to obtain a result of incorrect identification. Therefore, the validity test of the travel lane identified by the image sensor 134 is the most important determination condition for invalidity determination.

[0127] Whether the left and right lanes are parallel to each other is determined by calculating the directions of the left and right lanes existing in the range of 5m to 10m in front of the host vehicle and determining the difference in direction between the left and right lanes 1510.

[0128] Whether the host vehicle deviates from the road width is determined by calculating the side offset amount of the left and right lanes based on the center of the host vehicle, calculating the difference between the side offset amounts, and comparing the difference with the usual road width d 1520.

[0129] Therefore, the use of the result of the road boundary false detection or the travel lane false identification due to the error of the output of the image sensor and the output of the distance sensor can be limited by the function of determining invalidity. Therefore, the travel lane identification rate is improved.

[0130] In addition, the error of the output of the image sensor mainly occurs during lane change. The error of the output of the image sensor directly affects the false identification of the travel lane. Therefore, the false identification result due to the error of the output of the image sensor can be treated as an anomaly by invalidity determination condition 3.

[0131] In addition, the error of the output of the distance sensor (especially LiDAR sensor) is unpredictable, which also directly affects the false identification of the travel lane. Therefore, the false identification result due to the error of the output of the distance sensor can be treated as an anomaly by invalidity determination conditions 1 and 2.

[0132] The test results according to invalidity determination conditions 1 to 3 are shown in the following table.

[0133] Table 1

[0134]

[0135]

[0136] The processing time was obtained using the Matlab program of Mathworks, Inc.

[0137] Meanwhile, even in a case where a lane in which the distance sensor output is hardly generated, previous frames can be accumulated to maintain the lane boundary information. That is, even when the lane boundary information is lost by surrounding vehicles, the lane boundary can be detected by accumulating the previous lane boundary information.

[0138] Further, the operations of the methods or algorithms described in some forms of the present disclosure can be implemented in the form of program commands that can be executed by various computer means, such as a microprocessor, a processor, a central processing unit (CPU), etc., and can be recorded in a computer readable medium. The computer readable medium can include program (command) codes, data files, data structures, etc., alone or in combination.

[0139] The program (command) codes recorded in the computer readable recording medium can be specifically designed and configured for some forms of the present disclosure, or can be known and available to those skilled in the computer software field. Examples of the computer readable recording medium can include magnetic media such as a hard disk, a floppy disk, a magnetic tape, etc.; optical media such as a compact disc read only memory (CD-ROM), a digital versatile disc (DVD), a Blu-ray disc, etc.; and semiconductor memory devices specifically configured to store and execute program (command) codes, such as a read only memory (ROM), a random access memory (RAM), a flash memory, etc.

[0140] Here, examples of the program (command) codes include machine language codes generated by a compiler, and high-level language codes executable by a computer using an interpreter, etc. The above-described hardware devices can be configured to operate as one or more software modules in order to perform the operations of the present disclosure, and vice versa.

[0141] According to the present disclosure, the effect is that the road boundary can be accurately detected through road boundary determination by matching with a precise map even when the shape of the road boundary is complex or there is a large amount of noise outside or inside the road boundary.

[0142] Further, as another effect of the present disclosure, the use of the result of the erroneous detection of the road boundary or the erroneous recognition of the travel lane due to the error of the output of the camera sensor and the output of the light detection and ranging (LiDAR) sensor can be limited by the function of determining invalidity, so that the travel lane recognition rate can be improved.

[0143] While the present disclosure has been described with reference to the figures, it will be apparent the skilled person that various changes and modifications can be made without departing from the spirit and scope of the present disclosure, and that the disclosure is not to be limited to some forms described. It is therefore noted that such variations or modifications are to be considered within the scope of the present disclosure, and that the scope of the present disclosure is to be interpreted only based on the appended claims.

Claims

1. A device for identifying driving lanes based on multiple sensors, the device comprising: The first sensor is configured to calculate road information; The second sensor is configured to calculate information about moving obstacles; The third sensor is configured to calculate the vehicle's movement information; as well as The controller is configured as follows: Remove the moving obstacle information from the road information to extract only the road boundary data; Accumulate the road boundary data to calculate multiple candidate location information for the vehicle based on the movement information; and Select the final candidate location information from the multiple candidate location information. The fourth sensor is configured to calculate lane information. The controller is configured as follows: The multiple candidate location information are calculated using the driving lane information obtained based on the current vehicle location information or the preset precise map lane information. The precise map road boundary information searched based on the multiple candidate location information is matched with the accumulated road boundary data; Assign a matching score to each of the multiple candidate location information; and The lane with the highest matching score among the final candidate location information is selected as the driving lane.

2. The device according to claim 1, wherein, The controller is configured to: The motion information is used to extract the road boundary data from multiple previous frames and accumulate the road boundary data in the current frame.

3. The device according to claim 1, wherein, The controller is configured to: A grid is formed by dividing predetermined regions of interest placed on the boundaries of the precise map lane information at regular intervals; and The multiple candidate location information are calculated based on the grid.

4. The device according to claim 3, wherein, The controller is configured to: Forming the grid having multiple lateral directions and multiple transverse directions; and The multiple side directions and the multiple lateral directions are set as multiple lateral candidate position information and multiple side candidate position information.

5. The device according to claim 4, wherein, The controller is configured to: The grid with multiple heading angles is formed; The multiple heading angles are generated by dividing the region of interest at regular intervals based on the center point of the vehicle. as well as The multiple heading angles are set as multiple heading angle candidate position information.

6. The device according to claim 5, wherein, The controller is configured to: The multiple candidate position information is obtained by multiplying the multiple side candidate position information, the multiple lateral candidate position information, and the multiple heading angle candidate position information.

7. The device according to claim 1, wherein, Taking into account the vehicle's position error, the multiple candidate position information includes a preset number of lateral candidate position information based on the lateral direction.

8. The device according to claim 7, wherein, The controller is configured to: The position of the vehicle within a road boundary is determined based on the ratio of the lateral offset calculated between the driving lane and the center of the vehicle.

9. The device according to claim 8, wherein, The controller is configured to: Based on the ratio of the lateral offset, a preset number of lateral candidate position information for each lane is determined based on the lane positions and the number of lanes identified in a preset region of interest in the precise map lane information.

10. The device according to claim 8, wherein, The controller is configured to: The heading angle of each of the preset number of lateral candidate position information is calculated using the difference between a first average value and a second average value. The first average value is the average of the heading angles of the left lane and the right lane in front of the vehicle, identified by the second sensor. The second average value is the average of the heading angles of the left lane and the right lane in the preset number of lateral candidate position information, calculated using the precise map lane information.

11. The device according to claim 1, wherein, The controller is configured to: The precise map road boundary information and accumulated road boundary data are converted into a coordinate system including Y-axis and X-axis coordinates; and The matching score is calculated by comparing the values ​​of the X-axis coordinates based on the index of the Y-axis coordinate.

12. The device according to claim 11, wherein, The controller is configured to: The matching score is calculated using multiple data points located near the road boundary information of the precise map. as well as The data points are generated by the first sensor, wherein the number of data points includes the number of left points, the number of right points, and the number of points between the road boundaries based on the precise map road boundary information.

13. The device according to claim 2, wherein, The controller is configured to: When the number of previous frames exceeds a preset value, remove the furthest previous frame and add a current frame to keep the cumulative number of frames constant.

14. The device according to claim 13, wherein, The controller is configured to: The cumulative number of frames can be variably set according to the vehicle speed or the distance the vehicle has traveled.

15. The device according to claim 1, wherein: The first sensor includes a distance sensor. The second sensor includes a vision sensor. The third sensor includes a motion sensor, and The fourth sensor includes a global positioning system sensor or an image sensor.

16. The device according to claim 15, wherein, The controller is configured to: A preset invalidation determination condition is applied to the final candidate location information to determine whether the final candidate location information is valid.

17. The device according to claim 16, wherein, The invalid determination condition includes any one of the following: whether the matching score of the final candidate location information is less than the preset first matching score comparison condition; Whether the remaining quantity of the distance sensor existing in the driving lane is greater than the comparison condition of the preset second matching degree in the final candidate position information; The system also determines whether the left lane direction and the right lane direction are parallel based on the difference between the left lane direction and the right lane direction in the final candidate position information, and determines whether the vehicle deviates from the road width by comparing the difference between the lateral offset of the left lane and the lateral offset of the right lane based on the center of the vehicle with a preset road width.

18. A method for identifying a driving lane based on multiple sensors, the method comprising: Road information and moving obstacle information are calculated by the first and second sensors; The vehicle's movement information is calculated using a third sensor; The controller removes the moving obstacle information from the road information to extract only the road boundary data; The controller accumulates the road boundary data to calculate multiple candidate location information for the vehicle based on the movement information; The controller selects the final candidate location information from the multiple candidate location information; The controller uses lane information obtained based on the current vehicle location information or preset precise map lane information to calculate the multiple candidate location information. The controller matches the precise map road boundary information searched based on the multiple candidate location information with the accumulated road boundary data; The controller assigns a matching score to each of the multiple candidate location information entries; as well as The controller selects the lane with the highest matching score from the final candidate location information as the driving lane.

Images