Apparatus for generating precision map and method thereof

Abstract

The present invention relates to a precision map generation device and a method thereof. The precision map generation device according to the present invention may include a communication unit that receives a time-series image and spatial information regarding the surrounding environment of a vehicle, a memory that stores the received time-series image and spatial information, and a processor that generates a precision map based on the time-series image and spatial information. The processor may extract lane information based on the time-series image, generate a polyline based on the extracted lane information, and generate the precision map for autonomous driving based on the polyline and the spatial information.

Description

Technology Field

[0001] The present invention relates to a precision map generation device for autonomous driving and a method thereof, and more specifically, to a technology for generating a precision map for autonomous driving of a vehicle. Background Technology

[0002] Recently, vehicles capable of autonomous driving and autonomous parking are being developed. High-performance lane recognition and precision map construction technologies are required for such autonomous driving and parking. Commonly known methods for constructing precision maps include Feature-Based Maps and Occupancy Grid Maps (OGM). In this context, core technologies for constructing precision maps include techniques that enhance positional accuracy through drift compensation and localization technologies designed to determine the vehicle's actual location.

[0003] In this case, feature maps have the disadvantage that positioning is possible based on specific landmarks. For example, there is a problem in that it is difficult to construct a map in environments such as a sports field where no features have been extracted. On the other hand, while occupancy grid maps (OGM) allow for positioning even without specific landmarks, they have the disadvantage of high memory usage and computational load because they store thousands to tens of thousands of points.

[0004] In addition, both of the aforementioned methods are map construction methods focused on tall objects. That is, since both of the aforementioned methods are difficult to recognize ground information where there are no tall objects, their usefulness is low in indoor and outdoor parking spaces where ground information recognition is important.

[0005] Furthermore, the two methods described above make map construction difficult in environments with many dynamic objects. For instance, the two methods have limitations in their technology for distinguishing between static and dynamic objects. For instance, since dynamic objects move simultaneously with vehicles, it is difficult to predict their behavioral information. Consequently, the two methods described above have the disadvantage of being difficult to remove noise from dynamic objects.

[0006] Therefore, there is a need for technology to construct precision maps that support autonomous driving and autonomous parking while simultaneously solving the aforementioned problems. Prior art literature

[0007] Korean Registered Patent Publication No. 10-2734906 (Published Nov. 28, 2024) The problem to be solved

[0008] Embodiments of the present invention aim to provide a precision map generation device for autonomous driving and a method thereof, which can generate precision maps in environments without landmarks and environments with many dynamic objects.

[0009] Another embodiment of the present invention aims to provide a precision map generation device for autonomous driving that improves positioning accuracy based on lane information, and a method thereof.

[0010] Another embodiment of the present invention aims to provide a precision map generation device and a method for autonomous driving that is effective for memory parking, as the precision map is stored using only lane information, thereby facilitating map reuse with a small amount of computation and memory capacity.

[0011] The technical problems of the present invention are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0012] A precision map generation device for autonomous driving according to an embodiment of the present invention may include a communication unit that receives time-series images and spatial information regarding the surrounding environment of a vehicle, a memory that stores the received time-series images and spatial information, and a processor that generates a precision map based on the time-series images and spatial information. The processor may extract lane information based on the time-series images, generate a polyline based on the extracted lane information, and generate the precision map for autonomous driving based on the polyline and spatial information.

[0013] In one embodiment, the processor may include an image preprocessing module that preprocesses the time series image by applying a deep learning model, a lane extraction module that detects lanes of a road in the preprocessed time series image and extracts lane information, a polyline generation module that generates the polyline based on the lane information and generates a polyline map based on the generated polyline, and a map generation module that generates the precision map by applying the spatial information to the polyline map.

[0014] In one embodiment, the image preprocessing module can generate a binary image by applying semantic segmentation to the time series image.

[0015] In one embodiment, the lane extraction module can extract the edge of the lane from the binary image, calculate the feature point of the lane from the extracted edge, calculate the similarity of the feature point by calculating the distance and angle between the extracted feature points, and classify the feature point based on the calculated similarity.

[0016] In one embodiment, the polyline generation module can generate the polyline by applying the Douglas-Peucker algorithm to the classified feature points, calculate the similarity of the polyline by calculating the distance and angle between the generated polylines, and classify the polyline based on the calculated similarity.

[0017] In one embodiment, the image preprocessing module can generate depth information for the time series image by applying a mono-depth estimation technique to the time series image.

[0018] In one embodiment, the polyline generation module can remove outliers based on the length of the polyline and the width of the lane, and can correct the position coordinates of the distorted polyline based on the depth information.

[0019] In one embodiment, the polyline generation module can fuse the polylines by applying a weighted line fitting algorithm to the polylines located in the overlapping areas of each frame of the time series image, and can generate the polyline map based on the fused polylines.

[0020] In one embodiment, the map generation module can calculate the location coordinates and driving direction of an object and a vehicle based on the spatial information, match the calculated location coordinates and driving direction to the polyline map, and generate a time-series polyline map based on the matching result.

[0021] In one embodiment, the map generation module can perform drift compensation on the time-series polyline map using a loop closing technique and generate the precision map by mapping the object to the time-series polyline map.

[0022] A method for generating a precision map for autonomous driving according to another embodiment of the present invention may include the steps of receiving a time-series image and spatial information regarding the surrounding environment of a vehicle, storing the received time-series image and spatial information, and generating a precision map based on the time-series image and spatial information. The step of generating the precision map may include the steps of extracting lane information based on the time-series image, generating a polyline based on the extracted lane information, and generating the precision map for autonomous driving based on the polyline and spatial information.

[0023] In another embodiment, the step of generating the precision map for autonomous driving may include the step of preprocessing the time series image by applying a deep learning model, the step of detecting lanes of a road in the preprocessed time series image and extracting lane information, the step of generating the polyline based on the lane information and generating a polyline map based on the generated polyline, and the step of generating the precision map by applying the spatial information to the polyline map.

[0024] In another embodiment, the step of preprocessing the time series image may include the step of generating a binary image by applying semantic segmentation to the time series image.

[0025] In another embodiment, the step of extracting lane information may include the step of extracting the edge of the lane from the binary image, the step of calculating the feature point of the lane from the extracted edge, the step of calculating the similarity of the feature point by calculating the distance and angle between the extracted feature points, and the step of classifying the feature point based on the calculated similarity.

[0026] In another embodiment, the step of generating the polyline map may include the step of generating the polyline by applying the Douglas-Peucker algorithm to the classified feature points, the step of calculating the similarity of the polyline by calculating the distance and angle between the generated polylines, and the step of classifying the polyline based on the calculated similarity.

[0027] In another embodiment, the step of preprocessing the time series image may further include the step of generating depth information for the time series image by applying a mono-depth estimation technique to the time series image.

[0028] In another embodiment, the step of generating the polyline map may further include the step of removing outliers based on the length of the polyline and the width of the lane, and the step of correcting the position coordinates of the distorted polyline based on the depth information.

[0029] In another embodiment, the step of generating the polyline map may further include the step of fusing the polylines by applying a weighted line fitting algorithm to the polylines located in the overlapping areas of each frame of the time series image, and the step of generating the polyline map based on the fused polylines.

[0030] In another embodiment, the step of generating the precision map may include the step of calculating the position coordinates and driving direction of an object and a vehicle based on the spatial information, the step of matching the calculated position coordinates and driving direction to the polyline map, and the step of generating a time-series polyline map based on the matching result.

[0031] In another embodiment, the step of generating the precision map may further include the step of performing drift compensation on the time series polyline map using a loop closing technique and the step of generating the precision map by mapping the object to the time series polyline map. Effects of the invention

[0032] This technology improves drift compensation and positioning accuracy, thereby enhancing the reliability of precision maps.

[0033] In addition, this technology can improve computational efficiency by reducing memory usage and processing time through the generation of a high-precision map using only lane information.

[0034] In addition, since this technology provides high-accuracy precision maps, it can improve the reliability of autonomous driving and autonomous parking of vehicles.

[0035] In addition, since this technology stores a precise map using only lane information, it facilitates map reuse with minimal computational load and memory capacity, making it effective for memory parking.

[0036] In addition, various effects that can be identified directly or indirectly through this document may be provided. Brief explanation of the drawing

[0037] FIG. 1 is a block diagram showing the configuration of an autonomous driving vehicle system including a precision map generating device according to one embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of a processor of a precision map generation device according to one embodiment of the present invention. FIG. 3 is an example of an image preprocessing module according to one embodiment of the present invention. FIG. 4 is an example of a polyline generation module according to one embodiment of the present invention. FIG. 5 is an example of a map generation module according to one embodiment of the present invention. FIG. 6 is a flowchart illustrating a method for generating a precise map according to an embodiment of the present invention. FIGS. 7 to 10 are flowcharts for explaining in detail a method for generating a precise map according to an embodiment of the present invention. FIG. 11 is a block diagram showing a computing system according to one embodiment of the present invention. Specific details for implementing the invention

[0038] Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the embodiments of the present invention, if it is determined that a detailed description of related known components or functions would hinder understanding of the embodiments of the present invention, such detailed description is omitted.

[0039] In describing the components of the embodiments of the present invention, terms such as first, second, A, B, (a), (b), etc., may be used. These terms are intended merely to distinguish the components from other components, and the essence, order, or sequence of the components is not limited by the terms. Furthermore, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0041] Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 1 to 11.

[0042] FIG. 1 is a block diagram showing the configuration of an autonomous driving vehicle system (1) including a precision map generating device (10) according to one embodiment of the present invention.

[0043] Referring to FIG. 1, the autonomous driving vehicle system (1) may include a precision map generating device (10), a Mobile Mapping System (MMS, 20), and an autonomous driving control system (1).

[0044] The precision map generating device (10) may include a processor (100), memory (200), and a communication unit (300).

[0045] In one embodiment, the processor (100) may receive time-series images and spatial information regarding the surrounding environment of the vehicle through the communication unit (300). Here, the vehicle may be an autonomous vehicle having an autonomous driving device. An autonomous driving device refers to a device that controls the steering and speed of the vehicle based on information obtained through a plurality of vehicle sensors (e.g., cameras, radar and / or lidar, etc.).

[0046] In one embodiment, a time-series image may be received from a camera sensor (210) of an MMS (20) equipped in a vehicle. The time-series image may be at least one image capturing the surrounding environment of the vehicle, and may be a series of consecutive image frames over time.

[0047] In one embodiment, spatial information may be received from the IMU module (220), GPS module (230), and lidar sensor of the MMS (20). The spatial information may include the vehicle's location information, object information of the surrounding environment, attitude information, and driving information.

[0048] Additionally, the processor (100) can extract lane information based on a time-series image. Here, the lane information may include lane position, lane width, etc., for the vehicle's driving road.

[0049] Subsequently, the processor (100) can generate a polyline based on the extracted lane information. A polyline may refer to a line recognized as a single object in a time-series image.

[0050] Subsequently, the processor (100) can generate a precision map for autonomous driving based on polyline and spatial information. Here, the precision map may be a precision map used for autonomous driving or autonomous parking of a vehicle.

[0051] Afterwards, the processor (100) can transmit the precision map generated through the communication unit (300) to the autonomous driving control system (30).

[0052] In one embodiment, the processor (100) can generate a precision map of a parking lot where a vehicle's parking slot is located. At the same time, the generated precision map can be stored in memory (200). When a vehicle arrives at the parking lot, the processor (100) can extract the precision map from memory (200). Subsequently, the processor (100) can transmit the precision map of the parking lot to an autonomous driving control system (30) via a communication unit (300). Accordingly, the autonomous driving control system (30) can easily perform autonomous parking in the vehicle's parking slot at the parking lot.

[0053] In one embodiment, the memory (200) can store time-series images and spatial information received through the communication unit (3000). Additionally, the memory (200) can store all information performed by the processor (100).

[0054] Additionally, the memory (200) may include at least one memory in which a program for performing the aforementioned operation and the operation described below is stored. Here, the memory may include ROM (Read Only Memory) and RAM (Random Access Memory).

[0055] In one embodiment, the communication unit (300) can perform CAN (Controller Area Network) communication or wired communication. For example, for the control of various control systems mounted in a vehicle and for communication between various control systems, a communication network including a body network, a multimedia network, and a chassis network is configured within the vehicle, and each of these separated networks can be connected by a processor (100) to transmit and receive CAN (Controller Area Network) communication messages to and from each other. That is, the communication unit (300) can transmit various information to the vehicle system based on the control signal of the processor (100) and can receive various information from the vehicle system.

[0056] A precision map generating device (10) for autonomous driving according to one embodiment of the present invention may be implemented inside a vehicle. At this time, the precision map generating device (10) may be formed integrally with the internal control units of the vehicle, or it may be implemented as a separate device and connected to the control units of the vehicle by a separate connection means. The precision map generating device (10) will be described in detail later in FIGS. 2 to 10.

[0057] In one embodiment, the MMS (20) may include a camera sensor (210), an IMU module (220), a GPS module (230), and a LiDAR sensor, etc. The MMS (20) may be a system implemented to integrate the various sensors described above to obtain location measurement and visual information of terrain features around the driving road simultaneously with driving information of the vehicle.

[0058] In one embodiment, the camera sensor (210) may include a wide-angle camera having multiple channels and may collect time-series images. For example, the camera sensor (210) may be a 4-channel wide-angle camera that captures the front / rear and left / right environments of a vehicle and may collect time-series RGB images. However, this is merely an example and the present disclosure is not limited thereto. Additionally, the camera sensor (210) may include at least one camera.

[0059] In one embodiment, the IMU module (220) and the GPS module (230) can collect spatial information of the vehicle. For example, the IMU module (220) and the GPS module (230) can collect location information, attitude information, and driving information of the vehicle. Additionally, the lidar sensor can collect object information of the surrounding environment. Here, the object information may include static objects such as streetlights and dynamic objects such as people and vehicles.

[0060] In one embodiment, the autonomous driving control system (30) may be a system that controls autonomous driving and autonomous parking of a vehicle. For example, the autonomous driving control system (30) may receive a precision map of a specific area from a processor (100). Subsequently, the autonomous driving control system (30) may facilitate autonomous driving and autonomous parking in the specific area based on the precision map.

[0061] As described above, the precision map generation device (10) for autonomous driving according to the present invention can improve the performance of autonomous driving and autonomous parking of a vehicle by recognizing and extracting line information (e.g., driving lanes, parking lines, stop lines, other road markers, etc.) present in the driving environment of the vehicle.

[0063] FIG. 2 is a block diagram showing the configuration of a processor (100) of a precision map generating device (10) according to one embodiment of the present invention.

[0064] And FIG. 3 is an example of an image preprocessing module (110) according to one embodiment of the present invention, FIG. 4 is an example of a polyline generation module (130) according to one embodiment of the present invention, and FIG. 5 is an example of a map generation module (140) according to one embodiment of the present invention.

[0065] Referring to FIG. 2, the processor (100) may include an image preprocessing module (110), a lane extraction module (120), a polyline generation module (130), and a map generation module (140).

[0066] In one embodiment, the image preprocessing module (110) can preprocess time-series images by applying a deep learning model. Here, the deep learning model may be semantic segmentation, but is not limited thereto. For example, referring to FIG. 3, the image preprocessing module (110) can receive time-series images from a 4-channel wide-angle camera and apply semantic segmentation to the received time-series images. Accordingly, the image preprocessing module (110) can classify the time-series images into a specific class based on pixel information. At this time, the image preprocessing module (110) can select the specific class as a lane. Subsequently, the image preprocessing module (110) can generate a binary image by setting the lane as a specific class. Here, the lane may include driving lanes, parking lines, stop lines, other road markers, etc. And within the specific class, driving lanes, parking lines, stop lines, other road markers, etc. can be set as detailed classes.

[0067] As described above, the precision map generating device (10) according to the present invention can give special characteristics to the precision map to be generated thereafter by setting all lane information in the surrounding environment of the vehicle to a specific class. At the same time, the precision map generating device (10) according to the present invention can easily correct the positioning of the vehicle.

[0068] In one embodiment, the image preprocessing module (110) can generate depth information for a time-series image by applying a mono-depth estimation technique to the time-series image. Subsequently, the image preprocessing module (110) can transmit the depth information to a polyline generation module (130).

[0069] Referring again to FIG. 2, the lane extraction module (120) can detect lanes of a road in a preprocessed time series image and extract lane information.

[0070] In one embodiment, the lane extraction module (120) can receive a binary image from the image preprocessing module (110) and extract lane edges from the binary image. Subsequently, the lane extraction module (120) can calculate lane feature points from the extracted edges. Here, the feature points may be feature points with respect to the vertical direction of the edge, but are not limited thereto. Subsequently, the lane extraction module (120) can calculate the similarity of the feature points by calculating the distance and angle between the extracted feature points. Here, the similarity may be the degree of similarity regarding the distance and angle between the feature points. Subsequently, the lane extraction module (120) can classify the feature points based on the calculated similarity. For example, the lane extraction module (120) can perform clustering of the feature points by including feature points with high similarity among the feature points into one group. Specifically, feature points satisfying the following two conditions can be clustered into the same group.

[0071] 1. distPoint(P n-1 , P n ) < maxDistThr1 (here, P n is the world coordinate system value (x, y) of the nth feature point, and distPoint() is the distance between the two points.)

[0072] 2. anyLine( , ) < maxAngtThr1 (here, ( , is P n-1 and P n The line segment connecting them; anyLine() is the angle formed by the two line segments.)

[0073] The lane extraction module (120) can assign an index to the feature point for the group to which the feature point belongs. Additionally, the lane extraction module (120) can define a starting feature point and an ending feature point within a single group.

[0074] As described above, the lane, which is a feature point, can be represented by two points. Accordingly, even if a large amount of lane information is stored, the precision map generation device (10) according to the present invention can reduce memory usage and computation processing time compared to conventional technology.

[0075] In one embodiment, the polyline generation module (130) can generate polylines based on lane information. For example, the polyline generation module (130) can generate polylines by applying the Douglas-Peucker algorithm to classified feature points. Here, the Douglas-Peucker algorithm is an algorithm that simplifies curves (or polygons) composed of line components. Subsequently, the polyline generation module (130) can calculate the similarity of polylines by calculating the distances and angles between the generated polylines. Here, similarity may be the degree of similarity regarding the distances and angles between polylines. Subsequently, the polyline generation module (130) can classify polylines based on the calculated similarity. For example, the polyline generation module (130) can perform clustering of polylines by including polylines with high similarity among polylines into one group. Specifically, polylines satisfying the following two conditions can be clustered into the same group.

[0076] 1. distPoint(PL m-1,e , PL m,s) < maxDistThr2 (here, PL m-1,e is the endpoint coordinate value of the 1st polyline, PL m,s is the coordinate value of the starting point of the m-th polyline.)

[0077] 2. anyLine( , ) < maxAngtThr2 (here, is the line segment connecting the start and end points of the m-th polyline, maxDistThr1 << maxDistThr2, maxAngtThr1 << maxAngtThr2.)

[0078] In one embodiment, the polyline generation module (130) can remove outliers based on the length of the polyline and the width of the lane. For example, the polyline generation module (130) can remove outliers based on a polyline of 2 m or more and a lane width of 15 to 20 cm. However, this is merely an example. The present disclosure is not limited thereto.

[0079] Referring to FIG. 4, the polyline generation module (130) can generate a polyline map based on the generated polyline.

[0080] In one embodiment, the polyline generation module (130) can correct the position coordinates of a distorted polyline based on depth information. For example, the polyline generation module (130) can receive depth information for a time-series image from the image preprocessing module (110). The position of the polyline may be distorted by ground gradient, and the polyline generation module (130) can correct the position coordinates of the polyline based on the received depth information to correct this distortion. Here, the position coordinates may be in the world coordinate system, but are not limited thereto. Specifically, the position coordinates of the polyline can be corrected using the following equation.

[0081] I2W(I n , D n ) = P n , where W2I(Pn ) = I n (Here, I n is P n The image coordinate system value corresponding to, P n is corrected P n The world coordinate system value of, I2W / W2I is the coordinate system transformation function, D n I n This is the depth value corresponding to.)

[0082] As described above, by utilizing depth information of an image to correct position information for lanes, a precise map of various environments, particularly spaces such as parking lots, can be generated.

[0083] In one embodiment, the polyline generation module (130) can extract polylines located in the overlapping areas of each frame of a time-series image. For example, a 4-channel time-series image is an image taken of the front / rear and left / right sides of a vehicle, and overlapping areas may exist in each frame of the time-series image. Accordingly, the polyline generation module (130) can apply a line fitting algorithm, which is an image processing technique, based on the weights of the polylines located in the overlapping areas. Through the process described above, the polyline generation module (130) can fuse the overlapping polylines into a single polyline. Subsequently, the polyline generation module (130) can generate a polyline map based on the fused polyline.

[0084] Referring again to FIG. 2, the map generation module (140) can generate a precise map by applying spatial information to a polyline map.

[0085] In one embodiment, the map generation module (140) can calculate the location coordinates and driving direction of an object and a vehicle based on spatial information. For example, the map generation module (140) can calculate the location coordinates, driving direction, speed, etc. of the vehicle. Subsequently, the map generation module (140) can match the calculated location coordinates and driving direction of the vehicle to a polyline map. Here, since the polyline map is generated from a time-series image having continuous frames over time, the polyline map can also have continuous frames over time. For example, the map generation module (140) can perform translation, rotation, etc., between the frames of the continuous polyline map. Subsequently, the map generation module (140) can track the movement of the vehicle based on the calculation results. That is, the map generation module (140) can match the location coordinates and driving direction of the vehicle one-to-one with the polyline map to which the movement of the vehicle is tracked.

[0086] Referring to FIG. 5 (a), through the process described above, the map generation module (140) can generate a time-series polyline map based on the matching result.

[0087] Additionally, the map generation module (140) can generate a polyline map in real time when a vehicle revisits a specific area where a precision map has been generated and stored. Subsequently, the map generation module (140) can accurately determine the location of the vehicle by matching the polyline map being generated in real time to the stored precision map.

[0088] In one embodiment, the map generation module (140) can perform drift compensation on a time-series polyline map using a loop closing technique. Here, drift may refer to a phenomenon where errors in the map accumulate and increase over time. For example, drift may occur as errors in vehicle movement estimation and errors caused by ground gradients accumulate. To generate a precise map, vehicle positioning correction is required, and to perform vehicle positioning correction, drift compensation must be performed. Additionally, the loop closing technique is also known as Loop Closure Detection and is a technique used for drift compensation based on position estimation.

[0089] For example, the map generation module (140) can set a time-series polyline map for the start point of the precision map as a QR map. In other words, the precision map generation device (10) according to the present invention can generate a precision map of a specific area. The start point of the precision map may include, but is not limited to, the start point when a vehicle enters a specific area. Subsequently, the map generation module (140) can perform drift correction by applying a loop closing technique to the set QR map. Specifically, it is assumed that the current vehicle revisits a specific area where a precision map has already been generated and stored. The map generation module (140) can perform drift compensation by calculating the relative position between the QR map and the vehicle's current polyline map using the ICP (Iterative Closest Point) technique. Accordingly, the map generation module (140) can precisely correct the vehicle's entire path for a specific area.

[0090] As described above, since position correction is performed based on the depth information of the image for lane information, the accuracy of drift correction using the loop closing technique can be improved.

[0091] Referring to FIG. 5(b), the map generation module (140) can generate a precise map by mapping objects onto a time-series polyline map. Here, the objects may include static objects such as stoppers, curbs, pillars, traffic cones, trees, etc. For example, the map generation module (140) can map static objects onto a time-series polyline map and perform mapping on the relative positions of the polyline. Accordingly, the map generation module (140) can correct the path to improve accuracy.

[0092] The map generation module (140) can add specificity to the precision map by mapping static objects commonly found in parking lots. In other words, the precision map generation device (10) can easily generate a precision map of a parking lot. In addition, the precision map generation device (10) can enhance the specificity of the precision map by setting static objects to a specific class, in addition to lanes.

[0093] As described above, the precision map generating device (10) for autonomous driving according to the present invention improves drift compensation and positioning accuracy, thereby improving the reliability of the precision map. In particular, the precision map generating device (10) can generate a precision map using only ground information even in environments where it is difficult to detect feature points by landmarks or environments with many dynamic objects.

[0094] In addition, the precision map generating device (10) can improve computational efficiency by reducing memory usage and computation processing time by generating a precision map using only lane information. In addition, the precision map generating device (10) can improve the reliability of autonomous driving and autonomous parking of a vehicle by providing a precision map with high accuracy.

[0096] Hereinafter, a method for generating a precise map for autonomous driving according to an embodiment of the present invention will be specifically described with reference to FIGS. 6 to 10.

[0097] In the following, it is assumed that the precision map generating device (10) of FIG. 2 performs the process of FIG. 6 to FIG. 10. Also, in the description of FIG. 6 to FIG. 10, the operation described as being performed by the device can be understood as being controlled by the processor (100) of the precision map generating device (10).

[0098] FIG. 6 is a flowchart illustrating a method for generating a precise map according to an embodiment of the present invention.

[0099] Referring to FIG. 6, in step S610, a time-series image can be preprocessed by applying a deep learning model. Here, the deep learning model may be a semantic segmentation model, but is not limited thereto.

[0100] Subsequently, in step S620, lane information can be extracted by detecting road lanes in the preprocessed time-series image. Here, the lanes may include driving lanes, parking lines, stop lines, other road markers, etc.

[0101] Subsequently, in step S630, a polyline can be generated based on lane information. For example, a polyline can be generated by applying the Douglas Picker algorithm to the lane information.

[0102] Subsequently, in step S640, a polyline map can be generated based on the generated polyline.

[0103] Subsequently, in step S650, a precision map can be generated by applying spatial information to the polyline map. For example, the vehicle's location coordinates and driving direction can be matched to the polyline map, and an object can be mapped to the polyline map. As a result of the matching, a precision map can be generated.

[0104] FIGS. 7 to 10 are flowcharts for explaining in detail a method for generating a precise map according to an embodiment of the present invention.

[0105] The precise map generation method of Fig. 7 can be performed by the image preprocessing module (110) of Fig. 2.

[0106] Referring to FIG. 7, in step S611, a binary image can be generated by applying semantic segmentation to a time-series image. For example, a binary image can be generated in which lanes are set to a specific class.

[0107] Then, in step S612, depth information for the time series image can be generated by applying a monocular depth estimation technique to the time series image. Subsequently, the depth information can be used for polyline position correction.

[0108] The precise map generation method of Fig. 8 can be performed by the lane extraction module (120) of Fig. 2.

[0109] Referring to Fig. 8, in step S621, lane edges can be extracted from the binary image.

[0110] Subsequently, in step S622, lane feature points can be calculated from the extracted edges.

[0111] Subsequently, in step S623, the similarity of feature points can be calculated by calculating the distance and angle between the extracted feature points. Here, the similarity may be the degree of similarity regarding the distance and angle between feature points.

[0112] Subsequently, in step S624, feature points can be classified based on the calculated similarity. Here, an index for the group to which the feature point belongs can be assigned to the feature point.

[0113] The precise map generation method of FIG. 9 can be performed by the polyline generation module (130) of FIG. 2.

[0114] Referring to Fig. 9, in step S631, a polyline can be generated by applying the Douglas Picker algorithm to the classified feature points.

[0115] Subsequently, in step S632, the similarity of the polylines can be calculated by computing the distance and angle between the generated polylines. Here, the similarity may be the degree of similarity regarding the distance and angle between the polylines.

[0116] Subsequently, in step S633, polylines can be classified based on the calculated similarity. Clustering of polylines can be performed by including polylines with high similarity into a single group.

[0117] Subsequently, in step S634, outliers may be removed based on the length of the polyline and the width of the lane. For example, outliers may be removed based on a polyline of 2 m or more and a lane width of 15 to 20 cm. However, this is merely an example. The present disclosure is not limited thereto.

[0118] Subsequently, in step S641, the position coordinates of the distorted polyline can be corrected based on depth information. For example, the position of the polyline may be distorted by ground gradient, and the position coordinates of the polyline can be corrected based on the received depth information.

[0119] Subsequently, in step S642, a weighted line fitting algorithm is applied to the polylines located in the overlapping region to fuse the polylines. In other words, overlapping polylines in the time-series image can be fused into a single polyline.

[0120] Subsequently, in step S643, a polyline map can be generated based on the fused polylines.

[0121] The precise map generation method of FIG. 10 can be performed by the map generation module (140) of FIG. 2.

[0122] Referring to FIG. 10, in step S651, the location coordinates and driving direction of an object and a vehicle can be calculated based on spatial information. For example, the location coordinates of the vehicle, the driving direction, speed, etc., can be calculated. In addition, static objects such as stoppers, curbs, pillars, traffic cones, and trees can be detected.

[0123] Subsequently, in step S652, the vehicle's position coordinates and driving direction calculated on the polyline map can be matched.

[0124] Subsequently, in step S653, a time series polyline map can be generated based on the matching results.

[0125] Subsequently, in step S654, drift correction can be performed on the time-series polyline map using a loop closing technique. Accordingly, the entire path of the vehicle for a specific area can be precisely corrected.

[0126] Subsequently, in step S655, a precision map can be generated by mapping objects onto a time-series polyline map.

[0127] As described above, the precision map generation method for autonomous driving according to the present invention can improve the reliability of the precision map by enhancing drift compensation and positioning accuracy. Furthermore, the precision map generation method can improve computational efficiency by reducing memory usage and computation processing time by generating a precision map using only lane information. Additionally, since the precision map generation method provides a precision map with high accuracy, it can improve the reliability of autonomous driving and autonomous parking of a vehicle.

[0128] FIG. 11 is a block diagram showing a computing system (1000) according to one embodiment of the present invention.

[0129] Referring to FIG. 11, the computing system (1000) may include at least one processor (1100), memory (1300), user interface input device (1400), user interface output device (1500), storage (1600), and network interface (1700) connected via a bus (1200).

[0130] The processor (1100) may be a semiconductor device that executes processing on instructions stored in a central processing unit (CPU) or memory (1300) and / or storage (1600). The memory (1300) and storage (1600) may include various types of volatile or non-volatile storage media. For example, the memory (1300) may include ROM (Read Only Memory) and RAM (Random Access Memory).

[0131] Accordingly, the steps of the method or algorithm described in connection with the embodiments disclosed in this specification may be directly implemented in hardware, software modules, or a combination of both, executed by the processor (1100). The software modules may reside in storage media such as RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs (i.e., memory (1300) and / or storage (1600)).

[0132] An exemplary storage medium is coupled to a processor (1100), and the processor (1100) can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium may be integrated with the processor (1100). The processor and the storage medium may reside within an application-specific integrated circuit (ASIC). The ASIC may reside within a user terminal. Alternatively, the processor and the storage medium may reside as separate components within the user terminal.

[0133] The user interface input device (1400) may include an input device that receives user input.

[0134] The user interface output device (1500) may include a display that displays various information related to the driving and / or functions of the vehicle, and a speaker that outputs various sounds related to the driving and / or functions of the vehicle.

[0135] Here, the display may provide a user interface for the occupant and the vehicle to interact. For example, the display may include a Liquid Crystal Display (LCD) panel and / or a Light Emitting Diode (LED).

[0136] The display can provide various information to the user based on control signals from the processor (1100). For example, the display may be installed in the center fascia, which is the central area of ​​the dashboard inside the vehicle, and the display unit may be a component of the head unit or a component of a navigation device provided separately from the head unit. Here, the head unit can process and output audio and video signals, and it is also possible to perform navigation functions. Therefore, the head unit is also referred to as an AVN (Audio Video Navigation) device. For example, the display can display a route guidance screen, that is, a screen necessary to perform navigation functions. In addition, the display can display additional screens necessary to perform audio functions, video functions, or phone call functions.

[0137] The network interface (1700) may include a long-distance communication module and / or a short-distance communication module that transmits and receives data with an external device (e.g., server (2), user terminal). For example, the network interface (1700) may refer to a communication module capable of performing wireless internet communication such as Wireless LAN (WLAN), Wireless Broadband (Wibro), WiFi, World Interoperability for Microwave Access (Wimax), and High Speed ​​Downlink Packet Access (HSDPA).

[0138] For example, the user can input a destination through a user interface input device (1400), and the user interface output device (1500) can provide a path to reach the destination.

[0139] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential characteristics of the present invention.

[0140] Accordingly, the embodiments disclosed in this invention are intended to illustrate, not limit, the technical concept of the invention, and the scope of the technical concept of the invention is not limited by these embodiments. The scope of protection of this invention shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of this invention.

Claims

Claim 1 A precision map generating device comprising: a communication unit for receiving time-series images and spatial information regarding the surrounding environment of a vehicle; a memory for storing the received time-series images and spatial information; and a processor for generating a precision map based on the time-series images and spatial information, wherein the processor extracts lane information based on the time-series images, generates a polyline based on the extracted lane information, and generates the precision map for autonomous driving based on the polyline and spatial information. Claim 2 A precision map generation device according to claim 1, wherein the processor comprises: an image preprocessing module that preprocesses the time-series image by applying a deep learning model; a lane extraction module that detects a road lane in the preprocessed time-series image and extracts lane information; a polyline generation module that generates the polyline based on the lane information and generates a polyline map based on the generated polyline; and a map generation module that generates the precision map by applying the spatial information to the polyline map. Claim 3 A precision map generation device according to claim 2, wherein the image preprocessing module generates a binary image by applying semantic segmentation to the time series image. Claim 4 A precision map generation device according to claim 3, wherein the lane extraction module extracts the edge of the lane from the binary image, calculates the feature point of the lane from the extracted edge, calculates the similarity of the feature point by calculating the distance and angle between the extracted feature points, and classifies the feature point based on the calculated similarity. Claim 5 A precision map generation device according to claim 4, wherein the polyline generation module generates the polyline by applying a Douglas-Peucker algorithm to the classified feature points, calculates the similarity of the polyline by calculating the distance and angle between the generated polylines, and classifies the polyline based on the calculated similarity. Claim 6 A precision map generation device according to claim 5, wherein the image preprocessing module generates depth information for the time series image by applying a mono-depth estimation technique to the time series image. Claim 7 A precision map generating device according to claim 6, wherein the polyline generating module removes outliers based on the length of the polyline and the width of the lane, and corrects the position coordinates of the distorted polyline based on the depth information. Claim 8 A precision map generation device according to claim 7, wherein the polyline generation module fuses the polylines by applying a weighted line fitting algorithm to the polylines located in the overlapping areas of each frame of the time series image, and generates the polyline map based on the fused polylines. Claim 9 A precision map generation device according to claim 8, wherein the map generation module calculates the position coordinates and driving direction of an object and a vehicle based on the spatial information, matches the calculated position coordinates and driving direction to the polyline map, and generates a time-series polyline map based on the matching result. Claim 10 A precision map generation device according to claim 9, wherein the map generation module performs drift compensation on the time-series polyline map using a loop closing technique and generates the precision map by mapping the object to the time-series polyline map. Claim 11 A method for generating a precision map, comprising: receiving time-series images and spatial information regarding the surrounding environment of a vehicle; storing the received time-series images and spatial information; and generating a precision map based on the time-series images and spatial information; wherein the step of generating the precision map comprises: extracting lane information based on the time-series images, generating a polyline based on the extracted lane information, and generating the precision map for autonomous driving based on the polyline and spatial information. Claim 12 A method for generating a precision map according to claim 11, wherein the step of generating the precision map for autonomous driving comprises: a step of preprocessing the time series image by applying a deep learning model; a step of detecting lanes of a road in the preprocessed time series image and extracting lane information; a step of generating the polyline based on the lane information and generating a polyline map based on the generated polyline; and a step of generating the precision map by applying the spatial information to the polyline map. Claim 13 A method for generating a precise map according to claim 12, wherein the step of preprocessing the time series image comprises the step of generating a binary image by applying semantic segmentation to the time series image. Claim 14 A method for generating a precise map according to claim 13, wherein the step of extracting lane information comprises: extracting an edge of the lane from the binary image; calculating a feature point of the lane from the extracted edge; calculating the distance and angle between the extracted feature points to calculate the similarity of the feature points; and classifying the feature points based on the calculated similarity. Claim 15 A method for generating a precise map according to claim 14, wherein the step of generating the polyline map comprises: generating the polyline by applying a Douglas-Peucker algorithm to the classified feature points; calculating the similarity of the polyline by calculating the distance and angle between the generated polylines; and classifying the polylines based on the calculated similarity. Claim 16 A method for generating a precise map according to claim 15, wherein the step of preprocessing the time series image further comprises the step of generating depth information for the time series image by applying a mono-depth estimation technique to the time series image. Claim 17 A method for generating a precise map according to claim 16, wherein the step of generating the polyline map further comprises: a step of removing outliers based on the length of the polyline and the width of the lane; and a step of correcting the position coordinates of the distorted polyline based on the depth information. Claim 18 A method for generating a precise map according to claim 17, wherein the step of generating the polyline map further comprises: a step of fusing the polylines by applying a line fitting algorithm based on weights to the polylines located in the overlapping areas of each frame of the time series image; and a step of generating the polyline map based on the fused polylines. Claim 19 A method for generating a precision map according to claim 18, wherein the step of generating the precision map comprises: a step of calculating the position coordinates of an object and a vehicle and a driving direction based on the spatial information; a step of matching the calculated position coordinates and the driving direction to the polyline map; and a step of generating a time-series polyline map based on the matching result. Claim 20 A method for generating a precision map according to claim 19, wherein the step of generating the precision map further comprises: a step of performing drift compensation on the time series polyline map using a loop closing technique; and a step of generating the precision map by mapping the object to the time series polyline map.

Images