A method for providing an auto-labeling assistance service for generating training data and a server using the same.

Abstract

This document provides a method and server for providing an auto-labeling support service for generating training data. [Solution] The method includes S120, in which the server causes the labeler to select a specific lane detection model suitable for a target image through a labeling interface displayed on the labeler terminal, and S130, in which, once first point information to the k-th point information (where k is an integer of 2 or more) on the specific lane selected by the labeler to label the specific lane among at least one lane located in the target image is obtained through the labeling interface, the server generates a first spline model having m spline points (where m is an integer of 2 or more) corresponding to the specific lane in the target image and displays it through the labeling interface, thereby providing an auto-labeling assistance service.

Description

Technical Field

[0001] The present invention relates to a method for providing an auto-labeling assistance service for learning data generation and a server using the same.

Background Art

[0002] Technologies utilizing artificial intelligence have been developed and utilized across all industries. For the utilization of artificial intelligence, learning of artificial intelligence models such as deep learning models is essential.

[0003] For the learning of artificial intelligence models, a large amount of learning data is required. To obtain learning data, labeling information regarding raw data and original data is necessary.

[0004] Since the labeling work must be performed by a labeler, research institutions and developers of artificial intelligence may sometimes have labelers who perform the labeling work.

[0005] However, since the labeling work is often a task that anyone can perform without specialized knowledge, platforms have emerged that distribute data to a large number of workers via the Internet and pay remuneration for the labeling work in order to reduce labor costs. For example, there is Amazon's Mechanical Turk.

[0006] On the other hand, for the labeling work for learning data generation, the labeler has to directly perform the labeling while checking the original data such as images one by one, so a lot of time is required for the labeling work, and a uniform quality for the labeling results cannot be guaranteed.

[0007] In particular, there are differences in the level of labeling skill among labelers, which leads to not only differences in work speed among labelers when performing labeling work on the same data, but also to problems with differences in the quality of the labeling results.

[0008] Furthermore, when the same labeler performs the labeling work, there are problems not only with differences in the speed of labeling for each data set, but also with differences in the quality of the labeling results for each data set. [Overview of the project] [Problems that the invention aims to solve]

[0009] The purpose of this invention is to solve all of the problems of the prior art described above.

[0010] Furthermore, the present invention also aims to reduce the time required for labeling work for generating training data through interaction with a labeler, and to improve the quality of the labeling work. Furthermore, another objective of the present invention is to minimize the speed and quality deviations of the labeling work for each labeler. Furthermore, another objective of the present invention is to minimize the speed and quality deviations of the labeling process for each data point. [Means for solving the problem]

[0011] According to one embodiment of the present invention, in a method for providing an auto-labeling assistance service for generating learning data, (a) when auto-labeling assistance service request information is obtained from a labeler terminal, the server provides a labeling interface to the labeler terminal, so that the labeler, through the labeling interface displayed on the labeler terminal, can refer to at least one lane detection result among the first lane detection results to the nth lane detection results obtained by applying each of the first lane detection model to the nth lane detection model (where n is an integer of 1 or more) to the target image, and select a specific lane detection model suitable for the target image from among the first lane detection model to the nth lane detection model; and (b) the labeler selects a specific lane among the at least one lane located in the target image. A method is provided which, once first point information to k-th point information (where k is an integer of 2 or more) on a particular lane selected for labeling is obtained through the labeling interface, the server provides an auto-labeling assistance service by using a semi-auto labeling module to generate a first spline model having m spline points (where m is an integer of 2 or more) corresponding to the particular lane in the target image, by referring to the first point information to k-th point information obtained through the labeling interface and at least a portion of the segmentation map and clustering map corresponding to the specific lane detection result of the particular lane model, and displaying the first spline model through the labeling interface.

[0012] In one example, (c) if the labeler changes the m spline points constituting the first spline model to m' (where m' is an integer greater than or less than m) in order to optimize the first spline model confirmed through the labeling interface, the server provides the auto-labeling assistance service by having a line pose regression module regress the shape of the first spline model to generate a second spline model having the m' spline points and displaying it through the labeling interface;

[0013] In one example, in step (c), the server uses the line pose regression module to optimize the pose coefficients and width coefficients of the first spline model for each of the m' spline points by referencing at least a portion of the edge map, gradient map, and magnitude map corresponding to the target image through a metaheuristic optimization algorithm, thereby generating the second spline model localized to the target image.

[0014] The method according to claim 3, in one example, the server causes the line pose regression module to generate optimized position information and optimized spline width for each of the m' spline points in the parameter map, which references at least a portion of the edge map, the gradient map, and the magnitude map, through the Nelder-Mead algorithm, which is a metaheuristic optimization algorithm, wherein each of the m' spline points has a function value that minimizes the cubic polynomial coefficients due to the pose of the first spline model and the linear polynomial coefficients due to the spline width of the first spline model.

[0015] In one example, in step (b), the server uses the semi-automatic labeling module to (i) cluster at least one lane segment edge for a specific lane of the target image (the edge represents at least a portion of both ends in the lane direction of the lane segment) using a feature map generated by referencing at least a portion of the segmentation map and the clustering map; (ii) generate at least two anchor points for connecting the lane segments; (iii) generate a graphical model for connecting the lane segments; (iv) determine the optimal path for connecting the lane segments by referring to the graphical model; and (v) generate the first spline model having the m spline points by determining the m spline points and corresponding m spline widths for generating splines for the lane segments through a metaheuristic algorithm.

[0016] In one example, the server uses the semi-automatic labeling module to determine the optimal route for connecting the lane segments by referring to the graphical model, using the A* algorithm to determine the optimal route based on the connection score of the lane segments in the graphical model.

[0017] In one example, the server uses the semi-automatic labeling module to determine the m spline points and m spline widths for generating the splines for the lane segment through a metaheuristic algorithm, wherein the metaheuristic algorithm uses the firefly algorithm to determine the m spline points and m spline widths.

[0018] In one example, in step (a) above, the server overlays each of the first lane detection results from the first lane detection model or the n-lane detection results from the n-lane detection model onto the target image via the labeling interface, so that the labeler can check each of the first lane detection results or the n-lane detection results overlaid on the target image and select a specific lane detection model that corresponds to the specific lane detection result suitable for the target image.

[0019] In one example, the first lane detection model to the n lane detection model includes at least a portion of the following: a cat's eye lane detection model, a zigzag lane detection model, a double lane detection model, a general lane detection model, a merging / diverging lane detection model, and an intersection lane detection model.

[0020] In one example, the segmentation map is a map obtained by segmenting at least one lane detected in the target image through the specific lane detection model, and the clustering map is a map obtained by clustering the at least one lane according to whether or not it is the same lane.

[0021] Furthermore, according to another embodiment of the present invention, a server providing an auto-labeling assistance service for generating learning data includes: a memory storing instructions for providing the auto-labeling assistance service for generating learning data; and a processor that performs operations for providing the auto-labeling assistance service for generating learning data based on the instructions stored in the memory, wherein the processor (i) when auto-labeling assistance service request information is obtained from a labeler terminal, provides a labeling interface to the labeler terminal, so that the labeler can, through the labeling interface displayed on the labeler terminal, refer to at least one lane detection result among the first lane detection results to the nth lane detection results obtained by applying each of the first lane detection models to the nth lane detection models (where n is an integer of 1 or more) to the target image and apply the first lane detection model to the nth lane detection model to the target image (II) A process is provided for selecting a suitable specific lane detection model, and (II) when first point information to the kth point information (where k is an integer of 2 or more) on the specific lane selected by the labeler to label the specific lane located in the target image is obtained through the labeling interface, a server is provided that performs the process of providing the auto-labeling assistance service by using a semi-auto-labeling module to generate a first spline model having m spline points (where m is an integer of 2 or more) corresponding to the specific lane in the target image by referring to the first point information to the kth point information obtained through the labeling interface and at least a portion of the segmentation map and clustering map corresponding to the specific lane detection result of the specific lane model, and displaying the first spline model through the labeling interface.

[0022] In one example, the processor further performs a process of providing the auto-labeling assistance service by changing m spline points that constitute the first spline model to m' (where m' is an integer greater than or less than m) to optimize the first spline model confirmed by the labeler through the labeling interface, and then using the line pose regression module to regress the shape of the first spline model to generate a second spline model having the m' spline points, and displaying the second spline model through the labeling interface.

[0023] In one example, in the process (III), the processor uses the line pose regression module to optimize the pose coefficients and width coefficients of the first spline model for each of the m' spline points by referring to at least a part of an edge map, a gradient map, and a magnitude map corresponding to the target image through a metaheuristic optimization algorithm, so as to generate the second spline model localized to the target image.

[0024] In one example, the processor uses the line pose regression module to generate, for each of the m' spline points in a parameter map that refers to at least a part of the edge map, the gradient map, and the magnitude map through the Nelder-Mead algorithm, which is the metaheuristic optimization algorithm, the optimized position information and the optimized spline width for each of the m' spline points, each having a function value that minimizes the cubic polynomial coefficient due to the pose of the first spline model and the linear polynomial coefficient due to the spline width of the first spline model.

[0025] In one example, in the (II) process, the processor uses the semi-automatic labeling module to: (i) cluster edges of at least one lane segment of the target image with respect to the specific lane using a feature map generated by referring to at least a part of the segmentation map and the clustering map (the edges indicate at least a part of both ends in the lane direction in the lane segment); (ii) generate at least two anchor points for connecting the lane segments; (iii) generate a graphical model for connecting the lane segments; (iv) determine an optimal path for connecting the lane segments by referring to the graphical model; and (v) generate the first spline model having the m spline points by determining the m spline points and the corresponding m spline widths for generating a spline for the lane segment through a metaheuristic algorithm.

[0026] In one example, in determining the optimal path for connecting the lane segments by referring to the graphical model in (v), the processor uses the A* algorithm to determine the optimal path based on the connection score of the lane segments in the graphical model using the semi-automatic labeling module.

[0027] In one example, in determining the m spline points and the m spline widths for generating the spline for the lane segment through a metaheuristic algorithm in (vi), the processor uses the firefly algorithm as the metaheuristic algorithm to determine the m spline points and the widths of the m splines using the semi-automatic labeling module.

[0028] In one example, the processor, in process (I), overlays each of the first lane detection result from the first lane detection model or the n-th lane detection result from the n-th lane detection model onto the target image through the labeling interface, so that the labeler can check each of the first lane detection result to the n-th lane detection result overlaid on the target image and select a specific lane detection model that corresponds to the specific lane detection result suitable for the target image.

[0029] The first lane detection model to the n lane detection model includes at least a portion of the following: a cat's eye lane detection model, a zigzag lane detection model, a double lane detection model, a general lane detection model, a merging / diverging lane detection model, and an intersection lane detection model.

[0030] In one example, the segmentation map is a map obtained by segmenting at least one lane detected in the target image through the specific lane detection model, and the clustering map is a map obtained by clustering the at least one lane according to whether or not it is the same lane. [Effects of the Invention]

[0031] According to the present invention, the time required for labeling work for generating training data can be reduced through interaction with the labeler, and the quality of the labeling work can be improved. According to the present invention, it becomes possible to minimize the speed deviation and quality deviation of the labeling work for each labeler. According to the present invention, it becomes possible to minimize the speed deviation and quality deviation of the labeling work for each data point. [Brief explanation of the drawing]

[0032] The following drawings, attached for use in describing embodiments of the present invention, represent only a portion of embodiments of the present invention, and a person with ordinary skill in the art to which the present invention pertains (hereinafter referred to as "ordinary art") can obtain the other drawings from these drawings without performing any inventive work. Figure 1 shows a simplified representation of a server providing an auto-labeling support service for generating training data according to one embodiment of the present invention. Figure 2 shows a simplified method for providing an auto-labeling support service for generating training data according to one embodiment of the present invention. Figure 3 shows a simplified example of a labeling interface provided to a labeler terminal in a method for providing an auto-labeling assistance service for generating learning data according to one embodiment of the present invention. Figure 4 illustrates the results of lane detection using a general lane detection model and the results of lane detection using a specific lane detection model according to the present invention, in different lane environments. Figures 5a to 5g show a simplified process of providing an auto-labeling support service through semi-auto-labeling in a method for providing an auto-labeling support service for generating learning data according to one embodiment of the present invention. Specific details for carrying out the invention

[0033] The detailed description of the present invention, as described below, refers to the accompanying drawings illustrating specific embodiments in which the present invention can be carried out. These embodiments are described in sufficient detail so that those skilled in the art can carry out the present invention. It should be understood that the various embodiments of the present invention are different from one another but do not need to be mutually exclusive. For example, certain shapes, structures and characteristics described herein may be embodied by modifying one embodiment to another without departing from the spirit and scope of the present invention. It should also be understood that the position or arrangement of individual components within each embodiment may be modified without departing from the spirit and scope of the present invention. Therefore, the detailed description below should not be taken as restrictive, and the scope of the present invention should be accepted as encompassing the scope claimed in the claims and all equivalent scopes thereto. Similar reference numerals in the drawings refer to parts that are identical or have similar functions across various aspects.

[0034] Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that persons with ordinary skill in the art to which the present invention pertains can easily implement the present invention.

[0035] Figure 1 is a simplified representation of a server providing an auto-labeling assistance service for generating training data according to one embodiment of the present invention. Referring to Figure 1, the server 1000 may include a memory 1100 that stores instructions for providing the auto-labeling assistance service for generating training data, and a processor 1200 that performs operations to provide the auto-labeling assistance service for generating training data based on the instructions stored in the memory 1100.

[0036] Specifically, the server 1000 may, but is not limited to, achieve desired system performance by utilizing a combination of typical computing devices (e.g., devices that may include computer processors, memory, storage, input and output devices, and other components of existing computing devices; electronic communication devices such as routers and switches; and electronic information storage systems such as networked storage (NAS) and storage area networks (SANs)) and computer software (i.e., instructions for making the computing devices function in a particular way).

[0037] Furthermore, the processor 1200 of server 1000 may include hardware configurations such as an MPU (Micro Processing Unit) or CPU (Central Processing Unit), cache memory, and data bus. The computing device may also further include operational structures and software configurations for applications that perform specific purposes.

[0038] However, this does not preclude the case where server 1000 includes an integrated processor in which medium, processor and memory are integrated for carrying out the present invention.

[0039] On the other hand, when the processor 1200 of the server 1000 obtains auto-labeling assistance service request information from the labeler terminal 2000 based on instructions stored in the memory 1100, it can provide a labeling interface to the labeler terminal 2000, thereby enabling the labeler to perform a process in which, through the labeling interface displayed on the labeler terminal 2000, refer to at least one lane detection result from the first lane detection result to the nth lane detection result obtained by applying each of the first lane detection model to the nth lane detection model to the target image, and select a specific lane detection model from the first lane detection model to the nth lane detection model that is suitable for the target image. Then, if the processor 1200 obtains first point information to the kth point information, which is at least two point pieces on a specific lane selected by the labeler to label a specific lane in the target image, through the labeling interface, the processor 1200 can perform a process to provide an auto-labeling support service by using a semi-auto-labeling module to generate a first spline model having two or more m spline points corresponding to a specific lane in the target image by referring to the first point information to the kth point information obtained through the labeling interface and at least a portion of the segmentation map and clustering map corresponding to the specific lane detection result of the specific lane model, and displaying the first spline model through the labeling interface.

[0040] In addition, the server 1000 can perform a process to provide an auto-labeling assistance service by, if the labeler changes the m spline points constituting the first spline model to a number greater than or less than m' in order to optimize the first spline model confirmed through the labeling interface, using the line pose regression module to regress the shape of the first spline model to generate a second spline model with m' spline points, and displaying it through the labeling interface.

[0041] On the other hand, although Figure 1 shows only one labeler terminal 2000, this is for the sake of explanation; multiple labelers corresponding to multiple labeler terminals can perform labeling work using the auto-labeling support service through multiple labeler terminals.

[0042] A more detailed explanation of how to provide an auto-labeling service for generating training data on the server 1000 configured in this way can be found in Figure 2, as follows.

[0043] First, once the labeler terminal 2000 receives information requesting the auto-labeling assistance service, the server 1000 can provide the labeling interface to the labeler terminal 2000 (S110).

[0044] This allows the labeler to, through the labeling interface displayed on the labeler terminal 2000, refer to at least one lane detection result from the first to the nth lane detection models, obtained by applying each of the first to the nth lane detection models to the target image, and select a specific lane detection model suitable for the target image from among the first to the nth lane detection models (S120).

[0045] As an example, referring to Figure 3, the labeling interface 10 may include a settings menu 11, a target image selection menu 12, a lane detection model selection menu 13, and a display area 14.

[0046] The labeler can configure the labeling work environment through the setting menu 11 of the labeling interface 10 displayed on the labeler terminal 2000. If the labeler inputs labeler information through the setting menu 11 of the labeling interface 10, the server 1000 can retrieve an image set for the labeling work corresponding to the labeler information from a database (not shown) and provide a list of the image set to the target image selection menu 12. At this time, the server 1000 can either display a list of the images included in the image set in the target image selection menu 12, or display thumbnails of the images. In this way, the labeler can select the target image to perform the current labeling work from the target image selection menu 12 of the labeling interface 10, and the server 1000 can then display the target image selected by the labeler in the display area 14 of the labeling interface 10.

[0047] The labeler can then select a specific lane detection model suitable for the target image by selecting a first lane detection model or an nth lane detection model provided through the lane detection model selection menu 13 of the labeling interface 10 and applying them to the target image. At this time, the labeler can select a specific lane detection model that is expected to be suitable for the target image through the model names of each of the first lane detection model or the nth lane detection model, or by repeatedly applying any lane detection model from the first lane detection model to the nth lane detection model to select a specific lane detection model suitable for the target image. It is also possible to select a specific lane detection model suitable for the target image before applying all of the lane detection models, i.e., each of the n lane detection models. Furthermore, although it has been assumed that the number of lane detection models provided through the lane detection model selection menu 13 is n, the number of lane detection models provided through the lane detection model selection menu 13 may be n', which is a number greater than n, and in this case, it may mean that n lane detection models have been selected from n'.

[0048] In other words, the server 100 overlays the first lane detection result from the first lane detection model or the nth lane detection result from the nth lane detection model onto the target image displayed in the display area 14 via the labeling interface 10. This allows the labeler to check the first lane detection result or the nth lane detection result overlaid on the target image and select a specific lane detection model that corresponds to a specific lane detection result suitable for the target image.

[0049] In this case, the first lane detection model to the nth lane detection model is a lane detection device that detects lanes in an image based on a deep learning network, and may include at least a part of a cat's eye lane detection model, a zigzag lane detection model, a double lane detection model, a general lane detection model, a merging / diverging lane detection model, or an intersection lane detection model.

[0050] As an example, referring to Figure 4, Figure 4 shows the results of detecting lanes using a general lane detection model and the results of detecting lanes using a specific lane detection model according to the present invention in different lane environments. The upper part of Figure 4 shows the results of detecting lanes using a general lane detection model and the results of detecting lanes using the zigzag lane detection model, which is a specific lane detection model according to the present invention, in an image of a zigzag lane. The lower part of Figure 4 shows the results of detecting lanes using a general lane detection model and the results of detecting lanes using the double lane detection model, which is a specific lane detection model according to the present invention, in an image of a double lane.

[0051] As shown in Figure 4, lane detection results using a general lane detection model cannot accurately detect lanes depending on the lane environment in the image, i.e., the type of lane, making it difficult for labelers to use them for labeling work. However, according to the present invention, by allowing labelers to directly check and select a specific lane detection model that can accurately detect lanes depending on the lane environment in the image, the accurate lane detection results of the specific lane detection model selected based on the lane environment can be used for lane labeling work, thereby minimizing the difference in difficulty for each image.

[0052] Next, referring again to Figure 2, once the labeler has acquired first point information to the kth point information, which is information about two or more points on a specific lane selected for labeling a specific lane within at least one lane located in the target image, through the labeling interface, the server 1000 can use a semi-automatic labeling module to generate a first spline model having two or more m spline points corresponding to the specific lane in the target image by referring to the first point information to the kth point information acquired through the labeling interface and at least a portion of the segmentation map and clustering map corresponding to the specific lane detection result of the specific lane model, and can provide an auto-labeling assistance service (S130) by displaying it through the labeling interface. At this time, the segmentation map is a map segmented from at least one lane detected from the target image through the specific lane detection model, and the clustering map may be a map clustered from at least one lane based on whether or not it is the same lane. At this time, the spline model may be a diagrammatic display of a spline curve for indicating a lane.

[0053] As an example, the process of providing an auto-labeling support service that performs semi-automatic labeling through a semi-automatic labeling module can be described as follows, with reference to Figures 5a to 5g.

[0054] As shown in Figure 5a, the labeler can select a first point and a second point on a specific lane 20 in order to label that specific lane 20 within the lanes on the target image.

[0055] In this way, as shown in Figure 5b, the server 1000 can use a semi-automatic labeling module to cluster the edges of at least one lane segment for a specific lane in the target image, using a feature map generated by referencing at least a portion of the segmentation map and clustering map based on the specific lane detection results of a specific lane detection model, thereby generating an initial specific lane group. At this time, the edges can represent at least a portion of both ends of the lane direction in the lane segment. That is, the server 1000 can use a semi-automatic labeling module to cluster edges 1 to 6, which are edges of lane segments having the same lane direction in the feature map, into a single cluster (not shown).

[0056] Then, as shown in Figure 5c, the server 1000 can generate at least two anchor points for connecting lane segments using a semi-automatic labeling module. For reference, Figure 5c shows that one anchor point was generated for each of the three lane segments, but the present invention is not limited to this. Anchor points can also be generated at uniform intervals according to the lane direction of a specific lane confirmed by lane edge clustering, or anchor points can be generated using grid cells corresponding to a specific lane. Here, it should be clarified that the concept of "lane direction" is conceptually represented as a single lane direction (i.e., belonging to the same lane), even if, for example, the lane direction (i.e., vector direction) physically changes depending on the position in the case of a curved road lane.

[0057] Thereafter, as shown in Figure 5d, the server 1000 can generate a graphical model for connecting lane segments using a semi-automatic labeling module. In this case, the graphical model models the probability that each lane segment will be connected to each other, and can be a probabilistic model that represents which lane segment a separated lane segment should be connected to. For reference, Figure 5d shows a simplified graphical model for three lane segments, illustrating a situation where the probability of the top lane segment being connected to the middle lane segment is 0.9, the probability of the top lane segment being connected to the bottom lane segment is 0.1, and the probability of the middle lane segment being connected to the bottom lane segment is 1.0.

[0058] As shown in Figure 5e, the server 1000 can use a semi-automatic labeling module to determine the optimal route for connecting lane segments by referring to a graphical model. In this case, the semi-automatic labeling module can determine the optimal route for connecting lane segments by referring to a graphical model, using the A* algorithm to determine the optimal route based on the lane segment connection score, i.e., the connection probability value, in the graphical model. For reference, in Figure 5e, the probability that the topmost lane segment is connected to the middle lane segment is 0.9, and the probability that the topmost lane segment is connected to the bottommost lane segment is 0.1. Therefore, the topmost lane segment can be determined to be connected to the middle lane segment, and the middle lane segment can be determined to be connected to the bottommost lane segment with a probability of 1.0. Thus, the determined optimal route in Figure 5e can be determined to be a route that connects the top lane segment, the middle lane segment, and the bottommost lane segment. Furthermore, although not shown in the diagram, assuming that one lane segment is detected at the top and the lane segment directly below it is detected as a double lane, the lane segment corresponding to the top lane segment within the double lane can be determined through a process similar to that described above.

[0059] Thereafter, as shown in Figure 5f, the server 1000 can generate a first spline model having m spline points by using a semi-automatic labeling module to determine m spline points and corresponding m spline widths for generating splines for lane segments through a metaheuristic algorithm.

[0060] At this time, the server 1000, using a semi-automatic labeling module, can determine m spline points and m spline widths for generating splines for lane segments through a metaheuristic algorithm, and can use the firefly algorithm in the metaheuristic algorithm to determine the m spline points and m spline widths.

[0061] In other words, the server 1000 can generate a first spline model by using the Firefly algorithm to determine the optimal number of splines and the spline width at each position in order to generate a spline model from the optimal path using a semi-automatic labeling module.

[0062] Through this, the server 1000 provides an auto-labeling support service through semi-auto-labeling, enabling the labeler to perform lane labeling with minimal input.

[0063] Next, referring again to Figure 2, if the labeler changes the m spline points constituting the first spline model to m', which are greater than or less than m, in order to optimize the first spline model confirmed through the labeling interface, the server 1000 can use the line pose regression module to regress the shape of the first spline model to generate a second spline model with m' spline points, and then provide an auto-labeling assistance service by displaying it through the labeling interface (S140).

[0064] In other words, the server 1000, using a line pose regression module, optimizes the pose coefficients and width coefficients of the first spline model for each of the m' spline points by referencing at least a portion of the edge map, gradient map, and magnitude map corresponding to the target image through a metaheuristic optimization algorithm. This allows the server to generate a second spline model that is localized to the target image, i.e., different from the specific lane detection results corresponding to a specific lane detection model, and in which the spline pose and width are determined in the domain of the target image.

[0065] At this time, the server 1000 uses a line pose regression module to generate optimized position information and optimized spline widths for each of the m' spline points in the parameter map, which references at least a portion of the edge map, gradient map, and magnitude map. This is done by generating optimized position information and optimized spline widths for each of the m' spline points that have function values ​​that minimize the cubic polynomial coefficients due to the pose of the first spline model and the linear polynomial coefficients due to the spline width of the first spline model. In this way, the server 1000 can generate a second spline model localized to the target image.

[0066] As an example, the x-coordinate in the first spline model and the pause coefficient (S) and width coefficient (W) of the first spline model with respect to the spline width can be shown as follows. JPEG2026116652000002.jpg9102

[0067] In this case, i is the index representing each of the m' spline points, and t is the y-coordinate of each of the m' spline points, C i0 C is the intercept. i1 C is the gradient of a straight line. i2 C is the curvature of a quadratic curve.i3 This can represent the curvature of a cubic curve. That is, the Pause coefficient can include straight lines, quadratic curves, and cubic curves, and the width coefficient can be assumed to be linear to the segments between spline points.

[0068] Therefore, by searching for the optimal solution using the Nelder-Mead algorithm, we can generate the positions of m' spline points and their corresponding spline widths. JPEG2026116652000003.jpg1577

[0069] Here, L is the feature map This can show JPEG2026116652000004.jpg179, where E is the edge map generated from the target image, G is the gradient map generated from the target image, and M is the magnitude map generated from the target image.

[0070] In this way, the server 1000 provides an auto-labeling support service through line pose regression via interaction with the labeler, enabling the labeler to easily optimize the semi-auto-labeled first spline model for the target image.

[0071] In other words, the labeler checks whether the semi-automatically labeled first spline model accurately matches the lanes of the target image. If the first spline model does not accurately match the lanes of the target image, the labeler can add or remove spline points. This allows the server 1000 to directly verify the results of optimizing the first spline model by adding or removing spline points, using an algorithm different from the algorithm that performed the semi-automatic labeling, and without being dependent on the algorithm that performed the semi-automatic labeling.

[0072] For example, as shown in Figure 5f, in the first spline model generated by semi-automatic labeling, the spline lines in the areas of the uppermost lane segment and the intermediate lane segments may not fit accurately to the actual lanes.

[0073] Therefore, as shown in Figure 5g, the labeler can verify at the image level whether the first spline model is accurately fitted to the actual lane, and if it is not accurately fitted, it can add or remove spline points, thereby allowing the server 1000 to perform line pose regression through interaction with the labeler to generate a second spline model in which the first spline model is accurately fitted to the actual lane at the image level.

[0074] The embodiments of the present invention described above are embodied in the form of program instructions that can be executed through various computer components and can be stored on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., individually or in combination. The program instructions stored on the computer-readable recording medium may be specifically designed and configured for the present invention, or may be publicly known and available to those skilled in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code, such as that produced by a compiler, but also high-level language code executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform the processing according to the present invention, and vice versa.

[0075] Although the present invention has been described above with reference to specific components and other details, as well as limited embodiments and drawings, these are provided only to aid in a more general understanding of the invention. The present invention is not limited to the above embodiments, and various modifications and variations can be made from this description by those with ordinary skill in the art to which the invention pertains.

[0076] Therefore, the concept of the present invention should not be limited to the embodiments described above, and it can be said that not only the claims described later, but also all modifications that are equivalent or equivalent to the claims of this invention, fall within the scope of the concept of the present invention. [Explanation of Symbols]

[0077] 1000: Server 1100: Memory 1200: Processor

Claims

1. In a method for providing an auto-labeling support service for generating training data, (a) When an auto-labeling assistance service request information is obtained from the labeler terminal, the server provides a labeling interface to the labeler terminal, so that the labeler, through the labeling interface displayed on the labeler terminal, can refer to at least one lane detection result among the first lane detection results to the nth lane detection results obtained by applying each of the first lane detection models to the nth lane detection models (where n is an integer of 1 or more) to the target image, and select a specific lane detection model from the first lane detection model to the nth lane detection model that is suitable for the target image; and (b) Once the labeler has obtained, through the labeling interface, first point information to the k-th point information (where k is an integer of 2 or more) on the specific lane selected for labeling the specific lane among at least one lane located in the target image, the server provides an auto-labeling assistance service by using a semi-auto-labeling module to generate a first spline model having m spline points (where m is an integer of 2 or more) corresponding to the specific lane in the target image, by referring to the first point information to the k-th point information obtained through the labeling interface and at least a portion of the segmentation map and clustering map corresponding to the specific lane detection result of the specific lane model, and displaying the first spline model through the labeling interface; A method that includes this.

2. (c) If the labeler changes the m spline points constituting the first spline model to m' (where m' is an integer greater than or less than m) in order to optimize the first spline model confirmed through the labeling interface, the server provides the auto-labeling assistance service by having the server regress the shape of the first spline model using a line pose regression module to generate a second spline model having the m' spline points and displaying it through the labeling interface; The method according to claim 1, further comprising:

3. In step (c) above, The method according to claim 2, wherein the server uses the line pose regression module to generate a second spline model localized to the target image by referencing at least a portion of the edge map, gradient map, and magnitude map corresponding to the target image through a metaheuristic optimization algorithm to optimize the pose coefficients and width coefficients of the first spline model for each of the m' spline points.

4. The method according to claim 3, wherein the server uses the line pose regression module to generate optimized position information and optimized spline width for each of the m' spline points in the parameter map, which references at least a portion of the edge map, the gradient map, and the magnitude map, through the Nelder-Mead algorithm, which is a metaheuristic optimization algorithm, wherein each of the m' spline points has a function value that minimizes the cubic polynomial coefficients due to the pose of the first spline model and the linear polynomial coefficients due to the spline width of the first spline model.

5. In step (b) above, The method according to claim 1, wherein the server uses the semi-automatic labeling module to (i) cluster at least one lane segment edge for a specific lane of the target image (the edge represents at least a portion of both ends in the lane direction of the lane segment) using a feature map generated by referencing at least a portion of the segmentation map and the clustering map; (ii) generate at least two anchor points for connecting the lane segments; (iii) generate a graphical model for connecting the lane segments; (iv) determine the optimal path for connecting the lane segments by referring to the graphical model; and (v) generate the first spline model having the m spline points by determining the m spline points and corresponding m spline widths for generating splines for the lane segments through a metaheuristic algorithm.

6. The method according to claim 5, wherein the server uses the semi-automatic labeling module to determine the optimal route for connecting the lane segments by referring to the graphical model, and uses the A* algorithm to determine the optimal route by the connection score of the lane segments in the graphical model.

7. The method according to claim 5, wherein the server uses the semi-automatic labeling module to determine the m spline points and m spline widths for generating the splines for the lane segment through a metaheuristic algorithm, wherein the metaheuristic algorithm uses the firefly algorithm to determine the m spline points and m spline widths.

8. In step (a) above, The method according to claim 1, wherein the server overlays each of the first lane detection results from the first lane detection model to the n-th lane detection results from the n-th lane detection model onto the target image via the labeling interface, so that the labeler can check each of the first lane detection results to the n-th lane detection results overlaid onto the target image and select a specific lane detection model corresponding to the specific lane detection result suitable for the target image.

9. The method according to claim 8, wherein the first lane detection model to the n lane detection model includes at least a portion of a cat's eye lane detection model, a zigzag lane detection model, a double lane detection model, a general lane detection model, a merging / diverging lane detection model, and an intersection lane detection model.

10. The method according to claim 1, wherein the segmentation map is a map obtained by segmenting at least one lane detected in the target image through the specific lane detection model, and the clustering map is a map obtained by clustering the at least one lane according to whether or not it is the same lane.

11. In a server that provides an auto-labeling assistance service for generating training data, A memory containing instructions for providing an auto-labeling assistance service for generating training data; and A processor that performs operations to provide the auto-labeling assistance service for generating training data based on the instructions stored in the memory; Includes, The processor (i) when it obtains an auto-labeling assistance service request from the labeler terminal, provides the labeler terminal with a labeling interface, which allows the labeler to select a specific lane detection model suitable for the target image from among the first lane detection model to the nth lane detection model (where n is an integer of 1 or more) by referring to at least one lane detection result among the first lane detection results to the nth lane detection results obtained by applying each of the first lane detection model to the nth lane detection model to the target image through the labeling interface displayed on the labeler terminal; and (ii) the specific lane selected by the labeler to label a specific lane among the at least one lane located in the target image. A server that performs the process of providing the auto-labeling support service by, once first point information to k-th point information (where k is an integer of 2 or more) on a lane is acquired through the labeling interface, using a semi-auto labeling module to generate a first spline model having m spline points (where m is an integer of 2 or more) corresponding to the specific lane in the target image by referring to the first point information to k-th point information acquired through the labeling interface and at least a portion of the segmentation map and clustering map corresponding to the specific lane detection result of the specific lane model, and displaying the first spline model through the labeling interface.

12. The server according to claim 11, further performing the process of providing the auto-labeling assistance service by (III) if the labeler changes the m spline points constituting the first spline model to m' (where m' is an integer greater than or less than m) in order to optimize the first spline model confirmed through the labeling interface, then using a line pose regression module to regress the shape of the first spline model to generate a second spline model having the m' spline points, and displaying it through the labeling interface.

13. The server according to claim 12, wherein the processor, in the (III) process, uses the line pose regression module to optimize the pose coefficients and width coefficients of the first spline model for each of the m' spline points by referencing at least a portion of the edge map, gradient map, and magnitude map corresponding to the target image through a metaheuristic optimization algorithm, thereby generating the second spline model localized to the target image.

14. The server according to claim 13, wherein the processor causes the line pose regression module to generate optimized position information and optimized spline width for each of the m' spline points in the parameter map, which references at least a portion of the edge map, the gradient map, and the magnitude map, through the Nelder-Mead algorithm, which is a metaheuristic optimization algorithm, the m' spline points having function values ​​that minimize the cubic polynomial coefficients due to the pose of the first spline model and the linear polynomial coefficients due to the spline width of the first spline model.

15. The server according to claim 11, wherein the processor, in process (II), uses the semi-automatic labeling module to (i) cluster at least one lane segment edge for a specific lane of the target image (the edge represents at least a portion of both ends in the lane direction of the lane segment) using a feature map generated by referencing at least a portion of the segmentation map and the clustering map; (ii) generate at least two anchor points for connecting the lane segments; (iii) generate a graphical model for connecting the lane segments; (iv) determine the optimal path for connecting the lane segments by referencing the graphical model; and (v) generate the first spline model having the m spline points by determining the m spline points and corresponding m spline widths for generating splines for the lane segments through a metaheuristic algorithm.

16. The server according to claim 15, wherein the processor, using the semi-automatic labeling module, determines the optimal path for connecting the lane segments by referring to the graphical model, and uses the A* algorithm to determine the optimal path by the connection score of the lane segments in the graphical model.

17. The server according to claim 15, wherein the processor causes the semi-automatic labeling module to determine the m spline points and m spline widths for generating the splines for the lane segment through a metaheuristic algorithm, wherein the metaheuristic algorithm uses a firefly algorithm to determine the m spline points and m spline widths.

18. The server according to claim 11, wherein the processor, in process (I), overlays each of the first lane detection result by the first lane detection model or the n-th lane detection result by the n-th lane detection model onto the target image through the labeling interface, so that the labeler can check each of the first lane detection result to the n-th lane detection result overlaid on the target image and select a specific lane detection model corresponding to the specific lane detection result suitable for the target image.

19. The server according to claim 18, wherein the first lane detection model to the n lane detection model includes at least a portion of a cat's eye lane detection model, a zigzag lane detection model, a double lane detection model, a general lane detection model, a merging / diverging lane detection model, and an intersection lane detection model.

20. The server according to claim 11, wherein the segmentation map is a map obtained by segmenting at least one lane detected in the target image through the specific lane detection model, and the clustering map is a map obtained by clustering the at least one lane according to whether or not it is the same lane.

Images