Method and system for training road image generation model

Abstract

The present disclosure describes a method for learning a road image generative model, performed by at least one processor, comprising the steps of receiving high-definition map data for a specific area, receiving a plurality of actual road images captured within the specific area, and learning a road image generative model based on the high-definition map data and the plurality of actual road images, wherein each of the plurality of actual road images includes location information.

Description

Technology Field

[0001] The present disclosure relates to a method and system for training a road image generation model, and specifically, to a method and system for training a road image generation model based on high-precision map data and actual road images for a specific area. Background Technology

[0002] Autonomous driving technology refers to a technology that enables a vehicle to drive autonomously with minimal or no human intervention by perceiving the surrounding environment using radar, LIDAR (light detection and ranging), GPS, cameras, etc. For vehicles equipped with autonomous driving technology, lane detection is essential to determine their location and identify drivable spaces, thereby ensuring safe driving on the road.

[0003] Since actual driving environments involve various factors affecting autonomous driving—such as vehicles and traffic structures within the road area and buildings outside the road area—in addition to lanes marked on the road, a vast amount of data captured in diverse environments is required to secure high-performance lane detection capabilities.

[0004] On the other hand, there are physical limitations to collecting data from various real-world road environments to ensure applicability to all scenarios. Even if data were collected from diverse real-world road environments, there is a problem in that significant time and cost are incurred due to the annotation work performed to identify lanes. The problem to be solved

[0005] The present disclosure provides a method for solving the above-mentioned problems, a computer program stored on a recording medium, and a device (system). means of solving the problem

[0006] The present disclosure may be implemented in various ways, including a method, an apparatus (system), or a computer program stored on a readable storage medium.

[0007] According to one embodiment of the present disclosure, a method for learning a road image generative model performed by at least one processor comprises the steps of receiving high-definition map data for a specific area, receiving a plurality of actual road images captured within the specific area, and learning a road image generative model based on the high-definition map data and the plurality of actual road images, wherein each of the plurality of actual road images includes location information.

[0008] According to one embodiment of the present disclosure, a road image generation method performed by at least one processor comprises the steps of receiving a target lane mask image and generating a plurality of synthetic road images associated with the target lane mask image using a road image generation model, wherein the road image generation model is learned based on high-precision map data for a specific area and a plurality of actual road images captured within the specific area, and each of the plurality of actual road images includes location information.

[0009] A computer program stored on a computer-readable recording medium is provided for executing a road image generation model learning method according to one embodiment of the present disclosure on a computer.

[0010] According to one embodiment of the present disclosure, an information processing system comprises a communication module, a memory, and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program includes instructions for receiving high-precision map data for a specific area, receiving a plurality of actual road images taken within the specific area, and learning a road image generation model based on the high-precision map data and the plurality of actual road images, and each of the plurality of actual road images includes location information.

[0011] According to one embodiment of the present disclosure, an information processing system comprises a communication module, a memory, and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program includes instructions for receiving a target lane mask image and generating a plurality of synthetic road images associated with the target lane mask image using a road image generation model, wherein the road image generation model is learned based on high-precision map data for a specific area and a plurality of actual road images captured within the specific area, and each of the plurality of actual road images includes location information. Effects of the invention

[0012] According to one embodiment of the present disclosure, by generating a synthetic road image dataset for training based on precise map data and actual road images and training a road image generation model, the time and cost required to annotate training data or collect large amounts of data for various environments can be reduced.

[0013] According to one embodiment of the present disclosure, by selecting diverse data from data with potential overlap based on similarity defined between a plurality of lane mask images, the time and cost required to train a road image generation model can be reduced and training performance can be improved.

[0014] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art to which the present disclosure pertains from the description in the claims (“person skilled in the art”). Brief explanation of the drawing

[0015] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein similar reference numerals indicate similar elements, but are not limited thereto. FIG. 1 is a diagram illustrating an example of a method for training a lane detection model using precise map data and a road image generation model according to one embodiment of the present disclosure. FIG. 2 is a schematic diagram showing a configuration in which an information processing system according to one embodiment of the present disclosure is connected to communicate with a plurality of user terminals. FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure. FIG. 4 is a diagram illustrating an example of a method for training a road image generation model according to one embodiment of the present disclosure. FIG. 5 is a diagram illustrating an example of a method for learning a lane detection model according to one embodiment of the present disclosure. FIG. 6 is a drawing showing an example of a plurality of lane mask images and a plurality of composite road images generated by a road image generation model according to one embodiment of the present disclosure. FIG. 7 is a diagram illustrating the efficiency of a method for determining learning data using a maximum diversity set according to one embodiment of the present disclosure. FIG. 8 is a flowchart illustrating an example of a lane detection model learning method according to one embodiment of the present disclosure. FIG. 9 is a flowchart illustrating an example of a road image generation model learning method and a road image generation method according to one embodiment of the present disclosure. FIG. 10 is a flowchart illustrating an example of a lane detection model learning method and a lane detection method according to one embodiment of the present disclosure. Specific details for implementing the invention

[0016] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present disclosure may be unnecessarily obscured.

[0017] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0018] The advantages and features of the disclosed embodiments and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms, and the embodiments provided are merely to make the present disclosure complete and to fully inform those skilled in the art of the scope of the invention.

[0019] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0020] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0021] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0022] According to one embodiment of the present disclosure, a ‘module’ or ‘part’ may be implemented as a processor and memory. The term ‘processor’ should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the term ‘processor’ may refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term ‘processor’ may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations. Additionally, the term ‘memory’ should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable-Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, Magnetic or Optical Data Storage Devices, Registers, etc. If a processor can read information from memory and / or write information to memory, the memory is said to be in an electronic communication state with the processor. Memory integrated into a processor is in an electronic communication state with the processor.

[0023] In the present disclosure, the 'system' may include at least one of a server device and a cloud device, but is not limited thereto. For example, the system may be composed of one or more server devices. As another example, the system may be composed of one or more cloud devices. As yet another example, the system may be configured and operated with both a server device and a cloud device.

[0024] In the present disclosure, 'display' may refer to any display device associated with a computing device, for example, any display device capable of displaying any information / data controlled by or provided by the computing device.

[0025] In the present disclosure, 'each of a plurality of A' or 'each of a plurality of A' may refer to each of all components included in a plurality of A, or each of some components included in a plurality of A.

[0026] FIG. 1 is a diagram illustrating an example of a method for training a lane detection model (150) using high-definition map data (110) and a road image generation model (120) according to one embodiment of the present disclosure. According to one embodiment, at least one processor of an information processing system can train a road image generation model (120) based on high-definition map data (110) for a specific area. Then, the processor can generate a data set for training using the road image generation model (120) and train a lane detection model (150) based on the generated data set.

[0027] In one embodiment, the precision map data (110) may be a three-dimensional model of a specific area containing three-dimensional geometric information and texture information expressed in absolute coordinate positions. Here, the three-dimensional geometric information may include three-dimensional geometric information for lanes. For example, the precision map data (110) may be expressed in the form of a lightweight vector map and may include information about the actual road environment, such as road layout, road DEM (Digital Elevation Model), lane geometric data, road connection information, road signs or road surface markings. In one embodiment, the precision map data (110) may be generated based on aerial photographs, but is not limited thereto.

[0028] The road image generation model (120) may be a model trained to receive a target lane mask image as input and generate a synthetic road image corresponding to the target lane mask image. In one embodiment, the road image generation model (120) may be trained based on precision map data and a plurality of actual road images. A detailed description of the training method of the road image generation model (120) will be described later with reference to FIGS. 4 and FIGS. 9.

[0029] A plurality of lane mask images (130) can be generated based on precision map data (110) and a plurality of actual road images. In one embodiment, a processor can generate a plurality of lane mask images (130) by using location information included in a plurality of actual road images and projecting at least some of the three-dimensional geometric information included in the precision map data (e.g., lane objects) onto a plurality of actual road images.

[0030] A synthetic road image (140) can be generated by inputting a plurality of lane mask images (130) into a learned road image generation model (120). For example, the plurality of synthetic road images (140) may include various synthetic road images associated with a specific lane mask image. In this case, each of the various synthetic road images may include the same lane layout associated with the specific lane mask image, but may include different backgrounds and objects.

[0031] According to one embodiment, the processor can train a lane detection model (150) based on a lane mask image (130) and a synthetic road image (140). Here, the lane detection model (150) may be a model trained to receive a target road image as input and detect lane regions within the target road image. A detailed description of the training method for the lane detection model (150) will be provided later with reference to FIGS. 5 and FIGS. 10.

[0032] With the aforementioned configuration, human annotation of lanes within images is not required, thereby significantly reducing time and costs. Furthermore, by utilizing an image generation model to generate images from lane labels, it is possible to provide a lane detection model that exhibits high lane detection performance even in road environments not present in the training dataset.

[0033] FIG. 2 is a schematic diagram showing a configuration in which an information processing system (230) according to one embodiment of the present disclosure is connected to communicate with a plurality of user terminals (210_1, 210_2, 210_3). As illustrated, the plurality of user terminals (210_1, 210_2, 210_3) may be connected to an information processing system (230) capable of providing map information services through a network (220). Here, the plurality of user terminals (210_1, 210_2, 210_3) may include a user terminal receiving map information services. Additionally, the plurality of user terminals (210_1, 210_2, 210_3) may be devices embedded in / mounted in a vehicle that captures actual road images on a road. In one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data related to providing map information services, etc., or one or more distributed computing devices and / or distributed databases based on cloud computing services.

[0034] The map information service provided by the information processing system (230) may be provided to the user through an application or web browser installed on each of the multiple user terminals (210_1, 210_2, 210_3). For example, the information processing system (230) may provide information corresponding to actual road image data, lane estimation requests, lane unit route guidance requests, etc. received from the user terminals (210_1, 210_2, 210_3) through an application, etc., or perform corresponding processing.

[0035] Multiple user terminals (210_1, 210_2, 210_3) can communicate with an information processing system (230) through a network (220). The network (220) can be configured to enable communication between the multiple user terminals (210_1, 210_2, 210_3) and the information processing system (230). Depending on the installation environment, the network (220) may be configured as a wired network such as Ethernet, Power Line Communication, telephone line communication device and RS-serial communication, a mobile communication network, a Wireless LAN (WLAN), Wi-Fi, Bluetooth and ZigBee, or a combination thereof. The communication method is not limited and may include not only communication methods utilizing communication networks that the network (220) may include (e.g., mobile communication network, wired internet, wireless internet, broadcasting network, satellite network, etc.) but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0036] In FIG. 2, a mobile phone terminal (210_1), a tablet terminal (210_2), and a PC terminal (210_3) are illustrated as examples of user terminals, but are not limited thereto. The user terminals (210_1, 210_2, 210_3) may be any computing device capable of wired and / or wireless communication and capable of installing and running applications or web browsers, etc. For example, user terminals may include an AI speaker, a smartphone, a mobile phone, a navigation system, a computer, a laptop, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a tablet PC, a game console, a wearable device, an IoT (Internet of Things) device, a VR (Virtual Reality) device, an AR (Augmented Reality) device, a set-top box, etc. Additionally, FIG. 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with an information processing system (230) through a network (220), but is not limited thereto, and may be configured so that a different number of user terminals communicate with an information processing system (230) through a network (220).

[0037] FIG. 3 is a block diagram showing the internal configuration of a user terminal (210) and an information processing system (230) according to one embodiment of the present disclosure. The user terminal (210) may refer to any computing device capable of running applications or web browsers, etc., and capable of wired / wireless communication, and may include, for example, the mobile phone terminal (210_1), tablet terminal (210_2), PC terminal (210_3) of FIG. 2. As illustrated, the user terminal (210) may include a memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include a memory (332), a processor (334), a communication module (336), and an input / output interface (338). As illustrated in FIG. 3, the user terminal (210) and the information processing system (230) may be configured to communicate information and / or data through the network (220) using their respective communication modules (316, 336). Additionally, the input / output device (320) may be configured to input information and / or data to the user terminal (210) or output information and / or data generated from the user terminal (210) through the input / output interface (318).

[0038] The memory (312, 332) may include any non-transient computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a non-perishable permanent mass storage device such as ROM (read-only memory), a disk drive, an SSD (solid-state drive), or a flash memory. As another example, a non-perishable permanent mass storage device such as ROM, an SSD, a flash memory, or a disk drive may be included in the user terminal (210) or the information processing system (230) as a separate permanent storage device distinct from the memory. Additionally, the memory (312, 332) may store an operating system and at least one program code (e.g., code for an application installed and running on the user terminal (210)).

[0039] These software components may be loaded from a computer-readable recording medium separate from memory (312, 332). This separate computer-readable recording medium may include a recording medium that can be directly connected to the user terminal (210) and the information processing system (230), for example, a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, or memory card. As another example, the software components may be loaded into memory (312, 332) via a communication module rather than a computer-readable recording medium. For example, at least one program may be loaded into memory (312, 332) based on a computer program installed by files provided through a network (220) by developers or a file distribution system that distributes installation files for the application.

[0040] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a recording device such as memory (312, 332).

[0041] The communication module (316, 336) may provide a configuration or function for the user terminal (210) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (210) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system). For example, a request or data (e.g., precise map data associated with a specific area and actual road data associated with a specific area) generated by the processor (314) of the user terminal (210) according to program code stored in a recording device such as memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, control signals or commands provided under the control of the processor (334) of the information processing system (230) can be received by the user terminal (210) through the communication module (336) and the network (220) via the communication module (316) of the user terminal (210). For example, the user terminal (210) can receive data related to lane detection, including a lane mask image, from the information processing system (230).

[0042] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera including an audio sensor and / or an image sensor, a keyboard, a microphone, or a mouse, and the output device may include a device such as a display, a speaker, or a haptic feedback device. As another example, the input / output interface (318) may be a means for interfacing with a device in which the configuration or function for performing input and output is integrated into one, such as a touchscreen. For example, when the processor (314) of the user terminal (210) processes instructions of a computer program loaded in memory (312), a service screen configured using information and / or data provided by an information processing system (230) or another user terminal may be displayed on a display through the input / output interface (318). In FIG. 3, the input / output device (320) is depicted as not being included in the user terminal (210), but is not limited thereto and may be configured as a single device with the user terminal (210). Additionally, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (230) or that the information processing system (230) may include. In FIG. 3, the input / output interface (318, 338) is shown as an element configured separately from the processor (314, 334), but is not limited thereto, and the input / output interface (318, 338) may be configured to be included in the processor (314, 334).

[0043] The user terminal (210) and the information processing system (230) may include more components than those of FIG. 3. However, it is not necessary to clearly illustrate most of the prior art components. According to one embodiment, the user terminal (210) may be implemented to include at least some of the input / output devices (320) described above. Additionally, the user terminal (210) may further include other components such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. For example, if the user terminal (210) is a smartphone, it may include components that are generally included in a smartphone, and may be implemented to further include various components such as an accelerometer, a gyroscope, an image sensor, a proximity sensor, a touch sensor, an ambient light sensor, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration. According to one embodiment, the processor (314) of the user terminal (210) may be configured to operate an application that provides map information services, etc. At this time, code associated with the said application and / or program may be loaded into the memory (312) of the user terminal (210).

[0044] While a program for an application that provides map information services is running, the processor (314) can receive text, images, video, voice and / or actions that are input or selected through an input device such as a touch screen, keyboard, audio sensor and / or image sensor, camera, microphone, etc. connected to an input / output interface (318), and can store the received text, images, video, voice and / or actions in memory (312) or provide them to an information processing system (230) through a communication module (316) and a network (220).

[0045] The processor (314) of the user terminal (210) may be configured to manage, process, and / or store information and / or data received from an input / output device (320), another user terminal, an information processing system (230), and / or multiple external systems. The information and / or data processed by the processor (314) may be provided to the information processing system (230) through a communication module (316) and a network (220). The processor (314) of the user terminal (210) may transmit information and / or data to the input / output device (320) through an input / output interface (318) to output it. For example, the processor (314) may display the received information and / or data on the screen of the user terminal.

[0046] The processor (334) of the information processing system (230) may be configured to manage, process, and / or store information and / or data received from a plurality of user terminals (210) and / or a plurality of external systems. The information and / or data processed by the processor (334) may be provided to the user terminals (210) through a communication module (336) and a network (220).

[0047] FIG. 4 is a diagram illustrating an example of a method for learning a road image generation model (440) according to one embodiment of the present disclosure. According to one embodiment, at least one processor (e.g., at least one processor of an information processing system or a user terminal) may receive precise map data (412) for a specific area and a plurality of actual road images (414) taken within the specific area in order to learn the road image generation model (440).

[0048] Here, the precision map data (412) is a 3D model of a specific area that includes 3D geometric information and texture information expressed as absolute coordinate positions, and the 3D geometric information may include 3D geometric information for lanes. Additionally, a plurality of actual road images (414) may include location information. Additionally, the captured plurality of actual road images (414) may be images taken while driving a vehicle in a specific area. For example, the location information included in the plurality of actual road images (414) may be location information obtained using the vehicle's GPS equipment and may be a parameter of 6 Degrees of Freedom (DoF). For example, the location information included in the plurality of actual road images (414) may include direction information (i.e., image capture direction information) for each of the plurality of actual road images (414).

[0049] According to one embodiment, the processor can generate a plurality of learning lane mask images (420) based on precision map data (412) and a plurality of actual road images (414). For example, the processor can generate a plurality of learning lane mask images (420) by projecting lane objects included in the precision map data (412) onto the plurality of actual road images (414) using location information included in the plurality of actual road images (414). Through the process of acquiring such a plurality of learning lane mask images (420), pixel location information corresponding to lane objects on the plurality of actual road images (414) can be obtained without any additional annotation work.

[0050] The processor may determine some of the multiple training lane mask images (420) as training lane mask data (430) to minimize overlapping lane layouts among the multiple training lane mask images (420) and to select data with various lane layouts. To this end, a data selection algorithm may be used. Here, the data selection algorithm may mean a method for determining a set of multiple vertices such that the sum of similarity (or dissimilarity) scores between all pairs of elements included in a fully weighted graph set is minimized (or maximum). For example, a set of vertices representing N data points , edges connecting each data point and the weight of each edge defined by the similarity between two data points connected by an edge A complete weighted graph defined as, It can be assumed that... In this case, a predetermined number (e.g., k) of vertex sets that minimize the sum of similarity scores between all pairs of elements included in the fully weighted graph set, or a predetermined number of vertex sets that are greater than (or less than) a threshold value can be determined in the same way as described in the algorithm in Table 1 below.

[0051] Algorithm 1 Finding a set of K vertices with the minimum sum of similarity Input: Complete weighted graph G and K = # of vertices 1: Find the vertex pair (i, j) which has minimum w(i, j) 2: Add i and j to Q 3: while |Q| < K do 4: Find vertex k which minimizes 5: end while Output: Q

[0052] According to one embodiment, the processor can calculate a similarity score or a dissimilarity score between a plurality of training lane mask images. For example, two training lane mask images (e.g., M) among a plurality of training lane mask images (420). i , M jA similarity function s can be defined between ). The similarity between two training lane mask images may refer to the similarity between lane objects represented by pixel location information within the images. For example, a lane is a set of 2D points uniformly sampled along the vertical direction of the image plane (e.g., It can be expressed as ). Here, x i represents the x-coordinate of the i-th sample, and N can represent the number of samples. Utilizing all L-lane curves from the entire training dataset, the lane matrix It can be constructed. When the SVD algorithm is applied to the constructed lane matrix A, it can be factored as shown in Equation 1 below.

[0053]

[0054] Here, and are orthogonal matrices, and sigma (Σ) can represent the singular value matrix. Therefore, each lane curve ℓ is a singular vector as shown in Equation 2 below. It can be approximated using an M-rank approximation.

[0055]

[0056] singular vector Each lane curve in the eigenline space spanned by coefficient vector for It can be expressed as shown in mathematical formula 3 below.

[0057]

[0058] Two-lane curve and The similarity can be projected onto the eigenline space and defined as the Euclidean distance as shown in Equation 4 below.

[0059]

[0060] Here, the similarity between lane curves can be extended to the similarity between lane mask images. Two lane mask images containing m and n lane curves, respectively. and Given , and assuming m ≥ n always holds, two lane mask images M i and M j The similarity s between them can be defined as shown in Equation 5 below.

[0061]

[0062] Here, can represent a penalty constant. For example, if the number of lane curves within the two lane mask images is different, A penalty can be applied to similarity by adding . Penalty constant in the training dataset It can be determined empirically.

[0063] According to the examples described above, a dataset of lane mask images that maximizes the diversity of lane layouts, i.e., a maximal diversity set, can be determined as the lane mask data (430) for training. In one embodiment, the maximal diversity set may be configured such that the sum of similarity scores between a predetermined number of lane mask images is minimized or the sum of dissimilarity scores is maximized. In another embodiment, the maximal diversity set may include a plurality of lane mask images that are above (or below) a predetermined threshold.

[0064] According to one embodiment, a processor can train a road image generation model (440) using training lane mask data (430), which is a maximum diversity set, and corresponding training actual road data. Then, the processor can generate a plurality of synthetic road images (450) based on a target lane mask image using the trained road image generation model (440). The training lane mask data (430) and the corresponding plurality of synthetic road images (450) can be used to train a lane detection model.

[0065] FIG. 5 is a diagram illustrating an example of a method for learning a lane detection model (550) according to one embodiment of the present disclosure. According to one embodiment, at least one processor (e.g., at least one processor of an information processing system or a user terminal) may receive a plurality of lane mask images (510) to learn the lane detection model (550). Here, the lane mask may represent a lane area within an image. Additionally, the plurality of lane mask images (510) may be generated by projecting lane objects included in precise map data onto a plurality of actual road images using location information included in a plurality of actual road images in the same or similar manner as described in FIG. 4.

[0066] A processor can generate a plurality of composite road images (530) for each of a plurality of lane mask images (510) using a road image generation model (520). Here, each of the plurality of lane mask images (510) may include different lane layouts. Additionally, a plurality of composite road images (530) associated with a specific lane mask image may include the same lane layout associated with the specific lane mask image, but may include different backgrounds and objects. In one embodiment, the processor can generate a plurality of composite road images (530), which are unique image data generated by resampling noise vectors based on the received plurality of lane mask images (510).

[0067] After generating synthetic road images, the processor may determine some of the multiple synthetic road images (530) as training road data (540) in order to minimize duplicate images among the multiple synthetic road images (530) and select data with various objects and backgrounds. To this end, a data selection algorithm identical or similar to that described in FIG. 4 may be used.

[0068] According to one embodiment, the processor may calculate a similarity score between a plurality of synthetic road images (530). Here, the similarity score may be calculated using MS-SSIM (Multi-Scale Structural Similarity Index Measure). Additionally, based on the calculated similarity score, the processor may determine a set of maximal diversity among a plurality of synthetic road images associated with a single lane mask image as training road data (540). The processor may determine a set of maximal diversity for each of the plurality of lane mask images (510) and use them as training road data (540). In one embodiment, the set of maximal diversity may be such that the sum of similarity scores between a predetermined number of synthetic road images is minimized or the sum of dissimilarity scores is maximized. In another embodiment, the set of maximal diversity may include synthetic road images that are above (or below) a predetermined threshold.

[0069] After that, the processor can learn a lane detection model (550) based on a plurality of lane mask images (510) and a plurality of corresponding synthetic road images (530).

[0070] FIG. 6 is a diagram illustrating an example of a plurality of lane mask images (610) and a plurality of composite road images (620) generated by a road image generation model according to one embodiment of the present disclosure. According to one embodiment, at least one processor (e.g., at least one processor of an information processing system or a user terminal) can generate a plurality of composite road images (620) for each of the plurality of lane mask images (610) using a road image generation model (e.g., 520 of FIG. 5).

[0071] Here, each of the plurality of lane mask images (610) may include different lane layouts. For example, the plurality of lane mask images (610) may include a first lane mask image (612), a second lane mask image (614), a third lane mask image (616), and a fourth lane mask image (618). In this case, the first to fourth lane mask images (612 to 618) may be mask images in which at least one element among the number of lanes, the position of the lanes, and the curve of the lanes is different.

[0072] Additionally, each of the plurality of synthetic road images (620) may be associated with a specific lane mask image. For example, the processor may use a road image generation model to generate a first set of synthetic road images (622) associated with a first lane mask image (612). Here, the first set of synthetic road images (622) may be the maximum diversity set among the plurality of synthetic road images associated with the first lane mask image (612). Additionally, the processor may use a road image generation model to generate a second set of synthetic road images (624) associated with a second lane mask image (614). Similarly, the processor may generate a third set of synthetic road images (626) associated with a third lane mask image (616) and a fourth set of synthetic road images (628) associated with a fourth lane mask image (618).

[0073] Here, each of the plurality of composite road images (620) may include the same lane layout associated with the associated lane mask image, but may include different backgrounds and objects. For example, the first set of composite road images (622) are all identical to the lane layout of the first lane mask image (612), but each of the first set of composite road images (622) may include different backgrounds and objects. It can be observed that each of the first set of composite road images (622) has the same lane layout, but the color, shading state, etc. of the road are different. To this end, according to one embodiment, a processor may generate a plurality of composite road images (620), which are unique image data generated by resampling noise vectors based on the plurality of lane mask images (610).

[0074] FIG. 6 illustrates examples of four lane mask images (610) and composite road images (620) associated with each of the four lane mask images (610), but the number of lane mask images (610) and the number of composite road images (620) are not limited thereto.

[0075] FIG. 7 is a diagram illustrating the efficiency of a method for determining training data using a maximum diversity set according to one embodiment of the present disclosure. The relative accuracy performance of a model for a synthetic dataset (LabsLane) comprising a plurality of synthetic road images (530) described in FIG. 5, generated based on images collected using a vehicle equipped with a high-precision GPS, can be verified. To this end, the CULane dataset and the TuSimple dataset were used as benchmark datasets, and Accuracy and Total F1-score were used as comparison metrics, respectively. Here, the CULane dataset is a large-scale dataset used for lane detection, consisting mostly of data collected in complex urban road environments. Additionally, the TuSimple dataset is a lane detection dataset that primarily collects highway driving camera images with annotated lane markings.

[0076] The dataset for verifying the accuracy performance of the model consisted of a dataset in which 10 samples were randomly selected from 100 synthetic data images associated with a single lane mask image from the synthetic dataset (LabsLane), 2 maximum diversity sets, 5 maximum diversity sets, and 10 maximum diversity sets. As shown in the figure, it can be confirmed that using the maximum diversity sets with the data selection algorithm described above yields superior performance compared to using randomly selected samples. Furthermore, even when using maximum diversity sets, it can be confirmed that the highest performance is achieved when using 5 maximum diversity sets associated with a single lane mask image.

[0077] In addition, to verify the performance of the framework of the present disclosure, the performance of each image detection model and lane detection model can be seen in Table 2 below. Referring to Table 2, it can be seen that there is a significant difference in performance between TUSimple and CULane. This is presumed to be due to the fact that the two datasets were acquired in very different environments, and the TUSimple dataset lacks diversity compared to the CULane dataset because it includes highway scenarios.

[0078] Experiment Setting Total Normal Crowded Dazzle Shadow No line Arrow Curve Cross-road Night Trained on CULane 78.8 93.2 77.2 71.2 77.9 53.6 89.6 75.9 1240 72.8 Trained on TUSimple 24.2 41 19.6 11.4 12.7 12.7 28.8 32.6 1240 9.1 (a) Ablation studies Trained on LabsLane 30.5 47.1 21.4 19.8 14.6 17.5 38 39.3 869 23.1 +Scene selection 33.9 51.5 28.0 15.9 21.8 14.6 43.0 38.9 1120 24.4 +With synthesized images (Proposed) 39.6 56.7 34.0 27.1 22.0 21.2 53.2 46.9 1275 30.1 (b) Applying proposed framework on TUSimple dataset +w / o LabsLane 31.3 48.0 26.6 23.2 17.4 17.8 39.0 40.5 1863 18.2 (c) Comparison with domain adaptation method MLDA 38.4 61.4 36.3 30.3 23.4 21.3 49.1 43.4 11386 27.4 Proposed (ERFNet) 38.2 56.8 32.5 32.2 22.3 23.8 51.2 40.9 4572 24.4

[0079] In relation to ablation studies on the performance of each stage of the proposed framework of the present disclosure, the effect of the diversity-enhanced LabsLane dataset can be confirmed. Training a lane detection model using the LabsLane dataset instead of the TUSimple dataset resulted in a performance improvement of approximately 26%. Considering that most images in the TUSimple dataset consist only of straight-line driving, it is presumed that the diversity of the training dataset contributed to this result. Furthermore, for a road image generation model trained with a training dataset determined using a maximum diversity set, further subdivision of the LabsLane dataset resulted in an additional performance improvement of approximately 11%. Additionally, when diversity was increased using a maximum diversity set based on synthetic road images generated using the trained road image generation model, a performance improvement of approximately 16% was observed.

[0080] When applying the proposed framework of the present disclosure to the TUSimple dataset, it can be seen that the total F1 score (F1-Score) is improved by approximately 29% compared to when training only with the TUSimple dataset.

[0081] When comparing the proposed framework of the present disclosure with a domain adaptation method, it can be confirmed that the proposed framework is not limited to a specific lane detection model and can be applied to other lane detection models.

[0082] In addition, the results for the TUSimple dataset of the proposed framework of the present disclosure can be seen in Table 3 below.

[0083] Accuracy FP FN Trained on TUSimple 95.8 2.09 2.8 Trained on CULane 60.9 31.6 55.2 MLDA 89.7 29.5 18.4 Proposed framework 92.4 6.7 9.4

[0084] Referring to Table 3, it can be seen that, similar to the case of the CULane dataset, there is a significant decrease in all evaluation metrics between the two benchmark datasets. Furthermore, it can be confirmed that the proposed framework outperforms MLDA, a domain adaptation-based method. In particular, when compared to the results for the CULane dataset, both MLDA and the proposed method show significant performance improvements. Since the CULane dataset has higher diversity than the TUSimple dataset, it is speculated that it can benefit more from the proposed framework, which includes configurations that enhance diversity.

[0085] Table 4 below shows the results of verifying whether the proposed framework works well with different image generation models. To this end, OASIS, which uses rich semantic segmentation labels to generate images, and Pix2PixHD, another algorithm capable of generating various images by resampling noise vectors, were used. The image generation models were trained using the TUSimple dataset, and the performance of the models was evaluated using the Total F1-score on the CULane dataset.

[0086] Total Trained on TUSimple 24.2 Pix2PixHD 26.7 OASIS 29.0

[0087] Referring to Table 4, it can be seen that there is about a 10% performance improvement when using Pix2PixHD compared to training on TUSimple. Since OASIS generates more realistic images than Pix2PixHD, it is believed that the performance difference is due to the difference in image quality.

[0088] FIG. 8 is a flowchart illustrating an example of a lane detection model learning method (800) according to one embodiment of the present disclosure. The method (800) may be initiated by a processor (e.g., at least one processor of an information processing system or a user terminal) learning a road image generation model based on precise map data for a specific area (S810). Then, the processor may input a plurality of lane mask images into the road image generation model to generate a plurality of synthetic road images (S820). The processor may learn a lane detection model based on the plurality of lane mask images and the plurality of synthetic road images (S830).

[0089] FIG. 9 is a flowchart illustrating an example of a road image generation model learning method and a road image generation method (900) according to one embodiment of the present disclosure. The method (900) may be initiated by a processor (e.g., at least one processor of an information processing system or a user terminal) receiving high-definition map data for a specific area (S910). In one embodiment, the high-definition map data is a three-dimensional model of a specific area including three-dimensional geometric information and texture information expressed as absolute coordinate positions, and the three-dimensional geometric information may include three-dimensional geometric information for lanes. In one embodiment, the high-definition map data may be generated based on aerial photographs.

[0090] Then, the processor can receive multiple real road images captured within a specific area (S920). Here, each of the multiple real road images may include location information. Additionally, the captured multiple real road images may be images taken while driving a vehicle in the specific area.

[0091] Then, the processor can train a road image generation model based on precision map data and multiple actual road images (S930). Here, the road image generation model may be a model trained to receive a target lane mask image as input and generate a synthetic road image corresponding to the target lane mask image.

[0092] According to one embodiment, a processor can generate a plurality of training lane mask images based on precision map data and a plurality of actual road images. For example, the processor can generate a plurality of training lane mask images by using location information included in a plurality of actual road images and projecting at least some of the three-dimensional geometric information included in the precision map data onto the plurality of actual road images.

[0093] Then, the processor can train a road image generation model based on a plurality of training lane mask images and a plurality of actual road images. In one embodiment, the processor can determine some of the plurality of actual road images associated with the training lane mask data as training actual road data. Then, the processor can train a road image generation model based on the training lane mask data and the training actual road data.

[0094] Specifically, the processor may determine some of the plurality of training lane mask images as training lane mask data. Specifically, the processor may calculate similarity scores or dissimilarity scores between the plurality of training lane mask images. For example, the processor may calculate similarity scores or dissimilarity scores based on the similarity or dissimilarity of curves for lanes included in each of the plurality of training lane mask images. Additionally or alternatively, the processor may calculate similarity scores or dissimilarity scores based on the number of lanes included in each of the plurality of training lane mask images. Furthermore, based on the calculated similarity scores or dissimilarity scores, the processor may determine a set of maximal diversity among the plurality of training lane mask images as training lane mask data. In one embodiment, the maximal diversity set may be such that the sum of similarity scores between a predetermined number of lane mask images is minimized or the sum of dissimilarity scores is maximized.

[0095] According to one embodiment, a processor can generate a plurality of synthetic road images based on a target lane mask image using a learned road image generation model. For example, the processor can receive a target lane mask image (S940). Then, the processor can generate a plurality of synthetic road images associated with the target lane mask image using a road image generation model (S950). Here, the road image generation model is learned based on high-precision map data for a specific area and a plurality of actual road images captured within the specific area, and each of the plurality of actual road images may include location information. Here, the generated plurality of synthetic road images may include lanes corresponding to the target lane mask image. Additionally, the generated plurality of synthetic road images may include different backgrounds and objects.

[0096] According to one embodiment, a plurality of synthetic road images generated using a learned road image generation model can be used to train a lane detection model. For example, after the road image generation model is trained, the processor may receive a first lane mask image and a second lane mask image. Then, the processor may generate a plurality of first synthetic road images associated with the first lane mask image using the road image generation model. Additionally, the processor may generate a plurality of second synthetic road images associated with the second lane mask image using the road image generation model. Then, the processor may train a lane detection model based on pairs of the first lane mask image and the plurality of first synthetic road images, and pairs of the second lane mask image and the plurality of second synthetic road images.

[0097] The flowchart of FIG. 9 and the description above are merely examples, and the scope of the present disclosure is not limited thereto. For example, at least one step may be added, changed, or deleted, or the order of each step may be changed.

[0098] FIG. 10 is a flowchart illustrating an example of a lane detection model learning method and a lane detection method (1000) according to an embodiment of the present disclosure. The method (1000) may be initiated by a processor (e.g., at least one processor of an information processing system or a user terminal) receiving a plurality of lane mask images (S1010). Here, the lane mask may represent a lane area within an image. In one embodiment, the plurality of lane mask images may be generated by projecting at least a portion of the three-dimensional geometric information included in the high-definition map data onto the plurality of actual road images using location information included in the high-definition map data for a specific area and the plurality of actual road images. In one embodiment, the high-definition map data may be generated based on aerial photographs. Additionally, the plurality of actual road images may be images taken while driving a vehicle in a specific area.

[0099] Additionally, the processor can generate multiple synthetic road images for each of the multiple lane mask images using a road image generation model (S1020). Here, the road image generation model may be a model trained to receive a target lane mask image as input and generate a synthetic road image corresponding to the target lane mask image. For example, the multiple lane mask images may include a first lane mask image and a second lane mask image. The processor can generate multiple first synthetic road images associated with the first lane mask image using the road image generation model. Additionally, the processor can generate multiple second synthetic road images associated with the second lane mask image using the road image generation model. Here, the generated multiple first synthetic road images may include lanes corresponding to the first lane mask image. Additionally, the generated multiple first synthetic road images may include different backgrounds and objects. Likewise and similarly, the generated multiple second synthetic road images may include lanes corresponding to the second lane mask image. Additionally, the generated multiple second synthetic road images may include different backgrounds and objects.

[0100] Then, the processor can train a lane detection model based on a plurality of lane mask images and a plurality of synthetic road images (S1030). Here, the lane detection model may be a model trained to receive a target road image as input and detect lane regions within the target road image.

[0101] In one embodiment, the processor can train a lane detection model based on a plurality of lane masks and a portion of a plurality of synthetic road images. For example, a portion of a plurality of first synthetic road images can be determined as first training road data. Additionally, the processor can determine a portion of a plurality of second synthetic road images as second training road data. Furthermore, the processor can train a lane detection model based on pairs of first lane mask images and first training road data, and pairs of second lane mask images and second training road data.

[0102] In one embodiment, the processor may determine some of the plurality of synthetic road images as training road data based on similarity scores between the plurality of synthetic road images. The processor may calculate similarity scores between the plurality of first synthetic road images. Then, based on the calculated similarity scores, the processor may determine a set of maximal diversity among the plurality of first synthetic road images as first training road data. Of course, the above example can be applied to determining some of the plurality of second synthetic road images as second training road data. Here, the set of maximal diversity may be determined such that the sum of similarity scores between a predetermined number of synthetic road images is minimized. In one embodiment, the similarity scores may be calculated using MS-SSIM (Multi-Scale Structural Similarity Index Measure).

[0103] In one embodiment, prior to the step of generating a plurality of synthetic road images, the processor can learn a road image generation model based on precise map data for a specific area and a plurality of actual road images taken within the specific area.

[0104] According to one embodiment, a processor can detect lane regions within an image based on a target actual road image using a trained lane detection model. For example, the processor can receive a target actual road image (S1040). Then, the processor can generate a target lane mask image associated with the target actual road image using the lane detection model (S1050). Here, the lane detection model may be a model trained based on a plurality of lane mask images and a plurality of synthetic road images. Additionally, the lane mask image represents a lane region within the image, and the plurality of synthetic road images may be images generated for each of the plurality of lane masks using a road image generation model.

[0105] The flowchart of FIG. 10 and the description above are merely examples, and the scope of the present disclosure is not limited thereto. For example, at least one step may be added, changed, or deleted, or the order of each step may be changed.

[0106] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a computer-executable program, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple hardware components, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Furthermore, other examples of media may include recording or storage media managed by sites, servers, etc., that supply or distribute various software, such as app stores that distribute applications.

[0107] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0108] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.

[0109] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.

[0110] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and may cause the processor(s) to perform specific aspects of the functions described in this disclosure.

[0111] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0112] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification. Explanation of the symbols

[0113] 110: Precision map data 120: Road image generation model 130: Lane mask image 140: Composite road image 150: Lane detection model

Claims

Claim 1 A method for learning a road image generation model performed by at least one processor, comprising: receiving high-definition map data for a specific area; receiving a plurality of actual road images captured within the specific area; and learning a road image generation model based on the high-definition map data and the plurality of actual road images, wherein each of the plurality of actual road images includes location information, and after the road image generation model is learned, further comprising: receiving a first lane mask image and a second lane mask image; generating a plurality of first synthetic road images associated with the first lane mask image using the road image generation model; generating a plurality of second synthetic road images associated with the second lane mask image using the road image generation model; and learning a lane detection model based on pairs of the first lane mask image and the plurality of first synthetic road images, and pairs of the second lane mask image and the plurality of second synthetic road images. Claim 2 A method for training a road image generation model according to claim 1, wherein the precision map data is a 3D model of a specific area including 3D geometric information and texture information expressed in absolute coordinate positions, and the 3D geometric information includes 3D geometric information for lanes. Claim 3 In claim 1, the above-described precision map data is a road image generation model learning method based on aerial photographs. Claim 4 A method for training a road image generation model, wherein, in claim 1, the plurality of actual road images captured are images captured while driving a vehicle in the specific area. Claim 5 A method for training a road image generation model according to claim 1, wherein the step of training the road image generation model includes the step of generating a plurality of training lane mask images based on the precision map data and the plurality of actual road images. Claim 6 A road image generation model learning method according to claim 5, wherein the step of generating a plurality of learning lane mask images comprises the step of generating a plurality of learning lane mask images by projecting at least a portion of the three-dimensional geometric information included in the precision map data onto the plurality of actual road images using location information included in the plurality of actual road images. Claim 7 A method for learning a road image generation model according to claim 5, wherein the step of learning the road image generation model further includes the step of learning the road image generation model based on the plurality of learning lane mask images and the plurality of actual road images. Claim 8 A method for training a road image generation model according to claim 5, wherein the step of training the road image generation model further comprises: a step of determining some of the plurality of training lane mask images as training lane mask data; a step of determining some of the plurality of actual road images associated with the training lane mask data as training actual road data; and a step of training the road image generation model based on the training lane mask data and the training actual road data. Claim 9 A method for training a road image generation model according to claim 8, wherein the step of determining some of the plurality of lane mask images as training lane mask data comprises: a step of calculating a similarity score or a dissimilarity score between the plurality of training lane mask images; and a step of determining a set of maximal diversity among the plurality of training lane mask images as training lane mask data based on the calculated similarity score or dissimilarity score. Claim 10 A road image generation model training method according to claim 9, wherein the maximum diversity set is such that the sum of similarity scores between a predetermined number of lane mask images is minimized or the sum of dissimilarity scores is maximized. Claim 11 A road image generation model learning method according to claim 9, wherein the step of calculating the similarity score or dissimilarity score comprises the step of calculating the similarity score or dissimilarity score based on the similarity or dissimilarity of the curves for the lanes included in each of the plurality of learning lane mask images. Claim 12 A road image generation model learning method according to claim 9, wherein the step of calculating the similarity score or dissimilarity score comprises the step of calculating the similarity score or dissimilarity score based on the number of lanes included in each of the plurality of learning lane mask images. Claim 13 A method for training a road image generation model according to claim 1, wherein the road image generation model is trained to receive a target lane mask image as input and generate a synthetic road image corresponding to the target lane mask image. Claim 14 delete Claim 15 A method for generating road images performed by at least one processor comprises: receiving a target lane mask image; and generating a plurality of synthetic road images associated with the target lane mask image using a road image generation model, wherein the target lane mask image is generated based on precision map data for a specific area and a plurality of actual road images captured within the specific area, and each of the plurality of actual road images includes location information, and the road image generation model is trained based on a dataset consisting of lane mask images that maximize the diversity of lane layouts among the generated target lane mask images, and after the road image generation model is trained, the method comprises: receiving a first lane mask image and a second lane mask image; generating a plurality of first synthetic road images associated with the first lane mask image using the road image generation model; and generating a plurality of second synthetic road images associated with the second lane mask image using the road image generation model. A road image generation method further comprising the step of learning a lane detection model based on the first lane mask image and the pairs of the plurality of first synthetic road images, and the second lane mask image and the pairs of the plurality of second synthetic road images. Claim 16 A road image generation method according to claim 15, wherein the generated plurality of composite road images include lanes corresponding to the target lane mask image. Claim 17 A road image generation method according to claim 15, wherein the generated plurality of composite road images include different backgrounds and objects. Claim 18 A computer program stored on a computer-readable recording medium for executing a method according to any one of paragraphs 1 through 13 and paragraphs 15 through 17 on a computer. Claim 19 An information processing system comprising: a communication module; a memory; and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program comprises instructions for receiving precision map data for a specific area, receiving a plurality of actual road images taken within the specific area, and learning a road image generation model based on the precision map data and the plurality of actual road images, wherein each of the plurality of actual road images includes location information, and after the road image generation model is learned, receiving a first lane mask image and a second lane mask image, generating a plurality of first composite road images associated with the first lane mask image using the road image generation model, generating a plurality of second composite road images associated with the second lane mask image using the road image generation model, and further comprising instructions for learning a lane detection model based on pairs of the first lane mask image and the plurality of first composite road images, and pairs of the second lane mask image and the plurality of second composite road images. Claim 20 As an information processing system, communication module; memory; and includes at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program includes instructions for receiving a target lane mask image and generating a plurality of composite road images associated with the target lane mask image using a road image generation model, wherein the target lane mask image is generated based on precision map data for a specific area and a plurality of actual road images captured within the specific area, and each of the plurality of actual road images includes location information, and the road image generation model is trained based on a dataset consisting of lane mask images that maximize the diversity of lane layouts among the generated target lane mask images, and after the road image generation model is trained, a first lane mask image and a second lane mask image are received, and a plurality of first composite road images associated with the first lane mask image are generated using the road image generation model, and a plurality of second composite road images associated with the second lane mask image are generated using the road image generation model, and pairs of the first lane mask image and the plurality of first composite road images, and the second lane mask image and the plurality of second composite road images An information processing system further comprising instructions for training a lane detection model based on pairs.

Images