Road image generative model training method and system

Abstract

A road image generative model training method performed by at least one processor comprises the steps of: receiving high-definition map data for a specific area; receiving a plurality of actual road images photographed in the specific area; and training a road image generative model on the basis of the high-definition map data and the plurality of actual road images, wherein each of the plurality of actual road images includes location information.

Description

Learning method and system for road image generation model

[0001] The present disclosure relates to a method and system for learning a road image generation model, and more particularly, to a method and system for learning a road image generation model based on precise map data and actual road images for a specific area.

[0002] Autonomous driving technology refers to technologies that enable vehicles to drive autonomously with minimal or no human intervention by recognizing their surroundings using radar, LIDAR (light detection and ranging), GPS, and cameras. Vehicles equipped with autonomous driving technology require lane detection to determine their location, identify available driving space, and drive safely on the road.

[0003] In real-world driving environments, various factors affect autonomous driving, including not only lane markings on the road but also vehicles, traffic structures, and buildings on the roadside. Therefore, securing high-performance lane detection requires a massive dataset captured in various environments.

[0004] Meanwhile, collecting data from diverse real-world road environments to cover all scenarios presents physical limitations. Even if data were collected from diverse real-world road environments, the annotation required to identify lanes would be time-consuming and costly.

[0005] The present disclosure provides a method for solving the above-described problems, a computer-readable non-transitory recording medium recording commands, and a device (system).

[0006] The present disclosure can be implemented in various ways, including a computer-readable, non-transitory recording medium having recorded thereon a method, device (system), or instructions.

[0007] According to one embodiment of the present disclosure, a method for learning a road image generation model performed by at least one processor includes the steps of receiving high-definition map data for a specific area, receiving a plurality of real road images captured within the specific area, and learning a road image generation model based on the high-definition map data and the plurality of real road images, wherein each of the plurality of real road images includes location information.

[0008] According to one embodiment of the present disclosure, a road image generation method performed by at least one processor includes the steps of receiving a target lane mask image and the step of generating a plurality of synthetic road images associated with the target lane mask image using a road image generation model, wherein the road image generation model is learned based on precision map data for a specific area and a plurality of actual road images captured within the specific area, and each of the plurality of actual road images includes location information.

[0009] A computer-readable non-transitory recording medium having recorded thereon commands for executing a road image generation model learning method according to one embodiment of the present disclosure on a computer is provided.

[0010] According to one embodiment of the present disclosure, an information processing system includes a communication module, a memory, and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program includes instructions for receiving precision map data for a specific area, receiving a plurality of actual road images captured within the specific area, and learning a road image generation model based on the precision map data and the plurality of actual road images, each of the plurality of actual road images including location information.

[0011] According to one embodiment of the present disclosure, an information processing system includes a communication module, a memory, and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program includes instructions for receiving a target lane mask image and generating a plurality of synthetic road images associated with the target lane mask image using a road image generation model, wherein the road image generation model is learned based on precise map data for a specific area and a plurality of actual road images captured within the specific area, and each of the plurality of actual road images includes location information.

[0012] According to one embodiment of the present disclosure, by generating a synthetic road image data set for learning based on precision map data and actual road images and training a road image generation model, the time and cost required for annotating training data or collecting a large amount of data for various environments can be reduced.

[0013] According to one embodiment of the present disclosure, by selecting data having diversity from data with a possibility of duplication based on a similarity defined between multiple lane mask images, the time and cost required for learning a road image generation model can be reduced, and learning performance can be improved.

[0014] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs (“ordinary skilled in the art”) from the description of the claims.

[0015] Embodiments of the present disclosure will be described below with reference to the accompanying drawings, wherein like reference numerals represent similar elements, but are not limited thereto.

[0016] FIG. 1 is a diagram illustrating an example of a method for training a lane detection model using precision map data and a road image generation model according to one embodiment of the present disclosure.

[0017] FIG. 2 is a schematic diagram showing a configuration in which an information processing system according to one embodiment of the present disclosure is connected to enable communication with a plurality of user terminals.

[0018] FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure.

[0019] FIG. 4 is a diagram illustrating an example of a method for learning a road image generation model according to one embodiment of the present disclosure.

[0020] FIG. 5 is a diagram illustrating an example of a method for learning a lane detection model according to one embodiment of the present disclosure.

[0021] FIG. 6 is a diagram illustrating an example of a plurality of synthetic road images generated by a plurality of lane mask images and a road image generation model according to one embodiment of the present disclosure.

[0022] FIG. 7 is a diagram for explaining the efficiency of a method for determining learning data using a maximum diversity set according to one embodiment of the present disclosure.

[0023] FIG. 8 is a flowchart illustrating an example of a lane detection model learning method according to one embodiment of the present disclosure.

[0024] FIG. 9 is a flowchart illustrating an example of a road image generation model learning method and a road image generation method according to one embodiment of the present disclosure.

[0025] FIG. 10 is a flowchart illustrating an example of a lane detection model learning method and a lane detection method according to one embodiment of the present disclosure.

[0026] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions of widely known functions or configurations will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0027] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0028] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the completeness of the disclosure and to fully inform those skilled in the art of the scope of the invention.

[0029] The terms used in this specification will be briefly explained, followed by a detailed description of the disclosed embodiments. The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of engineers working in the relevant field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0030] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly indicates otherwise. When a part of the specification is said to include a component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0031] Also, the term 'module' or 'part' used in the specification means a software or hardware component, and the 'module' or 'part' performs certain roles. However, the 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, as an example, the 'module' or 'part' may include at least one of components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, or variables. The functionality provided within the components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0032] According to one embodiment of the present disclosure, a 'module' or 'unit' may be implemented as a processor and a memory. 'Processor' should be broadly construed to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, a 'processor' may also refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), and the like. A 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations. In addition, 'memory' should be broadly construed to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with the processor if the processor can read information from, and / or write information to, the memory. Memory integrated in a processor is in electronic communication with the processor.

[0033] In the present disclosure, the "system" may include, but is not limited to, at least one of a server device and a cloud device. For example, the system may be comprised of one or more server devices. As another example, the system may be comprised of one or more cloud devices. As yet another example, the system may be configured and operated by a combination of a server device and a cloud device.

[0034] In the present disclosure, 'display' may refer to any display device associated with a computing device, for example, any display device capable of displaying any information / data controlled by or provided from the computing device.

[0035] In the present disclosure, 'each of the plurality of As' or 'each of the plurality of As' may refer to each of all components included in the plurality of As, or may refer to each of some components included in the plurality of As.

[0036] FIG. 1 is a diagram illustrating an example of a method for training a lane detection model (150) using high-definition map data (110) and a road image generation model (120) according to one embodiment of the present disclosure. According to one embodiment, at least one processor of an information processing system can train a road image generation model (120) based on high-definition map data (110) for a specific region. Thereafter, the processor can generate a data set for training using the road image generation model (120) and train a lane detection model (150) based on the generated data set.

[0037] In one embodiment, the precision map data (110) may be a 3D model of a specific area including 3D geometric information and texture information expressed in absolute coordinate positions. Here, the 3D geometric information may include 3D geometric information about lanes. For example, the precision map data (110) may be expressed in the form of a lightweight vector map and may include information about the actual road environment, such as a road layout, a road DEM (Digital Elevation Model), lane geometric data, road connection information, road signs, or road surface markings. In one embodiment, the precision map data (110) may be generated based on aerial photographs, but is not limited thereto.

[0038] The road image generation model (120) may be a model trained to receive a target lane mask image and generate a synthetic road image corresponding to the target lane mask image. In one embodiment, the road image generation model (120) may be trained based on precision map data and multiple actual road images. A specific description of the training method of the road image generation model (120) will be described below with reference to FIGS. 4 and 9 .

[0039] A plurality of lane mask images (130) may be generated based on the precision map data (110) and a plurality of actual road images. In one embodiment, the processor may generate the plurality of lane mask images (130) by projecting at least a portion of the three-dimensional geometric information (e.g., lane objects) included in the precision map data onto the plurality of actual road images, using positional information included in the plurality of actual road images.

[0040] A synthetic road image (140) can be generated by inputting multiple lane mask images (130) into a learned road image generation model (120). For example, the multiple synthetic road images (140) can include various synthetic road images associated with a specific lane mask image. In this case, each of the various synthetic road images can include the same lane layout associated with the specific lane mask image, but can include different backgrounds and objects.

[0041] According to one embodiment, the processor may train a lane detection model (150) based on a lane mask image (130) and a synthetic road image (140). Here, the lane detection model (150) may be a model trained to receive a target road image and detect a lane area within the target road image. A specific description of a training method for the lane detection model (150) will be described below with reference to FIGS. 5 and 10 .

[0042] The above-described configuration eliminates the need for human annotation of lanes within images, significantly reducing both time and cost. Furthermore, by utilizing an image generation model to generate images from lane labels, a lane detection model can be provided that demonstrates high lane detection performance even in road environments not included in the training dataset.

[0043] FIG. 2 is a schematic diagram showing a configuration in which an information processing system (230) according to one embodiment of the present disclosure is connected to a plurality of user terminals (210_1, 210_2, 210_3) so as to be able to communicate with them. As illustrated, the plurality of user terminals (210_1, 210_2, 210_3) may be connected to an information processing system (230) capable of providing map information services via a network (220). Here, the plurality of user terminals (210_1, 210_2, 210_3) may include terminals of users who receive map information services. Additionally, the plurality of user terminals (210_1, 210_2, 210_3) may be devices built into / mounted in a vehicle that captures actual road images on the road. In one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer executable programs (e.g., downloadable applications) and data related to providing map information services, or one or more distributed computing devices and / or distributed databases based on cloud computing services.

[0044] The map information service provided by the information processing system (230) may be provided to users through applications or web browsers installed on each of a plurality of user terminals (210_1, 210_2, 210_3). For example, the information processing system (230) may provide information corresponding to actual road image data, lane estimation requests, lane-level route guidance requests, etc. received from user terminals (210_1, 210_2, 210_3) through applications, etc., or perform corresponding processing.

[0045] A plurality of user terminals (210_1, 210_2, 210_3) can communicate with an information processing system (230) via a network (220). The network (220) can be configured to enable communication between the plurality of user terminals (210_1, 210_2, 210_3) and the information processing system (230). Depending on the installation environment, the network (220) can be configured as a wired network such as Ethernet, a wired home network (Power Line Communication), a telephone line communication device, and RS-serial communication, a wireless network such as a mobile communication network, WLAN (Wireless LAN), Wi-Fi, Bluetooth, and ZigBee, or a combination thereof. The communication method is not limited, and may include not only a communication method utilizing a communication network (e.g., a mobile communication network, wired Internet, wireless Internet, broadcasting network, satellite network, etc.) that the network (220) may include, but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0046] In FIG. 2, a mobile phone terminal (210_1), a tablet terminal (210_2), and a PC terminal (210_3) are illustrated as examples of user terminals, but are not limited thereto, and the user terminals (210_1, 210_2, 210_3) may be any computing device capable of wired and / or wireless communication and capable of installing and executing applications or web browsers. For example, the user terminals may include AI speakers, smartphones, mobile phones, navigation systems, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (virtual reality) devices, AR (augmented reality) devices, set-top boxes, etc. In addition, although FIG. 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with the information processing system (230) via the network (220), this is not limited thereto, and a different number of user terminals may be configured to communicate with the information processing system (230) via the network (220).

[0047] FIG. 3 is a block diagram showing the internal configuration of a user terminal (210) and an information processing system (230) according to one embodiment of the present disclosure. The user terminal (210) may refer to any computing device capable of executing an application or a web browser and capable of wired / wireless communication, and may include, for example, a mobile phone terminal (210_1), a tablet terminal (210_2), a PC terminal (210_3) of FIG. 2. As illustrated, the user terminal (210) may include a memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include a memory (332), a processor (334), a communication module (336), and an input / output interface (338). As illustrated in FIG. 3, the user terminal (210) and the information processing system (230) may be configured to communicate information and / or data via a network (220) using respective communication modules (316, 336). In addition, the input / output device (320) may be configured to input information and / or data to the user terminal (210) or output information and / or data generated from the user terminal (210) via the input / output interface (318).

[0048] The memory (312, 332) may include any non-transitory computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a permanent mass storage device such as a read-only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, a permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be included in the user terminal (210) or the information processing system (230) as a separate permanent storage device distinct from the memory. In addition, the memory (312, 332) may store an operating system and at least one program code (e.g., code for an application installed and run on the user terminal (210).

[0049] These software components may be loaded from a computer-readable recording medium separate from the memory (312, 332). This separate computer-readable recording medium may include a recording medium directly connectable to the user terminal (210) and the information processing system (230), and may include, for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. As another example, the software components may be loaded into the memory (312, 332) through a communication module other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (312, 332) based on a computer program that is installed by files provided by developers or a file distribution system that distributes installation files of applications through a network (220).

[0050] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by a memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a storage device such as the memory (312, 332).

[0051] The communication module (316, 336) may provide a configuration or function for the user terminal (210) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (210) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system, etc.). For example, a request or data (e.g., precise map data associated with a specific area and actual road data associated with a specific area, etc.) generated by the processor (314) of the user terminal (210) according to a program code stored in a recording device such as a memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, a control signal or command provided under the control of the processor (334) of the information processing system (230) may be received by the user terminal (210) through the communication module (316) of the user terminal (210) via the communication module (336) and the network (220). For example, the user terminal (210) may receive data related to lane detection, including a lane mask image, from the information processing system (230).

[0052] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera, keyboard, microphone, mouse, etc., including an audio sensor and / or an image sensor, and the output device may include a device such as a display, a speaker, a haptic feedback device, etc. As another example, the input / output interface (318) may be a means for interfacing with a device that has a configuration or function integrated into one for performing input and output, such as a touch screen. For example, when the processor (314) of the user terminal (210) processes a command of a computer program loaded into the memory (312), a service screen configured using information and / or data provided by the information processing system (230) or another user terminal may be displayed on the display through the input / output interface (318). In FIG. 3, the input / output device (320) is illustrated as not being included in the user terminal (210), but is not limited thereto, and may be configured as a single device with the user terminal (210). In addition, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (230) or that the information processing system (230) may include. In FIG. 3, the input / output interfaces (318, 338) are illustrated as elements configured separately from the processors (314, 334), but are not limited thereto, and the input / output interfaces (318, 338) may be configured to be included in the processors (314, 334).

[0053] The user terminal (210) and the information processing system (230) may include more components than those in FIG. 3. However, there is no need to explicitly illustrate most of the conventional components. According to one embodiment, the user terminal (210) may be implemented to include at least some of the input / output devices (320) described above. In addition, the user terminal (210) may further include other components, such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, a database, etc. For example, when the user terminal (210) is a smartphone, it may include components that a smartphone generally includes, and for example, various components, such as an acceleration sensor, a gyro sensor, an image sensor, a proximity sensor, a touch sensor, an illuminance sensor, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration, may be implemented to be further included in the user terminal (210). According to one embodiment, the processor (314) of the user terminal (210) may be configured to operate an application that provides a map information service, etc. At this time, code associated with the application and / or program may be loaded into the memory (312) of the user terminal (210).

[0054] While a program for an application providing a map information service is running, the processor (314) can receive text, images, videos, voices and / or actions, etc. input or selected through an input device such as a camera, microphone, including a touch screen, keyboard, audio sensor and / or image sensor connected to an input / output interface (318), and can store the received text, images, videos, voices and / or actions, etc. in a memory (312) or provide them to an information processing system (230) through a communication module (316) and a network (220).

[0055] The processor (314) of the user terminal (210) may be configured to manage, process, and / or store information and / or data received from an input / output device (320), another user terminal, an information processing system (230), and / or multiple external systems. The information and / or data processed by the processor (314) may be provided to the information processing system (230) via a communication module (316) and a network (220). The processor (314) of the user terminal (210) may transmit the information and / or data to the input / output device (320) via an input / output interface (318) and output the information and / or data. For example, the processor (314) may display the received information and / or data on the screen of the user terminal.

[0056] The processor (334) of the information processing system (230) may be configured to manage, process, and / or store information and / or data received from multiple user terminals (210) and / or multiple external systems. Information and / or data processed by the processor (334) may be provided to the user terminal (210) via a communication module (336) and a network (220).

[0057] FIG. 4 is a diagram illustrating an example of a method for learning a road image generation model (440) according to one embodiment of the present disclosure. According to one embodiment, at least one processor (e.g., at least one processor of an information processing system or a user terminal) may receive precise map data (412) for a specific region and a plurality of actual road images (414) captured within the specific region in order to learn the road image generation model (440).

[0058] Here, the precision map data (412) is a 3D model of a specific area including 3D geometric information and texture information expressed in absolute coordinate positions, and the 3D geometric information may include 3D geometric information about a lane. In addition, the plurality of real road images (414) may include location information. In addition, the plurality of captured real road images (414) may be images captured while driving a vehicle in a specific area. For example, the location information included in the plurality of real road images (414) may be location information acquired using the vehicle's GPS equipment and may be a parameter of 6 degrees of freedom (DoF). For example, the location information included in the plurality of real road images (414) may include direction information (i.e., image shooting direction information) for each of the plurality of real road images (414).

[0059] According to one embodiment, the processor may generate a plurality of learning lane mask images (420) based on the precision map data (412) and a plurality of actual road images (414). For example, the processor may use location information included in the plurality of actual road images (414) to project lane objects included in the precision map data (412) onto the plurality of actual road images (414) to generate a plurality of learning lane mask images (420). Through the process of obtaining a plurality of learning lane mask images (420), pixel location information corresponding to lane objects on the plurality of actual road images (414) can be obtained without any additional annotation work.

[0060] The processor may determine some of the plurality of learning lane mask images (420) as learning lane mask data (430) to minimize overlapping lane layouts among the plurality of learning lane mask images (420) and select data with various lane layouts. For this purpose, a data selection algorithm may be used. Here, the data selection algorithm may mean a method for determining a plurality of vertex sets in which the sum of similarity (or dissimilarity) scores between all pairs of elements included in a fully weighted graph set is minimum (or maximum). For example, a vertex set V = {v1, v2} representing N data points v2,…, v N}, the edge E connecting each data point = {(v i ,v j )} and the weight of each edge W = {w(i,j)} defined as the similarity between two data points connected by an edge. In this case, a predetermined number (e.g., k) of vertices such that the sum of the similarity scores between all pairs of elements included in the complete weighted graph set is minimized, or a set of k vertices that are greater than (or less than) a predetermined threshold can be determined, as described in the algorithm in Table 1 below.

[0061] Algorithm 1Finding a set of K vertices with the minimum sum of similarityInput:Complete weighted graph G and K = # of vertices 1: Find the vertex pair (i, j) which has minimum w(i, j)2: Add i and j to Q3:while|Q| < Kdo4: Find vertex k which minimizes 5:end whileOutput:Q

[0062] According to one embodiment, the processor may calculate a similarity score or a dissimilarity score between a plurality of training lane mask images. For example, two training lane mask images (e.g., M) among the plurality of training lane mask images (420) i , M j ) can be defined as a similarity function s between the two training lane mask images. The similarity between the two training lane mask images can mean the similarity between the lane objects represented by the pixel location information within the image. For example, a lane is a set of 2D points uniformly sampled along the vertical direction in the image plane (e.g., ) can be expressed as, where x i represents the x-coordinate of the i-th sample, and N can represent the number of samples. By utilizing all L lane curves from the entire training data set, the lane matrix can be constructed. If the SVD algorithm is applied to the constructed lane matrix A, it can be factorized as in mathematical expression 1 below.

[0063]

[0064] Here, and are orthogonal matrices, and sigma (Σ) can represent a singular value matrix. Therefore, each suboptimal curve is a singular vector as shown in the mathematical formula 2 below. It can be approximated using M-rank approximation.

[0065]

[0066] singular vector Each lane curve in the eigenline space spanned by Coefficient vector c for i can be expressed as in mathematical formula 3 below.

[0067]

[0068] Two-lane curve and The similarity can be projected onto the eigenline space and defined as the Euclidean distance as in the following mathematical equation 4.

[0069]

[0070] Here, the similarity between lane curves can be extended to the similarity between lane mask images. Two lane mask images each contain m and n lane curves. and Given two lane mask images M , and assuming that m ≥ n always holds, i and M j The similarity s between them can be defined as in mathematical expression 5 below.

[0071]

[0072] Here, can represent a penalty constant. For example, if the number of lane curves in two lane mask images is different, We can penalize the similarity by adding a penalty constant to the training data set. can be determined empirically.

[0073] According to the example described above, a data set for lane mask images that maximizes the diversity of lane layouts, i.e., a maximal diversity set, can be determined as the training lane mask data (430). In one embodiment, the maximal diversity set can be such that the sum of similarity scores between a predetermined number of lane mask images is minimized or the sum of dissimilarity scores is maximized. In another embodiment, the maximal diversity set can include a plurality of lane mask images that are greater than (or less than) a predetermined threshold.

[0074] According to one embodiment, the processor can train a road image generation model (440) using training lane mask data (430) as a maximum diversity set and corresponding training real road data. Thereafter, the processor can generate a plurality of synthetic road images (450) based on the target lane mask image using the trained road image generation model (440). The training lane mask data (430) and the corresponding plurality of synthetic road images (450) can be used to train a lane detection model.

[0075] FIG. 5 is a diagram illustrating an example of a method for learning a lane detection model (550) according to one embodiment of the present disclosure. According to one embodiment, at least one processor (e.g., at least one processor of an information processing system or a user terminal) may receive a plurality of lane mask images (510) to learn the lane detection model (550). Here, the lane mask may indicate a lane area within the image. In addition, here, the plurality of lane mask images (510) may be generated by projecting lane objects included in precision map data onto a plurality of actual road images using location information included in the plurality of actual road images in a method identical / similar to the method described above in FIG. 4.

[0076] The processor may use the road image generation model (520) to generate a plurality of synthetic road images (530) for each of the plurality of lane mask images (510). Here, each of the plurality of lane mask images (510) may include a different lane layout. In addition, the plurality of synthetic road images (530) associated with a specific lane mask image may include the same lane layout associated with the specific lane mask image, but may include different backgrounds and objects. In one embodiment, the processor may generate the plurality of synthetic road images (530), which are unique image data generated by resampling a noise vector based on the received plurality of lane mask images (510).

[0077] After generating the synthetic road images, the processor may determine some of the plurality of synthetic road images (530) as training road data (540) to minimize overlapping images among the plurality of synthetic road images (530) and select data with diverse objects and backgrounds. For this purpose, a data selection algorithm identical to / similar to that described in FIG. 4 may be used.

[0078] In one embodiment, the processor may calculate a similarity score between a plurality of synthetic road images (530). Here, the similarity score may be calculated using a Multi-Scale Structural Similarity Index Measure (MS-SSIM). In addition, the processor may determine a maximal diversity set among the plurality of synthetic road images associated with one lane mask image as training road data (540) based on the calculated similarity score. The processor may determine a maximal diversity set for each of the plurality of lane mask images (510) and use the set as training road data (540). In one embodiment, the maximal diversity set may be such that the sum of the similarity scores between a predetermined number of synthetic road images is minimized or the sum of the dissimilarity scores is maximized. In another embodiment, the maximal diversity set may include synthetic road images that are greater than (or less than) a predetermined threshold.

[0079] Thereafter, the processor can learn a lane detection model (550) based on a plurality of lane mask images (510) and a plurality of synthetic road images (530) corresponding thereto.

[0080] FIG. 6 is a diagram illustrating an example of a plurality of lane mask images (610) and a plurality of synthetic road images (620) generated by a road image generation model according to one embodiment of the present disclosure. According to one embodiment, at least one processor (e.g., at least one processor of an information processing system or a user terminal) may generate a plurality of synthetic road images (620) for each of a plurality of lane mask images (610) using a road image generation model (e.g., 520 of FIG. 5).

[0081] Here, each of the plurality of lane mask images (610) may include a different lane layout. For example, the plurality of lane mask images (610) may include a first lane mask image (612), a second lane mask image (614), a third lane mask image (616), and a fourth lane mask image (618). In this case, the first to fourth lane mask images (612 to 618) may be mask images in which at least one of the number of lanes, the location of the lanes, and the curve of the lanes is different.

[0082] Additionally, each of the plurality of synthetic road images (620) may be associated with a particular lane mask image. For example, the processor may use the road image generation model to generate a first set of synthetic road images (622) associated with the first lane mask image (612). Here, the first set of synthetic road images (622) may be a maximum diversity set of the plurality of synthetic road images associated with the first lane mask image (612). Additionally, the processor may use the road image generation model to generate a second set of synthetic road images (624) associated with the second lane mask image (614). Similarly, the processor may generate a third set of synthetic road images (626) associated with the third lane mask image (616) and a fourth set of synthetic road images (628) associated with the fourth lane mask image (618).

[0083] Here, each of the plurality of synthetic road images (620) may include the same lane layout associated with the associated lane mask image, but may include different backgrounds and objects. For example, the first set of synthetic road images (622) may have the same lane layout as the first lane mask image (612), but each of the first set of synthetic road images (622) may include different backgrounds and objects. It can be confirmed that each of the first set of synthetic road images (622) has the same lane layout, but different road colors, shading states, etc. To this end, according to one embodiment, the processor may generate a plurality of synthetic road images (620), which are unique image data generated by resampling a noise vector based on the plurality of lane mask images (610).

[0084] FIG. 6 illustrates examples of four lane mask images (610) and a synthetic road image (620) associated with each of the four lane mask images (610), but the number of lane mask images (610) and the number of synthetic road images (620) are not limited thereto.

[0085] FIG. 7 is a diagram illustrating the efficiency of a method for determining learning data using a maximum diversity set according to one embodiment of the present disclosure. The relative accuracy performance of the model for a synthetic dataset (LabsLane) comprising multiple synthetic road images (530) described in FIG. 5, generated based on images collected using a vehicle equipped with a high-precision GPS, can be verified. For this purpose, the CULane dataset and the TuSimple dataset were used as benchmark datasets, and Accuracy and Total F1-score were used as comparison metrics, respectively. Here, the CULane dataset is a large-scale dataset used for lane detection, consisting mostly of data collected in complex urban road environments. In addition, the TuSimple dataset is a dataset for lane detection that primarily collects highway driving camera images with annotated lane markings.

[0086] The dataset for verifying the accuracy performance of the model consisted of a dataset in which 10 samples were randomly selected from 100 synthetic data images associated with a single lane mask image from a synthetic dataset (LabsLane), two maximum diversity sets, five maximum diversity sets, and ten maximum diversity sets. As illustrated, it can be confirmed that using the maximum diversity set using the above-described data selection algorithm outperforms using a randomly selected sample. Furthermore, even when using the maximum diversity set, it can be confirmed that the highest performance is achieved when using five maximum diversity sets associated with a single lane mask image.

[0087] Additionally, to verify the performance of the framework of the present disclosure, the performance of each image detection model and lane detection model is shown in Table 2 below. Referring to Table 2, a significant performance difference can be observed between TUSimple and CULane. This is likely due to the fact that the two datasets were acquired in very different environments, and the TUSimple dataset lacks diversity compared to the CULane dataset, as it includes a large number of highway scenarios.

[0088] Experiment SettingTotalNormalCrowdedDazzleShadowNo lineArrowCurveCross-roadNightTrained on CULane78.893.277.271.277.953.689.675.9124072.8Trained on TUSimple24.24119.611.412.712.728.832.612409.1(a) Ablation studiesTrained on LabsLane30.547.121.419.814.617.53839.386923.1+Scene selection33.951.528.015.921.814.643.038.9112024.4+With synthesized images (Proposed)39.656.734.027.122.021.253.246.9127530.1(b) Applying proposed framework on TUSimple dataset+w / o LabsLane31.348.026.623.217.417.839.040.5186318.2(c) Comparison with domain adaptation methodMLDA38.461.436.330.323.421.349.143.41138627.4Proposed (ERFNet)38.256.832.532.222.323.851.240.9457224.4

[0089] Ablation studies on the performance of each stage of the proposed framework of this disclosure demonstrate the effectiveness of the enhanced diversity of the LabsLane dataset. Training a lane detection model using the LabsLane dataset instead of the TUSimple dataset yields approximately 26% performance improvement. Considering that most images in the TUSimple dataset contain only straight-line driving, the diversity of the training dataset is believed to have contributed to this result. Furthermore, for a road image generation model trained on a training dataset determined using the maximum diversity set, further subdividing the LabsLane dataset yields approximately 11% additional performance improvement. Furthermore, when synthetic road images generated using the trained road image generation model are enriched with diversity through the maximum diversity set, a performance improvement of approximately 16% is observed.

[0090] When applying the proposed framework of this disclosure to the TUSimple dataset, it can be seen that the total F1 score (F1-Score) is improved by approximately 29% compared to when training with only the TUSimple dataset.

[0091] Comparison with domain adaptation method of the proposed framework of the present disclosure confirms that the proposed framework is not limited to a specific lane detection model and can be applied to other lane detection models.

[0092] Additionally, the results for the TUSimple dataset of the proposed framework of the present disclosure can be found in Table 3 below.

[0093] AccuracyFPFNTrained on TUSimple95.82.092.8Trained on CULane60.931.655.2MLDA89.729.518.4Proposed framework92.46.79.4

[0094] Referring to Table 3, we can see that, similar to the CULane dataset, there is a significant decrease in all evaluation metrics between the two benchmark datasets. Furthermore, the proposed framework outperforms MLDA, a domain adaptation-based method. Specifically, compared to the results for the CULane dataset, both MLDA and the proposed method demonstrate significant performance improvements. Because the CULane dataset has higher diversity than the TUSimple dataset, we speculate that it will benefit more from the proposed framework, which includes configurations that enhance diversity.

[0095] Table 4 below shows the results verifying the proposed framework's performance on different image generation models. For this purpose, we used OASIS, which uses rich semantic segmentation labels to generate images, and Pix2PixHD, a different algorithm that can generate diverse images by resampling noise vectors. The image generation model was trained using the TUSimple dataset, and its performance was evaluated using the Total F1 score on the CULane dataset.

[0096] TotalTrained on TUSimple24.2Pix2PixHD26.7OASIS29.0

[0097] Referring to Table 4, we see that Pix2PixHD achieves approximately a 10% performance improvement compared to the TUSimple training. Since OASIS produces more realistic images than Pix2PixHDdp, the performance difference is likely due to differences in image quality.

[0098] FIG. 8 is a flowchart illustrating an example of a lane detection model learning method (800) according to one embodiment of the present disclosure. The method (800) may be initiated by a processor (e.g., at least one processor of an information processing system or a user terminal) learning a road image generation model based on precise map data for a specific region (S810). Furthermore, the processor may input a plurality of lane mask images into the road image generation model to generate a plurality of synthetic road images (S820). The processor may learn a lane detection model based on the plurality of lane mask images and the plurality of synthetic road images (S830).

[0099] FIG. 9 is a flowchart illustrating an example of a road image generation model learning method and a road image generation method (900) according to one embodiment of the present disclosure. The method (900) may be initiated by a processor (e.g., at least one processor of an information processing system or a user terminal) receiving high-definition map data for a specific region (S910). In one embodiment, the high-definition map data is a 3D model of a specific region including 3D geometric information and texture information expressed in absolute coordinate positions, and the 3D geometric information may include 3D geometric information for lanes. In one embodiment, the high-definition map data may be generated based on aerial photographs.

[0100] Then, the processor may receive multiple real road images captured within a specific area (S920). Each of the multiple real road images may include location information. Furthermore, the multiple captured real road images may be images captured while driving a vehicle within a specific area.

[0101] Then, the processor can learn a road image generation model based on the precision map data and multiple actual road images (S930). Here, the road image generation model may be a model trained to receive a target lane mask image and generate a synthetic road image corresponding to the target lane mask image.

[0102] According to one embodiment, the processor may generate multiple learning lane mask images based on precision map data and multiple real road images. For example, the processor may use positional information contained in the multiple real road images to project at least a portion of the three-dimensional geometric information contained in the precision map data onto the multiple real road images to generate multiple learning lane mask images.

[0103] Then, the processor can train a road image generation model based on a plurality of training lane mask images and a plurality of actual road images. In one embodiment, the processor can determine a portion of the plurality of actual road images associated with the training lane mask data as training actual road data. Then, the processor can train a road image generation model based on the training lane mask data and the training actual road data.

[0104] Specifically, the processor may determine some of the plurality of learning lane mask images as learning lane mask data. Specifically, the processor may calculate a similarity score or a dissimilarity score between the plurality of learning lane mask images. For example, the processor may calculate the similarity score or the dissimilarity score based on the similarity or dissimilarity of curves for lanes included in each of the plurality of learning lane mask images. Additionally or alternatively, the processor may calculate the similarity score or the dissimilarity score based on the number of lanes included in each of the plurality of learning lane mask images. Then, the processor may determine a maximal diversity set among the plurality of learning lane mask images as learning lane mask data based on the calculated similarity score or dissimilarity score. In one embodiment, the maximal diversity set may be such that the sum of the similarity scores between a predetermined number of lane mask images is minimized or the sum of the dissimilarity scores is maximized.

[0105] According to one embodiment, the processor may generate a plurality of synthetic road images based on a target lane mask image using a learned road image generation model. For example, the processor may receive a target lane mask image (S940). Then, the processor may generate a plurality of synthetic road images associated with the target lane mask image using the road image generation model (S950). Here, the road image generation model is learned based on precise map data for a specific region and a plurality of actual road images captured within the specific region, and each of the plurality of actual road images may include location information. Here, the plurality of generated synthetic road images may include lanes corresponding to the target lane mask image. In addition, the plurality of generated synthetic road images may include different backgrounds and objects.

[0106] According to one embodiment, a plurality of synthetic road images generated using a learned road image generation model can be used to train a lane detection model. For example, after the road image generation model is trained, the processor can receive a first lane mask image and a second lane mask image. Then, the processor can generate a plurality of first synthetic road images associated with the first lane mask image using the road image generation model. Furthermore, the processor can generate a plurality of second synthetic road images associated with the second lane mask image using the road image generation model. Then, the processor can train a lane detection model based on pairs of the first lane mask image and the plurality of first synthetic road images, and pairs of the second lane mask image and the plurality of second synthetic road images.

[0107] The flowchart of FIG. 9 and the above description are merely examples, and the scope of the present disclosure is not limited thereto. For example, at least one step may be added / changed / deleted, or the order of each step may be changed.

[0108] FIG. 10 is a flowchart illustrating an example of a lane detection model learning method and a lane detection method (1000) according to one embodiment of the present disclosure. The method (1000) may be initiated by a processor (e.g., at least one processor of an information processing system or a user terminal) receiving a plurality of lane mask images (S1010). Here, the lane mask may indicate a lane area within the image. In one embodiment, the plurality of lane mask images may be generated by using high-definition map data for a specific region and location information included in a plurality of real road images, and projecting at least a portion of the 3D geometric information included in the high-definition map data onto a plurality of real road images. In one embodiment, the high-definition map data may be generated based on aerial photographs. In addition, the plurality of real road images may be images captured while driving a vehicle in a specific region.

[0109] And, the processor may generate a plurality of synthetic road images for each of the plurality of lane mask images using the road image generation model (S1020). Here, the road image generation model may be a model trained to receive a target lane mask image as input and generate a synthetic road image corresponding to the target lane mask image. For example, the plurality of lane mask images may include a first lane mask image and a second lane mask image. The processor may generate a plurality of first synthetic road images associated with the first lane mask image using the road image generation model. In addition, the processor may generate a plurality of second synthetic road images associated with the second lane mask image using the road image generation model. Here, the plurality of generated first synthetic road images may include a lane corresponding to the first lane mask image. In addition, the plurality of generated first synthetic road images may include different backgrounds and objects from each other. Similarly and identically, the plurality of generated second synthetic road images may include a lane corresponding to the second lane mask image. In addition, the plurality of generated second synthetic road images may include different backgrounds and objects from each other.

[0110] Then, the processor can learn a lane detection model based on multiple lane mask images and multiple synthetic road images (S1030). Here, the lane detection model may be a model trained to receive a target road image and detect lane areas within the target road image.

[0111] In one embodiment, the processor can train a lane detection model based on a plurality of lane masks and portions of a plurality of synthetic road images. For example, a portion of the plurality of first synthetic road images can be determined as first training road data. Additionally, the processor can determine a portion of the plurality of second synthetic road images as second training road data. Furthermore, the processor can train the lane detection model based on pairs of the first lane mask image and the first training road data, and pairs of the second lane mask image and the second training road data.

[0112] In one embodiment, the processor may determine some of the plurality of synthetic road images as training road data based on similarity scores between the plurality of synthetic road images. The processor may calculate similarity scores between the plurality of first synthetic road images. Then, the processor may determine a maximal diversity set among the plurality of first synthetic road images as the first training road data based on the calculated similarity scores. The above-described example may of course be applied to determining some of the plurality of second synthetic road images as the second training road data. Here, the maximal diversity set may be determined such that the sum of the similarity scores between a predetermined number of synthetic road images is minimized. In one embodiment, the similarity scores may be calculated using a Multi-Scale Structural Similarity Index Measure (MS-SSIM).

[0113] In one embodiment, prior to generating a plurality of synthetic road images, the processor may train a road image generation model based on precision map data for a specific area and a plurality of actual road images captured within the specific area.

[0114] According to one embodiment, the processor may detect a lane area within an image based on a target actual road image using a learned lane detection model. For example, the processor may receive a target actual road image (S1040). Then, the processor may generate a target lane mask image associated with the target actual road image using the lane detection model (S1050). Here, the lane detection model may be a model learned based on a plurality of lane mask images and a plurality of synthetic road images. In addition, the lane mask images may represent lane areas within the image, and the plurality of synthetic road images may be images generated for each of a plurality of lane masks using the road image generation model.

[0115] The flowchart of FIG. 10 and the above description are merely examples, and the scope of the present disclosure is not limited thereto. For example, at least one step may be added / changed / deleted, or the order of each step may be changed.

[0116] The above-described method may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0117] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and the design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0118] In a hardware implementation, the processing units used to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, a computer, or a combination thereof.

[0119] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0120] In a firmware and / or software implementation, the techniques may be implemented as instructions stored on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a compact disc (CD), a magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described herein.

[0121] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the present disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include personal computers, network servers, and portable devices.

[0122] While the present disclosure has been described in connection with certain embodiments herein, various modifications and variations may be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations are intended to fall within the scope of the claims appended to this specification.

Claims

1. A method for learning a road image generation model (generative model) performed by at least one processor, A step of receiving high-definition map data for a specific area; A step of receiving a plurality of real road images captured within the above specific area; and A step of learning a road image generation model based on the above-described precision map data and the plurality of actual road images. Including, A method for learning a road image generation model, wherein each of the plurality of real road images includes location information.

2. In paragraph 1, The above precision map data is a 3D model of a specific area including 3D geometric information and texture information expressed in absolute coordinate positions, A method for learning a road image generation model, wherein the above 3D geometric information includes 3D geometric information about lanes.

3. In paragraph 1, The above detailed map data is a road image generation model learning method generated based on aerial photographs.

4. In paragraph 1, A method for learning a road image generation model, wherein the plurality of actual road images photographed above are images photographed while driving a vehicle in the specific area.

5. In paragraph 1, The step of learning the above road image generation model is: A step of generating a plurality of learning lane mask images based on the above-described precision map data and the plurality of actual road images. A method for learning a road image generation model, including:

6. In paragraph 5, The step of generating the plurality of learning lane mask images is as follows: A step of generating a plurality of learning lane mask images by projecting at least some of the 3D geometric information included in the precision map data onto the plurality of actual road images using the location information included in the plurality of actual road images. A method for learning a road image generation model, including:

7. In paragraph 5, The step of learning the above road image generation model is: A step of learning the road image generation model based on the plurality of learning lane mask images and the plurality of actual road images. A method for learning a road image generation model, which further includes:

8. In paragraph 5, The step of learning the above road image generation model is: A step of determining some of the plurality of learning lane mask images as learning lane mask data; A step of determining a portion of the plurality of real road images associated with the learning lane mask data as learning real road data; and A step of learning the road image generation model based on the above learning lane mask data and the above learning actual road data. A method for learning a road image generation model, which further includes:

9. In paragraph 8, The step of determining some of the above multiple lane mask images as lane mask data for learning is: A step of calculating a similarity score or dissimilarity score between the plurality of learning lane mask images; and A step of determining a set with the maximum diversity among the plurality of learning lane mask images as learning lane mask data based on the similarity score or dissimilarity score calculated above. A method for learning a road image generation model, including:

10. In paragraph 9, A method for learning a road image generation model, wherein the above maximum diversity set is such that the sum of similarity scores between a predetermined number of lane mask images is minimized or the sum of dissimilarity scores is maximized.

11. In paragraph 9, The step of calculating the above similarity score or dissimilarity score is: A step of calculating a similarity score or a dissimilarity score based on the similarity or dissimilarity of curves for lanes included in each of the plurality of learning lane mask images. A method for learning a road image generation model, including:

12. In paragraph 9, The step of calculating the above similarity score or dissimilarity score is: A step of calculating a similarity score or dissimilarity score based on the number of lanes included in each of the plurality of learning lane mask images. A method for learning a road image generation model, including:

13. In paragraph 1, A road image generation model learning method, wherein the above road image generation model is trained to receive a target lane mask image as input and generate a synthetic road image corresponding to the target lane mask image.

14. In paragraph 1, After the above road image generation model is trained, A step of receiving a first lane mask image and a second lane mask image; A step of generating a plurality of first synthetic road images associated with the first lane mask image using the above road image generation model; A step of generating a plurality of second synthetic road images associated with the second lane mask image using the above road image generation model; and A step of learning a lane detection model based on pairs of the first lane mask image and the plurality of first synthetic road images, and pairs of the second lane mask image and the plurality of second synthetic road images. A method for learning a road image generation model, which further includes:

15. A method for generating a road image performed by at least one processor, A step of receiving a target lane mask image; and A step of generating multiple synthetic road images associated with the target lane mask image using a road image generation model. Including, The above road image generation model is learned based on precise map data for a specific area and multiple actual road images taken within the specific area, A method for generating a road image, wherein each of the plurality of actual road images includes location information.

16. In paragraph 15, A road image generation method, wherein the plurality of generated synthetic road images include lanes corresponding to the target lane mask image.

17. In paragraph 15, A method for generating a road image, wherein the plurality of synthetic road images generated above include different backgrounds and objects.

18. A computer-readable non-transitory recording medium recording commands for executing the method according to paragraph 1 on a computer.

19. As an information processing system, Communication module; memory; and At least one processor connected to said memory and configured to execute at least one computer-readable program contained in said memory Including, At least one program above, Receive precise map data for a specific area, Receive multiple real road images captured within the above specific area, Includes commands for learning a road image generation model based on the above-described precision map data and the plurality of actual road images, An information processing system, wherein each of the plurality of real road images includes location information.

20. As an information processing system, Communication module; memory; and At least one processor connected to said memory and configured to execute at least one computer-readable program contained in said memory Including, At least one program above, Receive the target lane mask image, Includes commands for generating a plurality of synthetic road images associated with the target lane mask image using a road image generation model, The above road image generation model is learned based on precise map data for a specific area and multiple actual road images taken within the specific area, An information processing system, wherein each of the plurality of real road images includes location information.

Images