Small sample lane line generation method and device, medium and equipment

Abstract

The invention discloses a small sample lane line generation method and device, a medium and equipment. The method comprises the following steps: acquiring common lane lines and target small sample lane lines, wherein the common lane lines and the target small sample lane lines are in one-to-one correspondence; training the pre-training diffusion model based on the common lane line and the corresponding target small sample lane line to obtain a small sample lane line generation model; the original training image set is input into the small sample lane line generation model to obtain a new training image set, and the proportion of the small sample lane line images in the new training image set is higher than the proportion of the small sample lane line images in the original training image set. According to the method, additional manual marking and data acquisition processes are not needed, the required small sample lane line image can be flexibly, efficiently and accurately generated according to the existing common lane line, the data size and diversity of the small sample lane lines in the training data required by lane line detection are improved, and the detection accuracy is improved. And the accuracy and robustness of lane line detection are ensured.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and particularly to a method, device, medium and equipment for generating small-sample lane lines. Background Art

[0002] In autonomous driving, existing autonomous driving systems usually identify the positions of lane lines and judge the meanings of lane lines based on lane line detection models to determine the position of the vehicle relative to the lane, so as to plan a driving path for the vehicle.

[0003] Among them, the lane line detection model is often obtained by training a deep learning model, and its detection accuracy depends on a large amount of diverse training data. However, in the real road environment, the data of common types of lane lines such as white solid lines, white dotted lines and double yellow lines account for more than 80% of the total, while the number of other uncommon or special-shaped lane lines is relatively small, belonging to small-sample lane lines. Therefore, the training data for the lane line detection model often lacks balance, resulting in the model having a strong recognition ability for common lane lines, while the detection of small-sample lane lines is extremely difficult, often resulting in misjudgment and missed judgment, and the accuracy and generalization ability of the model significantly decrease when processing small-sample lane line data, which is not conducive to the safety and driving comfort of relevant vehicles and personnel.

[0004] In response to the above problems, some solutions have been proposed currently. For example, specifically collecting and annotating a large amount of special-shaped and small-sample data to enrich the training data set; using simulation data to supplement special-shaped and small-sample data to increase the generalization ability of the model; artificially creating special-shaped and small-sample data and then collecting it at fixed points. However, the method of specifically collecting and annotating often takes a lot of time and effort, and it is difficult to completely eliminate the influence of common lane lines. For example, common lane lines often exist in the images of small-sample lane lines, interfering with the feature extraction of small-sample lane lines; the data collected by simulation often has an obvious deviation from the real data, resulting in unstable performance of the model in actual applications; the artificially created small-sample data is costly and has poor generalization, making it difficult to meet the needs of actual applications. Summary of the Invention

[0005] This application mainly provides a method, device, medium and equipment for generating small-sample lane lines, aiming to solve the technical problem of difficult acquisition of small-sample lane line data.

[0006] To solve the above technical problems, the technical solution adopted in this application is: to provide a small-sample lane line generation method. The small-sample lane line generation method includes: obtaining common lane lines and target small-sample lane lines, where the common lane lines and the target small-sample lane lines correspond one by one; training a pre-trained diffusion model based on the common lane lines and the corresponding target small-sample lane lines to obtain a small-sample lane line generation model; inputting the original training atlas into the small-sample lane line generation model to obtain a new training atlas, where the proportion of small-sample lane line images in the new training atlas is higher than the proportion of small-sample lane line images in the original training atlas.

[0007] In some embodiments, the obtaining of the common lane lines and the target small-sample lane lines includes: obtaining the common lane lines; preprocessing the common lane lines to obtain the target small-sample lane lines that have the same background features as the common lane lines and have preset small-sample lane line features.

[0008] In some embodiments, the preprocessing includes at least one of lane line color modification, lane line element erasure, lane line segmentation, and lane line background fusion.

[0009] In some embodiments, the training of the pre-trained diffusion model based on the common lane lines and the corresponding target small-sample lane lines to obtain a small-sample lane line generation model includes: determining a prompt for converting the common lane lines into the corresponding small-sample lane lines based on the categories of the common lane lines and the target small-sample lane lines; invoking the pre-trained diffusion model based on the prompt to perform inference on the common lane lines to obtain predicted small-sample lane lines; calculating the loss value between the corresponding target small-sample lane lines and the predicted small-sample lane lines, and iteratively fine-tuning the pre-trained diffusion model based on the loss value to obtain the small-sample lane line generation model.

[0010] In some embodiments, before invoking the pre-trained diffusion model based on the prompt to perform inference on the common lane lines to obtain predicted small-sample lane lines, it further includes: normalizing the common lane lines and superimposing random noise on the data of the common lane lines to update the common lane lines.

[0011] In some embodiments, the loss value between the target small-sample lane lines and the predicted small-sample lane lines is the root mean square of the errors between each pixel point of the target small-sample lane lines and each corresponding pixel point in the predicted small-sample lane lines.

[0012] In some embodiments, after inputting the original training atlas into the few-shot lane line generation model to obtain a new training atlas, the method further includes: determining that a training set adjustment instruction is received, and adjusting the proportion and data volume of the few-shot lane line images in the new training atlas based on the training set adjustment instruction.

[0013] To solve the above technical problems, another technical solution adopted by this application is: to provide a few-shot lane line generation device, which includes: an acquisition module, configured to acquire common lane lines and target few-shot lane line; a training module, configured to train a pre-trained diffusion model based on the common lane lines and the target few-shot lane lines to obtain a few-shot lane line generation model; an inference module, configured to input the original training atlas into the few-shot lane line generation model to obtain a new training atlas, where the proportion of few-shot lane line images in the new training atlas is higher than that in the original training atlas.

[0014] To solve the above technical problems, another technical solution adopted by this application is: to provide a storage medium, on which program data is stored, and characterized in that when the program data is executed by a processor, the steps of the few-shot lane line generation method as described above are implemented.

[0015] To solve the above technical problems, another technical solution adopted by this application is: to provide a computer device, which includes a processor and a memory connected to each other, the memory stores a computer program, and when the processor executes the computer program, the steps of the few-shot lane line generation method as described above are implemented.

[0016] The beneficial effects of this application are: different from the prior art, this application discloses a few-shot lane line generation method, device, medium and equipment. This application trains a pre-trained diffusion model with common lane lines and target few-shot lane lines to obtain a few-shot lane line generation model, without additional manual annotation and data collection processes, which is highly generalizable. Subsequently, only the common lane line images need to be input into the model to flexibly, efficiently and accurately generate the required few-shot lane line images, which can increase the data volume and diversity of few-shot lane lines in the training data required by the lane line detection model, reduce the cost of data collection and annotation of few-shot lane lines, and is conducive to ensuring the accuracy and robustness of lane line detection, thereby improving the overall performance and safety of the autonomous driving system. Description of the Drawings

[0017] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings, where:

[0018] Figure 1 is a schematic flowchart of an embodiment of the small-sample lane line generation method provided by the present application;

[0019] Figure 2 is Figure 1 a schematic flowchart of an embodiment of step 10 in the embodiment;

[0020] Figure 3 is Figure 1 a schematic flowchart of an embodiment of step 20 in the embodiment;

[0021] Figure 4 is a schematic structural diagram of an embodiment of the small-sample lane line generation device provided by the present application;

[0022] Figure 5 is a schematic structural diagram of an embodiment of the storage medium provided by the present application;

[0023] Figure 6 is a schematic structural diagram of an embodiment of the computer device provided by the present application. Detailed implementation manners

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0025] The terms "first", "second", and "third" in the embodiments of the present application are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", and "third" may explicitly or implicitly include at least one of such features. In the description of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0026] Reference to "embodiment" in this context means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0027] The present application provides a method for generating small-sample lane lines. Refer to Figure 1 , Figure 1 which is a schematic flowchart of an embodiment of the method for generating small-sample lane lines provided by the present application. The method for generating small-sample lane lines includes:

[0028] Step 10: Obtain common lane lines and target small-sample lane lines, where the common lane lines and the target small-sample lane lines correspond one by one.

[0029] In this embodiment, common lane lines refer to types of lane lines that appear frequently in road traffic, such as white solid lines, white dashed lines, double yellow lines, and left-dashed right-solid white lines. Target small-sample lane lines, on the other hand, refer to those lane lines with a low appearance frequency and significant differences in shape or color from common lane lines, such as colored lane lines, lane lines with special markings, etc. These lane lines can be obtained through various means, such as screening from existing traffic image databases, synthesizing through image processing techniques, or taking on-site photos under specific conditions. In this embodiment, the common lane lines and target small-sample lane lines used are paired, and the common lane lines and target small-sample lane lines correspond one by one to reflect the differential features between the target small-sample lane lines and the common lane lines, enabling the subsequent trained model to automatically process the common lane lines into small-sample lane lines based on these differential features and achieve the generation of small-sample lane lines.

[0030] Optionally, refer to Figure 2, in some embodiments, obtaining common lane lines and target small-sample lane lines can be performed according to the following steps:

[0031] Step 11: Obtain common lane lines.

[0032] Step 12: Preprocess the common lane lines to obtain target small-sample lane lines that have the same background features as the common lane lines and have preset small-sample lane line features.

[0033] In this optional embodiment, the method for obtaining the corresponding target small-sample lane lines by preprocessing the common lane lines is specified. Among them, preprocessing can ensure that the common lane lines and the corresponding target small-sample lane lines have the same background features, thereby highlighting the differences in the foreground lane line features, so that in subsequent steps, the model's recognition ability for lane line features can be further enhanced. Specifically, there are various preprocessing methods for common lane lines. For example, the brightness and contrast of the lane lines can be adjusted to simulate lane line images under different lighting conditions, the image distortion technology, such as affine transformation, can be applied to simulate the deformation of lane lines at different perspectives, noise or blur effects can be added to simulate lane line images under bad weather conditions, and image synthesis technology can be used to synthesize specific markings or color features of target small-sample lane lines into common lane line images to create target small-sample lane line images with specific features. As long as the common lane line images can be processed into target small-sample lane line images with the required target small-sample lane features and the background consistency is ensured.

[0034] Optionally, in some embodiments, the preprocessing includes at least one of lane line color modification, lane line element erasure, lane line segmentation, and lane line background fusion.

[0035] In this optional embodiment, a variety of methods are provided that are very suitable for processing common lane lines into target small-sample lane lines. Among them, lane line color modification refers to changing the color of common lane lines to a color similar to or the same as that of target small-sample lane lines; lane line element erasure refers to removing some areas of lane lines in the image to simulate the missing or incomplete features of target small-sample lane lines relative to common lane lines; lane line segmentation refers to dividing lane lines into multiple parts to simulate the discontinuous or non-continuous features of target small-sample lane lines; lane line background fusion refers to fusing the background of target small-sample lane lines with the background of common lane lines to simulate the visual effect of target small-sample lane lines in a specific background. Through these preprocessing steps, various target small-sample lane lines can be effectively simulated, providing rich training samples for subsequent lane line generation.

[0036] In this alternative embodiment, preprocessing such as lane line color modification, lane line element erasure, lane line segmentation, and lane line background fusion can be implemented in various ways. For example, it can be processed through image editing software. Image editing software such as Photoshop usually supports manual operation processing of the above preprocessing steps, and advanced image editing software already supports instruction control based on large language models. It can also be processed through a dedicated processing model. For example, the Inpaint model can be used to erase lane line elements, and the Segmentation model can be used to segment lane lines. Or it can also be preprocessed by writing specific algorithm scripts in combination with specific color space conversion techniques, image restoration algorithms, image segmentation algorithms, and background fusion algorithms.

[0037] Taking the processing of common fishbone double white lines into small-sample left fishbone right smooth double yellow lines as an example in this alternative embodiment, the fishbone lines can be filled with yellow using Photoshop software first, and then the fishbone on the right side of the lane line can be segmented through a specific image segmentation algorithm. Immediately, the segmented fishbone can be erased using the Inpaint model, and the erasure can be filled through the background fusion algorithm to adapt to its background image, obtaining paired data of fishbone double white lines - left fishbone right smooth double yellow lines.

[0038] By obtaining common lane lines and corresponding target small-sample lane lines through the above steps in this alternative embodiment, it can ensure that the common lane lines and the corresponding target small-sample lane lines have the same background characteristics, thereby effectively highlighting the differences in the foreground lane line characteristics, simulating the diversity of the target small-sample lane lines effectively, ensuring that the small-sample lane line images obtained through preprocessing have a high similarity to the common lanes obtained in the real scene visually, enabling the model's recognition ability of lane line characteristics to be further enhanced in subsequent steps, and improving the accuracy, robustness, and generalization ability of the model in practical applications.

[0039] Step 20: Train a pre-trained diffusion model based on the common lane lines and the target small-sample lane lines to obtain a small-sample lane line generation model.

[0040] In this embodiment, after obtaining the corresponding common lane lines and target small-sample lane lines one by one, the pre-trained diffusion model can be trained using this one-to-one corresponding data. Among them, the diffusion model is a generative model based on the probabilistic diffusion process. It gradually adds noise to the data and then learns how to recover the data from the noise to generate new data samples. The pre-trained diffusion model is a diffusion model that has been trained with a large amount of data. It can capture the distribution characteristics of the data and generate a lane line image that meets the requirements under a given prompt. The training of this pre-trained diffusion model is a process of fine-tuning for a specific task. In this embodiment, it specifically refers to iteratively fine-tuning the pre-trained diffusion model through common lane lines and corresponding target small-sample lane lines to meet the need for generating small-sample lane line images.

[0041] In this embodiment, the fine-tuning based on the common lane lines and the corresponding target small-sample lane lines can be specifically implemented through a variety of optimization algorithms. For example, techniques such as the gradient descent method, the stochastic gradient descent method, the Adam optimization algorithm, and the loss-based backpropagation algorithm can be used to guide the model to learn. These algorithms can help the model converge quickly during the training process and find the optimal parameter configuration to improve the accuracy and efficiency of generating small-sample lane lines. During the training process, the model will continuously adjust its internal parameters to minimize the difference between the generated lane lines and the real lane lines, so that the model can finally output the corresponding small-sample lane line image only by inputting the common lane line image.

[0042] Optionally, refer to Figure 3 , in some embodiments, training the pre-trained diffusion model based on the common lane lines and the corresponding target small-sample lane lines to obtain a small-sample lane line generation model can be performed according to the following steps:

[0043] Step 21: Determine the prompt for converting the common lane line into the corresponding small-sample lane line based on the category of the common lane line and the category of the target small-sample lane line.

[0044] Step 22: Based on the prompt, call the pre-trained diffusion model to infer the common lane line to obtain the predicted small-sample lane line.

[0045] Step 23: Calculate the loss value between the corresponding target small-sample lane line and the predicted small-sample lane line, and iteratively fine-tune the pre-trained diffusion model based on the loss value to obtain the small-sample lane line generation model.

[0046] In this alternative embodiment, a processing method for model training based on prompts is specified. Among them, a prompt is an instruction or description used to guide a pre-trained diffusion model to generate specific small-sample lane lines. It does not need to be extremely detailed. Based on the powerful semantic understanding ability of the pre-trained diffusion model, it only needs to semantically indicate the requirement of converting common lane lines into corresponding small-sample lane lines. For example, for the prompt corresponding to processing the common fishbone double white lines into the small-sample left fishbone right smooth double yellow lines shown in the relevant descriptions of steps 11 and 12 above, it can specifically be a description of the processing process, such as "fill the fishbone lines with yellow and adjust the right side of the fishbone lines to be smooth", or it can also be a description of the requirement, such as directly "process the fishbone double white lines into left fishbone right smooth double yellow lines". Usually, the semantic parsing ability of some pre-trained diffusion models is sufficient to achieve this. According to the specific categories and characteristics of the target small-sample lane lines, different prompts can be designed to adapt to different conversion requirements and match the actual semantic parsing characteristics of the pre-trained diffusion model, so that the pre-trained diffusion model can generate small-sample lane line images with specific characteristics according to different prompts.

[0047] In this alternative embodiment, after the prompt is confirmed, the pre-trained diffusion model can, based on its built-in knowledge or the invocation of an external knowledge base, reason about common lane lines and generate predicted small-sample lane lines. During the reasoning process, the model will consider the category information and feature descriptions contained in the prompt to ensure that the generated lane lines conform to the characteristics of the target small-sample lane lines.

[0048] In this alternative embodiment, after the above steps, the loss value between the predicted small-sample lane lines and the actual target small-sample lane lines can be calculated. This loss value can specifically be the Mean-Square Error (MSE), the Structural Similarity Index (SSIM), or other suitable metrics. This loss value reflects the degree of difference between the lane lines generated by the model and the real lane lines. Through the feedback of the loss value, the model can perform iterative fine-tuning and continuously optimize its parameters to reduce the difference between the generated lane lines and the real lane lines. Eventually, after multiple iterative fine-tuning, the pre-trained diffusion model will be adjusted to a model that can accurately generate small-sample lane lines, that is, a small-sample lane line generation model.

[0049] In the process of obtaining the small-sample lane line generation model in this alternative embodiment, no additional manual annotation and data collection processes are required, which is highly generalizable. Subsequently, only common lane line images need to be input into the model to flexibly, efficiently, and accurately generate the required small-sample lane line images, which can increase the data volume and diversity of small-sample lane lines in the training data required for the lane line detection model, reduce the cost of collecting and annotating small-sample lane line data, and is conducive to ensuring the accuracy and robustness of lane line detection, thereby improving the overall performance and safety of the autonomous driving system.

[0050] Optionally, in some embodiments, before inferring the predicted small-sample lane lines by invoking the pre-trained diffusion model based on the prompt, it further includes: standardizing the common lane lines and superimposing random noise on the data of the common lane lines to update the common lane lines.

[0051] In this alternative embodiment, standardization means converting the data of the common lane lines into a unified format or range to ensure that the model can process this data more effectively. Standardization can include one or more of format conversion processing, color space conversion processing, color space adjustment, tensor conversion processing, normalization processing, and size compression processing. Through these processing methods, the data can be adjusted to a format suitable for model input. For example, through the processing of VAE (Variational Autoencoder), the common lane lines can be downloaded and compressed, such as compressing their image size from 512×512 to 64x64, converting them into a latent variable representation in a low-dimensional space, and then the decoder can restore these latent variables to the standardized common lane line images subsequently.

[0052] In this alternative embodiment, superimposing random noise is to simulate the random interference that images in the real world may be subject to, such as camera jitter, electronic noise, etc. This helps to improve the robustness of the model so that it can still maintain good performance when facing noise in actual applications. These noises can specifically be noises generated by specific algorithms such as Gaussian random noise and salt-and-pepper random noise, or noise samples randomly collected from real-world scenarios. In this way, the model can learn how to extract useful information from the noise during the training process, so as to accurately identify and predict lane lines in actual applications. In addition, the process of superimposing noise can also control the generalization ability of the model by adjusting the intensity and type of the noise, ensuring that the model can maintain stable performance under different noise levels.

[0053] Optionally, in some embodiments, the loss value between the target small-sample lane lines and the predicted small-sample lane lines is the root mean square of the average of the errors between each pixel point of the target small-sample lane lines and each corresponding pixel point in the predicted small-sample lane lines.

[0054] In this alternative embodiment, a method for calculating the loss value based on the root mean square of the error is specified. The error between each pixel point of the target small-sample lane line and the corresponding pixel points in the predicted small-sample lane line refers to the difference in grayscale values or color values between the corresponding pixels of the two, and this difference reflects the difference between the two lane line images at the pixel level. The root mean square is the value obtained by averaging the squares of these differences and then taking the square root. This calculation method can effectively measure the overall difference between the predicted lane line and the target lane line, and has a certain inhibitory effect on outliers, enabling the model to pay more attention to the overall image quality rather than the errors of individual pixels during the training process.

[0055] In this alternative embodiment, by calculating the root mean square, a single value can be obtained, which can quantify the overall difference between the predicted lane line and the target lane line. This calculation method helps the model to more accurately adjust the parameters during the training process to reduce the error between the predicted lane line and the true lane line. In practical applications, this method of calculating the loss value can provide a clear optimization goal, guiding the model to continuously improve through the iterative process until the generated lane line image is close enough to the target lane line image. Finally, the optimized model can generate high-quality small-sample lane line images, providing strong support for improving the performance of lane line detection and autonomous driving systems.

[0056] Step 30: Input the original training atlas into the small-sample lane line generation model to obtain a new training atlas, where the proportion of small-sample lane line images in the new training atlas is higher than that in the original training atlas.

[0057] In this embodiment, based on the previously obtained small-sample lane line generation model, the original training atlas can be input into the small-sample lane line generation model to obtain a new training atlas. In this way, the number and diversity of small-sample lane line images in the training data can be significantly increased, thereby improving the model's ability to recognize the features of small-sample lane lines. Therefore, the proportion of small-sample lane line images in the new training atlas is higher than that in the original training atlas. Through this method, a data augmentation strategy is realized, which can effectively alleviate the problem of insufficient small-sample lane line data and improve the lane line detection performance of the autonomous driving system in complex scenarios. In addition, the generation process of the new training atlas is automated and does not require manual intervention, which not only improves the efficiency of data processing but also reduces the cost of data processing. Finally, by using the new training atlas to train the lane line detection model, a more robust and accurate lane line detection model can be obtained, providing solid technical support for the safe operation of the autonomous driving system.

[0058] Optionally, in some embodiments, after inputting the original training atlas into the few-shot lane line generation model to obtain a new training atlas, it further includes: determining that a training set adjustment instruction is received, and adjusting the proportion and data volume of the few-shot lane line images in the new training atlas based on the training set adjustment instruction.

[0059] In this optional embodiment, in order to make the data used for training the lane line detection model better adapt to the requirements of different scenarios, a mechanism for dynamically adjusting the new training atlas is provided. The training set adjustment instruction can be based on specific performance metrics, such as the accuracy, recall, or F1 score of the model on the validation set, or it can be based on the diversity requirements of lane lines in the actual application scenario. Through the adjustment instruction, the proportion and data volume of the few-shot lane line images in the new training atlas can be optimized to ensure that the model can maintain good performance under various complex conditions. For example, if the recognition accuracy of the model for smooth lane lines is low in a certain specific scenario, more smooth lane line images can be generated and added to the new training atlas to improve the model's recognition ability for this type of lane line. Similarly, if it is found that the performance of the model decreases at night or under low light conditions, the lane line image data under these conditions can be increased to enhance the generalization ability of the model. Through this flexible data adjustment strategy, it can be ensured that the model always faces diverse data during the training process, thereby effectively avoiding overfitting and improving the performance of the model in actual applications. Finally, the optimized new training atlas will provide richer and higher-quality training data for the lane line detection model, contributing to the construction of a more superior autonomous driving system.

[0060] Refer to Figure 4 , Figure 4 is a schematic structural diagram of an embodiment of the few-shot lane line generation device provided by this application.

[0061] The few-shot lane line generation device 40 includes: an acquisition module 41, configured to acquire common lane lines and target few-shot lane lines, where the common lane lines and the target few-shot lane lines correspond one by one; a training module 42, configured to train a pre-trained diffusion model based on the common lane lines and the corresponding target few-shot lane lines to obtain a few-shot lane line generation model; an inference module 43, configured to input the original training atlas into the few-shot lane line generation model to obtain a new training atlas, where the proportion of the few-shot lane line images in the new training atlas is higher than the proportion of the few-shot lane line images in the original training atlas.

[0062] Optionally, in some embodiments, the acquisition module 41 is further specifically configured to: acquire common lane lines; perform preprocessing on the common lane lines to obtain target few-shot lane lines that have the same background characteristics as the common lane lines and have preset few-shot lane line characteristics.

[0063] Optionally, in some embodiments, the preprocessing includes at least one of lane line color modification, lane line element erasure, lane line segmentation, and lane line background fusion.

[0064] Optionally, in some embodiments, the training module 42 is further specifically configured to: determine a prompt for converting a common lane line into a corresponding small-sample lane line based on the categories of common lane lines and the categories of target small-sample lane lines; call a pre-trained diffusion model to perform inference on the common lane line based on the prompt to obtain a predicted small-sample lane line; calculate the loss value between the target small-sample lane line and the predicted small-sample lane line, and iteratively fine-tune the pre-trained diffusion model based on the loss value to obtain a small-sample lane line generation model.

[0065] Optionally, in some embodiments, before calling the pre-trained diffusion model to perform inference on the common lane line based on the prompt to obtain a predicted small-sample lane line, it further includes: normalizing the common lane line and superimposing random noise on the data of the common lane line to update the common lane line.

[0066] Optionally, in some embodiments, the loss value between the target small-sample lane line and the predicted small-sample lane line is the root mean square of the errors between each pixel point of the target small-sample lane line and each corresponding pixel point in the predicted small-sample lane line.

[0067] Optionally, in some embodiments, after inputting the original training atlas into the small-sample lane line generation model to obtain a new training atlas, it further includes: determining that a training set adjustment instruction is received, and adjusting the proportion and data volume of the small-sample lane line images in the new training atlas based on the training set adjustment instruction.

[0068] Since the embodiments of the apparatus part correspond to the embodiments of the above method, the introduction of the small-sample lane line generation apparatus 40 provided in the embodiments of the present invention may refer to the above method embodiments, and the embodiments of the present invention will not be described in detail herein, and it has the same beneficial effects as the above small-sample lane line generation method.

[0069] Refer to Figure 5 , Figure 5 which is a schematic structural diagram of an embodiment of the storage medium provided in the present application.

[0070] The storage medium 50 stores program data 51, and when the program data 51 is executed by a processor, it implements the small-sample lane line generation method as Figures 1 to 3 described.

[0071] The program data 51 is stored in a storage medium 50, including several instructions for causing a network device (such as a router, a personal computer, a server, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0072] Optionally, the storage medium 50 can be various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store the program data 51.

[0073] Refer to Figure 6 , Figure 6 is a schematic structural diagram of an embodiment of the computer device provided by the present application.

[0074] The computer device 60 includes a processor 62 and a memory 61 that are interconnected. The memory 61 stores a computer program. When the processor 62 executes the computer program, the method for generating small-sample lane lines as described in Figures 1 to 3 is implemented. Among them, the memory X1 can include the storage medium 50 or can be other separately developed memories.

[0075] Different from the prior art, the present application discloses a method, device, medium, and device for generating small-sample lane lines. By using common lane lines and corresponding target small-sample lane lines to train a pre-trained diffusion model, a small-sample lane line generation model is obtained, without additional manual annotation and data collection processes, which is highly generalizable. Subsequently, only the common lane line images need to be input into the model to flexibly, efficiently, and accurately generate the required small-sample lane line images, which can increase the data volume and diversity of small-sample lane lines in the training data required for the lane line detection model, reduce the cost of small-sample lane line data collection and annotation, and is beneficial to ensuring the accuracy and robustness of lane line detection, thereby improving the overall performance and safety of the autonomous driving system.

[0076] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments. In particular, for the device, storage medium embodiments, and computer device embodiments, since they are basically similar to the method embodiments, the descriptions are relatively simple, and the relevant parts can refer to the partial descriptions of the method embodiments.

[0077] The present application can be used in many general or special computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, small computers, large computers, distributed computing environments including any of the above systems or devices, and so on.

[0078] In several embodiments provided in the present application, it should be understood that the disclosed methods, devices, storage media, and computer devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0079] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0080] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0081] The above are only the embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A small sample lane line generation method, characterized in that: include: Acquire common lane lines and target small sample lane lines, wherein the common lane lines and the target small sample lane lines correspond to each other one by one; The pre-trained diffusion model is trained based on the common lane lines and the corresponding target small sample lane lines to obtain a small sample lane line generation model; The original training atlas is input into the small sample lane line generation model to obtain a new training atlas, wherein the proportion of small sample lane line images in the new training atlas is higher than the proportion of small sample lane line images in the original training atlas.

2. The small sample lane line generation method according to claim 1, characterized in that: The obtaining of common lane lines and target small sample lane lines includes: Obtaining the common lane line; The common lane lines are preprocessed to obtain the target small sample lane lines having the same background features as the common lane lines and having preset small sample lane line features.

3. The small sample lane line generation method according to claim 2, characterized in that: The preprocessing includes at least one of lane line color modification, lane line element erasing, lane line segmentation and lane line background fusion.

4. The small sample lane line generation method according to claim 1, characterized in that: The pre-trained diffusion model is trained based on the common lane lines and the corresponding target small sample lane lines to obtain a small sample lane line generation model, including: Based on the category of the common lane line and the category of the target small sample lane line, determining to convert the common lane line into a prompt corresponding to the small sample lane line; Based on the prompt, the pre-trained diffusion model is called to infer the common lane line to obtain a predicted small sample lane line; The loss values ​​of the corresponding target small sample lane lines and the predicted small sample lane lines are calculated, and the pre-trained diffusion model is iteratively fine-tuned based on the loss values ​​to obtain the small sample lane line generation model.

5. The small sample lane line generation method according to claim 4, characterized in that: Before the pre-trained diffusion model is called based on the prompt to infer the common lane line to obtain the predicted small sample lane line, the method further includes: The common lane lines are standardized, and random noise is superimposed on data of the common lane lines to update the common lane lines.

6. The small sample lane line generation method according to claim 4, characterized in that: The loss value between the target small sample lane line and the predicted small sample lane line is the average root mean square of the error between each pixel point of the target small sample lane line and each corresponding pixel point in the predicted small sample lane line.

7. The small sample lane line generation method according to claim 1, characterized in that: After the original training atlas is input into the small sample lane line generation model to obtain a new training atlas, the method further includes: It is determined that a training set adjustment instruction is received, and based on the training set adjustment instruction, a proportion and a data amount of small sample lane line images in the new training set are adjusted.

8. A small sample lane line generation device, characterized in that: include: An acquisition module, used to acquire common lane lines and target small sample lane lines, wherein the common lane lines and the target small sample lane lines correspond to each other one by one; A training module, used for training a pre-trained diffusion model based on the common lane lines and the corresponding target small sample lane lines to obtain a small sample lane line generation model; An inference module is used to input the original training atlas into the small sample lane line generation model to obtain a new training atlas, wherein the proportion of small sample lane line images in the new training atlas is higher than the proportion of small sample lane line images in the original training atlas.

9. A storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the steps of the small sample lane line generation method as described in any one of claims 1 to 7 are implemented.

10. A computer device, characterized in that: The method comprises a processor and a memory connected to each other, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the small sample lane line generation method according to any one of claims 1 to 7 are implemented.

Images