Lane line semantic segmentation method and apparatus based on model dynamic correction

Abstract

This application relates to a lane line semantic segmentation method and apparatus based on model dynamic correction. The method includes: acquiring a sample image; inputting the sample image into a preset semantic segmentation network model to obtain a first lane line result, wherein the preset semantic segmentation network model outputs target lane lines based on the sample image; performing regression calculations on the first lane line result based on a preset regression algorithm to obtain a second lane line result; and correcting the sample image based on a preset threshold and the second lane line result. The solution provided in this application can improve the accuracy of lane line segmentation results for road images and correct errors in the sample image.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and in particular to a lane line semantic segmentation method and apparatus based on model dynamic correction. Background Technology

[0002] Lane lines are a core element of the experience layer in high-precision maps. Currently, lane lines are mostly generated semi-automatically based on traffic signs, which doesn't conform to human driving habits. Another method generates anthropomorphic lane lines based on the aggregation of obstacle-perceiving vehicle trajectories, but due to limitations in the number of obstacle vehicles deployed, some lane lines may be missing. Currently, various image segmentation models are emerging for segmenting various types of images. Among these image segmentation models is an important one: the image semantic segmentation model.

[0003] Image semantic segmentation models typically classify images at the pixel level, enabling fine-grained segmentation. Traditional image semantic segmentation methods segment images based on low-level features such as color, texture information, and spatial structure. When the frequency of different target categories varies significantly, the model may not converge simultaneously across all categories, resulting in inconsistent semantic segmentation performance and poor segmentation results.

[0004] To address the class imbalance problem in image semantic segmentation tasks, this application provides a lane line semantic segmentation and apparatus based on dynamic model correction, and the results of a real-time conditional semantic segmentation network. Summary of the Invention

[0005] To address or partially address the problems existing in related technologies, this application provides a lane line semantic segmentation method and apparatus based on model dynamic correction, which can improve the accuracy of lane line segmentation results of corrected road images.

[0006] The first aspect of this application provides a lane line semantic segmentation method based on dynamic model correction, the method comprising:

[0007] Acquire a sample image and input the sample image into a preset semantic segmentation network model to obtain the first lane line result. The preset semantic segmentation network model outputs the target lane line based on the sample image.

[0008] Based on a preset regression algorithm, regression calculation is performed on the first lane line result to obtain the second lane line result;

[0009] The sample image is corrected based on the preset threshold and the second lane line results.

[0010] Optionally, the method further includes obtaining a preset semantic segmentation network, which includes:

[0011] The sample image is input into the initial neural network to obtain the predicted image;

[0012] Calculate the loss function of the initial neural network based on the predicted image and the sample image;

[0013] The initial neural network is adjusted according to the loss function to obtain the preset semantic segmentation network.

[0014] Optionally, based on a preset regression algorithm, regression calculation is performed on the first lane line result to obtain the second lane line result, including:

[0015] The first lane line result and the sample image are input into a preset regression algorithm to obtain the first regression loss value;

[0016] The first lane line result is corrected based on the first regression loss value to obtain the second lane line result.

[0017] Optionally, the second lane line result includes the probability value of each lane line pixel being a lane line.

[0018] Optionally, the preset threshold includes a first threshold and a second threshold. Based on the preset threshold and the second lane line result, the sample image is corrected, including:

[0019] If the probability value of a pixel in the second lane line result is greater than the first threshold, then find the sample pixel corresponding to the pixel. If the sample pixel is an unlabeled lane line, then modify the sample pixel to a sample lane line.

[0020] If the probability value of a pixel in the second lane line result is less than the second threshold, then find the sample pixel corresponding to the pixel. If the sample pixel is a lane line that has been marked, modify the lane line background of the sample pixel.

[0021] Optionally, after correcting the sample image, the following may also be included:

[0022] The corrected sample image is input into a preset semantic segmentation network to obtain the corrected predicted image;

[0023] The loss function of the preset semantic segmentation network is calculated based on the corrected predicted image and the sample image;

[0024] The target semantic segmentation network is obtained by iteratively applying the loss function of the preset semantic segmentation network.

[0025] A second aspect of this application provides a lane line semantic segmentation device based on model dynamic correction, comprising:

[0026] The acquisition module is used to acquire sample images and input the sample images into a preset semantic segmentation network model to obtain the first lane line result. The preset semantic segmentation network model outputs the target lane line based on the sample images.

[0027] The first correction module is used to perform regression calculation on the first lane line result based on a preset regression algorithm to obtain the second lane line result;

[0028] The second correction module is used to correct the sample image based on a preset threshold and the second lane line result.

[0029] Optionally, the device further includes a third correction module, which is used to input the corrected sample image into a preset semantic segmentation network to obtain a corrected prediction image; calculate the loss function of the preset semantic segmentation network based on the corrected prediction image and the sample image; and iterate the loss function of the preset semantic segmentation network to obtain a target semantic segmentation network.

[0030] A third aspect of this application provides an electronic device, comprising:

[0031] Processor; and

[0032] A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0033] A fourth aspect of this application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method described above.

[0034] The technical solution provided in this application can include the following beneficial effects: This application acquires sample images, inputs the sample images into a preset semantic segmentation network model to obtain a first lane line result, the preset semantic segmentation network model outputs target lane lines based on the sample images; based on a preset regression algorithm, a regression operation is performed on the first lane line result to obtain a second lane line result; the sample image is corrected based on a preset threshold and the second lane line result. The regression operation is equivalent to a secondary correction of the preset semantic segmentation network model. The solution provided in this application can train a preliminarily qualified semantic segmentation network, thereby correcting unreasonable sample labels in the sample data, and further correcting the preset semantic segmentation network model based on the corrected sample data, thus achieving the purpose of dynamically training the semantic segmentation network model and dynamically correcting the sample images.

[0035] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0036] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0037] Figure 1 This is a schematic flowchart illustrating the lane line semantic segmentation method based on model dynamic correction in an embodiment of this application;

[0038] Figure 2 This is a schematic diagram of the structure of the lane line semantic segmentation device based on model dynamic correction shown in the embodiments of this application;

[0039] Figure 3 This is a schematic diagram of the vehicle structure shown in the embodiments of this application. Detailed Implementation

[0040] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0041] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0042] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0043] To facilitate a better understanding of the technical solutions of this application by those skilled in the art, the terms involved in this application are introduced below.

[0044] 1. Artificial Intelligence (AI): This refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0045] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0046] 2. Machine Learning (ML): This is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0047] 3. Convolutional Neural Network (CNN): A type of feedforward neural network whose artificial neurons can respond to a portion of the surrounding units within their coverage area, exhibiting excellent performance in large-scale image processing. A CNN consists of one or more convolutional layers and a fully connected layer at the top (corresponding to a classic neural network), and also includes associated weights and pooling layers.

[0048] 4. Deep features: Image features extracted through deep networks contain abstract information about the image.

[0049] 5. Semantic segmentation: Assign a corresponding category label to each pixel in the image based on the object of interest to which it belongs.

[0050] 6. Semantic Image: The result obtained by assigning a category tag to each pixel in the image.

[0051] 7. Mask image: In this embodiment of the application, the mask image is used to represent the road image. For example, the mask image can be a binary image. The binary image includes a first type of pixel with a first value and a second type of pixel with a second value. For example, if the value of a pixel in the binary image is 0, it means that the pixel is not selected. If the value of a pixel in the binary image is 1, it means that the pixel is selected.

[0052] 8. Conditional Generative Adversarial Nets (CGAN): An improvement on GAN, CGAN adds additional conditional information to the generator and discriminator of the original GAN ​​to achieve a conditional generative model. This additional conditional information can be class labels or other auxiliary information.

[0053] 9. ImageNet database: A large-scale database containing 1,000 categories.

[0054] 10. MobileNetV2: A commonly used lightweight network model architecture that is trained on the ImageNet database and can be used to extract image features.

[0055] 11. Image Classification and Categories: Image classification refers to an image processing method that distinguishes different categories of targets based on their different features reflected in image information. It utilizes computers to quantitatively analyze images, classifying each pixel or region within an image into one of several categories to replace human visual interpretation. Categories are also called classifications. In the embodiments of this application, there can be two or more categories, such as vehicles, roads, etc. When the image semantic segmentation model is applicable to different scenarios, the corresponding categories to be labeled can be different. Each target in an image is actually composed of pixels; the category of a pixel corresponds to the category of that target.

[0056] 12. Sample image and target image: Both are images. In this embodiment, the image used to train the model is called the sample image, and the image used for subsequent processing by the model is called the target image.

[0057] 13. Edge Information and Edge Pixels: Edge information describes pixels in an image whose grayscale values ​​do not change continuously within their neighborhood. These pixels are called edge pixels. Edge information can specifically include the grayscale value of each edge pixel and the shape formed by these edge pixels. Edges are widely present between objects and the background, and between objects themselves. Edge information in an image can be obtained through image edge detection.

[0058] 14. Conditional random fields (CRFs) are discriminative probabilistic models, a type of random field, commonly used for labeling or analyzing sequence data, such as natural language text or biological sequences. A CRF is a conditional probability distribution model P(Y|X), representing a Markov random field where, given a set of input random variables X, there is another set of output random variables Y. In other words, a characteristic of CRFs is the assumption that the output random variables constitute a Markov random field. CRFs can be seen as a generalization of the maximum entropy Markov model to the labeling problem. Like Markov random fields, CRFs are undirected graphical models where vertices represent random variables, and the lines connecting vertices represent dependencies between random variables. In a CRF, the distribution of random variable Y is a conditional probability, and the given observation is the random variable X. In principle, the graphical model layout of a Conditional Random Field (CRF) can be arbitrarily given. A commonly used layout is a chained architecture, which offers efficient algorithms for training, inference, and decoding. A CRF is a typical discriminative model, and its joint probability can be written as a product of several potential functions, with the linear-chain CRF being the most common.

[0059] Autonomous vehicles need to know the precise location of lane lines when driving or creating high-precision maps to determine safe driving zones or make decisions based on lane lines. Currently, various image segmentation models have emerged for segmenting various types of images. One important image segmentation model is the semantic image segmentation model. Semantic image segmentation models typically classify images at the pixel level, allowing for fine-grained segmentation. Before semantically segmenting lane lines, each pixel in the lane line portion of the image needs to be manually labeled. Labeling is difficult to achieve pixel-level precision, making it hard to guarantee that every pixel is accurately labeled. Therefore, manually labeled data often contains some incorrectly labeled pixels. To achieve pixel-level precision, on the one hand, labelers need to spend more time, increasing labeling costs; on the other hand, even with significant time investment, it's still difficult to guarantee that every pixel is correctly labeled.

[0060] To address the aforementioned issues, this application provides a lane line semantic segmentation method based on dynamic model correction, which improves the efficiency of segmenting lane lines from road images.

[0061] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0062] See Figure 1 This is a flowchart illustrating a lane line semantic segmentation method based on dynamic model correction, as shown in an embodiment of this application. It mainly includes steps S101 to S103, as explained below:

[0063] Step S101: Obtain a sample image and input the sample image into a preset semantic segmentation network model to obtain the first lane line result. The preset semantic segmentation network model outputs the target lane line based on the sample image.

[0064] Before acquiring the sample image, the method further includes preprocessing the sample image by performing grayscale processing on the lane image, detecting lane lines in the grayscale processed vehicle image, and marking lane line pixels with lane line identifiers. In this embodiment, the lane line identifier can be a numerical identifier of 0 or 1 (taking lane line as 1 and background as 0 as an example), or the lane line identifier can be the distribution probability of the lane line corresponding to the lane line pixel. In step S101, the output target lane line is the sample lane line in the sample image carrying the lane line identifier.

[0065] In one embodiment, obtaining the lane line probability of a lane line pixel includes employing a Conditional Random Field (CRF) algorithm. Specifically, this involves: setting a feature function for the CRF; calculating feature scores for road images in the image training set based on the feature function; weighted summing of the feature scores for each road image in the image training set to determine a feature function set; and obtaining the probability that a pixel in the image training set is a lane line pixel based on the feature function set, thereby obtaining the sample probability.

[0066] Specifically, each element in the sample image is compared with the label assigned in the labeled image. A feature function (which can be a probability matrix) is introduced using a Conditional Random Field (CRF) to calculate the transition probability of each pixel becoming a lane line pixel, thus obtaining the label and probability score corresponding to each pixel in the sample image. CRFs are discriminative probabilistic models, a type of random field, commonly used for labeling or analyzing sequence data, such as natural language text or biological sequences. A CRF is a conditional probability distribution model P(Y|X), representing a Markov random field where, given a set of input random variables X, there is another set of output random variables Y. In other words, a characteristic of CRFs is the assumption that the output random variables constitute a Markov random field.

[0067] In this embodiment, a feature function is introduced through a conditional random field, and the initial score is input into the feature function to calculate the probability that each pixel in the sample image becomes a lane line pixel, thereby obtaining a probability score to obtain the label probability distribution of each pixel, that is, to obtain the similarity between each element and the label.

[0068] In one embodiment, the method further includes obtaining a preset semantic segmentation network, which includes: inputting a sample image into an initial neural network to obtain a predicted image; calculating a loss function of the initial neural network based on the predicted image and the sample image; and adjusting the initial neural network based on the loss function to obtain the preset semantic segmentation network.

[0069] In this embodiment, given that the lane line identification corresponds to the sample distribution probability of the lane line, the predicted distribution probability of the identified lane line pixels in the acquired predicted image is used. For the predicted distribution probability and sample probability of the same lane line points, a loss function for the initial neural network is calculated. The initial network is then trained based on this loss function until a predicted image within the output probability error range is reached.

[0070] In one embodiment, the sample image uses lane line pixels outside the lane line edge as training samples for the initial network, thereby improving the accuracy of the initial network's training sample set. Specifically, this includes: removing pixels within the lane line edge region to obtain sample lane lines and their corresponding lane line identifiers. For example, the lane line edge region is set to 8 pixels inward from the lane edge pixels, and 8 pixels from both ends of the lane line are removed to generate sample lane lines.

[0071] Step S102: Based on the preset regression algorithm, process the first lane line result to obtain the second lane line result.

[0072] In one embodiment, a regression operation is performed on the first lane line result based on a preset regression algorithm to obtain a second lane line result, including: inputting the first lane line result and the sample image into the preset regression algorithm to obtain a first regression loss value; and correcting the first lane line result according to the first regression loss value to obtain the second lane line result.

[0073] In one embodiment, a pre-defined regression algorithm is used to calculate the classification loss of the sample image. The pre-defined regression algorithm can use the sofmax loss function to determine the classification loss value corresponding to the data sample (the classification loss value is the first regression loss value mentioned above). If the classification loss value does not reach the preset convergence, the pre-defined semantic segmentation network is retrained based on the classification loss value to obtain a new lane line segmentation result, i.e., a second lane line result. The second lane line result includes lane line pixels and their corresponding second lane line identifiers. In other words, the sample image is corrected using the second lane line result, so that the first lane line result output by the corrected sample image converges after passing through the pre-defined regression algorithm.

[0074] This application further evaluates the segmentation results of the preset semantic segmentation network model through regression calculations, thereby making the output results of the preset semantic segmentation network model more accurate. Specifically, the first lane line result of this application includes pixels and the probability value of each pixel being a lane line, and the second lane line result includes pixels and the probability value of each pixel being a lane line. The classification loss value is determined by calculating the probability values ​​in the first and second lane line results. For each training sample in each data training sample, after obtaining the true classification probability of the lane line of the training sample (the true classification probability of the lane line is the probability that the sample pixel is identified as a lane line in the sample image), the embodiments of the present invention can predict the sample class to which the training sample belongs through a loss function such as Softmax loss, and obtain the classification prediction probability of the training sample (i.e., the probability that the sample pixel is a true lane line predicted by softmax loss). Thus, based on the classification prediction probability and the true classification probability of the training sample, the classification loss value corresponding to the training sample can be determined (such as taking the difference between the classification prediction probability and the classification target probability of the training sample). Furthermore, based on the classification loss value corresponding to each training sample in the batch of data training samples, the classification loss value corresponding to each data training sample can be determined (such as taking the average of the classification loss values ​​of each training sample).

[0075] As can be seen, the classification loss value corresponding to each training data sample is the indicator of the semantic segmentation network model's inter-class classification of lane lines and non-lane lines. The classification loss value corresponding to each training data sample can represent the difference between lane lines and non-lane lines (inter-class difference). Using the classification loss value corresponding to each training data sample as part of the optimization objective of the semantic segmentation network model can enable the optimized semantic segmentation network model to have higher performance in distinguishing between lane lines and non-lane lines.

[0076] Based on this, in each iteration, the central loss value corresponding to the training data sample represents the probability difference between the lane line classification of the training sample (i.e., determining whether a pixel belongs to a lane line) and the predicted distribution probability of the sample class to which the training sample belongs. Therefore, the classification loss value corresponding to the training data sample can illustrate the difference between the sample distribution probability of the training sample and the predicted distribution probability of its class. It can represent the difference in the intra-class sample distribution probability of the training samples in each sample class. Therefore, using the classification loss value corresponding to each training data sample as part of the optimization objective of the semantic segmentation network model can make the optimized semantic segmentation network model invariant to the intra-class differences of lane lines (such as the intra-class differences between lane lines in different scenarios), thereby improving the robustness of lane line detection.

[0077] Specifically, in this embodiment, the training of the preset semantic segmentation network model can include classification training and regression training, which is a joint training process. The loss value formed by the center loss value and the classification loss value corresponding to each training data sample in each iteration can be considered the optimization objective of classification training, such as minimizing the loss value formed by the center loss value and the classification loss value corresponding to each training data sample, as the optimization objective of classification training in the semantic segmentation network model training. In each iteration, the optimization objective of regression training in the semantic segmentation network model training can be formed by the lane frame coordinate regression loss value corresponding to each training data sample. By jointly considering the center loss value, classification loss value, and lane frame coordinate regression loss value corresponding to each training data sample in each iteration, the target loss value of the semantic segmentation network model can be formed to represent the optimization objective of the semantic segmentation network model training. Optionally, in this embodiment, the target loss value of the semantic segmentation network model can be obtained by summing the product of the center loss value and the first set weight corresponding to each training data sample, the product of the lane frame coordinate regression loss value and the second set weight corresponding to each training data sample, and the classification loss value corresponding to each training data sample.

[0078] In one embodiment, the weights of the loss function of the preset semantic segmentation network model and the loss function of the regression operation in obtaining the second lane line result can be set.

[0079] In one embodiment, the first lane line result is processed based on a preset regression algorithm to obtain a second lane line result. The method further includes obtaining the first lane line result, fitting the first lane line using the preset regression algorithm, removing lane line pixels with large differences, and obtaining the second lane line result.

[0080] Specifically, this includes: obtaining the coordinates of the first lane line pixels and their individual pixel coordinates; obtaining the maximum number of rows m and the maximum number of columns n in the image; initializing the edge image matrix V(i,j) = 0; and performing the following operations on the middle pixels of V row by row and column by column using a preset regression algorithm:

[0081] Where A(i,j) is the middle pixel

[0082] Where A(i,j) is the middle pixel, Y s For A(i,j), there are 8 domains;

[0083] After obtaining the initial edge image V, the edge image is binarized. Pixels below a specified threshold are set to zero, and pixels above the specified threshold are assigned a grayscale value, resulting in a new grayscale image. The new grayscale image is then searched for the number of connected pixels. If the number of connected pixels of a pixel is lower than the threshold, the pixel is identified as noise and removed from the edge image, thus obtaining the second lane line result.

[0084] Step S103: Correct the sample image based on the preset threshold and the second lane line result.

[0085] In one embodiment, the preset threshold includes a first threshold and a second threshold. Correcting the sample image based on the preset threshold and the second lane line result includes: if the probability value of a pixel in the second lane line result is greater than the first threshold, then finding the sample pixel corresponding to the pixel; if the sample pixel is an unlabeled lane line, then modifying the sample pixel to a sample lane line; if the probability value of a pixel in the second lane line result is less than the second threshold, then finding the sample pixel corresponding to the pixel; if the sample pixel is a labeled lane line, then modifying the sample pixel to a lane line background.

[0086] In this embodiment, the first threshold is used to find the deviation of lane line pixels. If the probability value of a pixel in the second lane line result is greater than the first threshold, then the sample pixel corresponding to the pixel is found. If the sample pixel is an unlabeled lane line, the sample pixel is modified to a sample lane line. The second threshold is used to differ the deviation of background pixels. If the probability value of a pixel in the second lane line result is less than the second threshold, then the sample pixel corresponding to the pixel is found. If the sample pixel is a labeled lane line, the sample pixel is modified to a lane line background.

[0087] In one embodiment, after correcting the sample image, the method further includes: inputting the corrected sample image into a preset semantic segmentation network to obtain a corrected predicted image; calculating the loss function of the preset semantic segmentation network based on the corrected predicted image and the sample image; and iterating the loss function of the preset semantic segmentation network to obtain a target semantic segmentation network.

[0088] In this embodiment, the final target semantic segmentation network, based on the corrected sample images, yields a better neural network.

[0089] The technical solution provided in this application can include the following beneficial effects: This application acquires sample images, inputs the sample images into a preset semantic segmentation network model to obtain a first lane line result, the preset semantic segmentation network model outputs target lane lines based on the sample images; based on a preset regression algorithm, a regression operation is performed on the first lane line result to obtain a second lane line result; the sample image is corrected based on a preset threshold and the second lane line result. The regression operation is equivalent to a secondary correction of the preset semantic segmentation network model. The solution provided in this application can train a preliminarily qualified semantic segmentation network, thereby correcting unreasonable sample labels in the sample data, and further correcting the preset semantic segmentation network model based on the corrected sample data, thus achieving the purpose of dynamically training the semantic segmentation network model and dynamically correcting the sample images. Figure 2 As shown, a second aspect of this application provides a lane line semantic segmentation device based on model dynamic correction, comprising:

[0090] Acquisition module 201 is used to acquire a sample image, wherein the sample image includes sample lane lines;

[0091] A second aspect of this application provides a lane line semantic segmentation device based on model dynamic correction, comprising:

[0092] The acquisition module 201 is used to acquire sample images and input the sample images into a preset semantic segmentation network model to obtain the first lane line result. The preset semantic segmentation network model outputs the target lane line based on the sample images.

[0093] The first correction module 202 is used to perform regression calculation on the first lane line result based on a preset regression algorithm to obtain the second lane line result;

[0094] The second correction module 203 is used to correct the sample image based on a preset threshold and the second lane line result.

[0095] In one embodiment, the device further includes a third correction module, which is used to input the corrected sample image into a preset semantic segmentation network to obtain a corrected predicted image; calculate the loss function of the preset semantic segmentation network based on the corrected predicted image and the sample image; and iterate the loss function of the preset semantic segmentation network to obtain a target semantic segmentation network.

[0096] See Figure 3 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. The electronic device 300 includes a memory 310 and a processor 320.

[0097] The processor 320 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0098] Memory 310 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by processor 320 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 310 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, memory 310 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0099] The memory 310 stores executable code, which, when processed by the processor 320, can cause the processor 320 to execute part or all of the methods described above.

[0100] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0101] Alternatively, this application may also be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) thereon, which, when executed by a processor of a vehicle (or server, etc.), causes the processor to perform part or all of the steps of the above-described method according to this application.

[0102] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A lane line semantic segmentation method based on model dynamic correction, characterized in that, The method comprises: acquiring a sample image, inputting the sample image into a preset semantic segmentation network model to obtain a first lane line result, and the preset semantic segmentation network model is used for outputting a target lane line according to the sample image; based on a preset regression algorithm, the first lane line result is subjected to regression operation to obtain a second lane line result; according to a preset threshold and the second lane line result, the sample image is corrected; the method further comprises acquiring the preset semantic segmentation network model, and the acquiring the preset semantic segmentation network model comprises: inputting the sample image into an initial neural network to obtain a prediction image; according to the prediction image and the sample image, a loss function of the initial neural network is calculated; 2. The method of claim 1, wherein, according to the loss function, the initial neural network is adjusted to obtain the preset semantic segmentation network model. the second lane line result comprises a probability value of each pixel point being a lane line. the preset threshold comprises a first threshold and a second threshold, and the correcting the sample image according to the preset threshold and the second lane line result comprises: if the probability value of a pixel point in the second lane line result is greater than the first threshold, a sample pixel point corresponding to the pixel point is searched, and in a case that the sample pixel point is not labeled lane line, the sample pixel point is modified as a sample lane line; 3. The method of claim 1, wherein, if the probability value of a pixel point in the second lane line result is less than the second threshold, a sample pixel point corresponding to the pixel point is searched, and in a case that the sample pixel point is labeled lane line, the sample pixel point is modified as lane line background.

4. The method of claim 3, wherein, after the sample image is corrected, the method further comprises: inputting the corrected sample image into the preset semantic segmentation network to obtain a corrected prediction image; according to the corrected prediction image and the sample image, a loss function of the preset semantic segmentation network is calculated; 5. The method of claim 1, wherein, iterating the loss function of the preset semantic segmentation network to obtain a target semantic segmentation network. comprises: an acquiring module, configured to acquire a sample image, input the sample image into a preset semantic segmentation network model to obtain a first lane line result, and the preset semantic segmentation network model is used for outputting a target lane line according to the sample image; a first correction module, configured to perform regression operation on the first lane line result based on a preset regression algorithm to obtain a second lane line result; 6.A lane line semantic segmentation device based on model dynamic correction, characterized in that, a second correction module, configured to correct the sample image according to a preset threshold and the second lane line result; the method further comprises acquiring the preset semantic segmentation network model, and the acquiring the preset semantic segmentation network model comprises: inputting the sample image into an initial neural network to obtain a prediction image; according to the prediction image and the sample image, a loss function of the initial neural network is calculated; according to the loss function, the initial neural network is adjusted to obtain the preset semantic segmentation network model. the second lane line result comprises a probability value of each pixel point being a lane line. the preset threshold comprises a first threshold and a second threshold, and the correcting the sample image according to the preset threshold and the second lane line result comprises: if the probability value of a pixel point in the second lane line result is greater than the first threshold, a sample pixel point corresponding to the pixel point is searched, and in a case that the sample pixel point is not labeled lane line, the sample pixel point is modified as a sample lane line; if the probability value of a pixel point in the second lane line result is less than the second threshold, a sample pixel point corresponding to the pixel point is searched, and in a case that the sample pixel point is labeled lane line, the sample pixel point is modified as lane line background. after the sample image is corrected, the method further comprises: inputting the corrected sample image into the preset semantic segmentation network to obtain a corrected prediction image; according to the corrected prediction image and the sample image, a loss function of the preset semantic segmentation network is calculated; iterating the loss function of the preset semantic segmentation network to obtain a target semantic segmentation network. The first lane line result is corrected according to the first regression loss value, and a second lane line result is obtained.

7. The apparatus of claim 6, wherein, The device further comprises a third correction module configured to input the corrected sample image into the preset semantic segmentation network to obtain a corrected prediction image; and calculate a loss function of the preset semantic segmentation network according to the corrected prediction image and the sample image. The loss function of the preset semantic segmentation network is iterated to obtain a target semantic segmentation network.

8. An electronic device, comprising: comprising: a processor; and a memory having stored thereon executable code that, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, a memory having stored thereon executable code that, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 5.

Images