A training method and device for lane detection network

Abstract

The present disclosure provides a training method and device for a lane line detection network, wherein the method includes: obtaining a sample image of a target domain, and inputting the sample image into an initial lane line detection network obtained by training based on a source domain image of a source domain, and determining an initial feature map of the sample image; generating a probability feature map of the sample image based on the initial feature map; wherein the probability feature map is used to represent the probability that the pixel points in the sample image belong to a plurality of pre-set lane line detection categories; determining a pseudo label corresponding to the sample image based on the probability feature map and preset probability thresholds corresponding to the plurality of lane line detection categories, and determining a lane line detection result corresponding to the sample image based on the probability feature map; training the initial lane line detection network based on the pseudo label corresponding to the sample image and the lane line detection result, and obtaining a target lane line detection network corresponding to the target domain.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a training method and device for a lane detection network. Background Art

[0002] In the field of autonomous driving, it is usually necessary to collect sample images of lane lines and identify the lane line information of the sample images through a lane line detection network, so as to generate decision information based on the lane line information to control the vehicle.

[0003] However, since different lane lines usually have different appearances, different backgrounds, and different ratios between lane lines and backgrounds, if the lane line detection network trained based on sample images in the source domain is directly applied to the target domain, the identified lane line information will have large errors. In actual applications, it may not be possible to obtain training labels for sample data in the target domain. Therefore, how to train the lane line detection network in the target domain has become an urgent problem to be solved. Summary of the Invention

[0004] The embodiments of the present disclosure at least provide a method and apparatus for training a lane detection network.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for training a lane detection network, comprising:

[0006] Obtaining a sample image of the target domain, and inputting the sample image into an initial lane detection network trained based on source domain images of the source domain, to determine an initial feature map of the sample image;

[0007] Generating a probability feature map of the sample image based on the initial feature map; wherein the probability feature map is used to represent the probability that the pixel points in the sample image belong to a plurality of preset lane line detection categories;

[0008] Determining a pseudo label corresponding to the sample image based on the probability feature map and preset probability thresholds corresponding to the multiple lane line detection categories, and determining a lane line detection result corresponding to the sample image based on the probability feature map;

[0009] Based on the pseudo labels corresponding to the sample images and the lane line detection results, the initial lane line detection network is trained to obtain a target lane line detection network corresponding to the target domain.

[0010] In the above method, pseudo labels can be automatically generated based on sample images, and the initial lane line detection network can be trained based on the pseudo labels to solve the problem of unsupervised data of sample images in the target domain; and when determining the pseudo labels, different lane line detection categories correspond to their own probability thresholds, thereby adapting to the probability distribution differences of different lane line detection categories in the target domain to train the lane line detection network's ability to distinguish between each lane line and the background, and further improving the detection accuracy of the trained lane line detection network.

[0011] In a possible implementation, determining the initial feature map of the sample image includes:

[0012] Encoding the sample image to obtain an initial feature map of the sample image;

[0013] Generating the probability feature map of the sample image based on the initial feature map includes:

[0014] Performing deep feature extraction on the initial feature map to obtain a deep feature map of the sample image;

[0015] A probabilistic feature map of the sample image is determined based on the deep feature map.

[0016] In this way, since the deep feature map contains more feature information of the sample image, determining the probability feature map based on the deep feature map can make the generated probability feature map more accurate.

[0017] In one possible implementation, the training of the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results includes:

[0018] The initial lane detection network is trained based on the pseudo labels corresponding to the sample images, the lane detection results, the probability feature map, and the deep feature map.

[0019] Using this method, the initial lane line detection network can be trained at the instance level (i.e., lane line level) and pixel level, thereby improving the accuracy of the initial lane line detection network.

[0020] In one possible implementation, the training of the initial lane detection network based on the pseudo labels corresponding to the sample images, the lane detection results, the probabilistic feature map, and the deep feature map includes:

[0021] Calculating a pixel prediction loss value corresponding to the initial lane detection network based on the pseudo label corresponding to the sample image and the lane detection result; and

[0022] Calculating an instance prediction loss value corresponding to the initial lane detection network based on the probability feature map, the deep feature map, and the pseudo labels corresponding to the sample images; and

[0023] Adjusting network parameters of the initial lane detection network based on the pixel prediction loss value and the instance prediction loss value.

[0024] In one possible implementation, calculating the instance prediction loss value corresponding to the initial lane detection network based on the probability feature map, the deep feature map, and the pseudo label corresponding to the sample image includes:

[0025] Determining position information of a plurality of real lane lines in the sample image based on the pseudo labels corresponding to the sample image;

[0026] sequentially using the real lane lines as reference lane lines, and determining target lane lines other than the reference lane lines from the plurality of real lane lines; and, based on the pseudo labels corresponding to the sample images, determining a target region of interest that does not include a lane line, and determining a false lane line corresponding to the reference lane line from the target region of interest;

[0027] Based on the position information of the target lane line, the position information of the reference lane line and the probability feature map, boundary pooling processing is performed on the target lane line and the reference lane line; the boundary pooling processing is used to improve the integrity of the lane line;

[0028] The lane line features corresponding to the reference lane line after boundary pooling processing, the target lane line after boundary pooling processing, and the false lane line are determined from the deep feature map, and the instance prediction loss value is determined based on the lane line features.

[0029] By adopting this method, the initial lane line detection network can automatically determine the target lane line and the false lane line. By adjusting the embedded feature vectors of the target lane line and the baseline lane line to be closer, and the distance between the embedded feature vectors of the false lane line and the baseline lane line to be farther, the recognition ability of the initial lane line detection network can be made more accurate.

[0030] In one possible implementation, determining the target region of interest that does not include a lane line based on the pseudo label corresponding to the sample image includes:

[0031] Based on the pseudo-label, determining a region of interest where lane lines exist in the sample image;

[0032] Eliminate lane line points corresponding to the pseudo labels in the region of interest to determine a target region of interest;

[0033] The determining of a false lane line corresponding to the reference lane line from the target region of interest includes:

[0034] Line detection is performed in the target region of interest to determine the false lane line.

[0035] By adopting this method, false lane lines can be determined from the area of interest where real lane lines may exist, so that the characteristics of the false lane lines are closer to those of the real lane lines, thereby training the initial lane line detection network's ability to distinguish false lane lines.

[0036] In one possible implementation, the performing boundary pooling processing on the target lane line and the reference lane line based on the position information of the target lane line, the position information of the reference lane line, and the probability feature map includes:

[0037] For any target lane line point on the lane line to be processed, based on the position information of the target lane line point, determine the pixel points to be optimized adjacent to the target lane line point; the lane line to be processed is the target lane line or the reference lane line;

[0038] Based on the target probability corresponding to the pixel point to be optimized in the probability feature map and a preset optimization probability threshold, boundary pooling processing is performed on the lane line to be processed, wherein the target probability is the probability that the pixel point to be optimized is a point on the lane line to be processed.

[0039] In one possible implementation, the boundary pooling process for the lane line to be processed based on the target probability corresponding to the pixel to be optimized in the probability feature map and a preset optimization probability threshold includes:

[0040] The pixel points to be optimized whose corresponding target probabilities exceed the preset optimization probability threshold are used as target lane line points on the lane line to be processed after boundary pooling processing.

[0041] Since the lane lines detected by the initial lane line detection network are at the pixel level and the detection accuracy of the initial lane line detection network is limited, the determined lane lines to be processed usually have breakpoints. Using this method, the lanes to be processed can be made more complete, thereby improving the training effect of the subsequent training of the initial lane line detection network.

[0042] In one possible implementation, the training of the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results includes:

[0043] Calculating a pixel prediction loss value corresponding to the initial lane detection network based on the pseudo label corresponding to the sample image and the lane detection result; and

[0044] Calculating a category prediction loss value corresponding to the initial lane detection network based on the initial feature map and preset category thresholds corresponding to the multiple lane detection categories;

[0045] Based on the pixel prediction loss value and the category prediction loss value, network parameters of the initial lane detection network are adjusted.

[0046] By adopting this method, the initial lane line detection network can be trained based on the level of the lane line detection category contained in the sample image, thereby improving the accuracy of the initial lane line detection network.

[0047] In one possible implementation, calculating the category prediction loss value corresponding to the initial lane detection network based on the initial feature map and preset category thresholds corresponding to the multiple lane detection categories includes:

[0048] Determining a class prediction probability of the sample image based on the initial feature map; wherein the class prediction probability is used to represent the probability of each lane line detection class being included in the sample image;

[0049] Determining a category pseudo label for the sample image based on the category prediction probability and the preset category threshold; and determining a category prediction result corresponding to the sample image based on the category prediction probability;

[0050] The category prediction loss value is determined based on the category prediction result and the category pseudo label.

[0051] By adopting this method, the initial lane line detection network can automatically generate category pseudo labels for training. The initial lane line detection network is trained using the category pseudo labels, thereby improving the training effect of the initial lane line detection network for sample images of the target domain.

[0052] In one possible implementation, the training of the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results includes:

[0053] Obtain source domain images carrying labeled information;

[0054] Inputting the source domain image into the initial lane line detection network to obtain an original prediction result;

[0055] The initial lane line detection network is trained based on the marking information, the original prediction result, the pseudo label corresponding to the sample image and the lane line detection result.

[0056] In a second aspect, an embodiment of the present disclosure further provides a training device for a lane detection network, comprising:

[0057] an acquisition module, configured to acquire a sample image of a target domain, and input the sample image into an initial lane detection network trained based on a source domain image of a source domain, to determine an initial feature map of the sample image;

[0058] A generating module, configured to generate a probability feature map of the sample image based on the initial feature map; wherein the probability feature map is used to represent the probability that the pixel points in the sample image belong to a plurality of preset lane line detection categories;

[0059] a determination module, configured to determine a pseudo label corresponding to the sample image based on the probability feature map and preset probability thresholds corresponding to the plurality of lane line detection categories, and to determine a lane line detection result corresponding to the sample image based on the probability feature map;

[0060] A training module is used to train the initial lane line detection network based on the pseudo labels corresponding to the sample images and the lane line detection results to obtain a target lane line detection network corresponding to the target domain.

[0061] In a possible implementation, the acquisition module, when determining the initial feature map of the sample image, is configured to:

[0062] Encoding the sample image to obtain an initial feature map of the sample image;

[0063] The generating module is configured to:

[0064] Performing deep feature extraction on the initial feature map to obtain a deep feature map of the sample image;

[0065] A probabilistic feature map of the sample image is determined based on the deep feature map.

[0066] In one possible implementation, the training module, when training the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results, is configured to:

[0067] The initial lane detection network is trained based on the pseudo labels corresponding to the sample images, the lane detection results, the probability feature map, and the deep feature map.

[0068] In one possible implementation, the training module, when training the initial lane detection network based on the pseudo labels corresponding to the sample images, the lane detection results, the probabilistic feature map, and the deep feature map, is configured to:

[0069] Calculating a pixel prediction loss value corresponding to the initial lane detection network based on the pseudo label corresponding to the sample image and the lane detection result; and

[0070] Calculating an instance prediction loss value corresponding to the initial lane detection network based on the probability feature map, the deep feature map, and the pseudo labels corresponding to the sample images; and

[0071] Adjusting network parameters of the initial lane detection network based on the pixel prediction loss value and the instance prediction loss value.

[0072] In one possible implementation, the training module, when calculating the instance prediction loss value corresponding to the initial lane detection network based on the probability feature map, the deep feature map, and the pseudo labels corresponding to the sample images, is configured to:

[0073] Determining position information of a plurality of real lane lines in the sample image based on the pseudo labels corresponding to the sample image;

[0074] sequentially using the real lane lines as reference lane lines, and determining target lane lines other than the reference lane lines from the plurality of real lane lines; and, based on the pseudo labels corresponding to the sample images, determining a target region of interest that does not include a lane line, and determining a false lane line corresponding to the reference lane line from the target region of interest;

[0075] Based on the position information of the target lane line, the position information of the reference lane line and the probability feature map, boundary pooling processing is performed on the target lane line and the reference lane line; the boundary pooling processing is used to improve the integrity of the lane line;

[0076] The lane line features corresponding to the reference lane line after boundary pooling processing, the target lane line after boundary pooling processing, and the false lane line are determined from the deep feature map, and the instance prediction loss value is determined based on the lane line features.

[0077] In one possible implementation, the training module, when determining the target region of interest that does not include a lane line based on the pseudo-label corresponding to the sample image, is configured to:

[0078] Based on the pseudo-label, determining a region of interest where lane lines exist in the sample image;

[0079] Eliminate lane line points corresponding to the pseudo labels in the region of interest to determine a target region of interest;

[0080] The training module is configured to, when determining a false lane line corresponding to the reference lane line from the target region of interest,

[0081] Line detection is performed in the target region of interest to determine the false lane line.

[0082] In one possible implementation, the training module, when performing boundary pooling processing on the target lane line and the reference lane line based on the position information of the target lane line, the position information of the reference lane line, and the probability feature map, is configured to:

[0083] For any target lane line point on the lane line to be processed, based on the position information of the target lane line point, determine the pixel points to be optimized adjacent to the target lane line point; the lane line to be processed is the target lane line or the reference lane line;

[0084] Based on the target probability corresponding to the pixel point to be optimized in the probability feature map and a preset optimization probability threshold, boundary pooling processing is performed on the lane line to be processed, wherein the target probability is the probability that the pixel point to be optimized is a point on the lane line to be processed.

[0085] In one possible implementation, the training module, when performing boundary pooling processing on the lane line to be processed based on the target probability corresponding to the pixel to be optimized in the probability feature map and a preset optimization probability threshold, is configured to:

[0086] The pixel points to be optimized whose corresponding target probabilities exceed the preset optimization probability threshold are used as target lane line points on the lane line to be processed after boundary pooling processing.

[0087] In one possible implementation, the training module, when training the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results, is configured to:

[0088] Calculating a pixel prediction loss value corresponding to the initial lane detection network based on the pseudo label corresponding to the sample image and the lane detection result; and

[0089] Calculating a category prediction loss value corresponding to the initial lane detection network based on the initial feature map and preset category thresholds corresponding to the multiple lane detection categories;

[0090] Based on the pixel prediction loss value and the category prediction loss value, network parameters of the initial lane detection network are adjusted.

[0091] In one possible implementation, the training module, when calculating the category prediction loss value corresponding to the initial lane detection network based on the initial feature map and the preset category thresholds corresponding to the multiple lane detection categories, is configured to:

[0092] Determining a class prediction probability of the sample image based on the initial feature map; wherein the class prediction probability is used to represent the probability of each lane line detection class being included in the sample image;

[0093] Determining a category pseudo label for the sample image based on the category prediction probability and the preset category threshold; and determining a category prediction result corresponding to the sample image based on the category prediction probability;

[0094] The category prediction loss value is determined based on the category prediction result and the category pseudo label.

[0095] In one possible implementation, the training module, when training the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results, is configured to:

[0096] Obtain source domain images carrying labeled information;

[0097] Inputting the source domain image into the initial lane line detection network to obtain an original prediction result;

[0098] The initial lane line detection network is trained based on the marking information, the original prediction result, the pseudo label corresponding to the sample image and the lane line detection result.

[0099] In a third aspect, an embodiment of the present disclosure further provides a computer device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are performed.

[0100] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0101] For a description of the effects of the training device, computer equipment, and computer-readable storage medium for the lane line detection network, please refer to the description of the training method for the lane line detection network, which will not be repeated here.

[0102] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present disclosure.

[0103] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] The accompanying drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0105] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0106] Figure 1 A flowchart of a lane detection network training method provided by an embodiment of the present disclosure is shown;

[0107] Figure 2 A schematic diagram of a lane detection category provided by an embodiment of the present disclosure is shown;

[0108] Figure 3 A schematic diagram showing a probability image corresponding to some pixel points provided by an embodiment of the present disclosure is shown;

[0109] Figure 4 A schematic diagram showing a process for determining a category prediction probability provided by an embodiment of the present disclosure is shown;

[0110] Figure 5 A schematic diagram illustrating a training process of a lane detection network provided by an embodiment of the present disclosure is shown;

[0111] Figure 6 A schematic diagram of the architecture of a lane detection network training device provided by an embodiment of the present disclosure is shown;

[0112] Figure 7A schematic structural diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0113] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0114] In related technologies, when training a lane detection network, a uniform probability threshold is set for each lane line and background. After generating predictions based on a sample image in the input target domain, pseudo labels are generated based on this uniform probability threshold, and the lane detection network is trained based on these pseudo labels. However, due to the significant differences in the probability distributions of background and lane lines in the target domain, setting a uniform probability threshold for both background and lane line types can reduce the lane detection network's ability to distinguish between background and lane lines, as well as its ability to distinguish between different lane lines.

[0115] In practical applications, the detection probability of the background in the target domain is often high, while the detection probability of the lane line is often low. If a unified probability threshold is used, most of the lane lines may be mistakenly detected as background, thereby affecting the network accuracy of the neural network.

[0116] Based on the above research, the present disclosure provides a training method and device for a lane line detection network, which can automatically generate pseudo labels based on sample images, and train the initial lane line detection network based on the pseudo labels to solve the problem of unsupervised data of sample images in the target domain; and when determining the pseudo labels, different lane line detection categories correspond to their own probability thresholds, thereby adapting to the probability distribution differences of different lane line detection categories in the target domain, so as to train the lane line detection network to distinguish between each lane line and the background, and further improve the detection accuracy of the trained lane line detection network.

[0117] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0118] The term "and / or" herein simply describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. In addition, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0119] To facilitate understanding of this embodiment, we first provide a detailed introduction to a lane detection network training method disclosed in an embodiment of the present disclosure. This lane detection network training method is generally performed by a computer device with sufficient computing power, such as a client, server, or other processing device. The client may include a personal computer, tablet computer, or the like. In some possible implementations, this lane detection network training method may be implemented by a processor invoking computer-readable instructions stored in a memory.

[0120] See also Figure 1 FIG. 1 is a flowchart of a method for training a lane detection network according to an embodiment of the present disclosure. The method includes steps 101 to 104, wherein:

[0121] Step 101: Obtain a sample image of the target domain, and input the sample image into an initial lane detection network trained based on source domain images of the source domain to determine an initial feature map of the sample image;

[0122] Step 102: generating a probability feature map of the sample image based on the initial feature map; wherein the probability feature map is used to represent the probability that the pixels in the sample image belong to a plurality of preset lane line detection categories;

[0123] Step 103: determining a pseudo label corresponding to the sample image based on the probability feature map and preset probability thresholds corresponding to the multiple lane detection categories, and determining a lane detection result corresponding to the sample image based on the probability feature map;

[0124] Step 104: Based on the pseudo labels corresponding to the sample images and the lane detection results, the initial lane detection network is trained to obtain a target lane detection network corresponding to the target domain.

[0125] The following are detailed instructions for the above steps:

[0126] For step 101,

[0127] The source domain and the target domain represent different scenes, such as the same or different lane lines at different times, in different backgrounds, or with different lane-to-background ratios. The target domain sample image and the source domain image are images containing a front view of lane lines. For example, the source domain image is a front view image containing lane line A, and the sample image is a front view image containing lane line B.

[0128] The source domain image can be collected by an image acquisition device installed on the vehicle during the driving process of the vehicle, and the sample image of the target domain can be collected by an image acquisition device on the autonomous driving device during the driving process of the autonomous driving device. The installation position of the image acquisition device installed on the vehicle can be the same as the installation position of the image acquisition device installed on the autonomous driving device. This ensures that the angles of the collected sample images and the source domain images are the same, thereby improving the recognition ability of the initial lane line detection network for the sample images of the target domain.

[0129] In one possible implementation, the source domain image may have corresponding labeling information, which is supervisory data and is used to indicate the lane detection category corresponding to each pixel in the source domain image. Exemplarily, the initial lane detection network is trained based on the source domain image according to the following method:

[0130] Step 1: First, a source domain image with labeled information can be obtained.

[0131] The marking information is used to identify the lane line detection category to which the pixel point in the source domain image belongs. The lane line detection category may exemplarily include background, middle lane line, left lane line, right lane line, etc.

[0132] Step 2: Input the source domain image into the lane detection network to be trained to obtain the original prediction result.

[0133] The original prediction result is the predicted lane line detection category and probability to which each pixel in the source domain image belongs. Specifically, the initial lane line detection network can estimate the probability that each pixel in the source domain image belongs to each of the multiple lane line detection categories. For any pixel, the probabilities of the pixel belonging to each lane line detection category are compared, and the target category with the highest probability is determined. The target category and the probability corresponding to the target category are used as the prediction result for the pixel. For example, the probability that pixel 01 is the leftmost lane line is 95%. Similarly, the prediction result for each pixel in the source domain image is obtained, thereby obtaining the original prediction result of the source domain image.

[0134] Step 3: Train the lane line detection network to be trained based on the marking information and the original prediction result to obtain the initial lane line detection network.

[0135] Specifically, a source domain prediction loss value can be calculated based on the label information and the original prediction result, and then the parameters of the lane line detection network to be trained can be adjusted based on the source domain prediction loss value. The source domain prediction loss value can be exemplarily a cross entropy loss.

[0136] Here, since the initial lane line detection network is trained based on source domain images, the detection accuracy of the initial lane line detection network for source domain images is high, but the detection accuracy for target domain images may be low due to regional differences. Therefore, the initial lane line detection network needs to be further trained.

[0137] In one possible implementation, when determining the initial feature map of the sample image, the sample image may be first encoded to obtain the initial feature map of the sample image. Specifically, when encoding the sample image, the image encoder in the initial lane detection network may be used for encoding.

[0138] For step 102,

[0139] For example, Figure 2 As shown, the pre-set multiple lane line detection categories may include the leftmost lane line, the leftmost lane line, the rightmost lane line, the rightmost lane line, and the background, etc. The probability feature map may be N-channel, where N is the number of lane line detection categories, and the values on each channel may respectively represent the probability of belonging to the corresponding lane line detection category. Exemplarily, for any feature point in the probability feature map, the value of the feature point on each channel represents the probability that the pixel point corresponding to the feature point belongs to each lane line detection category. For example, the values may be 80%, 10%, 5%, 3%, and 2%, indicating that the probability that the pixel point belongs to the leftmost lane line is 80%, the probability that it belongs to the leftmost lane line is 10%, the probability that it belongs to the rightmost lane line is 5%, the probability that it belongs to the rightmost lane line is 3%, and the probability that it belongs to the background is 2%.

[0140] In a possible embodiment, in order to improve the accuracy of the probability feature map, when generating the probability feature map of the sample image based on the initial feature map, deep feature extraction can be performed on the initial feature map to obtain the deep feature map of the sample image, and then the probability feature map of the sample image is determined based on the deep feature map.

[0141] Specifically, when determining the probabilistic feature map of the sample image based on the deep feature map, the deep feature map may be input into a normalized exponential function softmax function to generate the probabilistic feature map. In this way, since the deep feature map contains more feature information of the sample image, determining the probabilistic feature map based on the deep feature map can make the generated probabilistic feature map more accurate.

[0142] For step 103,

[0143] In one possible implementation, when determining the pseudo-label corresponding to the sample image, preset probability thresholds corresponding to the multiple lane line detection categories can be pre-set. Then, for any pixel point, the probability that the pixel point in the probability feature map belongs to each lane line detection category is compared with the preset probability thresholds corresponding to the multiple lane line detection categories. If any probability is higher than the corresponding probability threshold, the lane line detection category corresponding to the probability is used as the pseudo-label of the pixel point.

[0144] Exemplarily, the probability thresholds corresponding to the five lane line detection categories are pre-set as follows: 80% for the leftmost lane line, 75% for the leftmost lane line, 90% for the rightmost lane line, 95% for the rightmost lane line, and 88% for the background. From the probability feature map, it can be seen that the probability of any pixel point belonging to the leftmost lane line is 2%, the probability of belonging to the leftmost lane line is 3%, the probability of belonging to the rightmost lane line is 92%, the probability of belonging to the rightmost lane line is 1%, and the probability of belonging to the background is 2%. Since the probability of the pixel point belonging to the rightmost lane line is higher than the probability threshold corresponding to the rightmost lane line (92%>90%), the pseudo label of the pixel point is determined to be the rightmost lane line.

[0145] For example, the following formula may be used to compare the probability that the pixel point belongs to each lane line detection category and the preset probability thresholds corresponding to the multiple lane line detection categories:

[0146]

[0147] Among them, y t represents the probability that the pixel point in the sample image belongs to the multiple lane line detection categories and is greater than the probability threshold, i represents the horizontal coordinate of the pixel point in the sample image, j represents the vertical coordinate of the pixel point in the sample image, represents the probability in the probability feature map, c represents the number of channels in the probability feature map, α c Represents the probability threshold, and null means no pseudo label is generated.

[0148] In another possible implementation, when determining the pseudo-label corresponding to the sample image, the probabilities that the pixel point belongs to each lane line detection category can be compared first, and the lane line detection category corresponding to the maximum probability can be determined. Then, the maximum probability can be compared with the probability threshold of the lane line detection category corresponding to the maximum probability. If it is greater than, the lane line detection category corresponding to the maximum probability can be used as the pseudo-label for the pixel point.

[0149] Continuing with the previous example, we can first compare the probabilities of the pixel belonging to each lane line detection category and determine that the highest probability is the probability that the pixel belongs to the rightmost lane line. Then, we compare the probability of the pixel belonging to the right lane line (92%) with the probability threshold corresponding to the rightmost lane line (90%). Because the probability of the pixel belonging to the rightmost lane line is greater than the probability threshold corresponding to the rightmost lane line, the pseudo label of the pixel is determined to be the rightmost lane line.

[0150] In the above two methods for determining pseudo labels, the preset probability thresholds corresponding to the multiple lane line detection categories can be the same or different. Since the initial lane line detection network has different recognition capabilities for different lane line detection categories, if multiple different probability thresholds are set for the multiple lane line detection categories (such as setting different probability thresholds for the four lane lines and the background), the multiple lane line detection categories in the sample image can be better distinguished, thereby improving the accuracy of the generated pseudo labels.

[0151] Here, since the purpose of generating pseudo labels is to train the initial lane line detection network, the probability threshold is usually set high to ensure the accuracy of the generated pseudo labels. Therefore, only some pixels in the sample image can generate pseudo labels. However, after the initial lane line detection network is trained with sample images multiple times, the recognition ability of the initial lane line detection network will gradually improve, thereby generating more and more pseudo labels.

[0152] When determining the lane line detection result corresponding to the sample image based on the probability feature map, the probability that each pixel point in the sample image belongs to the multiple lane line detection categories can be first determined based on the probability feature map, and then for any pixel point, the probabilities of the pixel point belonging to the multiple lane line detection categories are compared, and the maximum probability and the lane line detection category corresponding to the maximum probability are taken as the lane line detection result of the pixel point, and so on, to obtain the lane line detection result corresponding to the sample image.

[0153] For example, if the probability of any pixel point belonging to the leftmost lane line is 2%, the probability of belonging to the left lane line is 3%, the probability of belonging to the rightmost lane line is 92%, the probability of belonging to the right lane line is 1%, and the probability of belonging to the background is 2%, then the lane line detection result of the pixel point is "92%, rightmost lane line".

[0154] For step 104,

[0155] In one possible implementation, when training the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results, a pixel prediction loss value corresponding to the initial lane detection network may be calculated based on the pseudo labels corresponding to the sample images and the lane detection results, and the initial lane detection network may be trained based on the pixel prediction loss value. The pixel prediction loss value may, for example, be a cross entropy loss value between the pseudo labels and the lane detection results.

[0156] In another possible implementation, since the initial lane line detection network is trained using pseudo labels for sample images of the target domain, and pseudo labels are not real labels and may have deviations, in order to avoid training the initial lane line detection network only using sample images of the target domain, which results in a large deviation of the initial lane line detection network, when training the initial lane line detection network based on the pseudo labels corresponding to the sample images and the lane line detection results, the initial lane line detection network can also be trained based on the marking information, the original prediction results, the pseudo labels corresponding to the sample images, and the lane line detection results.

[0157] Specifically, the source domain prediction loss value can be calculated based on the label information and the original prediction result, and the pixel prediction loss value can be calculated based on the pseudo label corresponding to the sample image and the lane line detection result. Then, the pixel prediction loss value and the source domain prediction loss value are weighted and summed to obtain a first target loss value, and the initial lane line detection network is trained based on the first target loss. The specific calculation formula of the first target loss value is as follows:

[0158]

[0159] Among them, L st represents the first target loss, represents the source domain prediction loss value, represents the pixel prediction loss value, s represents the encoding of the source domain image, S represents the set of source domain image encodings, Represents the original prediction result, y srepresents the label information, t represents the encoding of the target domain image, T represents the set of target domain images, λ st Represents the weight of the pixel prediction loss value, y t represents the pseudo label, Indicates the lane line detection result.

[0160] In a possible implementation, when the initial lane line detection network is trained based on the pseudo labels corresponding to the sample images and the lane line detection results, the initial lane line detection network can be trained based on the pseudo labels corresponding to the sample images, the lane line detection results, the probability feature map, and the deep feature map.

[0161] Specifically, based on the pseudo label corresponding to the sample image and the lane line detection result, the pixel prediction loss value corresponding to the initial lane line detection network can be calculated, and based on the probability feature map, the deep feature map and the pseudo label corresponding to the sample image, the instance prediction loss value corresponding to the initial lane line detection network can be calculated, and then based on the pixel prediction loss value and the instance prediction loss value, the network parameters of the initial lane line detection network can be adjusted.

[0162] Using this method, the initial lane line detection network can be trained at the instance level (i.e., lane line level) and pixel level, thereby improving the accuracy of the initial lane line detection network.

[0163] In a possible implementation, when calculating the instance prediction loss value, it can be specifically divided into steps A to D (wherein, the order of executing step B1 and step B2 is not particular):

[0164] Step A: determining position information of multiple real lane lines in the sample image based on the pseudo labels corresponding to the sample image;

[0165] Specifically, since the pseudo-label is used to represent the lane line detection category to which some pixels in the sample image belong, the set of pixel points contained in multiple real lane lines in the sample image can be determined based on the pseudo-label (such as set A represents the set of pixel points of the leftmost lane line, set B represents the set of pixel points of the left lane line, set C represents the set of pixel points of the right lane line, and set D represents the set of pixel points of the rightmost lane line). Then, the position information of the pixel points in multiple sets in the sample image can be determined, that is, the position information of the multiple real lane lines is determined.

[0166] Step B1: sequentially using the real lane lines as reference lane lines, and determining target lane lines other than the reference lane lines from the multiple real lane lines;

[0167] Exemplarily, the leftmost lane line, the rightmost lane line, the leftward lane line, and the rightward lane line can be used as the reference lane lines in sequence. When any one of the lane lines is used as the reference lane line, the remaining three lane lines are the target lane lines. For example, when the leftmost lane line is the reference lane line, the rightmost lane line, the leftward lane line, and the rightward lane line are the target lane lines.

[0168] Step B2: determining a target region of interest that does not include a lane line based on the pseudo label corresponding to the sample image, and determining a false lane line corresponding to the reference lane line from the target region of interest;

[0169] In one possible implementation, since the false lane line needs to be used as a negative example to train the initial lane line detection network, if the false lane line is closer to the real lane line, the training effect of the initial lane line detection network can be improved. Therefore, when determining the false lane line corresponding to the baseline lane line based on the pseudo label corresponding to the sample image, the region of interest (ROI) where the lane line exists in the sample image can be first determined based on the pseudo label, and then the lane line points corresponding to the pseudo label in the region of interest are removed to determine the target region of interest, and finally, straight line detection is performed in the target region of interest to determine the false lane line.

[0170] Specifically, the pseudo-label includes lane line pseudo-labels (such as the leftmost lane line, the rightmost lane line, the leftmost lane line, and the rightmost lane line) and background pseudo-labels. The region of interest can be the region where the pixels corresponding to the lane line pseudo-labels are located, or the height range of the region of interest can be determined according to the height of the pixels corresponding to the lane line pseudo-labels. The area within the height range is the region of interest, and then the pixels corresponding to the lane line pseudo-labels are removed from the region of interest to determine the target region of interest. Then, a line detection technology is used to determine a preset number of straight lines from the target region of interest, and the multiple straight lines are used as the false lane lines.

[0171] By adopting this method, false lane lines can be determined from the area of interest where real lane lines may exist, so that the characteristics of the false lane lines are closer to those of the real lane lines, thereby training the initial lane line detection network's ability to distinguish false lane lines.

[0172] Step C: performing boundary pooling processing on the target lane line and the baseline lane line based on the position information of the target lane line, the position information of the baseline lane line, and the probability feature map; the boundary pooling processing is used to improve the integrity of the lane line;

[0173] Specifically, when performing boundary pooling processing, you can first determine the pixel points to be optimized adjacent to any target lane line point on the lane line to be processed based on the position information of the target lane line point. The lane line to be processed is the target lane line or the reference lane line, and the adjacent pixel points to be optimized can be pixel points adjacent to the target lane point in at least any direction (such as up, down, left, and right).

[0174] Then, based on the target probability corresponding to the pixel to be optimized in the probability feature map and a preset optimization probability threshold, boundary pooling is performed on the lane line to be processed, where the target probability is the probability that the pixel to be optimized is a point on the lane line to be processed. Here, the preset optimization probability threshold is generally lower than the probability threshold.

[0175] In one possible implementation, when performing boundary pooling on the lane line to be processed based on the target probabilities corresponding to the pixels to be optimized in the probability feature map and a preset optimization probability threshold, the pixels to be optimized whose corresponding target probabilities exceed the preset optimization probability threshold may be used as target lane line points on the lane line to be processed after boundary pooling. The preset optimization probability thresholds for the multiple lane line detection categories may be the same or different.

[0176] For example, Figure 3 The probability image corresponding to some pixel points shown in the figure takes the pixel point to be optimized as the point adjacent to the left of the target lane line point as an example. If the probability threshold is 0.7, then Figure 3 The dark pixel in is the target lane line point. If the preset optimization probability is 0.19, the pixel to be optimized whose target probability is greater than 0.19 (such as 0.2, 0.3) is also used as the target lane line point. Figure 3 The medium and light-colored pixels are the target lane line points added after boundary pooling.

[0177] Since the lane lines detected by the initial lane line detection network are at the pixel level and the detection accuracy of the initial lane line detection network is limited, the determined lane lines to be processed usually have breakpoints. Using this method, the lanes to be processed can be made more complete, thereby improving the training effect of the subsequent training of the initial lane line detection network.

[0178] Step D: Determine the lane line features corresponding to the baseline lane line after boundary pooling processing, the target lane line after boundary pooling processing, and the false lane line from the deep feature map, and determine the instance prediction loss value based on the lane line features.

[0179] After determining the false lane line and the target lane line, the target lane line is a positive sample and the false lane line is a negative sample. The features extracted by the initial lane line detection network should make the features of the baseline lane line close to those of the target lane line and far from the features of the false lane line.

[0180] When determining the instance prediction loss value, the lane features at the corresponding positions can be obtained from the deep feature map based on the position information of the target lane points that constitute the baseline lane and the target lane. The instance prediction loss value is then determined based on the lane features. Because the lane features corresponding to each target lane point have different dimensions, it is necessary to convert the lane features into embedded feature vectors using an embedding module in the initial lane detection network, and then perform calculations based on the embedded feature vectors.

[0181] For example, the instance prediction loss value may be calculated based on the following formula:

[0182]

[0183] in, Represents the instance prediction loss value, λ tl represents the weight of the instance prediction loss value, M represents the number of the baseline lane lines, N represents the number of the false lane lines, i represents the number of the baseline lane line, j represents the number of the target lane line, and k represents the number of the false lane line. represents the embedded feature vector of the reference lane line numbered i, f(x j p ) represents the embedded feature vector of the target lane line numbered j, f(x k n ) represents the embedded feature vector of the false lane line numbered k, and β represents the hyperparameter.

[0184] In this way, after the instance prediction loss value is determined according to the above method, the parameters of the initial lane line detection network can be adjusted based on the instance prediction loss value.

[0185] By adopting this method, the initial lane line detection network can automatically determine the target lane line and the false lane line. By adjusting the embedded feature vectors of the target lane line and the baseline lane line to be closer, and the distance between the embedded feature vectors of the false lane line and the baseline lane line to be farther, the recognition ability of the initial lane line detection network can be made more accurate.

[0186] In one possible embodiment, when the initial lane line detection network is trained based on the pseudo labels corresponding to the sample images and the lane line detection results, the pixel prediction loss value corresponding to the initial lane line detection network can be calculated based on the pseudo labels corresponding to the sample images and the lane line detection results; and the category prediction loss value corresponding to the initial lane line detection network can be calculated based on the initial feature map and the preset category thresholds corresponding to the multiple lane line detection categories, and then the network parameters of the initial lane line detection network are adjusted based on the pixel prediction loss value and the category prediction loss value.

[0187] By adopting this method, the initial lane line detection network can be trained based on the level of the lane line detection category contained in the sample image, thereby improving the accuracy of the initial lane line detection network.

[0188] Specifically, when calculating the category prediction loss value corresponding to the initial lane line detection network, the category prediction probability of the sample image can be determined based on the initial feature map; wherein the category prediction probability is used to represent the probability that each lane line detection category is contained in the sample image; then, based on the category prediction probability and the preset category threshold, the category pseudo-label of the sample image is determined; and, based on the category prediction probability, the category prediction result corresponding to the sample image is determined; finally, the category prediction loss value is determined based on the category prediction result and the category pseudo-label.

[0189] Specifically, when determining the category prediction probability, if Figure 4 As shown, the initial feature map (such as H×W×C1, H represents height, W represents width, and C1 represents the number of channels) can be first processed by weight sharing (shareweights) to obtain a shared feature map (such as H×W×C2, C2 represents the number of channels).

[0190] The shared feature map is processed in both spatial and channel dimensions. For spatial dimension processing, the shared feature map can be fed into a convolution kernel to establish a connection between features and lane detection categories. The convolution kernel can be, for example, a symmetrical and separable large filter. A softmax function is then used to obtain spatial weights (e.g., H×W×1) for the spatial dimension.

[0191] For the channel dimension processing, the shared feature map can be subjected to maximum pooling and average pooling, and the two pooling results obtained are spliced to obtain a splicing result to achieve global information embedding, and then the splicing result is processed by a multilayer perceptron (MLP) to obtain a processing result, and then the processing result is input into the activation function sigmoid function to calculate the channel weight of the channel dimension (such as 1×1×C2).

[0192] Finally, the Hadamard product of the spatial weight and the shared feature map, as well as the Hadamard product of the channel weight and the shared feature map are calculated respectively, and the Hadamard product of the spatial weight and the Hadamard product of the channel weight are multiplied. The product result is input into the fully connected layer (FC), processed by the fully connected layer, and then input into the sigmoid function to obtain the category prediction probability. The category prediction probability is used to represent the probability of each lane line detection category being contained in the sample image.

[0193] The lane line detection category with a category prediction probability higher than the preset category threshold is used as the category pseudo-label, and the lane line detection category corresponding to the highest probability among the category prediction probabilities is used as the category prediction result. Then, a category prediction loss value is calculated based on the category pseudo-label and the category prediction result. The category prediction loss value can be exemplarily a cross entropy loss between the category prediction result and the category pseudo-label. Exemplarily, the category prediction loss value can be calculated using the following formula:

[0194]

[0195] in, Represents the category prediction loss value, λ ce Represents the weight of the category prediction loss value, Represents the category prediction result, z t represents the category pseudo label, t represents the number of the lane line detection category, and T represents the set of lane line detection category numbers.

[0196] By adopting this method, the initial lane line detection network can automatically generate category pseudo labels for training. The initial lane line detection network is trained using the category pseudo labels, thereby improving the training effect of the initial lane line detection network for sample images of the target domain.

[0197] It should be noted that when training the initial lane line detection network, in addition to the pixel prediction loss value, any one or more loss values among the instance prediction loss value, category prediction loss value, and source domain prediction loss value can also be combined to improve the training accuracy of the initial lane line detection network.

[0198] The following is a general description of the training method of the lane detection network with a specific flowchart. Figure 5 As shown, the training method of the initial lane line detection network includes the following steps:

[0199] Step 1: Obtain a source domain image and input the source domain image into the initial lane line detection network to obtain an original prediction result.

[0200] Step 2: Obtain a sample image, encode the sample image to obtain an initial feature map of the sample image, perform deep feature extraction on the initial feature map to obtain a deep feature map of the sample image, and determine a probabilistic feature map of the sample image based on the deep feature map. Determine a pseudo label corresponding to the sample image based on the probabilistic feature map and preset probability thresholds corresponding to the multiple lane detection categories.

[0201] Step 3.1: Calculate a source domain prediction loss value based on the original prediction result and the label information of the source domain image.

[0202] Step 3.2: Calculate the pixel prediction loss value corresponding to the initial lane detection network based on the pseudo label corresponding to the sample image and the lane detection result;

[0203] Step 3.3: First, based on the pseudo-label corresponding to the sample image, determine the baseline lane line, target lane line, and false lane line in the sample image, and perform boundary pooling processing on the baseline lane line and the target lane line; then, determine the lane line features corresponding to the baseline lane line after the boundary pooling processing, the target lane line after the boundary pooling processing, and the false lane line, respectively, from the deep feature map, and determine the instance prediction loss value based on the lane line features;

[0204] Step 3.4 determines the category prediction probability of the sample image based on the initial feature map, determines the category pseudo-label of the sample image based on the category prediction probability and a pre-set category threshold, and determines the category prediction result corresponding to the sample image based on the category prediction probability, and then determines the category prediction loss value based on the category prediction result and the category pseudo-label.

[0205] Step 4: Training the initial lane detection network based on the source domain prediction loss value, the pixel prediction loss value, the instance prediction loss value, and the category prediction loss value.

[0206] The detailed description of each of the above steps refers to the above embodiments and will not be repeated here.

[0207] The lane line detection network training method provided by the embodiment of the present disclosure can automatically generate pseudo labels based on sample images, and train the initial lane line detection network based on the pseudo labels to solve the problem of unsupervised data of sample images in the target domain; and when determining the pseudo labels, different lane line detection categories correspond to their own probability thresholds, thereby adapting to the probability distribution differences of different lane line detection categories in the target domain, so as to train the lane line detection network to distinguish between each lane line and the background, and further improve the detection accuracy of the trained lane line detection network.

[0208] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0209] Based on the same inventive concept, the embodiments of the present disclosure also provide a training device for a lane line detection network corresponding to the training method for the lane line detection network. Since the principle of solving the problem by the device in the embodiments of the present disclosure is similar to the training method for the lane line detection network in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0210] Reference Figure 6 , which is a schematic diagram of the architecture of a training device for a lane detection network provided by an embodiment of the present disclosure, the device includes: an acquisition module 601, a generation module 602, a determination module 603, and a training module 604; wherein,

[0211] An acquisition module 601 is configured to acquire a sample image of a target domain and input the sample image into an initial lane detection network trained based on a source domain image to determine an initial feature map of the sample image.

[0212] A generating module 602 is configured to generate a probability feature map of the sample image based on the initial feature map; wherein the probability feature map is used to represent the probability that the pixels in the sample image belong to a plurality of preset lane line detection categories;

[0213] a determination module 603 for determining a pseudo label corresponding to the sample image based on the probability feature map and preset probability thresholds corresponding to the plurality of lane detection categories, and determining a lane detection result corresponding to the sample image based on the probability feature map;

[0214] The training module 604 is configured to train the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results to obtain a target lane detection network corresponding to the target domain.

[0215] In a possible implementation, the acquisition module 601, when determining the initial feature map of the sample image, is configured to:

[0216] Encoding the sample image to obtain an initial feature map of the sample image;

[0217] The generating module 602, when generating the probability feature map of the sample image based on the initial feature map, is configured to:

[0218] Performing deep feature extraction on the initial feature map to obtain a deep feature map of the sample image;

[0219] A probabilistic feature map of the sample image is determined based on the deep feature map.

[0220] In one possible implementation, the training module 604, when training the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results, is configured to:

[0221] The initial lane detection network is trained based on the pseudo labels corresponding to the sample images, the lane detection results, the probability feature map, and the deep feature map.

[0222] In one possible implementation, the training module 604, when training the initial lane detection network based on the pseudo labels corresponding to the sample images, the lane detection results, the probabilistic feature map, and the deep feature map, is configured to:

[0223] Calculating a pixel prediction loss value corresponding to the initial lane detection network based on the pseudo label corresponding to the sample image and the lane detection result; and

[0224] Calculating an instance prediction loss value corresponding to the initial lane detection network based on the probability feature map, the deep feature map, and the pseudo labels corresponding to the sample images; and

[0225] Adjusting network parameters of the initial lane detection network based on the pixel prediction loss value and the instance prediction loss value.

[0226] In one possible implementation, the training module 604, when calculating the instance prediction loss value corresponding to the initial lane detection network based on the probability feature map, the deep feature map, and the pseudo labels corresponding to the sample images, is configured to:

[0227] Determining position information of a plurality of real lane lines in the sample image based on the pseudo labels corresponding to the sample image;

[0228] sequentially using the real lane lines as reference lane lines, and determining target lane lines other than the reference lane lines from the plurality of real lane lines; and, based on the pseudo labels corresponding to the sample images, determining a target region of interest that does not include a lane line, and determining a false lane line corresponding to the reference lane line from the target region of interest;

[0229] Based on the position information of the target lane line, the position information of the reference lane line and the probability feature map, boundary pooling processing is performed on the target lane line and the reference lane line; the boundary pooling processing is used to improve the integrity of the lane line;

[0230] The lane line features corresponding to the reference lane line after boundary pooling processing, the target lane line after boundary pooling processing, and the false lane line are determined from the deep feature map, and the instance prediction loss value is determined based on the lane line features.

[0231] In one possible implementation, the training module 604, when determining the target region of interest that does not include a lane line based on the pseudo-label corresponding to the sample image, is configured to:

[0232] Based on the pseudo-label, determining a region of interest where lane lines exist in the sample image;

[0233] Eliminate lane line points corresponding to the pseudo labels in the region of interest to determine a target region of interest;

[0234] The training module 604 is configured to, when determining the false lane line corresponding to the reference lane line from the target region of interest,

[0235] Line detection is performed in the target region of interest to determine the false lane line.

[0236] In one possible implementation, the training module 604, when performing boundary pooling processing on the target lane line and the reference lane line based on the position information of the target lane line, the position information of the reference lane line, and the probability feature map, is configured to:

[0237] For any target lane line point on the lane line to be processed, based on the position information of the target lane line point, determine the pixel points to be optimized adjacent to the target lane line point; the lane line to be processed is the target lane line or the reference lane line;

[0238] Based on the target probability corresponding to the pixel point to be optimized in the probability feature map and a preset optimization probability threshold, boundary pooling processing is performed on the lane line to be processed, wherein the target probability is the probability that the pixel point to be optimized is a point on the lane line to be processed.

[0239] In one possible implementation, the training module 604, when performing boundary pooling processing on the lane line to be processed based on the target probability corresponding to the pixel to be optimized in the probability feature map and a preset optimization probability threshold, is configured to:

[0240] The pixel points to be optimized whose corresponding target probabilities exceed the preset optimization probability threshold are used as target lane line points on the lane line to be processed after boundary pooling processing.

[0241] In one possible implementation, the training module 604, when training the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results, is configured to:

[0242] Calculating a pixel prediction loss value corresponding to the initial lane detection network based on the pseudo label corresponding to the sample image and the lane detection result; and

[0243] Based on the initial feature map and the preset category thresholds corresponding to the multiple lane line detection categories, a category prediction loss value corresponding to the initial lane line detection network is calculated.

[0244] In one possible implementation, the training module 604, when calculating the category prediction loss value corresponding to the initial lane detection network based on the initial feature map and the preset category thresholds corresponding to the multiple lane detection categories, is configured to:

[0245] Determining a class prediction probability of the sample image based on the initial feature map; wherein the class prediction probability is used to represent the probability of each lane line detection class being included in the sample image;

[0246] Determining a category pseudo label for the sample image based on the category prediction probability and the preset category threshold; and determining a category prediction result corresponding to the sample image based on the category prediction probability;

[0247] The category prediction loss value is determined based on the category prediction result and the category pseudo label.

[0248] In one possible implementation, the training module 604, when training the initial lane detection network based on the pseudo labels corresponding to the sample images and the lane detection results, is configured to:

[0249] Obtain source domain images carrying labeled information;

[0250] Inputting the source domain image into the initial lane line detection network to obtain an original prediction result;

[0251] The initial lane line detection network is trained based on the marking information, the original prediction result, the pseudo label corresponding to the sample image and the lane line detection result.

[0252] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0253] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 7 , which is a schematic diagram of the structure of a computer device 700 provided in an embodiment of the present disclosure, including a processor 701, a memory 702, and a bus 703. The memory 702 is used to store execution instructions and includes a memory 7021 and an external memory 7022. The memory 7021 is also referred to as internal memory and is used to temporarily store operation data in the processor 701 and data exchanged with an external memory 7022 such as a hard disk. The processor 701 exchanges data with the external memory 7022 via the memory 7021. When the computer device 700 is running, the processor 701 communicates with the memory 702 via the bus 703, so that the processor 701 executes the following instructions:

[0254] Obtaining a sample image of the target domain, and inputting the sample image into an initial lane detection network trained based on source domain images of the source domain, to determine an initial feature map of the sample image;

[0255] Generating a probability feature map of the sample image based on the initial feature map; wherein the probability feature map is used to represent the probability that the pixel points in the sample image belong to a plurality of preset lane line detection categories;

[0256] Determining a pseudo label corresponding to the sample image based on the probability feature map and preset probability thresholds corresponding to the multiple lane line detection categories, and determining a lane line detection result corresponding to the sample image based on the probability feature map;

[0257] Based on the pseudo labels corresponding to the sample images and the lane line detection results, the initial lane line detection network is trained to obtain a target lane line detection network corresponding to the target domain.

[0258] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the lane detection network training method described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0259] The embodiments of the present disclosure also provide a computer program product that carries program code. The instructions included in the program code can be used to execute the steps of the training method of the lane line detection network described in the above method embodiment. For details, please refer to the above method embodiment and will not be repeated here.

[0260] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0261] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0262] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0263] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0264] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0265] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

[0266] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the category of personal information processed.

Claims

1. A training method for a lane detection network, characterized in that: include: Obtaining a sample image of the target domain, and inputting the sample image into an initial lane detection network trained based on source domain images of the source domain, to determine an initial feature map of the sample image; Generating a probability feature map of the sample image based on the initial feature map; wherein the probability feature map is used to represent the probability that the pixel points in the sample image belong to a plurality of preset lane line detection categories; Determining a pseudo label corresponding to the sample image based on the probability feature map and preset probability thresholds corresponding to the multiple lane line detection categories, and determining a lane line detection result corresponding to the sample image based on the probability feature map; Based on the pseudo labels corresponding to the sample images and the lane line detection results, the initial lane line detection network is trained to obtain a target lane line detection network corresponding to the target domain.

2. The method according to claim 1, characterized in that Determining the initial feature map of the sample image includes: Encoding the sample image to obtain an initial feature map of the sample image; Generating the probability feature map of the sample image based on the initial feature map includes: Performing deep feature extraction on the initial feature map to obtain a deep feature map of the sample image; A probabilistic feature map of the sample image is determined based on the deep feature map.

3. The method according to claim 2, characterized in that The training of the initial lane line detection network based on the pseudo labels corresponding to the sample images and the lane line detection results includes: The initial lane detection network is trained based on the pseudo labels corresponding to the sample images, the lane detection results, the probability feature map, and the deep feature map.

4. The method according to claim 3, characterized in that The training of the initial lane detection network based on the pseudo labels corresponding to the sample images, the lane detection results, the probability feature map, and the deep feature map includes: Calculating a pixel prediction loss value corresponding to the initial lane detection network based on the pseudo label corresponding to the sample image and the lane detection result; and Calculating an instance prediction loss value corresponding to the initial lane detection network based on the probability feature map, the deep feature map, and the pseudo labels corresponding to the sample images; and Adjusting network parameters of the initial lane detection network based on the pixel prediction loss value and the instance prediction loss value.

5. The method according to claim 4, characterized in that The calculating, based on the probability feature map, the deep feature map, and the pseudo labels corresponding to the sample images, an instance prediction loss value corresponding to the initial lane line detection network includes: Determining position information of a plurality of real lane lines in the sample image based on the pseudo labels corresponding to the sample image; sequentially using the real lane lines as reference lane lines, and determining target lane lines other than the reference lane lines from the plurality of real lane lines; and, based on the pseudo labels corresponding to the sample images, determining a target region of interest that does not include a lane line, and determining a false lane line corresponding to the reference lane line from the target region of interest; Based on the position information of the target lane line, the position information of the reference lane line and the probability feature map, boundary pooling processing is performed on the target lane line and the reference lane line; the boundary pooling processing is used to improve the integrity of the lane line; The lane line features corresponding to the reference lane line after boundary pooling processing, the target lane line after boundary pooling processing, and the false lane line are determined from the deep feature map, and the instance prediction loss value is determined based on the lane line features.

6. The method according to claim 5, characterized in that The determining, based on the pseudo labels corresponding to the sample images, a target region of interest excluding lane lines, includes: Based on the pseudo-label, determining a region of interest where lane lines exist in the sample image; Eliminate lane line points corresponding to the pseudo labels in the region of interest to determine a target region of interest; The determining of a false lane line corresponding to the reference lane line from the target region of interest includes: Line detection is performed in the target region of interest to determine the false lane line.

7. The method according to claim 5 or 6, characterized in that The performing boundary pooling processing on the target lane line and the reference lane line based on the position information of the target lane line, the position information of the reference lane line, and the probability feature map includes: For any target lane line point on the lane line to be processed, based on the position information of the target lane line point, determine the pixel points to be optimized adjacent to the target lane line point; the lane line to be processed is the target lane line or the reference lane line; Based on the target probability corresponding to the pixel point to be optimized in the probability feature map and a preset optimization probability threshold, boundary pooling processing is performed on the lane line to be processed, wherein the target probability is the probability that the pixel point to be optimized is a point on the lane line to be processed.

8. The method according to claim 7, characterized in that The performing boundary pooling processing on the lane line to be processed based on the target probability corresponding to the pixel point to be optimized in the probability feature map and a preset optimization probability threshold includes: The pixel points to be optimized whose corresponding target probabilities exceed the preset optimization probability threshold are used as target lane line points on the lane line to be processed after boundary pooling processing.

9. The method according to claim 2, characterized in that The training of the initial lane line detection network based on the pseudo labels corresponding to the sample images and the lane line detection results includes: Calculating a pixel prediction loss value corresponding to the initial lane detection network based on the pseudo label corresponding to the sample image and the lane detection result; and Calculating a category prediction loss value corresponding to the initial lane detection network based on the initial feature map and preset category thresholds corresponding to the multiple lane detection categories; Based on the pixel prediction loss value and the category prediction loss value, network parameters of the initial lane detection network are adjusted.

10. The method according to claim 9, characterized in that The calculating, based on the initial feature map and the preset category thresholds corresponding to the plurality of lane line detection categories, a category prediction loss value corresponding to the initial lane line detection network includes: Determining a class prediction probability of the sample image based on the initial feature map; wherein the class prediction probability is used to represent the probability of each lane line detection class being included in the sample image; Determining a category pseudo label for the sample image based on the category prediction probability and the preset category threshold; and determining a category prediction result corresponding to the sample image based on the category prediction probability; The category prediction loss value is determined based on the category prediction result and the category pseudo label.

11. The method according to claim 1, wherein The training of the initial lane line detection network based on the pseudo labels corresponding to the sample images and the lane line detection results includes: Obtain source domain images carrying labeled information; Inputting the source domain image into the initial lane line detection network to obtain an original prediction result; The initial lane line detection network is trained based on the marking information, the original prediction result, the pseudo label corresponding to the sample image and the lane line detection result.

12. A training device for a lane detection network, characterized in that: include: an acquisition module, configured to acquire a sample image of a target domain, and input the sample image into an initial lane detection network trained based on a source domain image of a source domain, to determine an initial feature map of the sample image; A generating module, configured to generate a probability feature map of the sample image based on the initial feature map; wherein the probability feature map is used to represent the probability that the pixel points in the sample image belong to a plurality of preset lane line detection categories; a determination module, configured to determine a pseudo label corresponding to the sample image based on the probability feature map and preset probability thresholds corresponding to the plurality of lane line detection categories, and to determine a lane line detection result corresponding to the sample image based on the probability feature map; A training module is used to train the initial lane line detection network based on the pseudo labels corresponding to the sample images and the lane line detection results to obtain a target lane line detection network corresponding to the target domain.

13. A computer device, characterized in that: include: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of the lane detection network training method according to any one of claims 1 to 11 are performed.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the method for training a lane detection network according to any one of claims 1 to 11.

Images