Lane detection method, device, vehicle and storage medium

Abstract

The present disclosure relates to a lane line detection method, device, vehicle and storage medium. The lane line detection method acquires a target detection image; inputs the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image, wherein the preset lane line detection model is obtained by training a preset initial model including a segmentation network and a classification network, the segmentation network is used to determine a first predicted position of the lane line in the lane line detection sample image, and the classification network is used to determine a second predicted position of the lane line in the lane line detection sample image. Lane line detection is performed by using the preset lane line detection model, which can effectively improve the accuracy of the lane line detection result.

Description

Technical Field

[0001] The present disclosure relates to the technical field of intelligent connected vehicles, and in particular to a lane line detection method, device, vehicle and storage medium. Background Art

[0002] With the continuous development of science and technology, autonomous driving technology has gradually reached a climax. Safety has always been the focus of autonomous driving technology. The environmental perception ability of the autonomous driving system plays a decisive role in safety performance, and lane line detection is a key link in environmental perception.

[0003] During the lane line detection process, the lane lines in the captured road images are usually blurred due to dim light, reflections, obstructions, or the lane lines being washed by rain for a long time. In addition, the scenes where the lane lines are located are ever-changing, which makes lane line detection difficult and the detection results less accurate. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a lane line detection method, device, vehicle and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a lane line detection method is provided, the method comprising:

[0006] Get target detection image;

[0007] Inputting the target detection image into a preset lane line detection model so that the preset lane line detection model outputs the target position of the lane line in the target detection image;

[0008] The preset lane line detection model is trained in the following way:

[0009] Acquire a plurality of lane line detection sample images, wherein the lane line detection images include lane line marking positions;

[0010] Inputting each lane line detection sample image into a preset initial model, wherein the preset initial model includes a segmentation network and a classification network, wherein the segmentation network is used to determine a first predicted position of a lane line in the lane line detection sample image, and the classification network is used to determine a second predicted position of a lane line in the lane line detection sample image;

[0011] The preset initial model is trained according to the first predicted position, the second predicted position and the lane line marking position to obtain the preset lane line detection model.

[0012] Optionally, the preset initial model further includes a feature extraction network, the feature extraction network is coupled with the classification network and the segmentation network, and the preset initial model is used

[0013] Acquire a multi-scale sample feature map corresponding to each lane line detection sample image through the feature extraction network, and determine a first sample feature map including image global information according to the multi-scale sample feature map;

[0014] Determine, by the segmentation network, a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map;

[0015] The second predicted position of the lane line in the lane line detection sample image is determined by the classification network according to the multi-scale sample feature map.

[0016] Optionally, the preset lane line detection model is trained in the following manner:

[0017] Obtain a multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model, and determine a first sample feature map including image global information according to the minimum-scale sample feature map in the multi-scale sample feature map;

[0018] Determine, by the segmentation network, a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map;

[0019] Determine a first loss value corresponding to a first loss function according to the first predicted position and the lane line marking position;

[0020] Determining a second predicted position of a lane line in the lane line detection sample image according to the minimum scale sample feature map by the classification network;

[0021] Determine a second loss value corresponding to a second loss function according to the second predicted position and the lane line marking position;

[0022] Determine a third loss value corresponding to a third loss function according to the first loss value and the second loss value;

[0023] When the third loss value is greater than or equal to the preset loss value threshold, the model parameters of the preset initial model are adjusted to obtain an updated target model, and the step of obtaining the multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model is executed again, until the step of determining the third loss value corresponding to the third loss function according to the first loss value and the second loss value, until it is determined that the third loss value is less than the preset loss value threshold, the segmentation network in the current target model is deleted to obtain the preset lane line detection model.

[0024] Optionally, the feature extraction network includes a feature pyramid network and a global feature extraction network, the feature pyramid network includes a bottom-up subnetwork and a top-down subnetwork, the input end of the global feature extraction network is coupled with the output end of the bottom-up subnetwork, the output end of the global feature extraction network is coupled with the input end of the top-down subnetwork, the output end of the global feature extraction network is also coupled with the input end of the segmentation network, and the output end of the top-down subnetwork is also coupled with the input end of the segmentation network, and the preset initial model is used to,

[0025] Acquire a multi-scale sample feature map corresponding to the lane line detection sample image through the bottom-up sub-network in the feature pyramid network, and input the minimum-scale sample feature map in the multi-scale sample feature map into the global feature extraction network, so that the global feature extraction network outputs the first sample feature map to the segmentation network and the top-down sub-network;

[0026] A multi-scale second sample feature map is input into the segmentation network through the top-down sub-network, so that the segmentation network determines a first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map.

[0027] Optionally, the segmentation network determines a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map, including:

[0028] Performing upsampling processing on the first sample feature map and the second sample feature map through the segmentation network to obtain a sample feature map to be segmented;

[0029] Segmenting the sample feature map to be segmented according to a preset segmentation method to obtain a plurality of local feature maps;

[0030] Performing convolution and splicing processing on the multiple local feature maps to obtain a target sample feature map;

[0031] Determine a first probability that each pixel in the lane line detection sample image belongs to a lane line according to the target sample feature map;

[0032] A first predicted position of a lane line in the lane line detection sample image is determined according to the first probability of each pixel.

[0033] Optionally, the segmentation according to a preset manner includes:

[0034] Divide the sample feature map to be segmented into H C*W local feature maps, divide the sample feature map to be segmented into C H*W local feature maps, and divide the sample feature map to be segmented into at least one of W H*C local feature maps, wherein H is the number of layers of the sample feature map to be segmented, C is the length of the sample feature map to be segmented, and W is the width of the sample feature map to be segmented.

[0035] Optionally, performing convolution and concatenation processing on the multiple local feature maps to obtain a target sample feature map includes:

[0036] Obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain a specified local feature map after convolution;

[0037] After splicing the specified local feature map with the next local feature map corresponding to the current local feature map, an updated current local feature map is obtained;

[0038] Determine whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

[0039] In the case where it is determined that the next local feature map corresponding to the current local feature map is not the last one of the multiple local feature maps, the step of obtaining the current local feature map again and performing a convolution operation on the current local feature map to obtain a designated local feature map after the convolution, until the step of determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

[0040] When it is determined that the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps, the current local feature map is obtained, and a convolution operation is performed on the current local feature map to obtain a designated local feature map after the convolution, and the designated local feature map corresponding to the current local feature map is used as the target sample feature map.

[0041] Optionally, the output end of the bottom-up sub-network is coupled to the input end of the classification network, and the preset initial model is used to:

[0042] Inputting a minimum scale sample feature map into the classification network through the bottom-up sub-network;

[0043] The minimum scale sample feature map is divided into a plurality of anchor frames through the classification network, a second probability that each anchor frame belongs to the lane line is determined, and a second predicted position of the lane line in the lane line detection sample image is determined according to the second probability.

[0044] Optionally, inputting the target detection image into a preset lane line detection model so that the preset lane line detection model outputs a target position of the lane line in the target detection image, comprises:

[0045] Acquire a target feature map corresponding to the target detection image through the feature extraction network;

[0046] The target feature map is input into the classification network to obtain the target position of the lane line in the target detection image output by the classification network.

[0047] According to a second aspect of an embodiment of the present disclosure, a lane line detection device is provided, the device comprising:

[0048] An acquisition module is configured to acquire a target detection image;

[0049] A determination module is configured to input the target detection image into a preset lane line detection model so that the preset lane line detection model outputs a target position of the lane line in the target detection image;

[0050] The preset lane line detection model is trained in the following way:

[0051] Acquire a plurality of lane line detection sample images, wherein the lane line detection images include lane line marking positions;

[0052] Inputting each lane line detection sample image into a preset initial model, wherein the preset initial model includes a segmentation network and a classification network, wherein the segmentation network is used to determine a first predicted position of a lane line in the lane line detection sample image, and the classification network is used to determine a second predicted position of a lane line in the lane line detection sample image;

[0053] The preset initial model is trained according to the first predicted position, the second predicted position and the lane line marking position to obtain the preset lane line detection model.

[0054] Optionally, the preset initial model further includes a feature extraction network, the feature extraction network is coupled with the classification network and the segmentation network, and the preset initial model is used

[0055] Acquire a multi-scale sample feature map corresponding to each lane line detection sample image through the feature extraction network, and determine a first sample feature map including image global information according to the multi-scale sample feature map;

[0056] Determine, by the segmentation network, a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map;

[0057] The second predicted position of the lane line in the lane line detection sample image is determined by the classification network according to the multi-scale sample feature map.

[0058] Optionally, the preset lane line detection model is trained in the following manner:

[0059] Obtain a multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model, and determine a first sample feature map including image global information according to the minimum-scale sample feature map in the multi-scale sample feature map;

[0060] Determine, by the segmentation network, a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map;

[0061] Determine a first loss value corresponding to a first loss function according to the first predicted position and the lane line marking position;

[0062] Determining a second predicted position of a lane line in the lane line detection sample image according to the minimum scale sample feature map by the classification network;

[0063] Determine a second loss value corresponding to a second loss function according to the second predicted position and the lane line marking position;

[0064] Determine a third loss value corresponding to a third loss function according to the first loss value and the second loss value;

[0065] When the third loss value is greater than or equal to the preset loss value threshold, the model parameters of the preset initial model are adjusted to obtain an updated target model, and the step of obtaining the multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model is executed again, until the step of determining the third loss value corresponding to the third loss function according to the first loss value and the second loss value, until it is determined that the third loss value is less than the preset loss value threshold, the segmentation network in the current target model is deleted to obtain the preset lane line detection model.

[0066] Optionally, the feature extraction network includes a feature pyramid network and a global feature extraction network, the feature pyramid network includes a bottom-up subnetwork and a top-down subnetwork, the input end of the global feature extraction network is coupled with the output end of the bottom-up subnetwork, the output end of the global feature extraction network is coupled with the input end of the top-down subnetwork, the output end of the global feature extraction network is also coupled with the input end of the segmentation network, and the output end of the top-down subnetwork is also coupled with the input end of the segmentation network, and the preset initial model is used to,

[0067] Acquire a multi-scale sample feature map corresponding to the lane line detection sample image through the bottom-up sub-network in the feature pyramid network, and input the minimum-scale sample feature map in the multi-scale sample feature map into the global feature extraction network, so that the global feature extraction network outputs the first sample feature map to the segmentation network and the top-down sub-network;

[0068] A multi-scale second sample feature map is input into the segmentation network through the top-down sub-network, so that the segmentation network determines a first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map.

[0069] Optionally, determining a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map through the segmentation network includes:

[0070] Performing upsampling processing on the first sample feature map and the second sample feature map through the segmentation network to obtain a sample feature map to be segmented;

[0071] Segmenting the sample feature map to be segmented according to a preset segmentation method to obtain a plurality of local feature maps;

[0072] Performing convolution and splicing processing on the multiple local feature maps to obtain a target sample feature map;

[0073] Determine a first probability that each pixel in the lane line detection sample image belongs to a lane line according to the target sample feature map;

[0074] A first predicted position of a lane line in the lane line detection sample image is determined according to the first probability of each pixel.

[0075] Optionally, the segmentation according to a preset manner includes:

[0076] Divide the sample feature map to be segmented into H C*W local feature maps, divide the sample feature map to be segmented into C H*W local feature maps, and divide the sample feature map to be segmented into at least one of W H*C local feature maps, wherein H is the number of layers of the sample feature map to be segmented, C is the length of the sample feature map to be segmented, and W is the width of the sample feature map to be segmented.

[0077] Optionally, performing convolution and concatenation processing on the multiple local feature maps to obtain a target sample feature map includes:

[0078] Obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain a specified local feature map after convolution;

[0079] After splicing the specified local feature map with the next local feature map corresponding to the current local feature map, an updated current local feature map is obtained;

[0080] Determine whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

[0081] In the case where it is determined that the next local feature map corresponding to the current local feature map is not the last one of the multiple local feature maps, the step of obtaining the current local feature map again and performing a convolution operation on the current local feature map to obtain a designated local feature map after the convolution, until the step of determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

[0082] When it is determined that the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps, the current local feature map is obtained, and a convolution operation is performed on the current local feature map to obtain a designated local feature map after the convolution, and the designated local feature map corresponding to the current local feature map is used as the target sample feature map.

[0083] Optionally, the output end of the bottom-up sub-network is coupled to the input end of the classification network, and the preset initial model is used to:

[0084] Inputting a minimum scale sample feature map into the classification network through the bottom-up sub-network;

[0085] The minimum scale sample feature map is divided into a plurality of anchor frames through the classification network, a second probability that each anchor frame belongs to the lane line is determined, and a second predicted position of the lane line in the lane line detection sample image is determined according to the second probability.

[0086] Optionally, the determining module is configured to:

[0087] Acquire a target feature map corresponding to the target detection image through the feature extraction network;

[0088] The target feature map is input into the classification network to obtain the target position of the lane line in the target detection image output by the classification network.

[0089] According to a third aspect of an embodiment of the present disclosure, a vehicle is provided, wherein the vehicle includes the lane line detection device described in the second aspect above.

[0090] According to a fourth aspect of an embodiment of the present disclosure, a lane line detection device is provided, including:

[0091] a memory having a computer program stored thereon;

[0092] A processor is used to execute the computer program in the memory to implement the steps of the method described in the first aspect above.

[0093] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method described in the first aspect of the present disclosure are implemented.

[0094] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by acquiring a target detection image; inputting the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image, wherein the preset lane line detection model is obtained by training a preset initial model including a segmentation network and a classification network, and lane line detection is performed using the preset lane line detection model, which can effectively improve the accuracy of the lane line detection results.

[0095] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0097] Figure 1 is a flow chart of a lane line detection method shown in an exemplary embodiment of the present disclosure;

[0098] Figure 2 is a model structure block diagram of a preset initial model shown in an exemplary embodiment of the present disclosure;

[0099] Figure 3is a schematic diagram of a segmented network according to an exemplary embodiment of the present disclosure;

[0100] Figure 4 is a training flow chart of a preset lane line detection model shown in an exemplary embodiment of the present disclosure;

[0101] Figure 5 is based on Figure 4 A flow chart of a lane line detection method shown in the illustrated embodiment;

[0102] Figure 6 is a block diagram of a lane line detection device shown in an exemplary embodiment of the present disclosure;

[0103] Figure 7 The figure is a block diagram of a lane line detection device according to an exemplary embodiment. DETAILED DESCRIPTION

[0104] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0105] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located, and with the authorization given by the owner of the corresponding device.

[0106] Before introducing the specific implementation methods of the present disclosure in detail, first, the application scenarios of the present disclosure are described as follows. The present disclosure can be applied to the lane line detection process, and the lane line detection can be an important link for lane recognition when sensing the driving environment during autonomous driving, unmanned driving or route navigation. At present, the methods of lane recognition are mainly divided into two categories, image feature-based methods and model-based methods. The principle of image feature recognition is to utilize the difference in features between the edge of the lane marking line and the road conditions around the road image. The common feature differences include image texture, edge geometry, boundary continuity, grayscale value and contrast of the area of ​​interest, etc. The image features can be extracted using methods such as threshold segmentation. The algorithm is simple and easy to implement. The disadvantage is that it is easily affected by noise, tree shadow occlusion, illumination changes, discontinuous marking lines, etc., resulting in low accuracy of lane line detection results and poor reliability of lane recognition results. The principle of model matching is to establish a corresponding lane line model for the geometric features of the structured road, identify the road model parameters, and then identify the lane marking line. Due to the limitations of the image feature method, in actual research, the image feature method and the model matching method are generally combined to establish a suitable lane line detection model. This can improve the accuracy of lane line recognition on the one hand, and enhance the adaptability of the system on the other hand. However, there is currently no more complete model that can guarantee the accuracy of lane line detection results in various detection scenarios.

[0107] The present disclosure provides a lane line detection method, device, vehicle and storage medium. The lane line detection method acquires a target detection image; inputs the target detection image into a preset lane line detection model, so that the preset lane line detection model outputs the target position of the lane line in the target detection image, wherein the preset lane line detection model is trained by a preset initial model including a segmentation network and a classification network. Lane line detection is performed by using the preset lane line detection model, which can effectively improve the accuracy of lane line detection results in various detection scenarios.

[0108] The technical solution of the present disclosure is described in detail below in conjunction with specific embodiments.

[0109] Figure 1 is a flow chart of a lane line detection method shown in an exemplary embodiment of the present disclosure; Figure 1 As shown, the lane line detection method may include:

[0110] Step 101, obtaining a target detection image.

[0111] Among them, the target detection image can be any image to be detected in various lane line detection scenarios (such as straight lane line detection scene, hyperbolic lane line detection scene, B-spline curve lane line detection scene, etc.), and the image to be detected may include the lane line to be detected.

[0112] Step 102: input the target detection image into a preset lane line detection model so that the preset lane line detection model outputs the target position of the lane line in the target detection image.

[0113] In this step, the preset lane line detection model is trained by the following method shown in S11 to S13:

[0114] S11, obtaining multiple lane line detection sample images.

[0115] The lane line detection image includes the lane line marking position.

[0116] S12, input each lane line detection sample image into a preset initial model, the preset initial model includes a segmentation network and a classification network, the segmentation network is used to determine the first predicted position of the lane line in the lane line detection sample image, and the classification network is used to determine the second predicted position of the lane line in the lane line detection sample image.

[0117] Among them, the segmentation network may include multiple convolution layers and classification layers, and different convolution layers correspond to different numbers of convolution kernels. The segmentation network is used to divide the feature map corresponding to the lane line detection sample image into multiple local feature maps, and then convolve and splice the multiple local feature maps to obtain a target sample feature map, and then input the target sample feature map into the classification layer so that the classification layer outputs the second predicted position of the lane line in the lane line detection sample image.

[0118] It should be noted that the segmentation network can effectively obtain more feature map details by performing convolution operations on the local feature maps obtained through feature map division, and then classify the feature maps containing more details, which can effectively improve the accuracy of the classification results, and thus improve the accuracy of the lane line detection results. The segmentation network can divide the feature map by cutting the feature map into multiple local feature maps from different directions of the feature map. For example, a H*C*W feature map can be cut into H C*W local feature maps, C H*W local feature maps, or W H*C local feature maps, where H is the number of layers of the feature map, C is the length of the feature map, and W is the width of the feature map. In addition, the classification layer in the segmentation network can be a fully connected layer or other network modules used for classification. There are many network modules with classification functions in the prior art, and the present disclosure does not limit this.

[0119] S13, training the preset initial model according to the first predicted position, the second predicted position and the lane line marking position to obtain the preset lane line detection model.

[0120] It should be noted that, during the training process, the preset lane line detection model is obtained by training the preset initial model including the classification network and the segmentation network, but during the application process, the preset lane line detection model may include the segmentation network and the classification network, or may include the classification network without the segmentation network, or may include the segmentation network without the classification network.

[0121] The above technical solution obtains a target detection image; inputs the target detection image into a preset lane line detection model so that the preset lane line detection model outputs the target position of the lane line in the target detection image, wherein the preset lane line detection model is obtained by training a preset initial model including a segmentation network and a classification network. Lane line detection is performed using the preset lane line detection model, which can effectively improve the accuracy of lane line detection results in various detection scenarios.

[0122] Figure 2 is a model structure diagram of a preset initial model shown in an exemplary embodiment of the present disclosure, such as Figure 2 As shown, the preset initial model also includes a feature extraction network, which is coupled with the classification network and the segmentation network. The preset initial model is used to:

[0123] Acquire a multi-scale sample feature map corresponding to each lane line detection sample image through the feature extraction network, and determine a first sample feature map including image global information according to the multi-scale sample feature map;

[0124] Determine, by the segmentation network, a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map;

[0125] The second predicted position of the lane line in the lane line detection sample image is determined by the classification network according to the multi-scale sample feature map.

[0126] The feature extraction network may include a feature pyramid network and a global feature extraction network, the feature pyramid network includes a bottom-up subnetwork and a top-down subnetwork, the global feature extraction network may be an Encoder in a Transformer network, the input end of the global feature extraction network is coupled with the output end of the bottom-up subnetwork, the output end of the global feature extraction network is coupled with the input end of the top-down subnetwork, the output end of the global feature extraction network is also coupled with the input end of the segmentation network, the output end of the top-down subnetwork is also coupled with the input end of the segmentation network, the output end of the bottom-up subnetwork is coupled with the input end of the classification network, and the preset initial model is used for:

[0127] The multi-scale sample feature map corresponding to the lane line detection sample image is obtained through the bottom-up sub-network in the feature pyramid network (such as Figure 2 The feature map x1, feature map x2, feature map x3, feature map x4 in the multi-scale sample feature map are converted into the minimum scale sample feature map (such as Figure 2 The feature map x4 in the global feature extraction network is input to make the global feature extraction network output the first sample feature map (such as Figure 2 In the process, the feature map x4 is converted into the feature map X1) through the global feature extraction network;

[0128] The multi-scale second sample feature map (such as Figure 2 The feature map X1, feature map X2, feature map X3, feature map X4) are used to enable the segmentation network to determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map.

[0129] It should be noted that, in the feature pyramid network, the feature map at the low layer (shallow layer) has rich detail information, and the feature map at the high layer (deep layer) has rich semantic information. The implementation process of the segmentation network determining the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map may include the following steps shown in S21 to S25:

[0130] S21, upsampling the first sample feature map and the second sample feature map through the segmentation network to obtain a sample feature map to be segmented.

[0131] For example, the multi-scale second sample feature map may include: Figure 2The feature maps X1, X2, X3 and X4 in the image are upsampled respectively to obtain a feature map of the same size as the feature map X4, and the feature maps X1, X2 and X3 corresponding to the feature map X4 with the same size as the feature map X4 are concat (splicing) with the feature map to obtain the feature map of the sample to be segmented.

[0132] S22, segmenting the sample feature map to be segmented according to a preset segmentation method to obtain a plurality of local feature maps.

[0133] Among them, the preset segmentation method includes: dividing the sample feature map to be segmented into H C*W local feature maps, dividing the sample feature map to be segmented into C H*W local feature maps, dividing the sample feature map to be segmented into at least one of W H*C local feature maps, wherein H is the number of layers of the sample feature map to be segmented, C is the length of the sample feature map to be segmented, and W is the width of the sample feature map to be segmented.

[0134] For example, Figure 3 As shown, Figure 3 : is a schematic diagram of the principle of a segmentation network shown in an exemplary embodiment of the present disclosure, in which segmentation is performed from four directions, that is, the feature map of the sample to be segmented of H*C*W is regarded as a cube, and the feature map of H*C*W is divided into H local feature maps of C*W from top to bottom along the height (corresponding to H) of the cube (such as Figure 3 SCNN_D in the cube), and then divide the H*C*W feature map from bottom to top along the height of the cube into another H C*W local feature maps (such as Figure 3 SCNN_U in the figure), along the width of the cube (corresponding to W) from left to right, the H*C*W feature map is divided into W H*C local feature maps (such as Figure 3 SCNN_R in the figure), along the width of the cube (corresponding to W) from right to left, the feature map of H*C*W is divided into another W local feature maps of H*C (such as Figure 3 SCNN_L in (SCNN_L).

[0135] S23, performing convolution and concatenation processing on the multiple local feature maps to obtain a target sample feature map.

[0136] A possible implementation method of this step is:

[0137] Obtain a current local feature map, and perform a convolution operation on the current local feature map to obtain a designated local feature map after convolution; concatenate the designated local feature map with the next local feature map corresponding to the current local feature map to obtain an updated current local feature map; determine whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps; if it is determined that the next local feature map corresponding to the current local feature map is not the last one of the multiple local feature maps, perform the steps of obtaining the current local feature map again, and performing a convolution operation on the current local feature map to obtain a designated local feature map after convolution, until determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps; if it is determined that the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps, obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain a designated local feature map after convolution, and use the designated local feature map corresponding to the current local feature map as the target sample feature map.

[0138] For example, Figure 3Taking the example shown as an example, after obtaining H local feature maps of C*W from top to bottom (for example, C*W-1, C*W-2 to C*WH), the local feature map C*W-1 can be first convolved (the convolution kernel is 1*1), and then the convolution result corresponding to the local feature map C*W-1 (the specified local feature map corresponding to C*W-1) is spliced ​​with the local feature map C*W-2 to obtain the spliced ​​current local feature map, and then the current local feature map is convolved (the convolution kernel is 2 1*1) to obtain the specified local feature map corresponding to C*W-2. The specified local feature map corresponding to the local feature map C*W-2 is concatenated with the local feature map C*W-3 to obtain the updated current local feature map, and this cycle is repeated until the specified local feature map corresponding to C*WH is obtained, wherein the dimension of the specified local feature map corresponding to C*WH is H*C*W, and then the specified local feature map corresponding to C*WH (also a feature map of H*C*W) is sliced ​​from bottom to top along the height direction to obtain H C*W local feature maps from bottom to top (for example, C*W+1, C*W+2 To C*W+H), perform convolution splicing operation on C*W+1, C*W+2 to C*W+H to obtain the specified local feature map corresponding to C*W+H (the dimension is H*C*W), and then divide the feature map of H*C*W into W local feature maps of H*C from left to right along the width of the cube (corresponding to W) to obtain W local feature maps of H*C from left to right (for example, H*C-1, H*C-2 to H*CW), and perform preset convolution operation on H*C-1, H*C-2 to H*CW A convolutional splicing operation is performed to obtain a specified local feature map corresponding to H*CW (with a dimension of H*C*W), and then the specified local feature map corresponding to H*CW is divided into local feature maps from right to left (for example, H*C+1, H*C+2 to H*C+W) along the width of the cube (corresponding to W) from right to left, and a preset convolutional splicing operation is performed on H*C+1, H*C+2 to H*C+W to obtain the specified local feature map corresponding to H*C+W, and the specified local feature map corresponding to H*C+W is used as the target sample feature map.

[0139] S24, determining a first probability that each pixel in the lane line detection sample image belongs to a lane line according to the target sample feature map.

[0140] In this step, since the target sample feature map includes more detailed information in the lane line detection sample image, the first probability that each pixel in the lane line detection sample image belongs to the lane line obtained according to the target sample feature map is more accurate.

[0141] S25, determining a first predicted position of the lane line in the lane line detection sample image according to the first probability of each pixel.

[0142] In this step, the pixel position where the first probability is greater than or equal to the first preset probability threshold may be used as the first predicted position of the lane line in the lane line detection sample image.

[0143] Through the above S21 to S25, the segmentation network can be used to obtain a target sample feature map with more detailed information, and then the first probability that each pixel in the lane line detection sample image belongs to the lane line is determined through the target sample feature map. Determining the first predicted position based on the first probability can effectively improve the accuracy of the first predicted position.

[0144] Optionally, the preset initial model is also used for:

[0145] The minimum scale sample feature map (such as Figure 2 Feature map x4 in ;

[0146] The minimum scale sample feature map is divided into multiple anchor frames through the classification network, a second probability that each anchor frame belongs to the lane line is determined, and a second predicted position of the lane line in the lane line detection sample image is determined according to the second probability.

[0147] It should be noted that there are many ways to divide the feature map x4 into anchor frames in the prior art, and they are also relatively easy to obtain. This disclosure will not go into details here. The specific details of determining the second probability that each anchor frame belongs to the lane line through a classification network (which can be a fully connected layer) belong to a common calculation process in the field, and this disclosure does not limit this. In addition, the pixel position corresponding to the anchor frame whose second probability is greater than or equal to the second preset probability threshold can be used as the second predicted position.

[0148] The above technical solution can obtain the second predicted position of the lane line in the lane line detection sample image through the classification network. Since the second predicted position is obtained through the minimum scale sample feature map output by FPN (Feature Pyramid Networks), the minimum scale sample feature contains rich contextual feature information, and therefore can also effectively improve the accuracy of the second predicted position.

[0149] Figure 4 is a training flow chart of a preset lane line detection model shown in an exemplary embodiment of the present disclosure; Figure 4 As shown, the preset lane line detection model can be trained by the following steps:

[0150] Step 401, obtaining a plurality of lane line detection sample images, wherein the lane line detection images include lane line marking positions.

[0151] Among them, the lane line detection sample image can be any image in various lane line detection scenes (such as straight lane line detection scenes, hyperbolic lane line detection scenes, B-spline curve lane line detection scenes, etc.), and the lane line position can be marked in the lane line detection sample image.

[0152] Step 402, obtaining a multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model, and determining a first sample feature map including global image information according to the minimum scale sample feature map in the multi-scale sample feature map.

[0153] The feature extraction network may include a feature pyramid network and a global feature extraction network. The feature pyramid network includes a bottom-up subnetwork and a top-down subnetwork. The global feature extraction network may be an encoder in a Transformer network. The multi-scale sample feature map (such as Figure 2 The feature map x1, feature map x2, feature map x3, feature map x4 in the multi-scale sample feature map are converted into the minimum scale sample feature map (such as Figure 2 The feature map x4 in the image is input into the global feature extraction network so that the global feature extraction network outputs the first sample feature map to the segmentation network and the top-down sub-network.

[0154] Step 403: Determine a first predicted position of the lane line in the lane line detection sample image according to the first sample feature map through the segmentation network.

[0155] In this step, the first sample feature map is input into the top-down sub-network, so that the top-down sub-network inputs a multi-scale second sample feature map (such as Figure 2 The feature map X1, feature map X2, feature map X3, feature map X4) are used to enable the segmentation network to determine the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map.

[0156] Among them, the specific implementation method of the segmentation network determining the first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map can be referred to the steps shown in S21 to S25 above, and the present disclosure will not repeat them here.

[0157] Step 404: Determine a first loss value corresponding to a first loss function according to the first predicted position and the lane marking position.

[0158] Among them, the first loss function can be a cross entropy loss function, a focal loss loss function, or other loss functions in the prior art.

[0159] Step 405: Determine a second predicted position of the lane line in the lane line detection sample image according to the minimum scale sample feature map through the classification network.

[0160] Step 406: Determine a second loss value corresponding to a second loss function according to the second predicted position and the lane marking position.

[0161] The second loss function may also be a cross entropy loss function or a focal loss function, or other loss functions in the prior art.

[0162] Step 407: Determine a third loss value corresponding to a third loss function according to the first loss value and the second loss value.

[0163] For example, the third loss function may be L total =αL cls +βL seg , where L total represents the third loss value, α and β are weighting coefficients. cls is the second loss value corresponding to the second loss function, L seg is the first loss value corresponding to the first loss function.

[0164] Step 408: Determine whether the third loss value is greater than or equal to a preset loss value threshold.

[0165] In this step, if the third loss value is greater than or equal to the preset loss value threshold, step 409 is executed, and if it is determined that the third loss value is less than the preset loss value threshold, step 410 is executed.

[0166] Step 409: adjust the model parameters of the preset initial model to obtain an updated target model.

[0167] After this step, the process may jump to step 402 again to execute steps 402 to 408 again.

[0168] Step 410, delete the segmentation network in the current target model to obtain the preset lane line detection model.

[0169] The above technical solution can effectively improve the accuracy of the model detection results by jointly training the preset lane line detection model through the segmentation network and the classification network. In order to improve the model processing speed, the segmentation network can be deleted in the application stage, and only the feature extraction network and the classification network are retained, thereby not only ensuring the accuracy of the lane line detection results, but also effectively improving the detection efficiency of the model.

[0170] Optionally, through the above Figure 4 When the preset lane line detection model obtained in the steps shown in the figure determines the target position of the lane line in the target detection image, it can include the following Figure 5 The steps shown are: Figure 5 is based on Figure 4 The embodiment shown is a flow chart of a lane line detection method; Figure 5 As shown, the method includes:

[0171] Step 501, obtaining a target detection image.

[0172] Among them, the target detection image can be any image to be detected in various lane line detection scenarios (such as straight lane line detection scene, hyperbolic lane line detection scene, B-spline curve lane line detection scene, etc.), and the image to be detected may include the lane line to be detected.

[0173] Step 502: Obtain a target feature map corresponding to the target detection image through the feature extraction network.

[0174] The feature extraction network may include a feature pyramid network, which includes a bottom-up subnetwork and a top-down subnetwork, and the target feature map may be a minimum scale feature map input by the bottom-up subnetwork (such as Figure 2 The feature map x4 in .

[0175] Step 503: input the target feature map into the classification network to obtain the target position of the lane line in the target detection image output by the classification network.

[0176] Determining the target position of the lane line in the target detection image by the method shown in the above steps 501 to 503 can not only ensure the accuracy of the lane line detection result, but also effectively improve the detection efficiency of the model.

[0177] Figure 6 is a block diagram of a lane line detection device shown in an exemplary embodiment of the present disclosure. Figure 6 As shown, the device may include:

[0178] An acquisition module 601 is configured to acquire a target detection image;

[0179] The determination module 602 is configured to input the target detection image into a preset lane line detection model so that the preset lane line detection model outputs a target position of the lane line in the target detection image;

[0180] The preset lane line detection model is trained in the following way:

[0181] Acquire a plurality of lane line detection sample images, wherein the lane line detection images include lane line marking positions;

[0182] Input each lane line detection sample image into a preset initial model, where the preset initial model includes a segmentation network and a classification network, where the segmentation network is used to determine a first predicted position of a lane line in the lane line detection sample image, and the classification network is used to determine a second predicted position of a lane line in the lane line detection sample image;

[0183] The preset initial model is trained according to the first predicted position, the second predicted position and the lane line marking position to obtain the preset lane line detection model.

[0184] The above technical solution obtains a target detection image; inputs the target detection image into a preset lane line detection model so that the preset lane line detection model outputs the target position of the lane line in the target detection image, wherein the preset lane line detection model is obtained by training a preset initial model including a segmentation network and a classification network, and can effectively improve the accuracy of lane line detection results in various detection scenarios.

[0185] Optionally, the preset initial model further includes a feature extraction network, which is coupled with the classification network and the segmentation network. The preset initial model is used

[0186] Acquire a multi-scale sample feature map corresponding to each lane line detection sample image through the feature extraction network, and determine a first sample feature map including image global information according to the multi-scale sample feature map;

[0187] Determine, by the segmentation network, a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map;

[0188] The second predicted position of the lane line in the lane line detection sample image is determined by the classification network according to the multi-scale sample feature map.

[0189] Optionally, the preset lane line detection model is trained in the following manner:

[0190] Obtain a multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model, and determine a first sample feature map including image global information according to the minimum-scale sample feature map in the multi-scale sample feature map;

[0191] Determine, by the segmentation network, a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map;

[0192] Determine a first loss value corresponding to a first loss function according to the first predicted position and the lane line marking position;

[0193] Determining a second predicted position of a lane line in the lane line detection sample image according to the minimum scale sample feature map by the classification network;

[0194] Determine a second loss value corresponding to a second loss function according to the second predicted position and the lane line marking position;

[0195] Determine a third loss value corresponding to a third loss function according to the first loss value and the second loss value;

[0196] When the third loss value is greater than or equal to the preset loss value threshold, the model parameters of the preset initial model are adjusted to obtain an updated target model, and the step of obtaining the multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model is executed again, until the step of determining the third loss value corresponding to the third loss function according to the first loss value and the second loss value, until it is determined that the third loss value is less than the preset loss value threshold, the segmentation network in the current target model is deleted to obtain the preset lane line detection model.

[0197] Optionally, the feature extraction network includes a feature pyramid network and a global feature extraction network, the feature pyramid network includes a bottom-up subnetwork and a top-down subnetwork, the input end of the global feature extraction network is coupled with the output end of the bottom-up subnetwork, the output end of the global feature extraction network is coupled with the input end of the top-down subnetwork, the output end of the global feature extraction network is also coupled with the input end of the segmentation network, and the output end of the top-down subnetwork is also coupled with the input end of the segmentation network, and the preset initial model is used to,

[0198] Acquire a multi-scale sample feature map corresponding to the lane line detection sample image through the bottom-up sub-network in the feature pyramid network, and input the minimum-scale sample feature map in the multi-scale sample feature map into the global feature extraction network, so that the global feature extraction network outputs the first sample feature map to the segmentation network and the top-down sub-network;

[0199] A multi-scale second sample feature map is input into the segmentation network through the top-down subnetwork, so that the segmentation network determines a first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map.

[0200] Optionally, the segmentation network determines a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map, including:

[0201] Performing upsampling processing on the first sample feature map and the second sample feature map through the segmentation network to obtain a sample feature map to be segmented;

[0202] Segmenting the sample feature map to be segmented according to a preset segmentation method to obtain multiple local feature maps;

[0203] Performing convolution and splicing processing on the multiple local feature maps to obtain a target sample feature map;

[0204] Determine a first probability that each pixel in the lane line detection sample image belongs to the lane line according to the target sample feature map;

[0205] A first predicted position of the lane line in the lane line detection sample image is determined according to the first probability of each pixel.

[0206] Optionally, the segmentation according to a preset manner includes:

[0207] The sample feature map to be segmented is divided into H C*W local feature maps, the sample feature map to be segmented is divided into C H*W local feature maps, and the sample feature map to be segmented is divided into at least one of W H*C local feature maps, wherein H is the number of layers of the sample feature map to be segmented, C is the length of the sample feature map to be segmented, and W is the width of the sample feature map to be segmented.

[0208] Optionally, the multiple local feature maps are convolved and concatenated to obtain a target sample feature map, including:

[0209] Obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain a specified local feature map after convolution;

[0210] After splicing the specified local feature map with the next local feature map corresponding to the current local feature map, an updated current local feature map is obtained;

[0211] Determine whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps;

[0212] In the case where it is determined that the next local feature map corresponding to the current local feature map is not the last one among the multiple local feature maps, the step of obtaining the current local feature map again and performing a convolution operation on the current local feature map to obtain a designated local feature map after the convolution, until determining whether the next local feature map corresponding to the current local feature map is the last one among the multiple local feature maps;

[0213] When it is determined that the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps, the current local feature map is obtained, and a convolution operation is performed on the current local feature map to obtain a specified local feature map after the convolution, and the specified local feature map corresponding to the current local feature map is used as the target sample feature map.

[0214] Optionally, the output end of the bottom-up sub-network is coupled to the input end of the classification network, and the preset initial model is used to:

[0215] Inputting the minimum scale sample feature map into the classification network through the bottom-up sub-network;

[0216] The minimum scale sample feature map is divided into multiple anchor frames through the classification network, a second probability that each anchor frame belongs to the lane line is determined, and a second predicted position of the lane line in the lane line detection sample image is determined according to the second probability.

[0217] Optionally, the determination module is configured to:

[0218] Obtaining a target feature map corresponding to the target detection image through the feature extraction network;

[0219] The target feature map is input into the classification network to obtain the target position of the lane line in the target detection image output by the classification network.

[0220] The above technical solution can effectively improve the accuracy of the model detection results by jointly training the preset lane line detection model through the segmentation network and the classification network. In order to improve the model processing speed, the segmentation network can be deleted in the application stage, and only the feature extraction network and the classification network are retained, thereby not only ensuring the accuracy of the lane line detection results, but also effectively improving the detection efficiency of the model.

[0221] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0222] The present disclosure also provides a vehicle, the vehicle comprising the above Figure 6 The lane line detection device.

[0223] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and when the program instructions are executed by a processor, the steps of the lane line detection method provided by the present disclosure are implemented.

[0224] Figure 7 800 is a block diagram of a lane detection device according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0225] Reference Figure 7 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0226] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the lane line detection method described above. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0227] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0228] The power component 806 provides power to the various components of the device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0229] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0230] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0231] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0232] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0233] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0234] In an exemplary embodiment, the device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned lane line detection method.

[0235] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of the device 800 to complete the lane line detection method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0236] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above lane line detection method when executed by the programmable device.

[0237] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0238] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A lane line detection method, characterized in that: The method comprises: Get target detection image; Inputting the target detection image into a preset lane line detection model so that the preset lane line detection model outputs the target position of the lane line in the target detection image; The preset lane line detection model is trained in the following way: Acquire a plurality of lane line detection sample images, wherein the lane line detection sample images include lane line marking positions; Inputting each lane line detection sample image into a preset initial model, wherein the preset initial model includes a segmentation network and a classification network, wherein the segmentation network is used to determine a first predicted position of a lane line in the lane line detection sample image, and the classification network is used to determine a second predicted position of a lane line in the lane line detection sample image; Training the preset initial model according to the first predicted position, the second predicted position and the lane line marking position to obtain the preset lane line detection model; The preset initial model also includes a feature extraction network, and the feature extraction network is coupled with the classification network and the segmentation network; Inputting the target detection image into a preset lane line detection model so that the preset lane line detection model outputs a target position of the lane line in the target detection image, including: Acquire a target feature map corresponding to the target detection image through the feature extraction network; The target feature map is input into the classification network to obtain the target position of the lane line in the target detection image output by the classification network.

2. The method according to claim 1, characterized in that The preset initial model further includes a feature extraction network, which is coupled with the classification network and the segmentation network. The preset initial model is used to: Acquire a multi-scale sample feature map corresponding to each lane line detection sample image through the feature extraction network, and determine a first sample feature map including image global information according to the multi-scale sample feature map; Determine, by the segmentation network, a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map; The second predicted position of the lane line in the lane line detection sample image is determined by the classification network according to the multi-scale sample feature map.

3. The method according to claim 2, characterized in that The preset lane line detection model is trained in the following way: Obtain a multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model, and determine a first sample feature map including image global information according to the minimum-scale sample feature map in the multi-scale sample feature map; Determine, by the segmentation network, a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map; Determine a first loss value corresponding to a first loss function according to the first predicted position and the lane line marking position; Determine, by the classification network, a second predicted position of a lane line in the lane line detection sample image according to the minimum scale sample feature map; Determine a second loss value corresponding to a second loss function according to the second predicted position and the lane line marking position; Determine a third loss value corresponding to a third loss function according to the first loss value and the second loss value; When the third loss value is greater than or equal to the preset loss value threshold, the model parameters of the preset initial model are adjusted to obtain an updated target model, and the step of obtaining the multi-scale sample feature map corresponding to the lane line detection sample image through the feature extraction network in the preset initial model is executed again, until the step of determining the third loss value corresponding to the third loss function according to the first loss value and the second loss value, until it is determined that the third loss value is less than the preset loss value threshold, the segmentation network in the current target model is deleted to obtain the preset lane line detection model.

4. The method according to claim 2, characterized in that: The feature extraction network includes a feature pyramid network and a global feature extraction network, the feature pyramid network includes a bottom-up sub-network and a top-down sub-network, the input end of the global feature extraction network is coupled with the output end of the bottom-up sub-network, the output end of the global feature extraction network is coupled with the input end of the top-down sub-network, the output end of the global feature extraction network is also coupled with the input end of the segmentation network, and the output end of the top-down sub-network is also coupled with the input end of the segmentation network, and the preset initial model is used for: Acquire a multi-scale sample feature map corresponding to the lane line detection sample image through the bottom-up sub-network in the feature pyramid network, and input the minimum-scale sample feature map in the multi-scale sample feature map into the global feature extraction network, so that the global feature extraction network outputs the first sample feature map to the segmentation network and the top-down sub-network; A multi-scale second sample feature map is input into the segmentation network through the top-down sub-network, so that the segmentation network determines a first predicted position of the lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map.

5. The method according to claim 4, characterized in that The segmentation network determines a first predicted position of a lane line in the lane line detection sample image according to the first sample feature map and the second sample feature map, including: Performing upsampling processing on the first sample feature map and the second sample feature map through the segmentation network to obtain a sample feature map to be segmented; Segmenting the sample feature map to be segmented according to a preset segmentation method to obtain a plurality of local feature maps; Performing convolution and splicing processing on the multiple local feature maps to obtain a target sample feature map; Determine a first probability that each pixel in the lane line detection sample image belongs to a lane line according to the target sample feature map; A first predicted position of a lane line in the lane line detection sample image is determined according to the first probability of each pixel.

6. The method according to claim 5, characterized in that The segmentation method according to the preset method includes: Divide the sample feature map to be segmented into H C*W local feature maps, divide the sample feature map to be segmented into C H*W local feature maps, and divide the sample feature map to be segmented into at least one of W H*C local feature maps, wherein H is the number of layers of the sample feature map to be segmented, C is the length of the sample feature map to be segmented, and W is the width of the sample feature map to be segmented.

7. The method according to claim 5, characterized in that The convolution and concatenation processing is performed on the multiple local feature maps to obtain a target sample feature map, including: Obtain the current local feature map, and perform a convolution operation on the current local feature map to obtain a specified local feature map after convolution; After splicing the specified local feature map with the next local feature map corresponding to the current local feature map, an updated current local feature map is obtained; Determine whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps; In the case where it is determined that the next local feature map corresponding to the current local feature map is not the last one of the multiple local feature maps, the step of obtaining the current local feature map again and performing a convolution operation on the current local feature map to obtain a designated local feature map after the convolution, until the step of determining whether the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps; When it is determined that the next local feature map corresponding to the current local feature map is the last one of the multiple local feature maps, the current local feature map is obtained, and a convolution operation is performed on the current local feature map to obtain a designated local feature map after the convolution, and the designated local feature map corresponding to the current local feature map is used as the target sample feature map.

8. The method according to claim 4, characterized in that The output end of the bottom-up sub-network is coupled to the input end of the classification network, and the preset initial model is used to: Inputting a minimum scale sample feature map into the classification network through the bottom-up sub-network; The minimum scale sample feature map is divided into a plurality of anchor frames through the classification network, a second probability that each anchor frame belongs to the lane line is determined, and a second predicted position of the lane line in the lane line detection sample image is determined according to the second probability.

9. A lane line detection device, characterized in that: The device comprises: An acquisition module is configured to acquire a target detection image; A determination module is configured to input the target detection image into a preset lane line detection model so that the preset lane line detection model outputs a target position of the lane line in the target detection image; The preset lane line detection model is trained in the following way: Acquire a plurality of lane line detection sample images, wherein the lane line detection sample images include lane line marking positions; Inputting each lane line detection sample image into a preset initial model, wherein the preset initial model includes a segmentation network and a classification network, wherein the segmentation network is used to determine a first predicted position of a lane line in the lane line detection sample image, and the classification network is used to determine a second predicted position of a lane line in the lane line detection sample image; Training the preset initial model according to the first predicted position, the second predicted position and the lane line marking position to obtain the preset lane line detection model; The preset initial model also includes a feature extraction network, and the feature extraction network is coupled with the classification network and the segmentation network; Inputting the target detection image into a preset lane line detection model so that the preset lane line detection model outputs a target position of the lane line in the target detection image, including: Acquire a target feature map corresponding to the target detection image through the feature extraction network; The target feature map is input into the classification network to obtain the target position of the lane line in the target detection image output by the classification network.

10. A vehicle, characterized in that: The vehicle comprises the lane line detection device as described in claim 9 above.

11. A lane line detection device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

Images