A multi-task joint perception network model and detection method for traffic road surface information

Abstract

The application discloses a kind of multi-task joint perception network model and detection method for traffic road pavement information, lane line and drivable area can be detected simultaneously, coordinate attention mechanism is integrated into traditional feature extraction network, to ensure that the calculation amount is not increased while enhancing feature extraction effect, in neck network, the application innovatively proposes an inflation convolution residual module to enhance the prediction performance of network details, and the features of the drivable area are shared to the lane line detection in the decoder part to enhance the lane line detection effect under complex road conditions.In the training session, a novel alternating optimization training method is proposed to improve the overall segmentation performance of the model.The application achieves very high accuracy and excellent speed performance in the challenging BDD100K dataset, with a drivable area segmentation mIoU of 92.51%, a lane line detection accuracy of 87.29%, and an IoU of 32.12%.The overall inference speed is 39FPS, and the detection effect is better than that of traditional single-task network model.

Description

Technical Field

[0001] This invention belongs to the field of environmental perception technology for intelligent autonomous vehicles, and specifically designs a multi-task detection method for traffic road surface information. Background Technology

[0002] The environmental perception system of an intelligent vehicle needs to extract information from both dynamic and static obstacles and the road surface environment. Dynamic obstacles mainly refer to other vehicles, pedestrians, and dynamic or static objects on the ground that may affect the vehicle's safe driving. The road surface environment mainly refers to the geographical environment and road traffic information around the vehicle, such as lane lines and road boundary lines. For unstructured roads, road environment perception can be simplified to the detection of drivable areas. In autonomous driving perception systems, the detection of lane lines and drivable areas is crucial.

[0003] Currently, most network models are designed for autonomous driving perception tasks alone. There are many deep learning-based solutions for lane detection. CNN-based lane detection mainly falls into three categories: segmentation-based methods, anchoring-based methods, and curve fitting-based methods. Segmentation-based methods include the classic SCNN, which uses layer-by-layer convolution to pass messages between row and column pixels, and SAD-Enet, which uses a self-attention distillation module to aggregate contextual information. Anchoring-based methods include LaneATT, which proposes a novel anchor-based attention mechanism to aggregate global information; and UFLD, which uses row anchors to achieve ultra-fast lane detection speeds. This network was also the first to propose row classification, which, while ensuring detection speed, doesn't provide very good results, especially when vehicles are changing lanes, where it may fail to detect them. Curve fitting-based methods differ from simple point regression; they use parameters to model the lane line curve and then regress these parameters. A classic example of this method is PolyLaneNet. However, curve fitting methods can lead to lane line distortion when predicting higher-order coefficient errors.

[0004] Most methods for drivable region detection employ semantic segmentation, which provides pixel-level results. FCN was the first to introduce fully convolutional networks into semantic segmentation, extending image-level classification to pixel-level classification. However, this network doesn't adequately consider the temporal relationship between pixels, resulting in a lack of spatial consistency. SegNet eliminates the need for a separate upsampling learning process by calculating the corresponding nonlinear upsampling in the encoder during the decoder's max-pooling process. PSPNet integrates features from four different pyramid sizes to form a module containing information at different scales between sub-regions, thus fully utilizing contextual relationships. SNE-RoadSeg is a road drivable region segmentation method based on surface normal vector extraction. This network is similar to U-Net (which demonstrated that skip connections improve overall spatial resolution) in its encoding and decoding, incorporating a surface normal estimator (SNE) module in the upsampling part. This method performs exceptionally well in free space detection.

[0005] Although single-task processing has achieved good detection results, processing a single task sequentially inevitably takes longer than processing multiple tasks together. Deploying it on embedded devices often requires consideration of latency and limited computing resources. Summary of the Invention

[0006] To address this problem, this invention proposes a multi-task joint sensing network model, RSI-Net, for road surface information, such as... Figure 2 As shown, this network model can be used to simultaneously handle lane detection and drivable region segmentation tasks. The model includes a data preprocessing module, a focus slicing module, a backbone network feature extraction module, a feature fusion module, a dilated convolution residual module, a lane detection branch module, a drivable region segmentation branch module, and a feature sharing module.

[0007] First, the input two-dimensional image is preprocessed using the data preprocessing module. During the preprocessing process, data augmentation is used to change the image's luminance and geometry. Specifically, the image's hue and saturation are adjusted, and the image is randomly rotated, scaled, translated, cropped, and flipped.

[0008] Then, the Focus slicing module is used to slice the preprocessed image, concentrating the information of the H and W dimensions in the image into the channel space, thereby turning the original image into a double-downsampled image without information loss. Next, the processed image is transmitted to the backbone network feature extraction module.

[0009] Next, the improved CSPDarkNet53 is used as the backbone network feature extraction module to perform three downsampling operations on the image to extract more image features. CSPDarkNet53 already has excellent feature extraction capabilities, which can reduce the computational cost of the entire model without sacrificing detection accuracy. However, lane lines and drivable areas are deep targets, requiring the network model to capture more perceptual information about direction and position. Therefore, this invention integrates a coordinate attention mechanism between the CBL and CSP modules in CSPDarkNet53. In this way, the feature maps convolved from the CBL module are aggregated into two separate direction-aware feature maps along the vertical and horizontal directions, respectively. These two feature maps with embedded direction-specific information are then encoded into two attention maps, each capturing the long-distance dependencies of the input feature map along a spatial direction.

[0010] The feature information obtained from the image through the feature extraction network is then input into the feature fusion module, namely the Feature Pyramid Network (FPN), to further fuse the information.

[0011] A dilated convolutional residual module is designed after the FPN. Since the network model designed in this invention is relatively lightweight, while ensuring inference speed, it inevitably leads to insufficient receptive field. Therefore, this invention incorporates a dilated convolutional residual module to enhance the model's receptive field, ensuring that each convolutional output contains a larger range of information. This module uses four 3×3 convolutional layers, with the dilation rates of these four layers increasing in sequence to enhance the long-distance information connection between drivable region segmentation and lane detection tasks.

[0012] The back end of the dilated convolutional residual module consists of two decoders: a lane detection branch module and a drivable region segmentation branch module. These two decoders upsample the image features output from the neck network (i.e., the dilated convolutional residual module) three times to restore the image to its original size of 1280×720.

[0013] This invention also incorporates a feature-sharing module between the two decoders. This module performs an Add operation on the feature map first upsampled by the drivable region segmentation branch module and the feature map first upsampled by the lane detection branch module. The added feature map is then input into the lane detection branch module for a second upsampling. Since the drivable region and lane lines are intersecting targets, the feature-sharing module allows the feature information from the drivable region segmentation branch module to be shared with the lane detection branch module, thereby providing valuable information for lane detection and improving its accuracy.

[0014] In the entire network model training process, the traditional end-to-end training method is the simplest, but it consumes a lot of computation and training time. Therefore, in order to compress training time and optimize parameters, this invention designs a brand-new alternating optimization algorithm to gradually train the model of this invention.

[0015] This invention also proposes a multi-task detection method for traffic road surface information, comprising the following steps:

[0016] S1 image preprocessing;

[0017] S2 performs slicing on the preprocessed image;

[0018] S3 performs feature extraction using a fusion coordinate attention mechanism on the image sliced ​​by S2.

[0019] S4 performs feature fusion on the extracted features and enhances the receptive field;

[0020] S5 performs drivable region segmentation and lane line detection on the fused features, and performs three upsampling operations on the features that enhance the receptive field to restore the image to its original size;

[0021] S6 performs feature sharing on the feature map obtained from S5, adds the feature map obtained from the first upsampling during the segmentation of the drivable region to the feature map obtained from the first upsampling during lane detection, and then inputs the feature map after the Add operation into the lane detection branch for a second upsampling.

[0022] Furthermore, the preprocessing of S1 includes: resizing the input image and label, adjusting the image's hue and saturation, and randomly rotating, scaling, translating, cropping, and flipping the image, as well as normalizing and zero-padding the edges to obtain a digital image with three channels.

[0023] Furthermore, the slicing process in S2 obtains a value every pixel in an image, similar to nearest-neighbor downsampling. This process generates four images from one image, concentrating the W and H dimension information into the channel space. The input channel is expanded to four times its original size, ultimately resulting in a double-downsampled image without information loss.

[0024] Furthermore, the feature extraction in S3 involves using an improved CSPDarknet53 to extract image features. The sliced ​​image is downsampled three times. The improvement of CSPDarkNet53 is the integration of a coordinate attention mechanism between the CBL and CSP modules. The coordinate attention mechanism is an improvement on the SE and CBAM attention mechanisms. The coordinate attention mechanism uses two independent parallel 1D global pooling operations to aggregate the input features into two separate orientation-aware feature maps along the vertical and horizontal directions, respectively. These two feature maps with embedded orientation-specific information are then encoded into two attention maps. Each attention map captures the long-distance dependencies of the input feature map along a spatial direction. After the above operations, the positional information can be stored in the generated attention map. Then, the two attention maps are applied to the input feature map through multiplication to emphasize the representation of interest.

[0025] Furthermore, the feature fusion in S4 incorporates multi-scale feature information by adding a dilated convolutional residual module at the end of the feature pyramid network. A dilation rate is introduced during convolution, and the spacing between points in the convolution kernel is controlled by setting the dilation rate parameter. When the dilation rate d is 1, a 3×3 dilated convolution is equivalent to a standard convolution, with a single-layer receptive field of only 3×3. When the dilation rate d is 3, the single-layer receptive field increases to 7×7. The dilated convolutional residual module employs four layers of 3×3 convolutions, with the dilation rates of these four layers increasing sequentially to enhance the long-distance information connection between the drivable area segmentation task and the lane detection task. Three dilated convolutional residual modules are used to enhance the model's receptive field, ensuring that each convolution output contains a larger range of information.

[0026] The beneficial effects of this invention are:

[0027] 1) This invention proposes a multi-task joint perception network model RSI-Net that focuses on road surface information, which can be used to simultaneously detect lane lines and drivable areas.

[0028] 2) This invention integrates a coordinate attention mechanism into a traditional feature extraction network, ensuring enhanced feature extraction performance without increasing computational cost.

[0029] 3) In the neck network, this invention innovatively proposes a dilated convolutional residual module to enhance the network's prediction performance for details, while the decoder part shares the features of the drivable area with lane line detection to enhance the lane line detection effect under complex road conditions.

[0030] 4) In the training phase, this invention proposes a novel alternating optimization training method to improve the overall segmentation performance of the model.

[0031] 5) The model of this invention achieved very high accuracy and excellent speed performance on the challenging BDD100K dataset. The mIoU of drivable region segmentation reached 92.51%, the lane detection accuracy and IoU were 87.29% and 32.12% respectively, and the overall inference speed reached 39 FPS. The detection performance is better than that of traditional single-task network models. Attached Figure Description

[0032] Figure 1 Flowchart for Invention Implementation

[0033] Figure 2 Overall framework diagram of RSI-Net network

[0034] Figure 3 Schematic diagram of Focus slice

[0035] Figure 4 CSP module structure diagram

[0036] Figure 5 A diagram of the coordinate attention mechanism structure

[0037] Figure 6 Schematic diagram of receptive field under different expansion rates

[0038] Figure 7 Diagram of dilated convolution residual structure

[0039] Figure 8 Feature sharing module structure diagram Detailed Implementation

[0040] This invention proposes a multi-task joint perception network model and detection method for traffic road surface information. It can simultaneously detect lane lines and drivable areas. A coordinate attention mechanism is integrated into the traditional feature extraction network to enhance feature extraction performance without increasing computational load. In the neck network, this invention innovatively proposes a dilated convolutional residual module to enhance the network's prediction performance for details. The decoder shares features from the drivable area with lane line detection to improve lane line detection performance under complex road conditions. In the training phase, a novel alternating optimization training method is proposed to improve the overall segmentation performance of the model. This invention achieves very high accuracy and excellent speed performance on the challenging BDD100K dataset. The mIoU for drivable area segmentation reaches 92.51%, and the accuracy and IoU for lane line detection are 87.29% and 32.12%, respectively. Furthermore, the overall inference speed reaches 39 FPS, outperforming traditional single-task network models.

[0041] The invention will now be further described with reference to the accompanying drawings.

[0042] like Figure 1, 2 As shown, the implementation of the present invention includes the following steps:

[0043] Step 1: Data Preprocessing

[0044] First, prepare the labeled dataset for model training: the BDD100K dataset. During model training, image preprocessing is performed using the following methods: (1) Load the dataset from a local file. (2) Load the label file from a local file. (3) Resize the input images and labels. (4) Adjust the hue and saturation of the images and perform random rotation, scaling, translation, cropping, and flipping to enhance the robustness of the model. (5) Normalize and zero-padding the edges to obtain a digital image with three channels.

[0045] Step 2: Use the Focus slicing module to slice the image.

[0046] Before feature extraction, this invention used the Focus module to slice the image, such as... Figure 3 The diagram illustrates the Focus slicing operation. This operation obtains a value for every single pixel in an image, similar to nearest-neighbor downsampling. By obtaining four images from one image, the W and H dimensions of information are concentrated into the channel space, expanding the input channels fourfold. The final result is a double-downsampled image without information loss. The primary function of this module is to increase the receptive field without losing original information. It does not increase the network's accuracy but rather reduces the model's computational load and improves its inference speed.

[0047] Step 3: Feature Extraction Using the Coordinate Attention Mechanism

[0048] In the backbone network, this invention uses an improved CSPDarkNet53 to extract image features, performing three downsampling operations on the sliced ​​image. The traditional CSPDarkNet53 has demonstrated excellent feature extraction capabilities in YOLOv4. Using this network structure as the backbone network in this invention offers two main advantages: firstly, it enhances the feature extraction capabilities of the convolutional network without sacrificing detection accuracy, thereby improving the overall detection speed of the model; secondly, it reduces the computational cost of the entire model, enabling efficient learning and training even on a simple CPU.

[0049] CSPDarkNet53 mainly consists of two modules: the CBL module and the CSP module, such as... Figure 4The diagram shows the structure of the CSP module. The CBL module consists of convolutional layers, standard regularization, and the Leaky_relu activation function. The CSP module, which borrows the idea of ​​the CSPNet network structure, first divides the feature mapping of the base layer into two parts, and then merges them through a cross-stage hierarchical structure, which reduces the amount of computation while ensuring high accuracy.

[0050] The improvement of this invention over the traditional CSPDarkNet53 is the integration of a coordinate attention mechanism between the CBL and CSP modules, such as... Figure 5 The diagram shows the two-dimensional structure of the coordinate attention mechanism. The coordinate attention mechanism is an improvement upon the SE and CBAM attention mechanisms. The SE attention mechanism only focuses on the connections between channels, ignoring positional information, which is crucial for capturing target structure in visual tasks. CBAM uses the maximum and average values ​​of multiple channels at each position as weighting coefficients, thus only considering local information and failing to capture long-range dependencies. To avoid the loss of positional information caused by 2D pooling, the coordinate attention mechanism uses two independent parallel 1D global pooling operations to aggregate the input features into two separate orientation-aware feature maps along the vertical and horizontal directions, respectively. These two feature maps, embedded with orientation-specific information, are then encoded into two attention maps, each capturing the long-range dependencies of the input feature map along a spatial direction. After these operations, positional information is stored in the generated attention maps. The two attention maps are then multiplied and applied to the input feature map to emphasize the representation of interest. This method not only captures cross-channel feature information but also captures orientation-aware and position-aware information. Since lane lines and drivable areas are considered depth targets, incorporating a coordinate attention mechanism into the feature extraction network helps the model more accurately identify and locate lane line targets on the road surface and improve the segmentation effect of the drivable area.

[0051] Step 4: Design the neck network and perform feature fusion.

[0052] The neck network of this invention employs a classic feature pyramid network as its feature fusion module. Furthermore, this invention integrates a dilated convolutional residual module proposed in this invention at the end of the neck network. Since lane lines are elongated targets and the drivable area is a spatially deep target, the network model of this invention requires both high-level semantic features and local detailed features. Therefore, this invention uses a feature pyramid network to fuse multi-scale feature information. The CSP module in the neck network of this invention replaces the original multiple residual keys with multiple CBL modules, thereby reducing the computational load of the network model and enhancing the overall inference speed.

[0053] To address the issue of the lightweight nature of the network designed in this invention, which inevitably leads to insufficient receptive field while maintaining inference speed, this invention incorporates dilated convolution to enhance the model's receptive field, ensuring that each convolution output contains a wider range of information. Dilated convolution introduces a dilation rate during the convolution process. By setting the parameter value of the dilation rate, the spacing between points in the convolution kernel is controlled. Compared to ordinary convolution, with the same kernel size and unchanged number of parameters, dilated convolution provides a larger receptive field. The receptive field size of a single layer of dilated convolution follows the formula:

[0054] k d =k+(k-1)(d-1)#(1)

[0055] In the formula: k is the receptive field size of the original convolution kernel; k d d represents the size of the receptive field of the convolution kernel after dilation; d represents the dilation rate.

[0056] like Figure 6 As shown, when the dilation rate d is 1, the 3×3 dilated convolution is equivalent to the standard convolution, and the receptive field of a single layer is only 3×3 in size; when the dilation rate d is 3, the receptive field of a single layer increases to 7×7 in size.

[0057] This invention proposes a residual module based on dilated convolution, such as... Figure 7 As shown, four 3×3 convolutional layers are employed, with the dilation rates of these four layers increasing in sequence to enhance the long-distance information connection between drivable area segmentation and lane detection tasks. This invention uses three dilated convolutional residual modules in the encoding network to enhance the model's receptive field, thereby improving the network model's detection accuracy, especially for detecting long-distance lane lines and drivable areas.

[0058] Step 5 allows for the implementation of zone segmentation and lane detection branches.

[0059] The decoder section includes a functional region segmentation head and a lane detection segmentation head, both employing the same network structure. This invention feeds the bottom layer of the dilated convolutional residual module into two segmentation branches with a size of (W / 8, H / 8, 256). After three upsampling operations, this invention restores the feature map output from the neck network to a size of (W, H, 2). To save computational costs, this invention uses nearest-neighbor interpolation in the upsampling layers instead of conventional deconvolution.

[0060] Step 6 Feature Sharing

[0061] Furthermore, this invention also incorporates a feature-sharing mechanism, such as... Figure 8The diagram shows the operation of the feature sharing module. This invention performs an Add operation on the feature map initially upsampled by the feasible region segmentation head and the feature map initially upsampled by the lane line detection head. Then, the feature map after the Add operation is input into the lane line detection head for a second upsampling. The following formulas explain the Add operation.

[0062]

[0063] In the formula, X and Y represent channels, i represents the i-th channel, the number of channels is c, and K represents the weight of the corresponding channel.

[0064] Since the drivable area and lane lines are intertwined targets, the feature sharing module can share the feature information of the drivable area segmentation part with the lane line detection task, thereby providing useful information for lane line detection and improving the accuracy of lane line detection.

[0065] Step 7: Model Training and Loss Function Setting

[0066] Since the network of this invention has two decoders, the multi-task loss is the feasible region segmentation loss l da-seg Lane separation loss l ll-seg Both of these losses include the cross-entropy (Logits lce) loss, the purpose of which is to minimize the class error between the pixel output by the network model and the target. This invention incorporates IoU loss into the lane line loss:

[0067]

[0068] Where TP represents a positive sample predicted as positive by the model, FP represents a negative sample predicted as positive by the model, and FN represents a positive sample predicted as negative by the model.

[0069] l da-seg and l ll-seg They are defined as follows:

[0070] l da-seg =l ce #(4)

[0071] l ll-se =l ce +l IoU #(5)

[0072] In summary, the final loss of this invention is a weighted average of the two parts, as shown in the equation.

[0073] l all =γ1l da-deg +γ2l ll-s #(6)

[0074] Among them, γ1 and γ2 are adjustment factors used to balance all parts of the total loss.

[0075] Since the model of this invention is a multi-task joint detection network, different training methods are attempted to train the model during the training process. End-to-end training is the simplest, but it consumes significant computational and training time. Therefore, to compress training time and optimize parameters, this invention designs a novel alternating optimization algorithm to train the model step-by-step. In each step, the model can focus on one or more relevant tasks while ignoring irrelevant ones. As shown in Table 1, the training strategy of this invention is as follows: Step 1: Train the encoder and the drivable area segmentation head; Step 2: Freeze the training weights of the drivable area segmentation head and unlock the lane detection segmentation head; Step 3: Train the encoder and the lane detection segmentation head; Step 4: Jointly train the drivable area and lane detection segmentation heads; Step 5: Repeat the above training steps until optimal results are achieved.

[0076] Table 1

[0077]

[0078] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent methods or modifications that do not depart from the technology of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-task joint perception network model for traffic road pavement information, characterized in that, include: It includes a data preprocessing module, a focus slicing module, a backbone network feature extraction module, a feature fusion module, a dilated convolution residual module, a lane line detection branch module, a drivable area segmentation branch module, and a feature sharing module; The data preprocessing module preprocesses the input two-dimensional image. During the preprocessing process, data augmentation is used to change the image's luminance and geometry. Specifically, the operation involves adjusting the image's hue and saturation, as well as randomly rotating, scaling, translating, cropping, and flipping the image. The Focus slicing module slices the preprocessed image, concentrating the information in the H and W dimensions of the image into the channel space, thereby turning the original image into a double-downsampled image without information loss. The processed image is then transmitted to the backbone network feature extraction module. The backbone network feature extraction module performs three downsampling operations on the sliced ​​image to extract more image features; The feature fusion module receives the graphic features output by the backbone network feature extraction module, enabling further information fusion. The dilated convolutional residual module, located after the feature fusion module, is used to enhance the receptive field of the model, ensuring that each convolutional output contains a wider range of information; specifically as follows: By adding a dilated convolutional residual module at the end of the feature pyramid network, multi-scale feature information is fused. A dilation rate is introduced during the convolution process, and the spacing between points of the convolution kernel is controlled by setting the parameter value of the dilation rate. When the dilation rate d is 1, the 3×3 dilated convolution is equivalent to the standard convolution, and the receptive field of a single layer is only 3×3 in size. When the dilation rate d is 3, the receptive field of a single layer increases to 7×7. The dilated convolutional residual module uses four layers of 3×3 convolution. The dilation rates of these four layers are arranged in an increasing manner to enhance the long-distance information connection between the drivable area segmentation task and the lane line detection task. Three dilated convolutional residual modules are used to enhance the receptive field of the model, so that each convolution output contains a large range of information. The back end of the dilated convolution residual module consists of two decoders: a lane detection branch module and a drivable region segmentation branch module. These two decoders upsample the image features output from the dilated convolution residual module three times to restore the image to its original size. The feature sharing module, located between the two decoders, performs an Add operation on the feature map first upsampled by the drivable area segmentation branch module and the feature map first upsampled by the lane line detection branch module, and inputs the feature map after the Add operation into the lane line detection branch module for a second upsampling.

2. The multi-task joint perception network model for traffic road pavement information according to claim 1, wherein, The backbone network feature extraction module adopts an improved CSPDarkNet53, the application fuses a coordinate attention mechanism between a CBL module and a CSP module in the CSPDarkNet53, so that feature maps convolved from the CBL module are aggregated into two separate direction perception feature maps along a vertical direction and a horizontal direction respectively, and the two feature maps with embedded direction-specific information are then encoded into two attention maps respectively, and each attention map captures a long-distance dependency relationship of an input feature map along a spatial direction.

3. The multi-task joint perception network model for traffic road pavement information according to claim 1, wherein, The feature fusion module adopts a feature pyramid network (FPN).

4. The multi-task joint perception network model for traffic road pavement information according to claim 1, wherein, The dilated convolution residual module adopts four 3*3 convolutions, and the dilated rates of the four convolutions are in an increasing form to enhance the long-distance information connection of the drivable area segmentation task and the lane line detection task.

5. The multi-task joint perception network model for traffic road pavement information according to any one of claims 1-4, characterized in that, The loss function of the model includes a feasible region segmentation loss function. Lane segmentation loss function Both of these loss functions include cross-entropy (Logits). The loss function incorporates IoU loss into the lane loss function: then and are designed as: The final loss function of the model is a weighted sum of two loss functions: Wherein, γ1 and γ2 are adjustment factors; The training of the model adopts an alternating optimization algorithm, and the specific steps are as follows: The first step is to train the encoder and the drivable area segmentation head; The second step is to freeze the training weight of the drivable area segmentation head and unlock the lane line detection segmentation head; The third step is to train the encoder and the lane line detection segmentation head; The fourth step is to train the drivable area and the lane line detection segmentation head jointly; The fifth step is to repeat the above training steps until the best effect is obtained.

6. A multi-task detection method for traffic road surface information, characterized in that, The method comprises the following steps: S1, image preprocessing; S2, slicing the preprocessed image; S3, performing feature extraction of the fused coordinate attention mechanism on the image after the slicing in S2; S4, performing feature fusion and enhancing the receptive field of the extracted features; The feature fusion of S4 adds a dilated convolution residual module at the end of the feature pyramid network to fuse multi-scale feature information, introduces a dilated rate in the convolution process, controls the spacing between the points of the convolution kernel by setting the parameter value of the dilated rate, when the dilated rate d is 1, the 3*3 dilated convolution is equivalent to the standard convolution, and the single-layer receptive field is only 3*3; when the dilated convolution d is 3, the single-layer receptive field is increased to 7*7; wherein, the dilated convolution residual module adopts four 3*3 convolutions, the dilated rates of the four convolutions are in an increasing form to enhance the long-distance information connection of the drivable area segmentation task and the lane line detection task, and three dilated convolution residual modules are used to enhance the receptive field of the model so that each convolution output contains a larger range of information; S5, performing drivable area segmentation and lane line detection on the fused features, and performing three times of upsampling on the features with enhanced receptive field to restore the image to the original size; S6, performing feature sharing on the feature maps obtained in S5, performing Add operation on the feature maps obtained by the first time of upsampling in the drivable area segmentation and the feature maps obtained by the first time of upsampling in the lane line detection, and then inputting the feature maps after the Add operation into the lane line detection branch for the second time of upsampling.

7. The multi-task detection method for traffic road surface information according to claim 6, wherein, The preprocessing of the S1 includes: resizing the input image and label, adjusting the tone and saturation of the image, and randomly rotating, scaling, translating, cropping and flipping the image, normalizing and boundary zero padding to obtain a digital image with three channels. 8.The multi-task detection method for traffic road surface information of claim 6, wherein, The slicing processing of the S2 is to obtain a value every other pixel in a picture, similar to the adjacent down sampling, four pictures can be obtained from one picture, the W and H dimensional information can be concentrated in the channel space, the input channel is expanded to four times of the original, and finally the twice down sampling picture without information loss can be obtained. 9.The multi-task detection method for traffic road surface information of claim 6, wherein, The feature extraction of the S3: using the improved CSPDarkNet53 to extract the picture features, the sliced image is down sampled three times, the improvement of the CSPDarkNet53 is to integrate the coordinate attention mechanism between the CBL module and the CSP module, the coordinate attention mechanism is improved on the basis of SE and CBAM, the coordinate attention mechanism uses two independent parallel 1D global pooling operations, respectively along the vertical direction and the horizontal direction, the input features are aggregated into two separate direction perception feature maps, the two feature maps with embedded direction specific information are then encoded into two attention maps, each attention map captures the long distance dependence of the input feature map along one spatial direction, the position information can be saved in the generated attention map after the above operation, then the two attention maps are applied to the input feature map through multiplication to emphasize the representation of interest.

Images