Multi-task collaborative perception network model and detection method for traffic road surface information

Abstract

The present invention provides a multi-task joint perception network model and detection method for traffic road surface information, which can simultaneously detect lane markings and drivable areas, and integrate a coordinate attention mechanism into the traditional feature extraction network to ensure that the amount of calculation is not increased while enhancing the effect of feature extraction. In the neck network, the present invention creatively proposes a dilated convolution residual module to enhance the network's prediction performance of details, and the decoder part shares the features of the drivable area with lane marking detection to enhance the lane detection effect in complex road conditions. In the training session, a new alternating optimization training method is proposed to improve the segmentation performance of the entire model. The present invention achieves very high accuracy and excellent speed performance in the challenging BDD100K dataset, with the mIoU of drivable area segmentation reaching 92.51%, the lane marking detection accuracy rate and IoU being 87.29% and 32.12% respectively, and the overall inference speed reaching 39 FPS, and the detection effect is better than that of the traditional single-task network model.

Description

[Technical field]

[0001] The present invention belongs to the technical field of intelligent self-driving vehicle environment perception, and a multi-task detection method is especially designed for traffic road surface information. [Background technology]

[0002] The information that smart car environment perception system needs to extract is dynamic and static obstacles and road surface environment respectively. Dynamic and static obstacles mainly refer to the dynamic or static objects that may affect the vehicle's safe driving on other vehicles, pedestrians and ground around the vehicle, while road surface environment mainly refers to the geographic environment and road traffic information around the vehicle, such as lane markings, road boundaries, etc. For unstructured roads, road environment perception can be simplified as detection of the drivable area. In autonomous driving perception system, detection of two types of traffic road surface information, lane marking detection and drivable area detection, is very important.

[0003] At present, most network models handle autonomous driving perception tasks independently. There are many deep learning methods for the lane marking detection task, and there are three main methods for lane marking detection using CNN: segmentation method, anchoring method, and curve fitting method. As for the partitioning method, a typical SCNN can realize message transmission between row pixels and column pixels by layer-by-layer convolution, and there is also SAD-Enet, which aggregates context information using a self-attention distillation module. As for the anchoring method, LaneATT proposes a new attention mechanism by anchoring to aggregate global information. UFLD is a row anchoring method that achieves ultra-fast lane marking detection speed. At the same time, this network is the first method to propose row classification. Row classification guarantees the detection speed, but the detection effect is not good, especially when lane changes cannot be detected. Curve fitting methods are different from simple point regression, but use parameters to model the lane marking curve and regress these parameters. A typical network for this method is PolyLaneNet. However, when predicting high-order coefficient errors, the curve fitting method may cause the lane marking to deform.

[0004] Semantic segmentation methods are mostly used for the processing of drivable area detection tasks, and semantic segmentation can provide pixel-level results for drivable area detection tasks. FCN is the first to introduce a fully convolutional network into the field of semantic segmentation, which extends the graphic level classification to pixel level classification, but this network lacks spatial consistency because it does not fully consider the relationship between pixels and pixel times. SegNet calculates the nonlinear upsampling of the corresponding encoder in the maximum pooling process of the decoder to omit the upsampling learning process. PSPNet integrates four features with different pyramid sizes to form a module that contains different scale information between different sub-regions, and thus can fully utilize the contextual relationship. SNE-RoadSeg is a road drivable area segmentation method by extracting surface normal vectors. This network is similar to the encoding and decoding of U-Net (U-Net proves that the use of skip connections contributes to improving the overall resolution of the space), and integrates the surface normal estimator SNE module into the upsampling part, and this method has excellent performance in terms of free space detection.

[0005] Although single-task processing solutions have a good detection effect, single-task connection processing always consumes more time than multi-task joint processing, and deployment in embedded devices often requires consideration of latency and limited computing resources. Summary of the Invention

[0006] To address this problem, the present invention proposes a multi-task joint perception network model for road surface information, RSI-Net, for simultaneously processing lane marking detection tasks and drivable area segmentation tasks, as shown in Fig. 2. This network model includes a data preprocessing module, a focus slice module, a backbone network feature extraction module, a feature fusion module, a dilated convolution residual module, a lane marking detection branching module, a drivable area segmentation branching module, and a feature sharing module.

[0007] First, the data preprocessing module performs preprocessing on the input two-dimensional image. In the preprocessing process, the brightness and geometry of the image are changed through data enhancement. The specific operations are to adjust the tone and saturation of the image, and to randomly rotate, scale, move, crop, and flip the image, and to perform normalization and boundary zero padding processing.

[0008] Then, the focus slice module performs slicing on the preprocessed graphics, collects the H and W information of the image into the channel space, converts the original image into a double downsampling image without information loss, and then transmits the processed image to the backbone network feature extraction module.

[0009] Then, the improved CSPDarkNet53 is used as the backbone network feature extraction module, and the graphics are downsampled three times so as to extract more graphics features. CSPDarkNet53 originally has good feature extraction ability, and can reduce the computation loss of the whole model without losing detection accuracy. However, lane markings and drivable areas belong to depth targets, and the network model needs to capture more direction and position perception information, so the present invention integrates a coordinate attention mechanism between the CBL module and the CSP module in CSPDarkNet53, in this way, the convoluted feature map from the CBL module is aggregated into two direction perception feature mappings along the vertical and horizontal directions respectively, and these two feature mappings with embedded direction specific information are then coded as two attention mappings respectively, and each attention mapping captures the long-distance dependency of the input feature map in one spatial direction.

[0010] After that, the feature information obtained from the image through the feature extraction network is input into the feature pyramid network FPN, which is a feature fusion module, to further fuse the information.

[0011] A dilated convolution residual module is designed after FPN, and the network model designed in the present invention is relatively lightweight and ensures the inference speed, but at the same time, in the case of the model receptive field being inevitably insufficient, the present invention is designed to add a dilated convolution residual module to strengthen the receptive field of the model, so that each convolution output contains a large range of information.This module uses a 4-layer 3×3 convolution, and the expansion rate of this 4-layer convolution is cumulatively adjusted to strengthen the long-distance information linkage of the drivable area segmentation task and the lane marking detection task.

[0012] At the back end of the dilated convolution residual module are two decoders, namely, the lane marking detection branch module and the drivable area segmentation branch module. These two decoders respectively upsample the image features output from the neck network (i.e., the dilated convolution residual module) three times to reduce the image to the original size of 1280x720.

[0013] In the present invention, a feature sharing module is also designed between these two decoders, and an Add operation is performed on the feature map upsampled the first time in the drivable area division branching module and the feature map upsampled the first time in the lane marking line detection branching module, and then the feature map that has been Add operated is input into the lane marking line detection branching module for a second upsampling. Because the drivable area and lane marking lines are intertwined targets, the feature sharing module can be used to share feature information from the drivable area division branching module with the lane marking line detection branching module task, providing useful information for lane marking line detection and improving the accuracy of lane marking line detection.

[0014] In the whole network model training process, the traditional end-to-end training method is the simplest but takes a lot of calculation and training time. Therefore, in order to shorten the training time and optimize the parameters, a new alternating optimization algorithm is designed in the present invention to train the model of the present invention step by step.

[0015] The present invention relates to Step 1 is image pre-processing; Step 2 performs a slice process on the preprocessed image; Step 3 extracts features of the fusion coordinate attention mechanism for the sliced ​​images from step 2; Step 4 performs feature fusion on the extracted features to enhance the receptive field; Step 5 performs drivable area segmentation and lane marking detection on the fused features, samples the features with enhanced receptive fields three or more times, and reduces the image to its original size; A multi-task detection method for traffic road surface information is further presented, which includes step 6 of feature sharing the feature map obtained in step 5, performing an Add operation on the first upsampled feature map for drivable area division and the first upsampled feature map for lane marking detection, and then inputting the Add-operated feature map into the lane marking detection branch for a second upsample.

[0016] Furthermore, the pre-processing in step 1 includes performing size adjustment on the input image and label, adjusting the tone and saturation of the image, and performing random rotation, scaling, translation, cropping, and flipping on the image, as well as normalization and border zero padding to obtain a digital image with three channels.

[0017] Furthermore, in the slicing process of step 2, the value of every other pixel is obtained from one photo, and like proximity downsampling, four photos are obtained from one photo, and the information in the W and H directions can be collected in the channel space, so that the input channel is expanded to four times the original size, and finally a double downsampling diagram without information loss can be obtained.

[0018] Furthermore, in the feature extraction of step 3, an improved CSPDarknet53 is used to extract photo features, and the sliced ​​image is downsampled three times. The improvement of the CSPDarkNet53 is that a coordinate attention mechanism is integrated between the CBL module and the CSP module, and the coordinate attention mechanism is improved based on two attention mechanisms, namely SE and CBAM. The coordinate attention mechanism aggregates the input features along the vertical and horizontal directions respectively into two independent direction perception feature mappings through two independent parallel 1D global pooling operations. These two feature mappings with embedded direction specific information are then coded as two attention mappings respectively, and each attention mapping captures the long-distance dependency of the input feature map in one spatial direction. Through the above operation, the location information can be stored in the generated attention mapping. Then, the two attention mappings are applied to the input feature mapping by multiplication to highlight the preference indication.

[0019] Furthermore, in the feature fusion of step 4, a dilated convolution residual module is added at the end of the feature pyramid network to fuse multi-scale feature information; a dilation rate is introduced into the convolution process, and the parameter value of the dilation rate is set to control the distance between the points of the convolution kernel; when the dilation rate d is 1, the 3×3 dilated convolution corresponds to the reference convolution, and the receptive field of the single layer is only 3×3 in size; when the dilated convolution d is 3, the receptive field of the single layer is enlarged to 7×7 in size; here, a 4-layer 3×3 convolution is used as the dilated convolution residual module, and the dilation rate of the 4-layer convolution is cumulatively adjusted to enhance the long-distance information linkage of the drivable area segmentation task and the lane marking detection task; and three dilated convolution residual modules are used to enhance the receptive field of the model, so that each convolution output contains a large range of information.

[0020] Beneficial Effects of the Invention: 1) This invention presents a multi-task collaborative perception network model RSI-Net that focuses on road surface information to simultaneously detect lane markings and drivable areas. 2) The present invention integrates the coordinate attention mechanism into the traditional feature extraction network to ensure that the computational complexity is not increased, while at the same time enhancing the feature extraction effect. 3) In the present invention, a dilated convolution residual module is creatively proposed in the neck network to enhance the network's detailed prediction performance, and the decoder part shares the features of the drivable area with lane marking detection, thereby enhancing the effect of lane marking detection in complex road conditions. 4) In the present invention, a new alternating optimization training method is presented to improve the segmentation performance of the entire model during the training session. 5) The model of the present invention achieves extremely high accuracy and excellent speed performance on the challenging BDD100K dataset, with the mIoU of drivable area segmentation reaching 92.51%, the lane marking detection accuracy rate and IoU reaching 87.29% and 32.12%, respectively, and the overall inference speed reaching 39 FPS, making the detection effect superior to that of traditional single-task network models. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a flow chart showing an embodiment of the invention. [Diagram 2] Figure 2 shows the overall framework of the RSI-Net network. [Diagram 3] FIG. 3 is a schematic diagram of a focus slice. [Figure 4] FIG. 4 is a configuration diagram of a CSP module. [Diagram 5] Figure 5 shows the structure of the coordinate attention mechanism. [Figure 6] Figure 6 shows a schematic diagram of the receptive fields at different expansion rates. [Figure 7] FIG. 7 is a diagram showing the structure of the dilated convolution residual. [Figure 8] FIG. 8 is a structural diagram of the feature sharing module. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The present invention provides a multi-task joint perception network model and detection method for traffic road surface information, which can simultaneously detect lane markings and drivable areas, and integrate a coordinate attention mechanism into the traditional feature extraction network to ensure that the amount of calculation is not increased while enhancing the effect of feature extraction. In the neck network, the present invention creatively proposes a dilated convolution residual module to enhance the network's prediction performance of details, and the decoder part shares the features of the drivable area with lane marking detection to enhance the lane detection effect in complex road conditions. In the training session, a new alternating optimization training method is proposed to improve the segmentation performance of the entire model. The present invention achieves very high accuracy and excellent speed performance in the challenging BDD100K dataset, with the mIoU of drivable area segmentation reaching 92.51%, the lane marking detection accuracy rate and IoU being 87.29% and 32.12% respectively, and the overall inference speed reaching 39 FPS, and the detection effect is better than that of the traditional single-task network model.

[0023] The present invention will now be further described with reference to the drawings.

[0024] As shown in FIGS. 1 and 2, the implementation of the present invention includes the following steps.

[0025] Step 1: Data Preprocessing First, prepare the BDD100K dataset, which is an annotated dataset for model training. During model training, perform image preprocessing in the following manner: (1) Load the dataset from a local file; (2) Load the label file from a local file; (3) Perform size adjustment on the input images and labels; (4) Adjust the tone and saturation of the images to enhance model robustness, and perform random rotation, scaling, translation, cropping, and inversion on the images; (5) Normalize and zero-pad the boundaries to obtain digital images with three channels.

[0026] Step 2: Slice the image using the Focus Slice module. Before feature extraction, the present invention uses the Focus module to slice the graphics, as shown in Figure 3, which is a schematic diagram of the focus slice operation, in which the value of every other pixel is obtained from one picture, and like the proximity downsampling, four pictures can be obtained from one picture, and the information of the W and H directions can be collected in the channel space, and the input channel is expanded four times as large as the original, and finally a double downsampling picture without information loss can be obtained. The main function of the module is to improve the receptive field without losing the original information, which is not for increasing the accuracy of the network, but for reducing the amount of calculation of the model and increasing the inference speed of the model.

[0027] Step 3: Feature extraction for fusion coordinate attention mechanism In the present invention, the improved CSPDarknet53 is used in the backbone network to present photo features, and the sliced ​​image is downsampled three times. The conventional CSPDarkNet53 has excellent feature extraction ability in YOLOv4, and the advantages of using this network structure as the backbone network in the network model of the present invention mainly include two points: first, to improve the feature extraction ability of the convolutional network without losing detection accuracy, and to improve the detection speed of the entire model; second, to reduce the calculation loss of the entire model, and to allow the model to be efficiently learned and trained even on a simple CPU.

[0028] CSPDarkNet53 mainly contains two modules: CBL module and CSP module. As shown in Figure 4, this is a schematic diagram of the CSP module structure. The CBL module is composed of a convolution layer, a standard regularization function, and a Leaky_relu activation function. The CSP module learns the idea of ​​the CSPNet network structure, and first divides the feature mapping of the base layer into two parts, and then combines them through a cross-stage hierarchical structure to reduce the amount of calculation and ensure high accuracy.

[0029] In the present invention, as an improvement over the conventional CSPDarkNet53, a coordinate attention mechanism is integrated between the CBL module and the CSP module. As shown in FIG. 5, a two-dimensional structure diagram of the coordinate attention mechanism is shown. The coordinate attention mechanism is an improvement based on two attention mechanisms, namely SE and CBAM. The SE attention mechanism only focuses on the connections between channels and ignores location information, which is very important for capturing target structures in visual tasks, and the CBAM takes the maximum and average values ​​as weighting coefficients for multiple channels at each location, so such weighting only takes into account local range information, and therefore long range information cannot be obtained. In order to avoid the loss of location information caused by 2D pooling, the coordinate attention mechanism aggregates the input features along the vertical and horizontal directions into two independent direction perception feature mappings, respectively, through two independent parallel 1D global pooling operations, and these two feature mappings with embedded direction specific information are then coded as two attention mappings, respectively, and each attention mapping captures the long-distance dependency of the input feature map in one spatial direction. Through the above operations, the position information can be stored in the generated attention mapping, and then the two attention mappings are applied to the input feature mapping by multiplication to highlight the preferred indication. This method can not only obtain cross-channel feature information, but also capture direction perception information and position perception information. Since lane markings and drivable areas belong to depth targets, integrating a coordinate attention mechanism into the feature extraction network helps the model to more accurately identify and locate lane marking targets on the road surface, and improve the effect of dividing the drivable area.

[0030] Step 4: Design the neck network and perform feature fusion. For the neck network, the feature fusion module used in the present invention is a typical feature pyramid network, and at the same time, the present invention also combines the dilated convolution residual module proposed by the present invention at the end of the neck network. Since the lane markings belong to targets with elongated structures, and the drivable area belongs to space depth targets, this requires that the network model of the present invention needs both high-level semantic features and local detailed features, so the present invention uses a feature pyramid network to fuse multi-scale feature information. The CSP module in the neck network of the present invention replaces the original multiple residual keys with multiple CBL modules to reduce the computational complexity of the network model and enhance the overall reasoning speed.

[0031] The network model designed in the present invention is relatively lightweight, and guarantees the inference speed, but at the same time, the model receptive field is inevitably insufficient.In order to meet the situation, the present invention adds dilation convolution to enhance the receptive field of the model, so that each convolution output contains a large range of information.Dilation convolution refers to the introduction of dilation rate in the convolution process, and the distance between the points of the convolution kernel is controlled by setting the parameter value of dilation rate.Compared with ordinary convolution, when the size of the convolution kernel is the same, the parameter amount does not change, but the receptive field of dilation convolution is larger, and the size of the receptive field of single-layer dilation convolution is as follows:

number

[0032] In the formula, k is the size of the receptive field of the original convolution kernel, and k d is the size of the receptive field of the dilated convolution kernel, and d is the magnitude of the dilation rate.

[0033] As shown in Figure 6, when the dilation factor d is 1, the 3 × 3 dilated convolution corresponds to the reference convolution, and the receptive field of the single layer is only 3 × 3 in size; when the dilation factor d is 3, the receptive field of the single layer increases to 7 × 7 in size.

[0034] The present invention proposes a residual module by dilated convolution, in which a four-layer 3×3 convolution is used, and the dilation rate of the four-layer convolution is cumulatively adjusted to enhance the long-distance information linkage of the drivable area segmentation task and the lane marking detection task, as shown in Fig. 7. In the present invention, three dilated convolution residual modules are used in the encoding network to enhance the receptive field of the model, thus improving the detection accuracy of the network model, which plays an especially important role in the detection of long-distance lane markings and drivable areas.

[0035] Step 5: Dividing the drivable area and branching for lane marking detection The decoder part includes a drivable area division head and a lane marking detection division head, and the same network structure is used for these two division heads. In the present invention, the lowest layer of the dilated convolution residual module is fed into two division branches, and the size is (W / 8, H / 8, 256). After three times of upsampling, the present invention restores the feature map output from the neck network to a size of (W, H, 2). In order to save computational cost, the present invention uses the nearest interpolation method in the upsampling layer, rather than the general deconvolution.

[0036] Step 6: Feature sharing In addition, the present invention also designs a feature sharing process, which is shown in Fig. 8, which is a schematic diagram of the feature sharing module operation. In the present invention, the first upsampled feature map of the drivable area division head and the first upsampled feature map of the lane marking detection head are added, and then the feature map that has been added is input to the lane marking detection head for a second upsample. The following formulas respectively interpret the Add operation.

number

[0037] In the formula, X and Y each represent a channel, i represents the i-th channel, c represents the number of channels, and K represents the weight of the corresponding channel.

[0038] Since the drivable area and lane markings are intertwined targets, a feature sharing module can be used to share feature information of the divided portion of the drivable area with the lane marking detection task, providing useful information for lane marking detection and improving the accuracy of lane marking detection.

[0039] Step 7: Model training and loss function configuration In our network, there are two decoders, and the multitask loss is the drivable region segmentation loss.

number

number

number

number

[0040] Here, TP represents the positive samples predicted as the positive class by the model, FP represents the negative samples predicted as the positive class by the model, and FN represents the positive samples predicted as the negative class by the model.

[0041]

number

number

[0042] In essence, the net loss of the present invention is a two-part weighting, as shown in the equation.

number

[0043] where γ1 and γ2 are adjustment factors to balance all parts of the total loss.

[0044] Since the model of the present invention is a multi-task joint detection network, in the training process, the present invention tries different training methods to train the model of the present invention. The end-to-end training method is the simplest, but it takes a lot of calculation and training time. Therefore, in order to shorten the training time and optimize the parameters, the present invention designs a new alternating optimization algorithm to train the model of the present invention step by step. At each step, the model can focus on one or more related tasks without considering unrelated tasks. As shown in Table 1, the training strategy of the present invention specifically includes step 1 of training the encoder and drivable area segmentation head, step 2 of freezing the training weights of the drivable area segmentation head and unlocking the lane marking detection segmentation head, step 3 of training the encoder and lane marking detection segmentation head, step 4 of jointly training the drivable area and lane marking detection segmentation head, and step 5 of repeating the above training steps until the effect is the best.

[0045] [Table 1]

[0046] The above detailed description is merely a specific description of the possible embodiments of the present invention, and is not intended to limit the protection scope of the present invention, and any equivalent method or modification that does not deviate from the technology of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-task collaborative perception network model for traffic road surface information, comprising: A data pre-processing module performs pre-processing on the input two-dimensional image, and in the pre-processing process, changes the brightness and geometry of the image through data enhancement, specifically adjusting the tone and saturation of the image, and randomly rotating, scaling, moving, cropping, and flipping the image, and performing normalization and border zero padding processing on the image; A focused slice module performs slicing on the preprocessed graphics, collects H and W information in the image into a channel space, converts the original image into a double downsampling image without information loss, and then transmits the processed image to the backbone network feature extraction module; A backbone network feature extraction module that downsamples the sliced ​​image three times to extract more image features; a feature fusion module that receives the graphics features output from the backbone network feature extraction module and further fuses the information; A dilated convolution residual module, located after the feature fusion module, is used to enhance the receptive field of the model so that each convolution output contains a large range of information; a lane marking detection branching module and a drivable area division branching module located at the rear end of the dilated convolution residual module, the lane marking detection branching module and the drivable area division branching module respectively upsampling the image features output from the dilated convolution residual module three times to reduce the image to its original size; a feature sharing module that is located between the lane marking detection branching module and the drivable area division branching module, performs an add operation on the feature map that is upsampled a first time by the drivable area division branching module and the feature map that is upsampled a first time by the lane marking detection branching module, inputs the added feature map into the lane marking detection branching module, and performs a second upsample operation. A multi-task collaborative perception network model for traffic road surface information characterized by:

2. The multi-task joint perception network model for traffic road surface information as claimed in claim 1, comprising a backbone network feature extraction module using an improved CSPDarkNet 53, in which the present invention integrates a coordinate attention mechanism between the CBL module and the CSP module in the CSPDarkNet 53, in such a way that the convolved feature map from the CBL module is aggregated into two direction perception feature mappings along the vertical and horizontal directions respectively, and these two feature mappings with embedded direction specific information are then coded as two attention mappings respectively, and each attention mapping captures the long-distance dependency of the input feature map in one spatial direction.

3. The multi-task joint perception network model for traffic road surface information as claimed in claim 1, characterized in that it comprises a feature fusion module in which a feature pyramid network FPN is used.

4. The multi-task joint perception network model for traffic road surface information according to claim 1, characterized in that a four-layer 3x3 convolution is used, and the dilation rate of the four-layer convolution is provided with a dilated convolution residual module that is cumulatively adjusted to strengthen the long-distance information coordination of the drivable area segmentation task and the lane marking detection task.

5. The loss function of the model is a drivable area division loss function [0089] and lane marking division loss function ##EQU00011## These two loss functions include the cross-entropy Logits ##EQU00012## The loss function includes the IoU loss function. ##EQU00013## is added, And then, ##EQU00014## are designed as follows, ##EQU00015## The final loss function of the model is the weighted sum of the two loss functions, ##EQU00016## Here, γ １ and γ ２ is the adjustment factor, An alternating optimization algorithm is used to train the model, specifically: Step 1 of training the encoder and drivable area segmentation head; Step 2: freezing the training weights of the drivable area segmentation head and unlocking the lane marking detection segmentation head; Step 3 of training the encoder and lane marking detection segmentation head; Step 4 of jointly training drivable area and lane marking detection segmentation heads; and step 5 of repeating steps 1 to 4 until the effect is the best. The multi-task joint perception network model for traffic road surface information according to any one of claims 1 to 4.

6. A computer-implemented multi-task detection method for traffic road surface information, comprising: Step 1 is image pre-processing; Step 2: performing a slice process on the pre-processed image; Step 3 extracts features of the fusion coordinate attention mechanism for the sliced ​​images from step 2; Step 4 performs feature fusion on the extracted features to strengthen the receptive field; Step 5: performing drivable area segmentation and lane marking detection on the fused features, sampling the features with enhanced receptive fields three or more times, and reducing the image to its original size; and step 6 of sharing features of the feature map obtained in step 5, adding the first upsampled feature map for drivable area division and the first upsampled feature map for lane marking detection, and then inputting the added feature map into a lane marking detection branch for a second upsample.

7. The multi-task detection method for traffic road surface information as described in claim 6, characterized in that the pre-processing in step 1 includes performing size adjustment processing on the input image and label, adjusting the tone and saturation of the image, and performing random rotation, scaling, translation, cropping, and inversion processing on the image, and performing normalization and boundary zero padding processing to obtain a digital image having three channels.

8. The multi-task detection method for traffic road surface information as described in claim 6, characterized in that in the slicing process of step 2, the value of every other pixel is obtained from one image, and four images can be obtained from one image like proximity downsampling, the information of the W and H directions can be collected in the channel space, the input channel is expanded four times as original, and finally a double downsampling image without information loss can be obtained.

9. 7. The method for detecting traffic road surface information according to claim 6, wherein in the feature extraction of step 3, an improved CSPDarkNet 53 is used to extract photo features, and the sliced ​​image is downsampled three times; the improvement of the CSPDarkNet 53 is to integrate a coordinate attention mechanism between the CBL module and the CSP module, and the coordinate attention mechanism is improved based on two attention mechanisms, namely, SE and CBAM; the coordinate attention mechanism is to aggregate the input features along the vertical and horizontal directions respectively into two independent direction perception feature mappings by two independent parallel 1D global pooling operations; these two feature mappings with embedded direction specific information are then coded as two attention mappings respectively; each attention mapping captures the long-distance dependency of the input feature map in one spatial direction; and the location information can be stored in the generated attention mapping through the 1D global pooling operation; and then the two attention mappings are applied to the input feature mapping by multiplication to highlight preference indications.

10. 7. The method of claim 6, wherein in the feature fusion of step 4, a dilated convolution residual module is added at the end of the feature pyramid network to fuse multi-scale feature information; a dilation factor is introduced into the convolution process; a parameter value of the dilation factor is set to control the distance between points of the convolution kernel; when the dilation factor d is 1, the 3×3 dilated convolution corresponds to the reference convolution, and the receptive field of the single layer is only 3×3 in size; when the dilated convolution d is 3, the receptive field of the single layer is enlarged to 7×7 in size; wherein, a 4-layer 3×3 convolution is used as the dilated convolution residual module; the dilation factor of the 4-layer convolution is cumulatively adjusted to enhance the long-distance information linkage of the drivable area segmentation task and the lane marking detection task; and three dilated convolution residual modules are used to enhance the receptive field of the model, so that each convolution output contains a large range of information.

Images