A road environment perception system and method based on improved yolop

Abstract

The application relates to the fields of electronic information engineering and image processing technology, and particularly relates to a road environment perception system based on an improved YOLOP, comprising a YOLOP network model, adopting the YOLOP network model as a basic network, and setting a space and channel feature extraction strengthening extractor in the YOLOP network model; the application also provides a road environment perception method based on the improved YOLOP. The application improves the YOLOP network model by using an attention mechanism module ECA and an attention mechanism module SA, so that the YOLOP network model becomes adaptable to multi-scale targets. The application can identify multi-task targets in a closed park and a semi-closed park, and can realize multi-target detection and segmentation; after training and testing, the improved YOLOP network model achieves better effects, and realizes the lightweight, accuracy and multi-task target identification of the YOLOP network model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of electronic information engineering and image processing, and more specifically relates to a road environment perception system based on an improved YOLOP and an algorithm thereof. BACKGROUND

[0002] With the wide application of 5G technology and the rapid development of artificial intelligence, countries around the world are accelerating the innovation and practical application of intelligent robots. For trackless intelligent inspection robots in the park scene, how to quickly and accurately perceive the surrounding environment information becomes the cornerstone for the robot to make a safe and accurate route.

[0003] At present, the network model for environment perception is constantly updated and has excellent performance, but most network model development is for single target recognition task. If applied to the road multi-target perception task of robots in the park scene, the parallel execution of multiple tasks will increase the burden of hardware, which not only requires constant increase of hardware performance to increase additional costs, but also needs constant optimization of network algorithms to improve target recognition speed, therefore, the disadvantages of multi-node parallelism cannot be ignored. SUMMARY

[0004] To achieve the above purpose, the technical scheme adopted by the application is to provide a road environment perception system based on an improved YOLOP, comprising a YOLOP network model, adopting the YOLOP network model as a basic network, and setting a space and channel feature extraction enhancement extractor in the YOLOP network model, wherein the space and channel feature extraction enhancement extractor comprises an attention mechanism module ECA and a spatial attention mechanism module SA.

[0005] Optionally, the YOLOP network model comprises an image input module, a Backbone layer, a Neck layer and an output layer, the output layer comprises a target detection head, a road edge detection head and a road driving area head;

[0006] The road driving area head is used for detecting the road drivable area.

[0007] The target detection head is used for detecting vehicles and pedestrians.

[0008] The road edge detection head is used for detecting the road edge.

[0009] Two space and channel feature extraction enhancement extractors are arranged in the Backbone layer, which are a first space and channel feature extraction enhancement extractor and a second space and channel feature extraction enhancement extractor.

[0010] Optionally, the image input module is configured to collect images of the to-be-detected region and transmit the images to the Backbone layer and the spatial attention mechanism module SA outside the Backbone layer respectively, and the spatial attention mechanism module SA outside the Backbone layer transmits feature maps to the road lane detection head and the road driving area head of the output layer.

[0011] The Backbone layer is configured to process the images of the to-be-detected region to obtain feature maps of the input images, and transmit the feature maps to the standard convolution module Conv outside the Backbone layer and the Neck layer respectively, wherein the spatial pyramid pooling module SPP is arranged between the Backbone layer and the Neck layer, and the standard convolution module Conv outside the Backbone layer transmits the feature maps to the spatial attention mechanism module SA outside the Backbone layer.

[0012] The Neck layer is configured to perform multi-scale feature fusion on the feature maps from the Backbone layer and transmit the feature maps to the target detection head of the output layer.

[0013] The application also provides a road environment perception algorithm based on an improved YOLOP, which is calculated by using the above system and includes the following steps:

[0014] Step S1: using a YOLOP network model as a basic network, setting an attention mechanism module ECA and a spatial attention mechanism module SA in the YOLOP network model, and establishing an improved YOLOP road environment perception system;

[0015] Step S2: training the improved YOLOP road environment perception system by using an ApolloScape dataset and an augmented dataset to obtain a trained improved YOLOP road environment perception system;

[0016] Step S3: detecting and segmenting targets in a to-be-detected region by using the trained improved YOLOP road environment perception system, and performing stereo frame labeling on the images of the to-be-detected region detected by the image input module.

[0017] Optionally, in step S1, the improved YOLOP road environment perception system is used for calculation, including the following steps:

[0018] Step S11: an image input module collects images of a to-be-detected region and inputs the images to a Backbone structure of a YOLOP network model, and step S12 is performed.

[0019] Step S12: The Backbone structure receives the image of the to-be-tested region sent by the image input module for image preprocessing and feature extraction of the input image, obtains a feature map of the input image, inputs the extracted feature map into the Neck layer, and executes step S13.

[0020] Step S13: The Neck layer performs multi-scale feature fusion on the received feature map and transmits the fused feature map to the target detection head, the road edge detection head, and the road driving area head of the output layer.

[0021] Optionally, in step 12, the Backbone structure adopts a spatial and channel feature extraction enhancement extractor to enhance the features extracted by the Backbone structure, and the spatial and channel feature extraction enhancement extractor includes an attention mechanism module ECA and a spatial attention mechanism module SA.

[0022] Optionally, the feature enhancement step of the attention mechanism module ECA is as follows:

[0023] Step S111: Global average pooling is performed on the input feature map.

[0024] Step S112: One-dimensional convolution operation is performed, and the weight is obtained through a Sigmoid activation function. ;

[0025] Step S113: The weight is multiplied by the corresponding element of the original input feature map in step S211 to obtain a final output feature enhancement map.

[0026] Optionally, after global average pooling is completed, one-dimensional convolution with a convolution kernel size of is used to extract features, and the channel weight is obtained through a Sigmoid activation function, and the calculation formula is as shown in equation (1):

[0027] (1)

[0028] In the formula, one-dimensional convolution with a convolution kernel size of ;

[0029] output of the feature map after GAP;

[0030] channel weight.

[0031] Optionally, the feature enhancement step of the attention mechanism module SA is as follows:

[0032] Step S121: The feature map ​The input is processed in global max pooling Maxpooling and global average pooling Averagepooling to obtain attention information in space and ;

[0033] In step S122, the attention information and is input into the concatenation module Concat inside the attention mechanism module SA for integration to obtain integrated attention information.

[0034] In step S123, the integrated attention information is sequentially input into the standard convolution module Conv and the Sigmoid activation function to obtain the weight The weight is multiplied by the feature map input in step S221 to obtain the fused feature map .

[0035] Optionally, in step S123, the calculation formula of the weight is shown in formula (2):

[0036] (2)

[0037] In formula (3), is a convolution calculation function;

[0038] is adaptive global average pooling;

[0039] is adaptive global max pooling;

[0040] The calculation formula of the fused feature map is shown in formula (3):

[0041] (3).

[0042] The present application improves the YOLOP network model by using the attention mechanism module ECA and the attention mechanism module SA, so that it becomes adaptable to multi-scale targets; it can identify multi-task targets in closed and semi-closed parks, and can realize multi-target detection and segmentation; after training and testing, the improved YOLOP network model has achieved better results, realizing the lightweight, accuracy and multi-task target recognition of the YOLOP network model. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0044] Figure 1 The structural block diagram of the road environment perception algorithm based on the improved YOLOP is shown in the figure.

[0045] Figure 2 The ECA diagram of the attention mechanism module is shown in the figure.

[0046] Figure 3 The SA diagram of the spatial attention mechanism module is shown in the figure.

[0047] Figure 4 The training loss curve is shown in the figure.

[0048] Figure 5 The test result graph in the general weather condition is shown in the figure. DETAILED DESCRIPTION

[0049] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clear and obvious, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0050] The road environment perception system and algorithm based on the improved YOLOP provided by the embodiments of the present application will be described. Referring to Figure 1 A road environment perception system based on the improved YOLOP, comprising a YOLOP network model, adopting the YOLOP network model as a basic network, setting a spatial and channel feature extraction enhancement extractor in the YOLOP network model, the spatial and channel feature extraction enhancement extractor comprising an attention mechanism module ECA and a spatial attention mechanism module SA, the attention mechanism module ECA being used to increase the feature extraction accuracy on the channel, and the spatial attention mechanism module SA being used to increase the feature extraction accuracy on the space.

[0051] The YOLOP network model comprises an image input module, a Backbone layer, a Neck layer and an output layer, the output layer comprising a target detection head, a road edge detection head and a road driving area head.

[0052] The road driving area head is used to detect the road drivable area.

[0053] The target detection head is used for detecting vehicles and pedestrians; further, the target detection head adopts a scale adaptive matching attention mechanism module SA to improve the detection performance of small-scale targets.

[0054] The curb detection head is used for detecting a curb.

[0055] The image input module is used for collecting images of a to-be-detected area and transmitting the images to the Backbone layer and the spatial attention mechanism module SA outside the Backbone layer respectively, and the spatial attention mechanism module SA outside the Backbone layer transmits feature maps to the curb detection head and the road driving area head of the output layer.

[0056] The Backbone layer is used for processing images of a to-be-detected area to obtain feature maps of the input images, and transmitting the feature maps to the standard convolution module Conv outside the Backbone layer and the Neck layer respectively, wherein a spatial pyramid pooling module SPP is arranged between the Backbone layer and the Neck layer; the standard convolution module Conv outside the Backbone layer transmits the feature maps to the spatial attention mechanism module SA outside the Backbone layer.

[0057] The Neck layer is used for performing multi-scale feature fusion on the feature maps from the Backbone layer and transmitting the feature maps to the target detection head of the output layer.

[0058] The Backbone layer is provided with two spatial and channel feature extraction enhancement extractors, namely a first spatial and channel feature extraction enhancement extractor and a second spatial and channel feature extraction enhancement extractor.

[0059] The Backbone layer comprises, in sequence, a preprocessing module, a focusing module Foucs, a first standard convolution module Conv inside the Backbone layer, a first cross-stage local bottleneck module bottleneckCSP inside the Backbone layer, a first spatial and channel feature extraction enhancement extractor, a second standard convolution module Conv inside the Backbone layer, a second cross-stage local bottleneck module bottleneckCSP inside the Backbone layer, a second spatial and channel feature extraction enhancement extractor, and a third standard convolution module Conv inside the Backbone layer.

[0060] The Neck layer comprises a first cross-stage partial bottleneck module bottleneckCSP, a first standard convolution module Conv, a first upsampling module Upsample, a first concatenation module Concat, a second cross-stage partial bottleneck module bottleneckCSP, a second standard convolution module Conv, a second upsampling module Upsample, and a second concatenation module Concat in the Neck layer;

[0061] The third standard convolution module Conv in the Backbone layer transmits the feature map fused by the Backbone layer to the first cross-stage partial bottleneck module bottleneckCSP in the Neck layer through the spatial pyramid pooling module SPP, and the first cross-stage partial bottleneck module bottleneckCSP in the Neck layer can be transmitted to the target detection head and the first standard convolution module Conv in the Neck layer, respectively;

[0062] The first standard convolution module Conv in the Neck layer transmits the feature map processed by the first cross-stage partial bottleneck module bottleneckCSP in the Neck layer to the target detection head and the first upsampling module Upsample in the Neck layer, respectively;

[0063] The first upsampling module Upsample in the Neck layer transmits the feature map processed by the first concatenation module Concat in the Neck layer, the second cross-stage partial bottleneck module bottleneckCSP in the Neck layer, to the second standard convolution module Conv in the Neck layer, respectively;

[0064] The second standard convolution module Conv in the Neck layer transmits the feature map processed by the second cross-stage partial bottleneck module bottleneckCSP in the Neck layer to the target detection head and the second upsampling module Upsample in the Neck layer, respectively;

[0065] The second upsampling module Upsample in the Neck layer transmits the feature map fused by the second concatenation module Concat in the Neck layer to the road edge detection head and the road driving area head.

[0066] The feature map output by the Neck layer after multi-scale fusion is taken as the input of the road driving area head and the road edge detection head, but after too much down-up sampling operation, too much feature information is lost, therefore, the feature map extracted by the third standard convolution module Conv in the Backbone layer is taken as the enhancement of the feature input of the target segmentation module, and a small convolution (i.e. the standard convolution module Conv outside the Backbone layer) and the spatial attention mechanism module SA outside the Backbone layer are used to enhance the enhanced feature, and then the two kinds of features are fused as the input of the road edge detection head and the road segmentation head.

[0067] The target detection head comprises, arranged in sequence, a first cross-stage local bottleneck module bottleneckCSP in the target detection head, a first spatial attention mechanism module SA in the target detection head, a first standard convolution module Conv in the target detection head, a first splicing module Concat in the target detection head, a second cross-stage local bottleneck module bottleneckCSP in the target detection head, a second spatial attention mechanism module SA in the target detection head, a second standard convolution module Conv in the target detection head, a second splicing module Concat in the target detection head, a third cross-stage local bottleneck module bottleneckCSP in the target detection head, a third spatial attention mechanism module SA in the target detection head, and a detection module Detect.

[0068] The first splicing module Concat in the target detection head receives the feature map transmitted by the first standard convolution module Conv in the Neck layer; and the second splicing module Concat in the target detection head receives the feature map transmitted by the second standard convolution module Conv in the Neck layer.

[0069] The application also provides a road environment perception algorithm based on the improved YOLOP, which is calculated by using the above system and comprises the following steps:

[0070] In step S1, a YOLOP network model is used as a basic network, an attention mechanism module ECA and a spatial attention mechanism module SA are arranged in the YOLOP network model, and an improved YOLOP road environment perception system is established;

[0071] Referring to Figure 1 The improved YOLOP road environment perception system is used for calculation, and the following steps are included:

[0072] In step 11, an image input module collects images of a region to be measured and inputs the images to the Backbone structure of the YOLOP network model, and step S12 is performed.

[0073] Step 12: The Backbone structure receives the image of the to-be-tested region sent by the image input module for image preprocessing and feature extraction of the input image, obtains a feature map of the input image, inputs the extracted feature map into the Neck layer, and executes step S13.

[0074] In step 12, the Backbone structure inputs the received image of the to-be-tested region into the Neck layer after extracting a feature map through the spatial pyramid pooling module SPP.

[0075] In step 12, the Backbone structure further transmits the received image of the to-be-tested region to the spatial attention mechanism module SA outside the Backbone layer after the standard convolution module Conv outside the Backbone layer, and then transmits the image to the road edge detection head and the road driving area head through the spatial attention mechanism module SA.

[0076] In step S13, the Neck layer performs multi-scale feature fusion on the received feature map and transmits the fused feature map to the target detection head, the road edge detection head and the road driving area head of the output layer.

[0077] Further, in step 12, the Backbone structure uses a spatial and channel feature extraction enhancer to enhance the features extracted by the Backbone structure, and the spatial and channel feature extraction enhancer includes an attention mechanism module ECA and a spatial attention mechanism module SA.

[0078] Referring to Figure 2 , the feature enhancement step of the attention mechanism module ECA is as follows:

[0079] Step S111, global average pooling of the input feature map;

[0080] Step S112, 1D convolution operation is performed, and the weight w is obtained through the Sigmoid activation function .

[0081] Step S113, multiply the weight w and the corresponding elements of the original input feature map in step S211 to obtain the final output feature enhancement map.

[0082] The feature maps obtained by the first cross-stage local bottleneck module bottleneckCSP in the Backbone layer and the second cross-stage local bottleneck module bottleneckCSP in the Backbone layer are subjected to a global average pooling (GAP) operation, for counting channel information. Wherein, the channel dimension of the input feature map is C, the height is H, and the width is W. The present application adopts global average pooling instead of global maximum pooling, because global average pooling can avoid overfitting. After global average pooling, a 1D convolution with a convolution kernel size of is used to extract features, and the channel weight is obtained through a Sigmoid activation function, as shown in formula (1):

[0083] (1)

[0084] wherein is a 1D convolution with a convolution kernel size of

[0085] is the output of the feature map after GAP;

[0086] is the channel weight.

[0087] The weight and each element of the input feature map are subjected to a multiplication operation, and the final feature map is obtained through the operation of the ECA attention mechanism and is used as the input of the subsequent network operation.

[0088] Referring to Figure 3 , the feature enhancement step of the attention mechanism module SA is as follows:

[0089] In step S121, the feature map is input into the global maximum pooling Maxpooling and the global average pooling Averagepooling for processing, to obtain spatial attention information and ;

[0090] In step S122, the attention information and are input into the splicing module Concat inside the attention mechanism module SA for integration, to obtain integrated attention information;

[0091] In step S123, the integrated attention information is sequentially input into the standard convolution module Conv and the Sigmoid activation function, to obtain the weight , and the weight ​The feature map input in step S221 is multiplied by the feature map input in step S222 to obtain a fused feature map. .

[0092] The feature map input in step S221 is multiplied by the feature map input in step S222 to obtain a fused feature map. The fused feature map is input into a global maximum pooling Maxpooling operation and a global average pooling Averagepooling operation to obtain spatial attention information. The two kinds of information are integrated by a concatenation module Concat in an attention mechanism module SA, and then input into a second standard convolution module Conv in the Backbone layer and a third standard convolution module Conv in the Backbone layer. The second standard convolution module Conv in the Backbone layer and the third standard convolution module Conv in the Backbone layer are input into a Sigmoid activation function to obtain a weight. The weight is multiplied by the input feature map to obtain a fused feature map. The fused feature map is obtained by multiplying the weight by the input feature map. .

[0093] The weight is calculated according to formula (2).

[0094] (2)

[0095] In formula (3), is a convolution calculation function.

[0096] is adaptive global average pooling.

[0097] is adaptive global maximum pooling.

[0098] The fused feature map is calculated according to formula (3).

[0099] (3)

[0100] Figure 3 In formula (3), Maxpooling represents global maximum pooling, Averagepooling represents global average pooling, Concat represents a concatenation module, Conv1x1 represents a 1x1 small convolutional layer, and Sigmoid represents a Sigmoid activation function.

[0101] ​​​​​​Step S2: training the improved YOLOP road environment perception system using the ApolloScape dataset and the augmented dataset to obtain the trained improved YOLOP road environment perception system;

[0102] The improved YOLOP road environment perception system uses a YOLOP network model as a base model, and the YOLOP network model is a multi-scale adaptive multi-task recognition network with sufficient lightweight characteristics.

[0103] The multi-scale adaptive multi-task recognition network is used for network training on the selected dataset and setting of training parameters.

[0104] Among them, the size of Epoch is set to 300, Batchsize is set to 16, that is, 16 images are trained at a time, One Cycle strategy is used as the network training learning rate adjustment strategy, and the initial learning rate is set to 0.01.

[0105] Further, the YOLOP network model is trained, and the loss function includes three parts, which are the losses of the three tasks. The target detection loss is the classic bounding box loss, target loss and class loss, each of which is added with a weight, and the other two semantic segmentation losses are cross-entropy losses. The total loss is the sum of the three losses, and the loss formula is as follows:

[0106] (4)

[0107] (5)

[0108] In the formula, , , respectively, are the classification loss, target loss and bounding box loss;

[0109] , are the feasible region loss and road edge segmentation loss;

[0110] , respectively, are the target detection loss and total loss;

[0111] Figure 4 In the formula, the horizontal coordinate represents the number of times the model is trained on the entire data of the training set, and the vertical coordinate represents the training loss value represented by formula (5). It can be seen from Figure 4 that the training loss curve of the improved YOLOP network model tends to be flat after 50 iterations, and finally completely fits, and the decline is fast, with good performance. The training loss after 300 iterations of fitting finally stabilizes at 0.018.

[0112] Step S3: detecting and segmenting the target in the to-be-tested region by using the trained improved YOLOP road environment perception system, and performing stereo frame labeling on the image of the to-be-tested region detected by the image input module.

[0113] The application adopts the YOLOP network model as the basis and further improves it to realize multi-target detection and segmentation of a single network; solves the problem of large scale variation of road and road edge targets and low recognition accuracy; improves the recognition accuracy of the network model; and further lightens the YOLOP network model, greatly improving the inference speed.

[0114] The ApolloScape data set is used to train the improved YOLOP network model and test the algorithm performance, and images close to the scene in the park are selected to form a data set for training and testing.

[0115] Further, considering that the application needs to face various weather scenes, including sunny, cloudy, rainy, foggy, and night, and the park road data set under these weather scenes is less, the application constructs a new data set based on the ApolloScape data set through data augmentation and other methods, which contains 6000 pictures under the above scenes, and divides the training set and the test set according to the ratio of 4:1, and based on this, the training and testing of the network model are completed, so that the trained model has stronger generalization ability.

[0116] Figure 5 is the test result of the improved YOLOP road environment perception algorithm under general weather conditions, and the result shows that the network designed in the application has good target detection effect.

[0117] Figure 1 The "input image" in is the "image input module" in this paper.

[0118] The application improves the YOLOP network model by using the attention mechanism module ECA and the attention mechanism module SA, so that it becomes adaptable to multi-scale targets. It can identify multi-task targets in closed and semi-closed parks, and can realize multi-target detection and segmentation. After training and testing, the improved YOLOP network model achieves good results, realizing the lightweight, accuracy and multi-task target recognition of the YOLOP network model. For example, the improved YOLOP network model has high recognition accuracy for targets with large scale variation such as roads and road edges, which has low recognition accuracy. For example, the improved YOLOP network model is designed to be lightweight, so as to realize high inference speed.

[0119] The above only describes preferred embodiments of the present application and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A road environment perception system based on improved YOLOP, comprising a YOLOP network model, characterized in that: The YOLOP network model is used as a basic network, and a spatial and channel feature extraction enhancer is arranged in the YOLOP network model, the spatial and channel feature extraction enhancer comprising an attention mechanism module ECA and a spatial attention mechanism module SA; The YOLOP network model comprises an image input module, a Backbone layer, a Neck layer and an output layer, and the output layer comprises a target detection head, a road edge detection head and a road drivable area head; The road drivable area head is configured to detect a road drivable area; The target detection head is configured to detect vehicles and pedestrians; The road edge detection head is configured to detect a road edge; The Backbone layer comprises a preprocessing module, a focusing module Foucs, a first standard convolution module Conv in the Backbone layer, a first cross-stage local bottleneck module bottleneckCSP in the Backbone layer, a first spatial and channel feature extraction enhancer, a second standard convolution module Conv in the Backbone layer, a second cross-stage local bottleneck module bottleneckCSP in the Backbone layer, a second spatial and channel feature extraction enhancer, and a third standard convolution module Conv in the Backbone layer, which are arranged in sequence; The image input module is configured to acquire images of a to-be-detected area and transmit the images to the Backbone layer and a spatial attention mechanism module SA outside the Backbone layer respectively, and the spatial attention mechanism module SA outside the Backbone layer transmits feature maps to the road edge detection head and the road drivable area head of the output layer; The Backbone layer is configured to process the images of the to-be-detected area to obtain feature maps of the input images, and transmit the feature maps to a standard convolution module Conv outside the Backbone layer and the Neck layer respectively, wherein a spatial pyramid pooling module SPP is arranged between the Backbone layer and the Neck layer, and the standard convolution module Conv outside the Backbone layer transmits the feature maps to the spatial attention mechanism module SA outside the Backbone layer; The Neck layer is configured to perform multi-scale feature fusion on the feature maps from the Backbone layer and transmit the feature maps to the target detection head of the output layer.

2. A road environment perception method based on improved YOLOP, characterized by: The improved YOLOP-based road environment perception system of claim 1 is used for calculation, comprising the following steps: Step S1: using a YOLOP network model as a basic network, and arranging an attention mechanism module ECA and a spatial attention mechanism module SA in the YOLOP network model to establish an improved YOLOP-based road environment perception system; Step S2: using an ApolloScape dataset and an augmented dataset to train the improved YOLOP-based road environment perception system to obtain a trained improved YOLOP-based road environment perception system; Step S3: using the trained improved YOLOP road environment perception system to detect and segment the target in the to-be-tested region, and performing stereo frame labeling on the image of the to-be-tested region detected by the image input module.

3. The road environment perception method based on improved YOLOP of claim 2, wherein: In step S1, the improved YOLOP road environment perception system is used for calculation, including the following steps: Step S11: the image input module collects the image of the to-be-tested region and inputs it into the Backbone structure of the YOLOP network model, and executes step S12; Step S12: the Backbone structure receives the image of the to-be-tested region sent by the image input module for image preprocessing and feature extraction of the input image, obtains the feature map of the input image, inputs the extracted feature map into the Neck layer, and executes step S13; Step S13, the Neck layer performs multi-scale feature fusion on the received feature map and transmits the fused feature map to the target detection head, the road edge detection head and the road driving area head of the output layer.

4. The road environment perception method based on improved YOLOP of claim 3, wherein: In step 12, the Backbone structure uses a spatial and channel feature extraction enhancement extractor to enhance the features extracted by the Backbone structure, and the spatial and channel feature extraction enhancement extractor includes an attention mechanism module ECA and a spatial attention mechanism module SA.

5. The road environment perception method based on improved YOLOP of claim 4, wherein: The feature enhancement steps of the attention mechanism module ECA are as follows: Step S111, global average pooling of the input feature map; Step S112, 1-dimensional convolution operation is performed, and the weight is obtained through a Sigmoid activation function ; Step S113, multiplying the weight with the corresponding element of the original input feature map in step S111, to obtain the final output feature enhancement map.

6. The road environment perception method based on improved YOLOP of claim 5, wherein: After completing the global average pooling, the feature extraction is performed using a 1-dimensional convolution with a kernel size of The channel weight is obtained by a Sigmoid activation function, and the calculation formula is shown in equation (1): （1） In the formula — the convolution kernel size is one-dimensional convolution; - the output of the feature map after GAP; - the weight of the channel.

7. The road environment perception method based on improved YOLOP of claim 4, wherein: The feature enhancement steps of the attention mechanism module SA are as follows: Step S121, the feature map is input into global maximum pooling Maxpooling and global average pooling Averagepooling for processing to obtain attention information in space and ; Step S122, attention information and The integrated attention information is obtained by integrating in the concatenation module Concat inside the input attention mechanism module SA. Step S123, the integrated attention information is input into the standard convolution module Conv and the Sigmoid activation function in turn, and the weight is obtained . Step S124, the weight is multiplied by the feature map input in step S121, and the fused feature map is obtained.

8. The road environment perception method based on improved YOLOP of claim 7, wherein: In step S123, the weight The calculation formula is shown in formula (2): （2） In equation (3), is a convolution computation function; adaptive global average pooling; adaptive global max pooling; fused feature map The calculation formula is shown as formula (3). （3）。

Images