Method for identifying drivable area in rural environment based on deep learning

Abstract

A deep learning-based method for identifying a drivable area in a rural environment comprises the following steps of S1, collecting a driving image of the rural environment by adopting a vehicle with a camera, and establishing a rural road data set; s2, building a deep learning semantic segmentation network model, wherein the deep learning semantic segmentation network model comprises a coding module, a neck module and a decoding module; s3, training a deep learning semantic segmentation model, inputting a to-be-detected target image into the network model, and obtaining a target region in the image; s4, calculating a loss function, wherein the loss function comprises a cross entropy loss function and a dice loss function; and S5, predicting and marking a drivable area. The method is used for solving the problem that the road recognition algorithm is interfered by the unclear road structure in the rural road environment, and realizing the task of stably recognizing the drivable area in the road in different light and weather. And the interference of image quality and other problems on the accuracy of identifying the drivable area is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and in particular to a method for identifying a drivable area in a rural environment based on deep learning. Background Art

[0002] Driving area segmentation is a prerequisite for safely passing through off-road environments. Its goal is to segment the road surface accessible to autonomous vehicles to assist in the route planning task of autonomous driving. Road segmentation usually relies on onboard sensors to collect road image data of different scenes and classify and predict each pixel of the image. Due to the complexity and diversity of road scenes, as well as other external factors such as weather and lighting conditions, the image quality obtained by the sensor is unstable. Compared with urban roads, rural roads often lack lane assist markings and clear road structures. Therefore, it is necessary to focus on identifying drivable areas in rural environments for autonomous driving.

[0003] Based on the above situation, how to adopt effective methods to accurately determine the drivable area in a rural environment and reduce the interference of image quality and other issues on the accuracy of identifying the drivable area is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present invention provides a method for identifying drivable areas in rural environments based on deep learning, which is used to solve the problem of interference to road recognition algorithms caused by unclear road structures in rural road environments, and to achieve the task of stably identifying drivable areas on roads under different light and weather conditions. It reduces the interference of image quality and other issues on the accuracy of identifying drivable areas.

[0005] A method for identifying drivable areas in a rural environment based on deep learning, comprising the following steps:

[0006] Step S1: Using a vehicle equipped with a camera to collect driving images of a rural environment and establish a rural road dataset;

[0007] Step S2: Building a deep learning semantic segmentation network model, including: an encoding module, a neck module and a decoding module;

[0008] Step S3: training a deep learning semantic segmentation model, inputting the target image to be detected into the network model, and obtaining the target area in the image;

[0009] Step S4: Calculate the loss function, which includes the cross entropy loss and the dice loss function;

[0010] Step S5: predict and mark the drivable area.

[0011] Furthermore, in step S1, the original image is preprocessed by random rotation, cropping, and normalization;

[0012] Use image processing functions to enhance or reduce the brightness, contrast, chroma, and sharpness of data sets to enrich data types;

[0013] Use data annotation software to construct a drivable area dataset.

[0014] Further, in step S2, obtaining the trained deep learning semantic segmentation network model includes:

[0015] Build an improved YOLO deep neural network semantic segmentation model in the PyTorch framework;

[0016] Train a semantic segmentation model on the drivable area dataset.

[0017] Furthermore, the YOLO deep neural network semantic segmentation model consists of the following three modules:

[0018] Encoding module: The CSPDarknet lightweight network is used as the backbone network. The neck network connecting the encoding module and the decoding module is composed of ASPP, gated deep interception model and FPN.

[0019] Gated Depthwise Separable Convolution Module: A gating mechanism module is used to help process information flow. The gated depthwise separable convolution module consists of two depthwise Inception convolutions. The depthwise convolution uses a parallel structure of three gated depthwise convolutions, including ordinary convolutions and depthwise separable convolutions with a kernel size of 1x1. The normalized NAM attention module is used to concatenate the output of each branch by channel, and finally the number of channels of the output feature is increased to 3N, where N is the number of channels of the input feature.

[0020] Decoding module: The CSP1_X structure proposed by Yolov4 is applied to divide the original input into two branches for convolution operation; the two branches are merged so that the number of channels in the structure output is half of the original input; in order to restore the prediction result to the size of the original image, the feature map is enlarged to three times the nearest upsampling.

[0021] Further, in step S3, the target image to be detected is input into the trained deep learning network, and the drivable area in the target image is obtained, including:

[0022] Inputting the image to be detected into the trained semantic segmentation model to predict the category of each pixel;

[0023] The semantic segmentation model extracts, enriches and fuses multi-scale features of the input image;

[0024] The semantic segmentation model outputs the drivable area of ​​the image to be tested.

[0025] Furthermore, in step S4, the cross entropy function and the dice loss function are used, wherein the addition of the dice loss function makes the semantic segmentation network prediction result closer to the actual situation.

[0026] Furthermore, in step S5, the method for predicting the drivable area is:

[0027] Use the visualization code to annotate the predicted drivable area and the actual drivable area in the input image.

[0028] The present invention can solve the problem of low accuracy in road recognition in rural environments. The present invention proposes solutions to problems such as unclear road structure and poor lighting conditions in rural environments, and improves the versatility of road recognition algorithms. It can identify drivable areas in real time under various lighting conditions and intuitively display them with visual results. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of the identification method of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described in detail below with reference to the accompanying drawings:

[0031] like Figure 1 As shown, a method for identifying drivable areas in a rural environment based on deep learning includes the following steps:

[0032] Step S1: Using a vehicle equipped with a camera to collect driving images of a rural environment and establish a rural road dataset;

[0033] Step S2: Building a deep learning semantic segmentation network model, including: an encoding module, a neck module and a decoding module;

[0034] Step S3: training a deep learning semantic segmentation model, inputting the target image to be detected into the network model, and obtaining the target area in the image;

[0035] Step S4: Calculate the loss function, which includes the cross entropy loss and the dice loss function;

[0036] Step S5: predict and mark the drivable area.

[0037] Specifically: In step S1, a vehicle equipped with a camera is used to collect driving videos of rural environments. Frames are extracted from the original video to obtain rural road images, and image random rotation, cropping, and normalization preprocessing are performed on the computer based on Python programming algorithms. Data annotation software is used to annotate the drivable areas in the image to establish the required rural drivable area dataset.

[0038] In step S2, a semantic segmentation network model based on the YOLO algorithm is built. The model mainly includes:

[0039] Encoding module: This module uses the CSPDarknet lightweight network as the backbone network. The neck network consists of ASPP (Atrous Spatial Pyramid Pooling), a gated deep interception model, and FPN (Feature Pyramid Networks). ASPP is a variant of the SPP (Spatial Pyramid Pooling) method, which applies toothless convolutions with different dilations to capture multi-scale features and increase the receptive field of the network. After the backbone network, we use ASPP with dilations of 6, 12, and 18 to obtain multi-scale features and fuse the feature maps together.

[0040] Gated Depthwise Separable Convolution Module: This module uses a gating mechanism module to help process information flow. The gated depthwise convolution model is mainly composed of two depthwise Inception convolutions. The depthwise convolution of this module adopts a parallel structure of three gated depthwise convolutions. It includes ordinary convolution and depthwise separable convolution with a kernel size of 1x1. It also uses a normalized attention module to splice the output of each branch by channel, and finally increase the number of channels of the output feature to 3N, where N is the number of channels of the input feature.

[0041] The function expression of the gated depthwise separable convolution module is:

[0042] putput=tanh(DIconv(x))*siigmoid(DIconv(x))

[0043] Among them, DIconv(x) is the input and output of the conv module intercepted by depth, the sigmoid function determines the weight of the transmitted features, the tanh function is the activation function of the output, and * is element multiplication.

[0044] The function expression of the normalized attention module is:

[0045]

[0046] Among them, w represents the weight of each channel after normalization, μ is the input feature vector, BN() is the function that normalizes the input features, and the output function y NAM It is the scaling factor after normalization of the weight multiplied by sigmoid.

[0047] Decoding module: This decoding module applies the CSP1_X structure proposed by Yolov4, which divides the original input into two branches for convolution operation. This module merges the two branches so that the number of channels in the structure output is half of the original input. In order to restore the prediction result to the size of the original image, the feature map is enlarged to three times the nearest upsampling.

[0048] The model is trained on the rural drivable area dataset to obtain the trained deep learning semantic segmentation network model.

[0049] In step S3, the image to be tested is input into the trained semantic segmentation model, and the semantic segmentation model extracts, enriches and fuses the input image to predict the category of each pixel, and finally the semantic segmentation model outputs the drivable area of ​​the image to be tested. In this example, the ASPP module is used to extract features for multi-scale fusion to capture contextual text at multiple scales. In addition, the method uses a gating mechanism unit to adjust feature weights to control information flow, aiming to reduce the impact of irrelevant features on road segmentation. After the gating module, the feature pyramid network structure integrates the multi-level features of the backbone network and connects the entire encoder and decoder parts. The encoder finally produces the final prediction result.

[0050] In step S4, the loss function L da Combined with the cross entropy loss function L ce and the dice loss function L dice :

[0051] L da =βL ce +γL dice

[0052]

[0053] Among them, TP, FP and FN are true positive, false positive and false negative respectively. i represents the true probability distribution, m i In order to make the network prediction result closer to the actual situation, the dice loss function is added to enrich the loss function, and the β parameter and α parameter are both set to 1.

[0054] In step S5, the method for predicting the drivable area is:

[0055] Use the visualization code to mark the predicted drivable area and the actual drivable area in the input image. The visualization results can be used as a more intuitive way to evaluate the algorithm.

[0056] The method for identifying drivable areas in rural environments based on deep learning of the present invention can solve the problem of low accuracy in road recognition in rural environments. Solutions are proposed for problems such as unclear road structure and poor lighting conditions in rural environments, which improves the versatility of road recognition algorithms. It can identify drivable areas in real time under various lighting conditions and intuitively display them with visual results.

Claims

1. A method for identifying drivable areas in rural environments based on deep learning, characterized in that The steps include: Step S1: Using a vehicle equipped with a camera to collect driving images of a rural environment and establish a rural road dataset; Step S2: Building a deep learning semantic segmentation network model, including: an encoding module, a neck module and a decoding module; Step S3: training a deep learning semantic segmentation model, inputting the target image to be detected into the network model, and obtaining the target area in the image; Step S4: Calculate the loss function, which includes the cross entropy loss and the dice loss function; Step S5: predict and mark the drivable area.

2. The method for identifying drivable areas in a rural environment based on deep learning according to claim 1, characterized in that In step S1, the original image is preprocessed by random rotation, cropping, and normalization; Use image processing functions to enhance or reduce the brightness, contrast, chroma, and sharpness of data sets to enrich data types; Use data annotation software to construct a drivable area dataset.

3. The method for identifying drivable areas in a rural environment based on deep learning according to claim 2, characterized in that In step S2, obtaining the trained deep learning semantic segmentation network model includes: Build an improved YOLO deep neural network semantic segmentation model in the PyTorch framework; Train a semantic segmentation model on the drivable area dataset.

4. The method for identifying drivable areas in a rural environment based on deep learning according to claim 3 is characterized in that The deep neural network semantic segmentation model consists of the following three modules: Encoding module: The CSPDarknet lightweight network is used as the backbone network. The neck network connecting the encoding module and the decoding module is composed of ASPP, gated deep interception model and FPN. Gated Depthwise Separable Convolution Module: A gating mechanism module is used to help process information flow. The gated depthwise separable convolution module consists of two depthwise Inception convolutions. The depthwise convolution uses a parallel structure of three gated depthwise convolutions, including ordinary convolutions and depthwise separable convolutions with a kernel size of 1x1. The normalized NAM attention module is used to concatenate the output of each branch by channel, and finally the number of channels of the output feature is increased to 3N, where N is the number of channels of the input feature. Decoding module: The CSP1_X structure proposed by Yolov4 is applied to divide the original input into two branches for convolution operation; the two branches are merged so that the number of channels in the structure output is half of the original input; in order to restore the prediction result to the size of the original image, the feature map is enlarged to three times the nearest upsampling.

5. The method for identifying drivable areas in a rural environment based on deep learning according to claim 4, characterized in that In step S3, the target image to be detected is input into the trained deep learning network, and the drivable area in the target image is obtained, including: Inputting the image to be detected into the trained semantic segmentation model to predict the category of each pixel; The semantic segmentation model extracts, enriches and fuses multi-scale features of the input image; The semantic segmentation model outputs the drivable area of ​​the image to be tested.

6. The method for identifying drivable areas in a rural environment based on deep learning according to claim 5, characterized in that In step S4, the cross entropy function and the dice loss function are used, wherein the addition of the dice loss function makes the semantic segmentation network prediction result closer to the actual situation.

7. The method for identifying drivable areas in a rural environment based on deep learning according to claim 6, characterized in that In step S5, the method for predicting the drivable area is: Use the visualization code to annotate the predicted drivable area and the actual drivable area in the input image.

Images