Terrain environment identification method based on ResNet residual network

Abstract

The invention relates to the technical field of deep learning, in particular to a terrain environment recognition method based on a ResNet residual network, and the method comprises the following steps: constructing a terrain environment recognition network based on the ResNet residual network and a CBAM attention mechanism; training and optimizing the terrain environment recognition network; acquiring a terrain environment image, and preprocessing the terrain environment image to obtain an input image; and inputting the input image into the terrain environment recognition network for terrain environment recognition to obtain a terrain environment recognition result. According to the invention, through combination of the ResNet residual network and the CBAM attention mechanism, the network training time is greatly reduced, faster and more accurate terrain environment classification can be realized, and powerful help is provided for outdoor path planning and navigation of the mobile robot.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a terrain environment recognition method based on a ResNet residual network. Background Art

[0002] Terrain environment recognition is a necessary function for mobile robots to achieve automatic cruising.

[0003] Existing terrain environment recognition is often implemented based on neural networks. After the mobile robot is equipped with hardware such as chips with computing capabilities, it collects environmental information through devices such as cameras and sensors, and processes it through a neural network to analyze the factors in the environment that affect the mobile robot's actions, and further realize functions such as path planning and navigation.

[0004] In related technologies, convolutional neural networks used for terrain environment recognition generally include AlexNet, VGG, etc. This type of convolutional neural network extracts features and performs classification tasks by stacking convolutional layers and pooling layers. VGG achieves better performance by using smaller convolution kernels and deeper network structures. However, as the number of network layers increases, networks such as AlexNet and VGG will experience problems such as gradient disappearance / explosion and increased training errors, which makes network training difficult and performance cannot be effectively improved with network depth, limiting the further development of terrain environment recognition technology. Summary of the invention

[0005] The present invention aims to solve the technical problems of network training difficulty and performance improvement difficulty existing in the existing terrain environment recognition method.

[0006] In order to solve the above technical problems, the present invention provides a terrain environment recognition method based on ResNet residual network, comprising the following steps:

[0007] Construct a terrain environment recognition network based on ResNet residual network and CBAM attention mechanism;

[0008] Training and optimizing the terrain environment recognition network;

[0009] Acquire a terrain environment image, and preprocess the terrain environment image to obtain an input image;

[0010] The input image is input into the terrain environment recognition network to perform terrain environment recognition to obtain a terrain environment recognition result.

[0011] Furthermore, the terrain environment recognition network includes, in order of logical input and output, the following:

[0012] Input layer, used to obtain the image to be recognized;

[0013] A convolution layer, used for performing convolution processing on the image to be recognized to obtain a first output feature;

[0014] A batch normalization layer, used for normalizing the first output features to obtain normalized features;

[0015] An activation function layer, used to perform excitation processing on the normalized features according to a preset activation function to obtain weight parameters of different feature channels in the normalized features;

[0016] A maximum pooling layer, used to perform maximum pooling processing on the weight parameters and extract significant features corresponding to the weight parameters;

[0017] A residual module, wherein the residual module includes the CBAM attention mechanism, and the residual module is used to perform feature refinement processing on the salient features to obtain terrain features;

[0018] An average pooling layer, used to compress the terrain features to obtain compressed terrain features;

[0019] A fully connected layer, used for restoring the dimension of the compressed terrain features to obtain a terrain feature recognition result;

[0020] The output layer is used to output the terrain feature recognition result.

[0021] Furthermore, the method for preprocessing the terrain environment image is specifically as follows:

[0022] The terrain environment image is adjusted to a preset size.

[0023] Furthermore, the maximum pooling layer satisfies the following expression:

[0024]

[0025] Among them, R ij represents the area where the pooling window corresponds to the coordinates (i, j) on the feature map of the normalized feature, x mn is the element value on the feature map of the normalized feature, y ij It is the element value with position coordinate (i, j) on the feature map of the outputted significant feature.

[0026] Furthermore, the average pooling layer satisfies the following expression:

[0027]

[0028] Among them, z c represents the compressed terrain feature, F sq represents the compression operation function, u crepresents the terrain feature, W and H represent the width and height of the feature output of the terrain feature respectively, i and j represent the coordinate position on the terrain feature respectively, and C represents the number of feature channels of the terrain feature respectively.

[0029] Furthermore, the input of the residual module is defined as x, the low-level mapping of the residual module is defined as H(x), and the residual mapping of the residual module is defined as F(x)=H(x)-x; the residual module satisfies:

[0030]

[0031] in, is the output of the residual module, h(x) is the identity mapping of x, W is a set of learning weights, and σ is a preset activation function.

[0032] Furthermore, the preset activation function is a ReLU activation function, which satisfies the following expression:

[0033]

[0034] in, for The activation value obtained by the activation layer function; U is the residual function.

[0035] Furthermore, the batch normalization layer satisfies the following expression:

[0036]

[0037] Among them, x is the input value, μ B is the data mean, is the data variance, ∈ is a constant, and γ and β are learnable parameters.

[0038] Furthermore, the CBAM attention mechanism includes a channel attention mechanism and a spatial attention mechanism, which satisfies the following expression:

[0039]

[0040] in, represents element-by-element multiplication, F represents the input feature map, F′ represents the output feature map of the channel attention mechanism, and F″ represents the output feature map of the final CBAM attention mechanism;

[0041] The channel attention mechanism satisfies:

[0042]

[0043] Among them, F represents the input feature map, avgpool represents global average pooling, maxpool represents global maximum pooling, MLP represents multi-layer perceptron, and Represent the weights of the two convolution operations, C is the number of channels, r is the reduction rate, and They represent the results of average pooling and maximum pooling respectively, σ represents the Sigmoid activation function, and m c is the obtained channel attention map;

[0044] The spatial attention mechanism satisfies:

[0045]

[0046] Where F represents the channel attention map output by the channel attention mechanism, f 7×7 represents a convolution operation with a convolution kernel size of 7×7. and They represent the results of average pooling and maximum pooling respectively, h and ω are the height and width of the feature map respectively, and m s is the obtained spatial attention map;

[0047] The Sigmoid activation function satisfies:

[0048]

[0049] Among them, e is a natural constant and x is an input value.

[0050] Furthermore, the terrain environment recognition network uses a Softmax regression function in the output layer to determine the terrain feature type of the terrain feature recognition result, and the Softmax regression function satisfies the following expression:

[0051]

[0052] Among them, (x i ,y i ) represents the training data, y i ∈{1, 2, …, k} represents the category to which the data belongs, n represents the number of samples, I{·} represents the logical expression function, and θ is the training set.

[0053] The beneficial effect achieved by the present invention is that a terrain environment recognition method based on ResNet residual network is proposed. This method greatly reduces the network training time by combining ResNet residual network and CBAM attention mechanism, and can achieve faster and more accurate terrain environment classification, which provides strong assistance for outdoor path planning and navigation of mobile robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the steps of a terrain environment recognition method based on a ResNet residual network provided by an embodiment of the present invention;

[0055] Figure 2 It is a schematic diagram of the structure of a terrain environment recognition network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0057] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of a step flow of a terrain environment recognition method based on a ResNet residual network provided by an embodiment of the present invention, wherein the terrain environment recognition method based on a ResNet residual network comprises the following steps:

[0058] S1. Construct a terrain environment recognition network based on ResNet residual network and CBAM attention mechanism;

[0059] S2, training and optimizing the terrain environment recognition network;

[0060] S3, acquiring a terrain environment image, and preprocessing the terrain environment image to obtain an input image;

[0061] S4. Input the input image into the terrain environment recognition network to perform terrain environment recognition to obtain a terrain environment recognition result.

[0062] For further information, please refer to Figure 2 , Figure 2 : is a schematic diagram of the structure of a terrain environment recognition network provided by an embodiment of the present invention, wherein the terrain environment recognition network includes, in order of logical input and output, the following:

[0063] Input layer, used to obtain the image to be recognized;

[0064] A convolution layer (Conv), used to perform convolution processing on the image to be recognized to obtain a first output feature;

[0065] A batch normalization layer (BN), used for normalizing the first output features to obtain normalized features;

[0066] An activation function layer (ReLU), used to perform excitation processing on the normalized features according to a preset activation function to obtain weight parameters of different feature channels in the normalized features;

[0067] A maximum pooling layer (MaxPool), used to perform maximum pooling processing on the weight parameters and extract significant features corresponding to the weight parameters;

[0068] A residual block, wherein the residual block includes the CBAM attention mechanism and is used to perform feature refinement processing on the salient features to obtain terrain features;

[0069] An average pooling layer (avg), used to compress the terrain features to obtain compressed terrain features;

[0070] A fully connected layer (fc), used to restore the dimension of the compressed terrain features to obtain a terrain feature recognition result;

[0071] The output layer (output) is used to output the terrain feature recognition result.

[0072] Furthermore, the method for preprocessing the terrain environment image is specifically as follows:

[0073] The terrain environment image is adjusted to a preset size.

[0074] Furthermore, the maximum pooling layer satisfies the following expression:

[0075]

[0076] Among them, R ij represents the area where the pooling window corresponds to the coordinates (i, j) on the feature map of the normalized feature, x mn is the element value on the feature map of the normalized feature, y ij It is the element value with position coordinate (i, j) on the feature map of the outputted significant feature.

[0077] Furthermore, the average pooling layer satisfies the following expression:

[0078]

[0079] Among them, z c represents the compressed terrain feature, F sq represents the compression operation function, u c represents the terrain feature, W and H represent the width and height of the feature output of the terrain feature respectively, i and j represent the coordinate position on the terrain feature respectively, and C represents the number of feature channels of the terrain feature respectively.

[0080] Furthermore, the input of the residual module is defined as x, the low-level mapping of the residual module is defined as H(x), and the residual mapping of the residual module is defined as F(x)=H(x)-x; the residual module satisfies:

[0081]

[0082] in, is the output of the residual module, h(x) is the identity mapping of x, W is a set of learning weights, and σ is a preset activation function.

[0083] Furthermore, the preset activation function is a ReLU activation function, which satisfies the following expression:

[0084]

[0085] in, for The activation value obtained by the activation layer function; U is the residual function.

[0086] Furthermore, the batch normalization layer satisfies the following expression:

[0087]

[0088] Among them, x is the input value, μ B is the data mean, is the data variance, ∈ is a constant, and γ and β are learnable parameters.

[0089] Furthermore, the CBAM attention mechanism includes a channel attention mechanism and a spatial attention mechanism, which satisfies the following expression:

[0090]

[0091] in, represents element-by-element multiplication, F represents the input feature map, F′ represents the output feature map of the channel attention mechanism, and F″ represents the output feature map of the final CBAM attention mechanism;

[0092] The channel attention mechanism satisfies:

[0093]

[0094] Among them, F represents the input feature map, avgpool represents global average pooling, maxpool represents global maximum pooling, MLP represents multi-layer perceptron, and Represent the weights of the two convolution operations, C is the number of channels, r is the reduction rate, and They represent the results of average pooling and maximum pooling respectively, σ represents the Sigmoid activation function, and m c is the obtained channel attention map;

[0095] The spatial attention mechanism satisfies:

[0096]

[0097] Where F represents the channel attention map output by the channel attention mechanism, f 7×7 represents a convolution operation with a convolution kernel size of 7×7. and They represent the results of average pooling and maximum pooling respectively, h and ω are the height and width of the feature map respectively, and m s is the obtained spatial attention map;

[0098] The Sigmoid activation function satisfies:

[0099]

[0100] Among them, e is a natural constant and x is an input value.

[0101] Furthermore, the terrain environment recognition network uses a Softmax regression function in the output layer to determine the terrain feature type of the terrain feature recognition result, and the Softmax regression function satisfies the following expression:

[0102]

[0103] Among them, (x i ,y i ) represents the training data, y i ∈{1, 2, …, k} represents the category to which the data belongs, n represents the number of samples, I{·} represents the logical expression function, and θ is the training set.

[0104] The beneficial effect achieved by the present invention is that a terrain environment recognition method based on ResNet residual network is proposed. This method greatly reduces the network training time by combining ResNet residual network and CBAM attention mechanism, and can achieve faster and more accurate terrain environment classification, which provides strong assistance for outdoor path planning and navigation of mobile robots.

[0105] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0106] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0107] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0108] The embodiments of the present invention are described above in conjunction with the accompanying drawings. What is disclosed is only the preferred embodiment of the present invention. However, the present invention is not limited to the above-mentioned specific implementation manner. The above-mentioned specific implementation manner is only illustrative rather than restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms and equivalent changes without departing from the scope of protection of the purpose of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A terrain environment recognition method based on ResNet residual network, characterized in that: The following steps are involved: Construct a terrain environment recognition network based on ResNet residual network and CBAM attention mechanism; Training and optimizing the terrain environment recognition network; Acquire a terrain environment image, and preprocess the terrain environment image to obtain an input image; The input image is input into the terrain environment recognition network to perform terrain environment recognition to obtain a terrain environment recognition result.

2. The terrain environment recognition method based on ResNet residual network according to claim 1 is characterized in that: The terrain environment recognition network includes, in order of logical input and output, the following: Input layer, used to obtain the image to be recognized; A convolution layer, used for performing convolution processing on the image to be recognized to obtain a first output feature; A batch normalization layer, used for normalizing the first output features to obtain normalized features; An activation function layer, used to perform excitation processing on the normalized features according to a preset activation function to obtain weight parameters of different feature channels in the normalized features; A maximum pooling layer, used to perform maximum pooling processing on the weight parameters and extract significant features corresponding to the weight parameters; A residual module, wherein the residual module includes the CBAM attention mechanism, and the residual module is used to perform feature refinement processing on the salient features to obtain terrain features; An average pooling layer, used to compress the terrain features to obtain compressed terrain features; A fully connected layer, used for restoring the dimension of the compressed terrain features to obtain a terrain feature recognition result; The output layer is used to output the terrain feature recognition result.

3. The terrain environment recognition method based on ResNet residual network according to claim 1 is characterized in that: The method for preprocessing the terrain environment image is specifically as follows: The terrain environment image is adjusted to a preset size.

4. The terrain environment recognition method based on ResNet residual network according to claim 2 is characterized in that: The maximum pooling layer satisfies the following expression: Among them, R ij represents the area where the pooling window corresponds to the coordinates (i, j) on the feature map of the normalized feature, x mn is the element value on the feature map of the normalized feature, y ij It is the element value with position coordinate (i, j) on the feature map of the outputted significant feature.

5. The terrain environment recognition method based on ResNet residual network according to claim 2 is characterized in that: The average pooling layer satisfies the following expression: Among them, z c represents the compressed terrain feature, F sq represents the compression operation function, u c represents the terrain feature, W and H represent the width and height of the feature output of the terrain feature respectively, i and j represent the coordinate position on the terrain feature respectively, and C represents the number of feature channels of the terrain feature respectively.

6. The terrain environment recognition method based on ResNet residual network according to claim 2 is characterized in that: The input of the residual module is defined as x, the low-level mapping of the residual module is defined as H(x), and the residual mapping of the residual module is defined as F(x)=H(x)-x; the residual module satisfies: in, is the output of the residual module, h(x) is the identity mapping of x, W is a set of learning weights, and σ is a preset activation function.

7. The terrain environment recognition method based on ResNet residual network according to claim 6 is characterized in that: The preset activation function is a ReLU activation function, which satisfies the following expression: in, for The activation value obtained by the activation layer function; U is the residual function.

8. The terrain environment recognition method based on ResNet residual network according to claim 2 is characterized in that: The batch normalization layer satisfies the following expression: Among them, x is the input value, μ B is the data mean, is the data variance, ∈ is a constant, and γ and β are learnable parameters.

9. The terrain environment recognition method based on ResNet residual network according to claim 2 is characterized in that: The CBAM attention mechanism includes a channel attention mechanism and a spatial attention mechanism, which satisfies the following expression: in, represents element-by-element multiplication, F represents the input feature map, F′ represents the output feature map of the channel attention mechanism, and F″ represents the output feature map of the final CBAM attention mechanism; The channel attention mechanism satisfies: Among them, F represents the input feature map, avgpool represents global average pooling, maxpool represents global maximum pooling, MLP represents multi-layer perceptron, and Represent the weights of the two convolution operations, C is the number of channels, r is the reduction rate, and Represent the results of average pooling and maximum pooling respectively, σ represents the Sigmoid activation function, m c is the obtained channel attention map; The spatial attention mechanism satisfies: Where F represents the channel attention map output by the channel attention mechanism, f 7×7 represents a convolution operation with a convolution kernel size of 7×7. and They represent the results of average pooling and maximum pooling respectively, h and ω are the height and width of the feature map respectively, and m s is the obtained spatial attention map; The Sigmoid activation function satisfies: Among them, e is a natural constant and x is an input value.

10. The terrain environment recognition method based on ResNet residual network according to claim 2 is characterized in that: The terrain environment recognition network uses a Softmax regression function in the output layer to determine the type of terrain features of the terrain feature recognition result, and the Softmax regression function satisfies the following expression: Among them, (x i ,y i ) represents the training data, y i ∈{1, 2, …, k} represents the category to which the data belongs, n represents the number of samples, I{} represents the logical expression function, and θ is the training set.

Images