Road surface type identification method and device based on lightweight residual network

Abstract

The invention provides a road surface type identification method and device based on a lightweight residual network, and relates to the field of traffic road surface identification, and the method comprises the steps: S1, constructing a lightweight residual network through a basic road surface image classification model; s2, obtaining a road surface classification image set, and preprocessing the road surface classification image set to obtain a training image set; s3, training the lightweight residual network through the training image set to obtain a trained lightweight residual network; and S4, recognizing the to-be-recognized pavement image through the trained lightweight residual network to obtain a pavement classification result. According to the method, a common convolutional layer conv in a residual module Conv Block and an identification module Identity Block is replaced by a deep separable convolutional layer DWconv, and lightweight DSC-Conv Block and DSC-Identity are obtained, so that a lightweight residual network is constructed, the parameter quantity of the network is reduced, and the road surface identification speed is improved.

Description

Technical Field

[0001] The present invention relates to the field of traffic pavement recognition, and in particular to a pavement type recognition method and device using a lightweight residual network. Background Art

[0002] With the rapid development of artificial intelligence, perception detection and other technologies, driverless technology has gradually matured and is moving towards commercial use. High-quality environmental perception information is a key prerequisite for the successful application of driverless technology. Environmental perception information can only be converted into control information through processing by the on-board computing platform. At present, most driverless cars use general heterogeneous on-board computing platforms (such as industrial computers, development boards, etc.) to handle various types of tasks, but the performance is not strong and the interface is limited.

[0003] More and more researchers are using sensing devices and deep neural networks to identify road types, but the number of parameters in deep neural network models is huge, resulting in a sharp increase in the amount of calculations. For example, ResNet50 contains 25 million parameters and requires 380 billion floating-point operations. The huge number of parameters and operations has led to a continuous increase in the computing resources required for the model, making it difficult to deploy on the vehicle computing platform. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a road type recognition method and device based on a lightweight residual network, so as to solve the technical problem that the existing ResNet50 model has a huge number of parameters.

[0005] The present invention provides a road type recognition method using a lightweight residual network, comprising:

[0006] S1: Build a lightweight residual network through the basic road image classification model;

[0007] S2: Acquire a road surface classification image set, preprocess the road surface classification image set, and obtain a training image set;

[0008] S3: Train the lightweight residual network through the training image set to obtain a trained lightweight residual network;

[0009] S4: The road image to be identified is identified through the trained lightweight residual network to obtain the road classification result.

[0010] Preferred:

[0011] The lightweight residual network includes Stage0 network, Stage1 network, Stage2 network, Stage3 network, Stage4 network and output layer connected in sequence;

[0012] The Stage0 network includes a Conv layer, a BN layer, a ReLU activation function layer, and a maximum pooling layer connected in sequence;

[0013] The Stage 1 network includes a first DSC-Conv Block unit, a first DSC-Identity Block unit, and a second DSC-Identity Block unit connected in sequence;

[0014] The Stage2 network includes a second DSC-Conv Block unit, a third DSC-Identity Block unit, a fourth DSC-Identity Block unit, and a fifth DSC-Identity Block unit connected in sequence;

[0015] The Stage3 network includes a third DSC-Conv Block unit, a sixth DSC-Identity Block unit, a seventh DSC-Identity Block unit, an eighth DSC-Identity Block unit, a ninth DSC-IdentityBlock unit, and a tenth DSC-Identity Block unit connected in sequence;

[0016] The Stage 4 network includes a fourth DSC-Conv Block unit, an eleventh DSC-IdentityBlock unit, and a twelfth DSC-Identity Block unit connected in sequence;

[0017] The output layer consists of an average pooling layer, a fully connected layer, and a softmax layer connected sequentially.

[0018] Preferred:

[0019] The structures of the first DSC-Identity Block unit to the twelfth DSC-Identity Block unit are the same;

[0020] The DSC-Identity Block unit includes a first 1×1conv module, a first 3×3DWconv module, and a second 1×1conv module connected in sequence;

[0021] The input of the first 1×1conv module and the output of the second 1×1conv module are added through the relu function as the output of the DSC-Identity Block unit.

[0022] Preferred:

[0023] The structures of the first DSC-Conv Block unit to the fourth DSC-Conv Block unit are the same;

[0024] The DSC-Conv Block unit includes: a third 1×1conv module, a second 3×3DWconv module, a fourth 1×1conv module, and a fifth 1×1conv module;

[0025] The third 1×1conv module, the second 3×3DWconv module and the fourth 1×1conv module are connected in sequence.

[0026] The input of the third 1×1conv module is connected to the fifth 1×1conv module, and the output of the fifth 1×1conv module is added to the output of the fourth 1×1conv module through the relu function as the output of the DSC-Conv Block unit.

[0027] Preferably, step S2 specifically comprises:

[0028] S21: adjusting the sizes of the images in the road surface classification image set to the same size, to obtain an adjusted road surface classification image set;

[0029] S22: extracting a preset number of road surface classification images from the adjusted road surface classification image set to form a training image set.

[0030] Preferably, step S3 specifically comprises:

[0031] S31: Input the training image into the lightweight residual network and calculate the cross entropy loss function;

[0032] S32: Optimize the parameters of the lightweight residual network through the Adam optimizer;

[0033] S33: Repeat steps S31-S32 until the cross entropy loss function is less than a preset value or the maximum number of training times is reached to obtain a trained lightweight residual network.

[0034] Preferred:

[0035] The expression of the cross entropy loss function L(p,q) is:

[0036]

[0037] Among them, x i is the i-th type of road surface, n is the total number of road surface types, p(x i ) is the true label value, q(x i ) is the prediction probability of the lightweight residual network for the i-th type of road surface.

[0038] A road type recognition device for a lightweight residual network comprises: a processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement the road type recognition method for the lightweight residual network.

[0039] The present invention has the following beneficial effects:

[0040] The ordinary convolution layer conv in the residual module Conv Block and the identification module Identity Block is replaced with the depthwise separable convolution layer DWconv to obtain the lightweight DSC-Conv Block and DSC-Identity to construct a lightweight residual network, reduce the number of network parameters, and improve the recognition speed of the road surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;

[0042] Figure 2 It is the structural diagram of the depth-separable convolutional layer DWconv;

[0043] Figure 3 This is the structural diagram of the lightweight residual network;

[0044] Figure 4 This is a system structure diagram of an embodiment of the present invention;

[0045] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0047] Reference Figure 1 The present invention provides a road type recognition method using a lightweight residual network, comprising:

[0048] S1: Build a lightweight residual network through the basic road image classification model;

[0049] Furthermore, a basic road image classification model ResNet50 is built, which includes five stages from Stage0 to Stage4. The input image first passes through the convolution layer, batch normalization, ReLU activation function, and maximum pooling layer of Stage0 to reduce the image size while extracting preliminary features. The structures of Stage1 to Stage4 are composed of the Conv Block, which is called the residual module, and the Identity Block structure of the recognition module. The Conv Block is mainly responsible for downsampling and changing the number of channels of image information. The Identity Block is mainly used to deepen the depth of the network while extracting deeper feature representations. Finally, after the average pooling layer, the image is converted into a vector form. Then it is classified through the fully connected layer and the softmax layer output.

[0050] Depthwise separable convolution consists of depthwise convolution and pointwise convolution, and has the feature of "plug and play". Its principle is as follows Figure 2 As shown in the figure. The 3×3 depth-separable convolutional layers are used to replace the 3×3 ordinary convolutional layers in the identity module and the residual module Conv Block. The Conv Block and Identity Block structures are lightened to DSC-Conv Block and DSC-Identity Block, respectively. Therefore, the traditional ResNet50 network structure is lightened to obtain a lightened residual network. Its network structure is as follows: Figure 3 As shown, the lightweight residual network is called DSC-ResNet50. Figure 3 The ordinary convolution layer is denoted as conv and the depthwise separable convolution layer is denoted as DWconv.

[0051] The lightweight residual network includes Stage0 network, Stage1 network, Stage2 network, Stage3 network, Stage4 network and output layer connected in sequence;

[0052] The Stage0 network includes a Conv layer, a BN layer, a ReLU activation function layer, and a maximum pooling layer connected in sequence;

[0053] The Stage 1 network includes a first DSC-Conv Block unit, a first DSC-Identity Block unit, and a second DSC-Identity Block unit connected in sequence;

[0054] The Stage2 network includes a second DSC-Conv Block unit, a third DSC-Identity Block unit, a fourth DSC-Identity Block unit, and a fifth DSC-Identity Block unit connected in sequence;

[0055] The Stage3 network includes a third DSC-Conv Block unit, a sixth DSC-Identity Block unit, a seventh DSC-Identity Block unit, an eighth DSC-Identity Block unit, a ninth DSC-IdentityBlock unit, and a tenth DSC-Identity Block unit connected in sequence;

[0056] The Stage 4 network includes a fourth DSC-Conv Block unit, an eleventh DSC-IdentityBlock unit, and a twelfth DSC-Identity Block unit connected in sequence;

[0057] The output layer consists of an average pooling layer, a fully connected layer, and a softmax layer connected sequentially.

[0058] Further, the structures of the first DSC-Identity Block unit to the twelfth DSC-Identity Block unit are the same;

[0059] The DSC-Identity Block unit includes a first 1×1conv module, a first 3×3DWconv module, and a second 1×1conv module connected in sequence;

[0060] The input of the first 1×1conv module and the output of the second 1×1conv module are added through the relu function as the output of the DSC-Identity Block unit.

[0061] Further, the structures of the first DSC-Conv Block unit to the fourth DSC-Conv Block unit are the same;

[0062] The DSC-Conv Block unit includes: a third 1×1conv module, a second 3×3DWconv module, a fourth 1×1conv module, and a fifth 1×1conv module;

[0063] The third 1×1conv module, the second 3×3DWconv module and the fourth 1×1conv module are connected in sequence.

[0064] The input of the third 1×1conv module is connected to the fifth 1×1conv module, and the output of the fifth 1×1conv module is added to the output of the fourth 1×1conv module through the relu function as the output of the DSC-Conv Block unit.

[0065] Specifically, in the depthwise separable convolutional layer, assuming that the size of the input feature map is W in ×H in ×M, where W in ×H in is the width and height of the input feature map, M is the number of channels of the input feature map, N is the number of output channels of the convolution layer, and the size of the convolution kernel is D k ×D k .

[0066] Since the traditional Conv Block and Identity Block structures are lightened to DSC-Conv Block and DSC-IdentityBlock respectively, the parameters of Stage 1 after lightening are calculated using the following formula (1):

[0067] Params DSCStage1 =Params DSCconv +Params DSCiden ×2 (1)

[0068] Among them, Params DSCconv Indicates the number of parameters of the DSC-Conv Block structure, Params DSCiden The parameter quantities representing the DSC-Identity Block structure are calculated as shown in equations (2) and (3) respectively.

[0069] Params DSCconv =Params DSCconv(1×1,1) +Params DSCconv(3×3,2) +Params DSCconv(1×1,3) +Params DSCconv(1×1,sam) (2)

[0070] Params DSCiden =Params DSCiden(1×1,1) +Params DSCiden(3×3,2) +Params DSCiden(1×1,3) (3)

[0071] Among them, Params DSCconv(1×1,1) Indicates the number of parameters of the first 1×1 convolutional layer of the DSC-Conv Block structure, Params DSCconv(3×3,2) Indicates the number of parameters of the second 3×3 convolutional layer of the DSC-Conv Block structure, ParamsDSCconv(1×1,3) Indicates the parameter quantity of the third 1×1 convolution layer of the DSC-Conv Block structure, Params DSCconv(1×1,sam) Indicates the parameter quantity of the 1×1 convolution layer on the branch path of the DSC-ConvBlock structure, Params DSCiden(1×1,1) 、Params DSCiden(3×3,2) 、Params DSCiden(1×1,3) Indicates the parameter quantities of the 1×1 convolution layer, 3×3 convolution layer, and 1×1 convolution layer in the DSC-Identity Block structure.

[0072] Since the DSC-Conv Block structure consists of two 1×1 convolutional layers, a 3×3 convolutional layer, and a 1×1 convolutional layer on the branch path, the parameters of each convolutional layer are calculated using formulas (4), (5), (6), and (7), as shown below:

[0073] Params DSCconv(1×1,1) =D k(1×1) ×D k(1×1) ×M S1,conv,1,1×1 ×N S1,conv,1,1×1 (4)

[0074] Params DSCconv(3×3,2) =D k(3×3) ×D k(3×3) ×M S1,conv,2,3×3 (5)

[0075] Params DSCconv(1×1,3) =D k(1×1) ×D k(1×1) ×M S1,conv,3,1×1 ×N S1,conv,3,1×1 (6)

[0076] Params DSCconv(1×1,sam) =D k(1×1) ×D k(1×1) ×M S1,conv,sam,1×1 ×N S1,conv,sam,1×1 (7)

[0077] Similarly, the parameters of each convolutional layer in the DSC-Identity Block structure are calculated using formulas (8), (9), and (10), as shown below:

[0078] Params DSCiden(1×1,1) =D k(1×1) ×D k(1×1) ×M S1,iden,1,1×1 ×N S1,iden,1,1×1 (8)

[0079] Params DSCiden(3×3,2) =D k(3×3) ×Dk(3×3) ×M S1,iden,2,3×3 (9)

[0080] Params DSCiden(1×1,3) =D k(1×1) ×D k(1×1) ×M S1,iden,3,1×1 ×N S1,iden,3,1×1 (10)

[0081] S2: Acquire a road surface classification image set, preprocess the road surface classification image set, and obtain a training image set;

[0082] Furthermore, step S2 is specifically as follows:

[0083] S21: adjusting the sizes of the images in the road surface classification image set to the same size, to obtain an adjusted road surface classification image set;

[0084] Specifically, the open source road surface image classification dataset (RSCD) is used for road surface classification. The road surface image classification dataset (RSCD) was released by the research team of the School of Vehicle Transportation of Tsinghua University in 2022. It includes 27 different road surface categories.

[0085] The open source road surface classification dataset (RSCD) is preprocessed. The preprocessing method includes: to meet the input and processing requirements of the residual network, the image is resized to 224×224.

[0086] S22: extracting a preset number of road surface classification images from the adjusted road surface classification image set to form a training image set.

[0087] Specifically, the preprocessed RSCD dataset is divided into a training set, a validation set, and a test set in proportion, wherein the training set is 970,000 images, the validation set is 20,000 images, and the test set is 50,000 images.

[0088] S3: Train the lightweight residual network through the training image set to obtain a trained lightweight residual network;

[0089] Furthermore, step S3 is specifically as follows:

[0090] S31: Input the training image into the lightweight residual network and calculate the cross entropy loss function;

[0091] Specifically, the expression of the cross entropy loss function L(p,q) is:

[0092]

[0093] Among them, x i is the i-th type of road surface, n is the total number of road surface types, p(x i) is the true label value, q(x i ) is the prediction probability of the lightweight residual network for the i-th type of road surface.

[0094] S32: Optimize the parameters of the lightweight residual network through the Adam optimizer;

[0095] S33: Repeat steps S31-S32 until the cross entropy loss function is less than a preset value or the maximum number of training times is reached to obtain a trained lightweight residual network.

[0096] S4: The road image to be identified is identified through the trained lightweight residual network to obtain the road classification result.

[0097] Specifically, the implementation effects of the road surface classification results are as follows:

[0098] Taking Stage 1 as an example, using formulas (1) to (10) and the proposed lightweight residual network, the reduction ratio of Stage 1 after lightweighting is 51.11%. Similarly, the number of parameters for Stage 2 to Stage 4 is reduced by 48.27%, 49.80%, and 47.28%, respectively. The total number of parameters for ResNet50 is reduced by 47.99%.

[0099] The open source road surface classification dataset (RSCD) is used for road surface classification. The number of training iterations is set to 50 epochs, the learning rate is set to 0.001, and the number of images used in each iteration during model training is batch_size 64. The results are shown in Table 1, which shows the number of parameters, computational complexity, accuracy, and average training time per epoch of DSC-ResNet50 and ResNet 50 respectively.

[0100] Table 1

[0101]

[0102] The experimental results show that the number of parameters of DSC-ResNet50 is reduced by 11.28M compared with ResNet50, with a reduction ratio of 47.87%. The amount of calculation is reduced by 117.59G, with a reduction ratio of 44.47%, which shows that the number of parameters and the amount of calculation of the network have been significantly reduced. At the same time, the classification accuracy of DSC-ResNet50 is improved by 1.07% compared with ResNet50. And the average training time per epoch is reduced by 705s, which shows that the training speed of the improved network has been improved, which has played a significant lightweight improvement effect.

[0103] See also Figure 4 , Figure 44 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, wherein the hardware device specifically comprises: a road type recognition device 401 of a lightweight residual network, a processor 402 and a storage medium 403.

[0104] A road surface type recognition device 401 for a lightweight residual network: The road surface type recognition device 401 for a lightweight residual network implements the road surface type recognition method for the lightweight residual network.

[0105] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the road type recognition method of the lightweight residual network.

[0106] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the road surface type recognition method of the lightweight residual network.

[0107] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0108] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.

[0109] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A road type recognition method based on a lightweight residual network, characterized in that: include: S1: Build a lightweight residual network through the basic road image classification model; S2: Acquire a road surface classification image set, preprocess the road surface classification image set, and obtain a training image set; S3: Train the lightweight residual network through the training image set to obtain a trained lightweight residual network; S4: The road image to be identified is identified through the trained lightweight residual network to obtain the road classification result.

2. The road type recognition method using a lightweight residual network according to claim 1 is characterized in that: The lightweight residual network includes Stage0 network, Stage1 network, Stage2 network, Stage3 network, Stage4 network and output layer connected in sequence; The Stage0 network includes a Conv layer, a BN layer, a ReLU activation function layer, and a maximum pooling layer connected in sequence; The Stage 1 network includes a first DSC-Conv Block unit, a first DSC-Identity Block unit, and a second DSC-Identity Block unit connected in sequence; The Stage2 network includes a second DSC-Conv Block unit, a third DSC-Identity Block unit, a fourth DSC-Identity Block unit, and a fifth DSC-Identity Block unit connected in sequence; The Stage3 network includes a third DSC-Conv Block unit, a sixth DSC-Identity Block unit, a seventh DSC-Identity Block unit, an eighth DSC-Identity Block unit, a ninth DSC-IdentityBlock unit, and a tenth DSC-Identity Block unit connected in sequence; The Stage 4 network includes a fourth DSC-Conv Block unit, an eleventh DSC-Identity Block unit, and a twelfth DSC-Identity Block unit connected in sequence; The output layer consists of an average pooling layer, a fully connected layer, and a softmax layer connected sequentially.

3. The road type recognition method using a lightweight residual network according to claim 2 is characterized in that: The structures of the first DSC-Identity Block unit to the twelfth DSC-Identity Block unit are the same; The DSC-Identity Block unit includes a first 1×1conv module, a first 3×3DWconv module, and a second 1×1conv module connected in sequence; The input of the first 1×1conv module and the output of the second 1×1conv module are added through the relu function as the output of the DSC-Identity Block unit.

4. The road type recognition method using a lightweight residual network according to claim 2 is characterized in that: The structures of the first DSC-Conv Block unit to the fourth DSC-Conv Block unit are the same; The DSC-Conv Block unit includes: a third 1×1conv module, a second 3×3DWconv module, a fourth 1×1conv module, and a fifth 1×1conv module; The third 1×1conv module, the second 3×3DWconv module and the fourth 1×1conv module are connected in sequence. The input of the third 1×1conv module is connected to the fifth 1×1conv module, and the output of the fifth 1×1conv module is added to the output of the fourth 1×1conv module through the relu function as the output of the DSC-Conv Block unit.

5. The road type recognition method using a lightweight residual network according to claim 1 is characterized in that: Step S2 is specifically as follows: S21: adjusting the sizes of the images in the road surface classification image set to the same size, to obtain an adjusted road surface classification image set; S22: extracting a preset number of road surface classification images from the adjusted road surface classification image set to form a training image set.

6. The road type recognition method using a lightweight residual network according to claim 1, characterized in that: Step S3 is specifically as follows: S31: Input the training image into the lightweight residual network and calculate the cross entropy loss function; S32: Optimize the parameters of the lightweight residual network through the Adam optimizer; S33: Repeat steps S31-S32 until the cross entropy loss function is less than a preset value or the maximum number of training times is reached to obtain a trained lightweight residual network.

7. The road type recognition method using a lightweight residual network according to claim 6 is characterized in that: The expression of the cross entropy loss function L(p,q) is: Among them, x i is the i-th type of road surface, n is the total number of road surface types, p(x i ) is the true label value, q(x i ) is the prediction probability of the lightweight residual network for the i-th type of road surface.

8. A road type recognition device based on a lightweight residual network, characterized in that: include: Processor and storage medium; the processor loads and executes instructions and data in the storage medium to implement the road type identification method of the lightweight residual network described in any one of claims 1 to 7.

Images