Road disease prediction method based on neural network architecture search and related device

Abstract

The invention belongs to the technical field of road disease prediction, and provides a road disease prediction method based on neural network architecture search and a related device. The method comprises the steps that a first particle swarm is determined in a scale adaptive search space, the first particle swarm comprises n second particle swarms, and the second particle swarms comprise m randomly generated third particle swarms; performing initialization processing on n second particle swarms in the first particle swarm by adopting a multi-particle swarm initialization strategy to obtain an optimal second particle swarm; performing optimization processing on m third particle swarms in the optimal second particle swarm by adopting a particle swarm dynamic adaptive updating strategy to obtain an optimal third particle swarm; determining the neural network containing the MBConv layer corresponding to the optimal third particle swarm as an optimal road disease prediction network; and performing road disease prediction by adopting the optimal road disease prediction network. According to the method, the neural network for road disease image prediction can be automatically and efficiently searched, so that the searched neural network not only can perform multi-scale feature extraction on the road disease image, but also can effectively balance the classification performance and complexity.

Description

Technical Field

[0001] The present invention relates to the technical field of road damage prediction, and in particular to a road damage prediction method and related devices based on neural network architecture search. Background Art

[0002] Road defects such as cracks, potholes and unevenness not only reduce the service life of roads and increase maintenance costs, but may also seriously threaten road traffic safety and affect driving comfort and efficiency. In the field of autonomous driving, road defect prediction technology is particularly critical. They can enhance the perception ability of autonomous driving systems, help vehicles identify road conditions in real time, and prevent traffic accidents. In recent years, machine learning and deep learning have improved the efficiency and safety of traffic systems due to their efficient road defect prediction performance. Although road defect prediction has achieved initial results, designing a deep neural network with good performance requires rich professional domain knowledge.

[0003] At present, the neural network search method based on particle swarm optimization is widely used due to its flexible design and easy implementation. However, due to the multi-scale nature of road damage images, the direct application of evolutionary neural network search algorithms based on particle swarm optimization for classification and prediction still faces the following challenges: (1) Unable to identify road damage areas of different scales: Most evolutionary neural network search methods based on particle swarm optimization are designed for natural image classification. The key difference between road damage images and natural images at different scales leads to low classification accuracy of existing methods; (2) Easy to fall into local optimality: The evolutionary neural network search method based on particle swarm optimization is prone to fall into local optimality, resulting in weak global search ability and inability to cope with dynamic environments; (3) High computational resource overhead and overly complex model: The network structure searched by the evolutionary neural network search method is complex, and the search process consumes a lot of computing resources and time overhead. Resource-constrained devices easily lead to the failure of the evolution process. Summary of the invention

[0004] In view of the defects in the prior art, the present invention provides a road damage prediction method and related devices based on neural network architecture search, which can automatically and efficiently search for neural networks for road damage image prediction, so that the searched neural network can not only extract multi-scale features of road damage images, but also effectively balance the classification performance of the neural network and the complexity of the neural network.

[0005] A road damage prediction method based on neural network architecture search, comprising:

[0006] Determine a first particle group in the scale adaptive search space, the first particle group includes n second particle groups, and the second particle group includes m randomly generated third particle groups;

[0007] The multi-particle swarm initialization strategy is used to initialize the n second particle swarms in the first particle swarm to obtain the best second particle swarm;

[0008] The particle swarm dynamic adaptive update strategy is used to optimize the m third particle swarms in the best second particle swarm to obtain the best third particle swarm;

[0009] The neural network including the MBConv layer corresponding to the best third particle group is determined as the optimal road disease prediction network;

[0010] The optimal road damage prediction network is used to predict road damage.

[0011] Further, the method also includes constructing a neural network including an MBConv layer according to the third particle group, and constructing a neural network including an MBConv layer according to the third particle group includes:

[0012] The number of particles in the third particle group is determined as the number of MBConv layers in the neural network;

[0013] The numerical values ​​of the particles in the third particle group are determined as the parameters of the MBConv layer in the neural network;

[0014] The MBConv layer in the neural network is constructed according to the number of layers of the MBConv layer in the neural network and the parameters of the MBConv layer in the neural network, so as to obtain a neural network including the MBConv layer.

[0015] Furthermore, a multi-particle swarm initialization strategy is used to initialize n second particle swarms in the first particle swarm to obtain the best second particle swarm, including:

[0016] Calculate the initial fitness values ​​of n second particle groups in the first particle group;

[0017] The second particle swarm with the highest initial fitness value is taken as the best second particle swarm.

[0018] Furthermore, the initial fitness values ​​of the n second particle groups in the first particle group are calculated, including:

[0019] The average classification performance of the n second particle groups in the first particle group is obtained by using zero sample evaluation, and the average parameters and average floating point numbers of the n second particle groups in the first particle group are obtained by calculation;

[0020] The average classification performance, average parameter and average floating point number of the n second particle swarms are weighted by using the optimal weight combination to obtain the initial fitness values ​​of the n second particle swarms.

[0021] Furthermore, before the particle swarm dynamic adaptive update strategy is used to optimize the m third particle swarms in the best second particle swarm, the method further includes:

[0022] The multi-objective performance evaluation function is used to obtain the evolutionary fitness value of each third particle group of the best second particle group during each iterative calculation of the best third particle group, and the current fitness value of each third particle group of the best second particle group is obtained;

[0023] According to the current fitness value of each third particle group of the best second particle group, the current average fitness value of the best second particle group in each iterative calculation of the best third particle group is obtained;

[0024] The solution environment of the best second particle group is determined by using the ratio of the current fitness value of each third particle group of the best second particle group to the current average fitness value of the best second particle group;

[0025] The calculation formula is:

[0026]

[0027] Among them, p i represents the solution environment of the best second particle group of the i-th third particle group at the t-th generation of evolution, exp represents the power of e, and x i (t) is the spatial distribution position of the ith third particle group after the tth generation of evolution, fit(·) is the current fitness value of the ith third particle group, and m is the number of third particle groups in the best second particle group.

[0028] Furthermore, the multi-objective performance evaluation function is characterized by the following formula:

[0029]

[0030] Wherein, Fitness(i) represents the evolutionary fitness value of the i-th third particle swarm, Zen-Score(·) represents the zero sample, #Params(i) represents the parameter quantity of the i-th third particle swarm, #FLOPs(i) represents the floating point operation quantity of the i-th third particle swarm, T p and T f They represent the maximum number of parameters and the maximum number of floating-point operations allowed, respectively. o and w q They respectively represent the trade-off between limiting the number of parameters and the amount of floating-point operations.

[0031] Furthermore, the particle swarm dynamic adaptive update strategy includes:

[0032] When p i >0.6, use "x i(t+1)=x i (t)+v i (t+1)” updates the spatial distribution position of the i-th third particle group, and the calculation formula is expressed as:

[0033]

[0034] When p i <0.6 or p i =0.6, use "x i (t+1)=w(t+1)x i (t)+(1-w)v i (t+1)” updates the spatial distribution position of the i-th third particle group, and the calculation formula is expressed as:

[0035]

[0036] Among them, t represents the current iteration number, v i (t) represents the velocity of the i-th third particle group, w(t+1) is the inertia weight of the best third particle group, (-1) γ is the dynamic reverse learning control factor, γ is a binary number {0, 1}, c1 is the individual learning factor, c2 is the social learning factor, r1, r2, β are random numbers between [0-1]. i It is represented as a local optimal solution of the third particle group in the second particle group, gbest i is represented as a global optimal solution of the third particle swarm in the second particle swarm, cos(·) represents the rotation of the third particle swarm, L is a random number in [0.4, 0.5], x i (t) represents the spatial distribution position of the i-th third particle group after the t-th generation of evolution.

[0037] A road damage prediction device based on neural network architecture search, comprising:

[0038] A first determining unit is used to determine a first particle group in the scale adaptive search space, the first particle group includes n second particle groups, and the second particle group includes m randomly generated third particle groups;

[0039] An initialization processing unit, used for initializing n second particle groups in the first particle group by adopting a multi-particle group initialization strategy to obtain an optimal second particle group;

[0040] An optimization processing unit, used for optimizing the m third particle groups in the best second particle group by adopting a particle group dynamic adaptive update strategy to obtain the best third particle group;

[0041] A second determining unit is used to determine the neural network including the MBConv layer corresponding to the best third particle group as the optimal road disease prediction network;

[0042] The prediction unit is used to predict road damage using an optimal road damage prediction network.

[0043] A terminal includes a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute any one of the above methods.

[0044] A computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes any one of the above methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific embodiments of the present invention, the following will briefly introduce the drawings required for the specific embodiments or the prior art description. In all the drawings, each element or part is not necessarily drawn according to the actual scale.

[0046] Figure 1 A flowchart of a road damage prediction method based on a neural network architecture search provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of the structure of a road damage prediction device based on a neural network architecture search provided by an embodiment of the present invention;

[0048] Figure 3 A comparison chart of experimental results of reliability analysis of road damage images using feature maps on Road, RSCD and RTK datasets using the road damage prediction method provided by an embodiment of the present invention and using FPSO-NAS, EDE-NAS, EPC-NAS and SLE-NAS methods;

[0049] Figure 4 This is a comparison chart of experimental results of reliability analysis of road damage sampling images using a heat map using the road damage prediction method provided by an embodiment of the present invention and using FPSO-NAS, EDE-NAS, EPC-NAS, and SLE-NAS methods. DETAILED DESCRIPTION

[0050] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0051] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0052] In one embodiment, if Figure 1 As shown, a road damage prediction method based on multi-particle swarm zero-shot neural architecture search is provided, including:

[0053] 1. Determine a first particle group in a scale-adaptive search space, wherein the first particle group includes n second particle groups, and the second particle group includes m randomly generated third particle groups;

[0054] Optionally, in one of the embodiments, it further includes constructing a neural network including an MBConv layer according to the third particle group,

[0055] According to the third particle group, a neural network including the MBConv layer is constructed, including:

[0056] (1) Determine the number of particles in the third particle group as the number of MBConv layers in the neural network;

[0057] (2) Determining the numerical values ​​of the particles in the third particle group as the parameters of the MBConv layer in the neural network;

[0058] (3) The MBConv layer in the neural network is constructed according to the number of MBConv layers in the neural network and the parameters of the MBConv layer in the neural network to obtain a neural network including the MBConv layer.

[0059] Specifically, a separation coding strategy is adopted for a neural network including an MBConv layer, and the neural network is mapped into a third particle group in a scale-adaptive search space. The number of particles in the third particle group is determined as the number of layers of the MBConv layer in the neural network; and the value of the particles in the third particle group is determined as the parameter of the MBConv layer in the neural network.

[0060] Preferably, in one of the embodiments, the parameters of the MBConv layer in the neural network are divided into the number of convolution kernels and other parameter configurations of the MBConv layer, the numerical values ​​of the particles in the third particle group are divided into an integer part and a decimal part, the integer part of the particles in the third particle group is used to represent the other parameter configurations of the MBConv layer in the neural network, and the decimal part of the particles in the third particle group is used to represent the number of convolution kernels of the MBConv layer in the neural network.

[0061] Preferably, other parameter configurations of the MBConv layer in the neural network include the convolution kernel size, the expansion ratio, and whether there is an attention mechanism.

[0062] Preferably, by permuting and combining different convolution kernel sizes, expansion ratios, and whether or not there is an attention mechanism, 13 different other parameter configurations of the MBConv layer are formed, as shown in Table 1.

[0063] Table 1

[0064] Encoded value Convolutional layer types Convolution operation configuration 0 Indentity — 1 MBConv K=3,e=3,w / oSE 2 MBConv K=3,e=3,w / SE 3 MBConv K=5,e=3,w / oSE 4 MBConv K=5,e=3,w / SE 5 MBConv K=3,e=4,w / oSE 6 MBConv K=3,e=4,w / SE 7 MBConv K=5,e=4,w / oSE 8 MBConv K=5,e=4,w / SE 9 MBConv K=3,e=6,w / oSE 10 MBConv K=3,e=6,w / SE 11 MBConv K=5,e=6,w / oSE 12 MBConv K=5,e=6,w / SE

[0065] For example, assuming that the value of the particle in the third particle group is 5.31, including the integer part 5 and the decimal part "31", the integer part 5 in the particle represents the other parameter configuration of the MBConv layer encoding value of 5 in the neural network, as shown in Table 1, that is, the convolution kernel size is 3, the expansion ratio is 4, there is no parameter configuration of the attention mechanism, and the MBConv layer in the neural network represented by the particle contains 31 convolution kernels.

[0066] Preferably, the integer part of the particles in the third particle group cannot exceed 12, and the decimal part of the particles in the third particle group cannot exceed 100, so as to prevent the scale adaptive search space from running too slowly.

[0067] This embodiment encodes the parameters of the MBConv layer in the neural network into the scale-adaptive search space corresponding to the example in the third particle group, which not only effectively separates the number of convolution kernels and other parameter configurations of the MBConv layer in the neural network in the one-dimensional space of the particles, but also maintains the logical relationship between them, which helps to improve the search efficiency and performance of the particle swarm optimization algorithm in the scale-adaptive search space, and also enhances the flexibility of the present invention in processing road disease image features of different scales. And the scale-adaptive search space proposed in this embodiment can process the input of road disease image information of different scales, and enhance the perception ability of the searched neural network to road disease image information of different scales.

[0068] 2. Use the multi-particle swarm initialization strategy to initialize the n second particle swarms in the first particle swarm to obtain the best second particle swarm;

[0069] Optionally, in one of the embodiments, a multi-particle swarm initialization strategy is used to initialize n second particle swarms in the first particle swarm to obtain an optimal second particle swarm, including:

[0070] (1) calculating the initial fitness values ​​of n second particle groups in the first particle group;

[0071] Calculating the initial fitness values ​​of n second particle groups in the first particle group includes:

[0072] ① Using zero sample evaluation to obtain the average classification performance of n second particle groups in the first particle group, and obtaining the average parameters and average floating point numbers of n second particle groups in the first particle group by calculation;

[0073] Optionally, in one of the embodiments, firstly, zero samples are used to evaluate m third particle groups in n second particle groups in the first particle group to obtain the classification performance of the neural network including the MBConv layer corresponding to each third particle group, and then the average classification performance of the n second particle groups is obtained according to the classification performance of the neural network corresponding to each third particle group and the second particle group to which each third particle group belongs;

[0074] That is, the classification performance of the neural network corresponding to the m third particle groups is first summed up, and then divided by m for average, to obtain the average classification performance of the n second particle groups.

[0075] Specifically, the third particle group is evaluated using zero samples to obtain the classification performance of the neural network containing the MBConv layer corresponding to each third particle group. The calculation is expressed as:

[0076]

[0077] Among them, Zen-Score(·) represents zero sample, F represents the neural network corresponding to each third particle group, Φ is Φ-Score, which is another zero sample evaluation method. represents the variance of the g-th BN layer and the j-th channel, and h represents the number of channels in the g-th BN layer.

[0078] Specifically, the neural network obtained after decoding the third particle group is first introduced into batch normalization (BatchNormalization, BN), and then the classification performance is evaluated. The zero-sample evaluation method in this application is an optimization based on the Φ-Score zero-sample evaluation, so batch normalization needs to be introduced to add the lost variance information back in each BN layer later to prevent overflow, and finally the classification performance is evaluated.

[0079] The existing technology is used to directly calculate the parameters of the neural network corresponding to each third particle group, and then the average parameters of the n second particle groups are obtained according to the parameters of the neural network corresponding to each third particle group and the second particle group to which each third particle group belongs. The same is true for obtaining the average classification performance of the n second particle groups (sum first and then average).

[0080] The existing technology is used to directly calculate the floating-point operation amount of the neural network corresponding to each third particle group, and then the average floating-point operation amount of the n second particle groups is calculated according to the floating-point operation amount of the neural network corresponding to each third particle group and the second particle group to which each third particle group belongs. The same is true for obtaining the average classification performance of n second particle groups (sum first and then average).

[0081] ② Using the best weight combination, weighted processing is performed on the average classification performance, average parameter and average floating point number of the n second particle groups respectively to obtain the initial fitness values ​​of the n second particle groups.

[0082] The average classification performance, average parameter and average floating point number of the n second particle swarms are weighted respectively by using the optimal weight combination to obtain the initial fitness values ​​of the n second particle swarms.

[0083] Optionally, in one embodiment, the weight w of the average parameter amount in the optimal weight combination p is 0.1, the average floating point operation weight w f is 0.1, the weight w of the average classification performance s is 0.8.

[0084] Optionally, the following formula is used to calculate the initial fitness value of n second particle groups:

[0085] Fitness n =w s P_Score(n)-w p Parameters(n)-w f Flops(n)

[0086] Fitness n represents the initial fitness value of the nth second particle swarm, P_Score(n) represents the average classification performance of the nth second particle swarm, Parameters(n) represents the average parameter value of the nth second particle swarm, and Flops(n) represents the average floating-point operation value of the nth second particle swarm.

[0087] (2) The second particle group with the highest initial fitness value is used as the optimal second particle group.

[0088] The Select function selects the second particle group with the highest initial fitness value from the n second particle groups of the first particle group, and uses it as the best second particle group for evolution.

[0089] 3. Adopting a particle swarm dynamic adaptive update strategy to optimize the m third particle swarms in the best second particle swarm to obtain the best third particle swarm;

[0090] Optionally, in one of the embodiments, before the particle swarm dynamic adaptive update strategy is used to optimize the m third particle swarms in the best second particle swarm, the method further includes:

[0091] (1) Using a multi-objective performance evaluation function to obtain the evolutionary fitness value of each third particle group of the best second particle group during each iterative calculation of the best third particle group, and obtaining the current fitness value of each third particle group of the best second particle group;

[0092] (2) according to the current fitness value of each third particle group of the best second particle group, obtain the current average fitness value of the best second particle group in each iterative calculation of the best third particle group;

[0093] (3) determining the solution environment of the best second particle swarm by using the ratio of the current fitness value of each third particle swarm of the best second particle swarm to the current average fitness value of the best second particle swarm;

[0094] The calculation formula is:

[0095]

[0096] Among them, p i represents the solution environment of the best second particle group of the i-th third particle group at the t-th generation of evolution, exp represents the power of e, and x i (t) is the spatial distribution position of the ith third particle group after the tth generation of evolution, fit(·) is the current fitness value of the ith third particle group, and m is the number of third particle groups in the best second particle group.

[0097] Optionally, in one of the embodiments, m=18, that is, the best second particle group includes 18 third particle groups, and accordingly, there are 18 neural networks including MBConv layers corresponding to the third particle groups. These 18 neural networks including MBConv layers are optimized using an improved particle swarm optimization algorithm, and updated and iterated 18 times to obtain the best third particle group.

[0098] In the above optional 18 iterations, the evolutionary fitness value of each third particle group in the best second particle group at each iteration is calculated to obtain the current fitness value of each third particle group in the best second particle group, and then the current average fitness value of the best second particle group at each iteration is calculated.

[0099] The solution environment p of the best second particle swarm is determined by using the ratio of the current fitness value of each third particle swarm in the best second particle swarm to the current average fitness value of the best second particle swarm. i ,

[0100] Preferably, the multi-objective performance evaluation function is characterized by the following formula:

[0101]

[0102] Wherein, Fitness(i) represents the evolutionary fitness value of the i-th third particle swarm, Zen-Score(·) represents the zero sample, #Params(i) represents the parameter quantity of the i-th third particle swarm, #FLOPs(i) represents the floating point operation quantity of the i-th third particle swarm, T p and T f They represent the maximum number of parameters and the maximum number of floating-point operations allowed, respectively. o and w q They respectively represent the trade-off between limiting the number of parameters and the amount of floating-point operations.

[0103] Constrain w in the following way o and w q

[0104]

[0105] The above-mentioned multi-objective fitness function adopts the weighted product method to combine zero-sample evaluation, parameter quantity calculation and floating-point operation quantity calculation together, so that the present application can not only evaluate the classification performance of the neural network containing the MBConv layer, but also take into account the parameter quantity and floating-point operation quantity, so that the selected neural network not only has a certain classification performance accuracy, but also can adapt to the specified environment and meet certain memory usage requirements, and finally obtain an efficient neural network for road disease image prediction.

[0106] Optionally, in one of the embodiments provided in the present application, a linear combination of local optimum and global optimum (pbest i +gbest i ) / 2 and (pbest i -gbest i ) / 2 replace pbest in the standard particle swarm optimization algorithm PSO formula i and gbest i , and use "x i (t+1)=x i (t)+v i (t+1)” updates the spatial distribution of the third particle group to improve the local exploration ability of the algorithm, and uses “x i (t+1)=w(t+1)x i (t)+(1-w)v i (t+1)” improves the global development capability of the algorithm and obtains an improved particle swarm optimization algorithm.

[0107] When the current fitness value of the i-th third particle group of the best second particle group is higher than the current average fitness value of the best second particle group, that is, when p i >0.6, use "x i (t+1)=x i (t)+v i (t+1)” updates the spatial distribution of the third particle swarm to ensure the local exploration capability of the improved particle swarm optimization algorithm. At the same time, the circling strategy and dynamic reverse learning are used to update the direction of the third particle swarm to reduce exploration and assist the third particle swarm to move closer to the global optimal solution, thus promoting the convergence of the improved particle swarm optimization algorithm.

[0108] The calculation formula is expressed as:

[0109]

[0110] When the current fitness value of the i-th third particle group of the best second particle group is lower than the current average fitness value of the best second particle group, that is, when p i <0.6 or p i =0.6, use "x i (t+1)=w(t+1)x i (t)+(1-w)v i (t+1)” updates the spatial distribution of the third particle swarm to enhance the global development capability of the algorithm, and uses a straight line strategy and dynamic reverse learning to update the direction of the third particle swarm to increase the exploration efforts to jump out of the local optimum and help the global optimal neural network search.

[0111] The calculation formula is expressed as:

[0112]

[0113] Among them, t represents the current iteration number, v i (t) represents the velocity of the i-th third particle group, w(t+1) is the inertia weight of the best third particle group, (-1) γ is the dynamic reverse learning control factor, γ is a binary number {0, 1}, c1 is the individual learning factor, c2 is the social learning factor, r1, r2, β are random numbers between [0-1]. i It is represented as a local optimal solution of the third particle group in the second particle group, gbest i is represented as a global optimal solution of the third particle swarm in the second particle swarm, cos(·) represents the rotation of the third particle swarm, L is a random number in [0.4, 0.5], x i (t) represents the spatial distribution position of the i-th third particle group after the t-th generation of evolution.

[0114] Optionally, in one of the embodiments, the inertia weight of the improved particle swarm optimization algorithm is adjusted by using nonlinear changes based on Logistic chaotic mapping plus Beta distribution perturbation. The inertia weight of the improved particle swarm optimization algorithm is expressed as:

[0115]

[0116] Among them, w max and w min are the maximum and minimum values ​​of the inertia weight of the improved particle swarm optimization algorithm, and r(t) is given by The random number generated by the iteration, t and T are the current iteration number and the maximum iteration number respectively, ε is the adjustment factor of the inertia weight of the improved particle swarm optimization algorithm, B(b1, b2) is the Beta distribution, the inertia weight and adjustment factor of the improved particle swarm optimization algorithm are disturbed to make their distribution more reasonable, and after fusion, the offset degree of the inertia weight of the improved particle swarm optimization algorithm is controlled, so that it can better improve the convergence speed and global search ability of the improved particle swarm optimization algorithm.

[0117] Optionally, in one of the embodiments, the improved particle swarm optimization algorithm further adjusts the learning factor, so that for the learning factor, the individual learning factor c1 is larger in the early stage, and the social learning factor c2 is smaller, so that the particles learn from themselves more and improve the global exploration ability of the algorithm. In the later stage, it is just the opposite, multi-directional global optimal learning is carried out to enhance the local search ability, so that the improved particle swarm optimization algorithm converges to the global optimum. The learning factor of the improved particle swarm optimization algorithm is expressed as:

[0118]

[0119] Among them, c1 is the individual learning factor, c2 is the social learning factor, and c max 、c min are the maximum and minimum values ​​of the learning factor respectively; t and T are the current and maximum iteration times respectively.

[0120] Repeat the above steps iteratively until the maximum number of iterations is reached, and output the best third particle swarm. The value of m is the same as the number of iterations.

[0121] 4. The neural network containing the MBConv layer corresponding to the best third particle group is determined as the optimal road disease prediction network;

[0122] Optionally, in one of the embodiments, the method further includes using the prepared road damage image dataset and hyperparameters to train the neural network containing only the MBConv layer corresponding to the best third particle group. During the training process, in order to avoid overfitting, the performance of the neural network on the validation set is continuously monitored.

[0123] 5. Use the optimal road damage prediction network to predict road damage.

[0124] like Figure 3 As shown, from the comparison chart of the experimental results of using feature maps to perform reliability analysis on road damage image datasets, it can be seen that using the road damage prediction method provided by the present invention to perform reliability analysis on road damage image datasets, the feature map has better performance in clarity and edge information, and can more accurately capture the characteristics of cracks and potholes.

[0125] In contrast, FPSO-NAS and EDE-NAS have problems with recognition errors and feature loss. Although EPC-NAS and SLE-NAS mostly perform well, they still have information loss and cannot accurately identify the locations of road cracks and potholes. The color and brightness distribution of the feature map of MPSZ-NAS is uniform, which reduces oversaturated or undersaturated areas, making it more accurate in learning and classifying tiny cracks and large potholes.

[0126] like Figure 4 As shown in the figure, the experimental results of reliability analysis of road disease sampling images are compared from the heat map (including the original image of road disease, in Figure 4 The first column of the figure shows that the road disease prediction method provided by the present invention is used to perform reliability analysis on the road disease sampling image. The thermal map can clearly identify the contours of cracks and potholes and accurately highlight the key areas of the disease.

[0127] In contrast, although the heat maps of FPSO-NAS and EDE-NAS have strong colors, the identification area is not accurate enough and is too broad; the heat maps of EPC-NAS and SLE-NAS have lighter colors, insufficient details, and it is difficult to accurately identify the diseased area. Therefore, the heat map of the road disease prediction method provided by the present invention is easier to understand and identify, which reflects the high efficiency of the present invention in road disease prediction.

[0128] Tables 2-4 also show the comparison results between the present invention and PSO-NAS, EDE-NAS, EPC-NAS, and SLE-NAS algorithms on Road, RSCD, and RTK data sets.

[0129] Each row of the table represents a different road damage prediction method. MPSZ-NAS is a road damage prediction method provided by an embodiment of the present invention, and the others are comparative methods. Each column represents a different experimental index.

[0130] Table 2

[0131]

[0132] Table 3

[0133]

[0134]

[0135] Table 4

[0136]

[0137]

[0138] It can be seen that on the Road dataset, MPSZ-NAS achieved an accuracy of 81%, which is significantly higher than other road damage prediction methods. On the RTK dataset, the accuracy of MPSZ-NAS reached 86%. In addition to the accuracy, other performance indicators such as Kappa coefficient, AUC, F1 score and exponential balance accuracy also provide a comprehensive evaluation of the performance of the neural network. It can be clearly seen from Table 2-4 that MPSZ-NAS also outperforms most neural networks in these indicators, showing better classification performance and consistency, which means that MPSZ-NAS can provide stable and reliable results when dealing with complex classification tasks. It is worth noting that MPSZ-NAS has the lowest search cost, with an average time of less than 1 hour on the three datasets. Due to the high efficiency of its search method and the reasonable design of the search space, it shows that MPSZ-NAS is more suitable for deployment in environments with limited computing resources.

[0139] The present invention integrates operations such as multi-convolution kernels, multi-pooling and attention, and can automatically and efficiently search for neural networks for road damage image prediction, so that the searched neural network can not only extract multi-scale features of road damage images, but also effectively balance the classification performance of the neural network and the complexity of the neural network. Experiments have proved that the present invention can ensure the dual challenges of low computing resource overhead and accurate prediction of road damage in an autonomous driving environment, and is both effective and practically applicable compared to other methods.

[0140] In one embodiment, a road damage prediction device based on neural network architecture search is also included, including:

[0141] A first determining unit is used to determine a first particle group in the scale adaptive search space, wherein the first particle group includes n second particle groups, and the second particle group includes m randomly generated third particle groups;

[0142] An initialization processing unit, used for initializing n second particle groups in the first particle group by adopting a multi-particle group initialization strategy to obtain an optimal second particle group;

[0143] An optimization processing unit, used for optimizing the m third particle groups in the best second particle group by adopting a particle group dynamic adaptive update strategy to obtain the best third particle group;

[0144] A second determining unit is used to determine the neural network including the MBConv layer corresponding to the best third particle group as the optimal road disease prediction network;

[0145] A prediction unit is used to use the optimal road damage prediction network to perform road damage prediction.

[0146] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any road damage prediction method based on neural architecture search as recorded in the above method embodiment.

[0147] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program enables a computer to execute part or all of the steps of any road damage prediction method based on neural architecture search as recorded in the above method embodiment.

[0148] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should be the common meanings understood by technicians in the field to which the present invention belongs. The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present disclosure described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. Unless otherwise specified, the term "multiple" means two or more. In the embodiments of the present disclosure, the character " / " indicates that the objects before and after are in an "or" relationship. For example, A / B means: A or B. The term "and / or" is a description of the association relationship of objects, indicating that there can be three relationships. For example, A and / or B means: A or B, or, A and B. The term "corresponding" can refer to an association relationship or a binding relationship, and A and B corresponding to each other means that there is an association relationship or a binding relationship between A and B.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A road damage prediction method based on neural network architecture search, characterized in that: include: Determine a first particle group in the scale adaptive search space, wherein the first particle group includes n second particle groups, and the second particle group includes m randomly generated third particle groups; The multi-particle swarm initialization strategy is used to initialize the n second particle swarms in the first particle swarm to obtain the best second particle swarm; Adopting a particle swarm dynamic adaptive update strategy to optimize the m third particle swarms in the best second particle swarm to obtain the best third particle swarm; The neural network including the MBConv layer corresponding to the best third particle group is determined as the optimal road disease prediction network; The optimal road damage prediction network is used to predict road damage.

2. A road damage prediction method based on neural network architecture search as claimed in claim 1, characterized in that: The method further comprises constructing a neural network including an MBConv layer according to a third particle group, According to the third particle group, a neural network including the MBConv layer is constructed, including: Determine the number of particles in the third particle group as the number of MBConv layers in the neural network; Determine the values ​​of particles in the third particle group as parameters of the MBConv layer in the neural network; The MBConv layer in the neural network is constructed according to the number of layers of the MBConv layer in the neural network and the parameters of the MBConv layer in the neural network, so as to obtain a neural network including the MBConv layer.

3. A road damage prediction method based on neural network architecture search as claimed in claim 2, characterized in that: The method of using a multi-particle swarm initialization strategy to initialize n second particle swarms in the first particle swarm to obtain the best second particle swarm includes: Calculating initial fitness values ​​of n second particle groups in the first particle group; The second particle group with the highest initial fitness value is used as the best second particle group.

4. A road damage prediction method based on neural network architecture search as claimed in claim 3, characterized in that: The calculating the initial fitness values ​​of n second particle groups in the first particle group includes: The average classification performance of the n second particle groups in the first particle group is obtained by using zero sample evaluation, and the average parameter and average floating point number of the n second particle groups in the first particle group are obtained by calculation; The average classification performance, average parameter and average floating point number of the n second particle groups are weighted by using the best weight combination to obtain the initial fitness values ​​of the n second particle groups.

5. The road damage prediction method based on neural network architecture search according to claim 1, characterized in that: Before optimizing the m third particle groups in the best second particle group by using the particle group dynamic adaptive update strategy, the method further includes: The multi-objective performance evaluation function is used to obtain the evolutionary fitness value of each third particle group of the best second particle group during each iterative calculation of the best third particle group, and the current fitness value of each third particle group of the best second particle group is obtained; According to the current fitness value of each third particle group of the best second particle group, the current average fitness value of the best second particle group in each iterative calculation of the best third particle group is obtained; Determine the solution environment of the best second particle swarm by using the ratio of the current fitness value of each third particle swarm of the best second particle swarm to the current average fitness value of the best second particle swarm; The calculation formula is: Among them, p i represents the solution environment of the best second particle group of the i-th third particle group at the t-th generation of evolution, exp represents the power of e, and x i (t) is the spatial distribution position of the ith third particle group after the tth generation of evolution, fit(·) is the current fitness value of the ith third particle group, and m is the number of third particle groups in the best second particle group.

6. A road damage prediction method based on neural network architecture search as claimed in claim 5, characterized in that: The multi-objective performance evaluation function is characterized by the following formula: Wherein, Fitness(i) represents the evolutionary fitness value of the i-th third particle swarm, Zen-Score(·) represents the zero sample, #Params(i) represents the parameter quantity of the i-th third particle swarm, #FLOPs(i) represents the floating point operation quantity of the i-th third particle swarm, T p and T f They represent the maximum number of parameters and the maximum number of floating-point operations allowed, respectively. o and w q They respectively represent the trade-off between limiting the number of parameters and the amount of floating-point operations.

7. A road damage prediction method based on neural network architecture search as claimed in claim 5, characterized in that: The particle swarm dynamic adaptive update strategy includes: When p i >0.6, use "x i (t+1)=x i (t)+v i (t+1)” updates the spatial distribution position of the i-th third particle group, and the calculation formula is expressed as: When p i <0.6 or p i =0.6, use "x i (t+1)=w(t+1)x i (t)+(1-w)v i (t+1)” updates the spatial distribution position of the i-th third particle group, and the calculation formula is expressed as: Among them, t represents the current iteration number, v i (t) represents the velocity of the i-th third particle group, w(t+1) is the inertia weight of the best third particle group, (-1) γ is the dynamic reverse learning control factor, γ is a binary number {0, 1}, c1 is the individual learning factor, c2 is the social learning factor, r1, r2, β are random numbers between [0-1]. i It is represented as a local optimal solution of the third particle group in the second particle group, gbest i is represented as a global optimal solution of the third particle swarm in the second particle swarm, cos(·) represents the rotation of the third particle swarm, L is a random number in [0.4, 0.5], x i (t) represents the spatial distribution position of the i-th third particle group after the t-th generation of evolution.

8. A road damage prediction device based on neural network architecture search, characterized in that: include: A first determining unit is used to determine a first particle group in the scale adaptive search space, wherein the first particle group includes n second particle groups, and the second particle group includes m randomly generated third particle groups; An initialization processing unit, used for initializing n second particle groups in the first particle group by adopting a multi-particle group initialization strategy to obtain an optimal second particle group; An optimization processing unit, used for optimizing the m third particle groups in the best second particle group by adopting a particle group dynamic adaptive update strategy to obtain the best third particle group; A second determining unit is used to determine the neural network including the MBConv layer corresponding to the best third particle group as the optimal road disease prediction network; A prediction unit is used to use the optimal road damage prediction network to predict road damage.

9. A terminal, characterized in that: The method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images