A method and system for detecting cracks in a road surface

Abstract

A kind of pavement crack detection method and system, the pavement image data is preprocessed, the neural network of supervised learning model trained is simultaneously pruned using reinforcement learning automatic pruning model, the preliminary prediction is carried out to preprocessed data using the supervised learning model after cutting, and initial result matrix is obtained;The environment parameter of preprocessed data and initial result matrix obtained is input into reinforcement learning auxiliary model to judge the current situation and output flip area matrix;According to flip area matrix, initial result matrix is changed to obtain the result matrix after first processing, and preprocessed data and the result matrix after first processing form new environment parameter;When reinforcement learning auxiliary model is executed to reach the maximum execution number set, or the flip area matrix output by reinforcement learning auxiliary model is empty, end the final prediction result is output by prediction.This application improves the recognition efficiency and recognition accuracy, and achieves the lightness of system as a whole.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a road surface crack detection method and system. BACKGROUND

[0002] With the increase of highway mileage and the extension of use time, due to the action of driving load and natural factors, the road surface gradually produces various damages, which leads to the fact that the maintenance task of the highway becomes more and more heavy. The damage of the road surface will cause adverse effects on the bearing capacity, durability of the road surface, and the driving speed, fuel consumption, mechanical wear, driving comfort, traffic safety, environmental protection, etc. of the vehicle.

[0003] Crack is one of the most common and important disease manifestations of asphalt pavement. Rapid pavement crack detection and efficient identification are the premise of timely road maintenance, and are an effective way to improve road service quality, prolong road service life and reduce road maintenance cost. Therefore, the research on the automatic identification technology of asphalt pavement cracks has important significance for the development of the transportation industry. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a road surface crack detection method and system to solve or partially solve the above technical problems.

[0005] In order to achieve the above purpose, the first aspect of the present application provides a road surface crack detection method, comprising:

[0006] The road surface image data is preprocessed, and the preprocessing includes image filling / cutting and image compression. The road surface image after image filling / cutting and image compression is normalized to obtain preprocessed data D;

[0007] The n-layer neural network of the supervised learning model after training is pruned simultaneously by using the reinforcement learning automatic pruning model. The supervised learning model after pruning is used to preliminarily predict the preprocessed data D, and an initial result matrix M0 is obtained;

[0008] The obtained preprocessed data D and initial result matrix M0 are input into the reinforcement learning auxiliary model Agent to judge the current situation and output a flip region matrix M s ;

[0009] According to the flip region matrix M s , the initial result matrix M0 is changed to obtain a first processed result matrix M1. The preprocessed data D and the first processed result matrix M1 are combined to form a new environment <D, M1>;

[0010] When the execution of the reinforcement learning auxiliary model Agent reaches a set maximum execution step number N, or the flip region matrix M output by the reinforcement learning auxiliary model Agent s When the time is empty, the final prediction result M is output n .

[0011] As a preferred solution of the road crack detection method, the image padding / cutting step includes:

[0012] Read the road image grayscale image, and when the size of the target road image grayscale image is greater than the size preset value, cut the target road image from the top left corner according to the size preset value;

[0013] When the size of the target road image grayscale image is less than the size preset value, use 255 pixel values to fill the right and bottom sides of the target road image grayscale image to obtain a road image grayscale image with uniform size.

[0014] As a preferred solution of the road crack detection method, the normalization step includes:

[0015] Obtain the pixel value p of the pixel point of the road image, calculate the pixel average value μ of the pixel point of the road image according to the pixel value p, and subtract the pixel average value μ from the pixel value p of each pixel point and divide by 255 to obtain the normalized preprocessed data D.

[0016] As a preferred solution of the road crack detection method, the specified number of 1 / 2 downsampling operations are performed on the road image data during the forward transmission process of the supervised learning model, and a result matrix M is output. Each value in the result matrix M represents a probability value of the existence of a disease in a specified pixel region position in the original road image region.

[0017] The supervised learning model is supervised trained using the Dice Loss loss function;

[0018]

[0019] In the formula, M is the result matrix, and T is the target matrix.

[0020] As a preferred solution of the road crack detection method, the reinforcement learning automatic pruning model Agent sets the output action as selecting a pruning ratio parameter for each layer of neural network at the same time, and the pruning ratio parameter ranges from 0 to 50%.

[0021] The second aspect of the present application provides a road crack detection system, comprising:

[0022] A data preprocessing module is configured to preprocess road image data, and the preprocessing includes image padding / cutting and image compression. The road image after image padding / cutting and image compression is normalized to obtain preprocessed data D.

[0023] a pruning processing module configured to simultaneously prune the n-layer neural network of the trained supervised learning model by using a reinforcement learning automatic pruning model;

[0024] a preliminary prediction module configured to perform preliminary prediction on the preprocessed data D by using the pruned supervised learning model to obtain an initial result matrix M0;

[0025] a flip processing module configured to input the obtained preprocessed data D and initial result matrix M0 into the reinforcement learning auxiliary model Agent to output a flip region matrix M s by the reinforcement learning auxiliary model according to the current situation;

[0026] an updating processing module configured to change the initial result matrix M0 according to the flip region matrix M s to obtain a first processed result matrix M1, and to input the preprocessed data D and the first processed result matrix M1 into a new environment <D, M1>;

[0027] a result prediction module configured to, when the reinforcement learning auxiliary model Agent is executed for a set maximum number of execution steps N or the flip region matrix M s output by the reinforcement learning auxiliary model Agent is empty, end the prediction and output a final prediction result M n .

[0028] As a preferred scheme of the road crack detection system, the data preprocessing module comprises an image padding / cutting submodule.

[0029] The image padding / cutting submodule is configured to read a road image grayscale image, and when a size of the target road image grayscale image is greater than a size preset value, cut the target road image from the top left corner according to the size preset value.

[0030] The image padding / cutting submodule is further configured to, when the size of the target road image grayscale image is less than the size preset value, pad the target road image grayscale image on the right side and the lower side with 255 pixel values to obtain a road image grayscale image with a uniform size.

[0031] As a preferred scheme of the road crack detection system, the data preprocessing module further comprises a normalization processing submodule.

[0032] The normalization processing submodule is configured to obtain a pixel value p of a pixel point of a road image, calculate a pixel average value μ of the pixel point of the road image according to the pixel value p, and obtain normalized preprocessed data D by subtracting the pixel average value μ from the pixel value p of each pixel point and then dividing by 255.

[0033] As a preferred scheme of the road crack detection system, the road crack detection system further comprises a supervised learning model training module.

[0034] In the supervised learning model training module, the road surface image data is subjected to a downsampling operation of a specified number of 1 / 2 in the forward transmission process of the supervised learning model, and a result matrix M is output, each value in the result matrix M representing a probability value of the existence of a disease in a specified pixel region position in the original road surface image region;

[0035] The supervised learning model training module uses a Dice Loss loss function to supervise the training of the supervised learning model.

[0036]

[0037] In the formula, M is a result matrix, and T is a target matrix.

[0038] As a preferred scheme of the road surface crack detection system, in the pruning processing module, the reinforcement learning automatic pruning model Agent sets the output action as simultaneously selecting a pruning proportion parameter for each layer of neural network, and the pruning proportion parameter ranges from 0 to 50%.

[0039] The third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the road surface crack detection method of the first aspect or any possible implementation manner thereof.

[0040] The fourth aspect of the present application provides a non-transitory computer readable storage medium, which stores computer instructions for causing a computer to execute the road surface crack detection method of the first aspect or any possible implementation manner thereof.

[0041] From the above, it can be seen that the technical scheme provided by the present application pre-processes the road surface image data, the pre-processing includes image filling / cutting and image compression, and the pre-processed road surface image after image filling / cutting and image compression is normalized to obtain pre-processing data; a reinforcement learning automatic pruning model is used to simultaneously perform pruning operation on the neural network of the supervised learning model after training, a supervised learning model after cutting is used to preliminarily predict the pre-processing data to obtain an initial result matrix; the obtained pre-processing data and initial result matrix are input into a reinforcement learning auxiliary model Agent as environment parameters, and the reinforcement learning auxiliary model is used to judge the current situation and output a flip area matrix; the initial result matrix is changed according to the flip area matrix to obtain a first processed result matrix, and the pre-processing data and the first processed result matrix are used to form new environment parameters; when the reinforcement learning auxiliary model Agent reaches the set maximum execution step number or the flip area matrix output by the reinforcement learning auxiliary model Agent is empty, the prediction is ended and the final prediction result is output. The present application improves the overall recognition efficiency and recognition accuracy, realizes the lightness of the system as a whole, can be migrated and used on most mobile devices, expands the use range of the system, improves the use efficiency of the system, and maximally reduces the physical configuration requirements of the system. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present application or related art, the drawings needed to be used in the embodiments or related art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0043] Figure 1 The road surface crack detection method flowchart provided by the embodiment of the present application;

[0044] Figure 2 The image pre-processing diagram in the road surface crack detection method provided by the embodiment of the present application;

[0045] Figure 3 The supervised learning model training diagram in the road surface crack detection method provided by the embodiment of the present application;

[0046] Figure 4 The pruning flowchart in the road surface crack detection method provided by the embodiment of the present application;

[0047] Figure 5 The auxiliary reinforcement learning diagram in the road surface crack detection method provided by the embodiment of the present application;

[0048] Figure 6 The road surface crack detection system architecture diagram provided by the embodiment of the present application;

[0049] Figure 7 Structure diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be described in further detail below with reference to specific embodiments and with reference to the accompanying drawings.

[0051] It should be noted that, unless otherwise defined, technical terms or scientific terms used in the embodiments of the present application should be understood as their common meanings to those of ordinary skill in the art to which the present application belongs. The terms such as "comprise" or "include" and the like used in the embodiments of the present application mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects.

[0052] Cracks are one of the most common and important forms of asphalt pavement diseases. Cracks indicate that the subgrade or base layer has diseases, such as wheel track band net cracks, which may indicate that the strength of the subgrade or base layer is weakened, and may also induce diseases of the subgrade or base layer. If rainwater enters the base layer along the cracks, the base layer or subgrade will be unstable, thus further exacerbating the pavement diseases. Meanwhile, if cracks are not treated in time, the service life of the highway will be shortened, and the comfort and safety of driving will be affected. Pavement crack detection is the basis of highway maintenance and management, and is the most important part of pavement diseases. Rapid pavement crack detection and efficient identification are the prerequisites for timely road maintenance, and are effective ways to improve road service quality, extend road service life, and reduce road maintenance costs. Therefore, the research on automatic identification technology of asphalt pavement cracks has important significance for the development of highway transportation.

[0053] For pavement data collection, people initially used the walking eye observation method to record the pavement condition, but the efficiency was low, the labor intensity was high, and the safety was poor. In the face of the growing demand for highway maintenance, since the 1990s, people have begun to develop pavement information detection systems, and have developed pavement detection equipment based on ultrasonic technology, ground penetrating radar, and camera measurement technology. Among them, the camera measurement technology can collect a wide range of damage information and is suitable for various pavements, so it has been widely applied, such as the Japanese Komatsu system, the American PCES system, the Swiss CREHOS system, and the Chinese CiCs system. These systems have gradually realized certain automatic detection functions, improved efficiency and research safety, but they are mostly based on area or line scanning technology to obtain two-dimensional pavement images. In recent years, some teams have used 3D laser scanning imaging technology to obtain pavement three-dimensional images with an accuracy of 1mm / pixel at high speed, which can more truly reflect the pavement morphology and significantly improve the quality of pavement images, but it has not been applied to actual production, and two-dimensional images are still used as the basis for data analysis.

[0054] In related manual observation technologies, most road maintenance companies still choose to use manual naked-eye observation to detect disease areas when they obtain first-hand road image data, which is inefficient and costly.

[0055] In related automatic detection technologies based on traditional image processing, filters and grayscale correction methods are used to eliminate noise and improve image quality, and traditional threshold segmentation algorithms and edge recognition algorithms are improved and researched, and various automatic crack image recognition algorithms such as seed recognition algorithm, machine learning algorithm, texture analysis algorithm and global optimization algorithm are proposed. Among them, the features used by Liu Fanfan et al. include the proportion of black pixels, the proportion of the largest connected black block of pixels, etc. After constructing the feature vector, a single-layer back propagation network is used for classification to identify cracks. Similarly, Varadharajan et al. construct a 138-dimensional feature vector for each picture as input to the classification model. Some researchers use the fitting degree of linear regression of the pixel points of the largest connected black block to mark whether the shape of the black block is elongated (cracks are generally elongated), but the accuracy does not meet the needs of practical application.

[0056] In related automatic detection technologies based on machine learning, deep learning technology is applied to road image disease area detection engineering, and classic deep supervision network models such as resnet and densenet are used for end-to-end black box training of automatic identification of road image disease areas.

[0057] However, for all current identification schemes, the current algorithms cannot well handle the interference of factors such as light and stains. For example, the CiCs system is still a semi-automatic detection technology based on manual or computer-aided manual evaluation, the main process is to first process based on traditional threshold segmentation algorithm, edge recognition algorithm, etc., then check the road images stored on the computer hard disk by the staff for manual evaluation, manually mark the cracks on the pictures for those that cannot meet the production requirements, and finally count by the computer. This kind of automatic detection technology has an efficiency bottleneck, and for deep learning technology, there is a learning ability bottleneck.

[0058] Therefore, in order to solve the problem of model lightweight, especially in the context of the popularity of mobile devices today, to make the model more lightweight and convenient; break through the learning bottleneck of traditional deep learning and improve the ability of artificial intelligence automatic detection of road diseases; the embodiment of the present application provides a road crack detection method and system, and the following is the specific content of the invention embodiment.

[0059] Reference Figure 1 , Figure 2 ,Figure 3 、 Figure 4 and Figure 5 The embodiment of the application provides a road surface crack detection method, comprising the following steps:

[0060] S1, pre-processing road surface image data, the pre-processing comprising image padding / cutting and image compression, and performing normalization on the road surface image after image padding / cutting and image compression to obtain pre-processing data D;

[0061] S2, performing pruning operation on the n-layer neural network of the supervised learning model after training by using a reinforcement learning automatic pruning model, performing preliminary prediction on the pre-processing data D by using the pruned supervised learning model, and obtaining an initial result matrix M0;

[0062] S3, inputting the obtained pre-processing data D and initial result matrix M0 into a reinforcement learning auxiliary model Agent, judging the current situation by using the reinforcement learning auxiliary model, and outputting a flip region matrix M s ;

[0063] S4, changing the initial result matrix M0 to obtain a first processed result matrix M1 according to the flip region matrix M s , and inputting the pre-processing data D and the first processed result matrix M1 into a new environment<D, M1>;

[0064] S5, when the reinforcement learning auxiliary model Agent reaches the set maximum execution step number N, or the flip region matrix M s output by the reinforcement learning auxiliary model Agent is empty, ending the prediction and outputting a final prediction result M n .

[0065] The mainstream method for current automatic recognition of road surface image disease areas is still to construct a recognition model based on traditional supervised learning for end-to-end recognition. However, due to different model structures, different training parameters and training methods, and other reasons, different models have different performances in the automatic detection task of road surface image disease areas under different parameter and training method settings. In order to eliminate the influence of other interference factors, the embodiment of the application uniformly processes data processing, training parameters and training method settings, and realizes the learning bottleneck of different structure neural networks in the automatic recognition task of road surface image disease areas under the unified parameter background.

[0066] auxiliary Figure 2In this embodiment, the road surface image data is first preprocessed, the road surface image is filled and compressed, the size of the original road surface image is set as: height H*length L*channel number 3, the image gray scale H*L is read, when the size of the target image gray scale is greater than 2200*3400, cutting is performed from the upper left corner according to the size of 2200*3400, when the size of the target image gray scale is less than 2200*3400, 255 pixel values are used to fill the right side and the lower side of the image gray scale, and finally the image size is unified to 2200*3400 gray scale.

[0067] Then the image is compressed to 704*1088, and finally normalized to obtain the pixel value p of the road surface image pixel point. The pixel average value μ of the road surface image pixel point is calculated according to the pixel value p. The pixel value p of each pixel point is subtracted from the pixel average value μ and then divided by 255 to obtain the normalized preprocessed data D. The data preprocessing is ended: D=(p-μ) / 255.

[0068] Auxiliary Figure 3 In this embodiment, a modified supervised learning model is used for training. In the forward transmission process of the supervised learning model, 5 times of 1 / 2 downsampling operation is performed on the road surface image data, and the dimension of the final output result matrix M is 704 / 2 5 *1088 / 2 5 That is, 22*34. In this way, each value in the result matrix represents the probability value of the existence of disease in a 100*100 pixel region at a certain position in the original image 2200*3400 region. The label used for supervised training is also a 22*34 target matrix T. The Dice Loss loss function is used to supervise the training of the model:

[0069]

[0070] until the supervised learning model no longer converges.

[0071] In this embodiment, in order to improve the overall recognition efficiency, the efficiency of the traditional supervised learning model is optimized, and the overall recognition efficiency is improved.

[0072] Specifically, a reinforcement learning automatic pruning model is selected to prune the supervised learning model trained in the early stage. The supervised learning model trained is composed of a layer of parameter weights weight and bias. In the forward transmission, not all parameters work, only a small part of parameter operations can activate the lower layer neurons, but all neurons for calculation will increase the calculation burden. Therefore, the method of deactivating (setting the value to 0) part of the neurons is selected to improve the overall running efficiency of the corresponding system.

[0073] Auxiliary Figure 4wherein the supervised learning model neural network has n layers in total, starting from the first layer, are respectively denoted as L1, L2,..., Ln n represent the n layers of the neural network in turn. The reinforcement learning automatic pruning model simultaneously performs pruning operations on each layer of the neural network. The reinforcement learning automatic pruning model agent sets the output action as simultaneously selecting a corresponding parameter pruning ratio for each layer, with the range being 0-50%. The environment parameter is set as the current input image data. A control group is set, i.e., a model without pruning. The changes in running efficiency FLOPs (floating point of operations) and Dice loss are recorded. When the New Dice Loss of the reinforcement learning automatic pruning model is not less than 5% compared with the Dice Loss calculated by the unpruned model, the running efficiency New FLOPs of the reinforcement learning automatic pruning model is increased compared with the FLOPs calculated by the unpruned model, then a positive reward +1 is given to the reinforcement learning agent, otherwise a negative reward -1 is given to the agent:

[0074] i.e.,

[0075] In this way, the reinforcement learning automatic pruning model agent is updated once for the total reward reward iteration every m steps, until the reinforcement learning automatic pruning model no longer converges.

[0076] auxiliary Figure 5 In the embodiment, in order to break the learning bottleneck of the supervised learning model in automatic identification of disease areas of road surface images, the identification result is further optimized on the basis of the identification result of the supervised learning model, so as to improve the overall identification effect.

[0077] Specifically, the reinforcement learning auxiliary model is trained on the basis of the supervised learning model and the reinforcement learning automatic pruning model, and the judgment of the marked area of the increased / decreased crack is performed on the basis of the output result in the early stage and the original image. The action output by the agent of the reinforcement learning auxiliary model is whether the judgment result in the initial result matrix of the judgment dimension 22*34 needs to be flipped, i.e., the value flipping operation is performed on part of the area in the initial result matrix. The value range of each point in the initial result matrix is 0-1.0, representing the possibility of disease in the 100*100 pixel area of the original image. Let M s be the selected value area in the initial result matrix: M n = 1-M s i.e., the flipped prediction value M n + the original prediction value equals 1.

[0078] The environment setting of the reinforcement learning auxiliary model is the current preprocessed data D and the result matrix M obtained last time n The combination of the result matrix processed by the reinforcement learning auxiliary model n times is represented as <D, M n During the training of the reinforcement learning auxiliary model, the step limit N is set, and when the deployment n of the reinforcement learning auxiliary model reaches N times or the current prediction selected value area is empty, the prediction is ended, and the result matrix M n The current result matrix M n is taken as the final prediction result matrix, and the Dice Loss difference percentage of the last processing result matrix M n-1 is taken as the reward for accumulation, and the agent of the reinforcement learning auxiliary model is updated according to the reward after n times of operation: reward = reward + (NewDiceLoss-DiceLoss) / DiceLoss.

[0079] In summary, the road image data is preprocessed in the present application, which includes image padding / cutting and image compression. The road image after image padding / cutting and image compression is normalized to obtain preprocessed data. The reinforcement learning automatic pruning model is used to prune the neural network of the trained supervised learning model, and the pruned supervised learning model is used to preliminarily predict the preprocessed data to obtain an initial result matrix. The preprocessed data and the initial result matrix are input into the reinforcement learning auxiliary model Agent as environment parameters, and the reinforcement learning auxiliary model is used to judge the current situation and output a flip region matrix. The initial result matrix is changed according to the flip region matrix to obtain a first processed result matrix, and the preprocessed data and the first processed result matrix are used as new environment parameters. When the reinforcement learning auxiliary model Agent reaches the set maximum execution step, or the flip region matrix output by the reinforcement learning auxiliary model Agent is empty, the prediction is ended, and the final prediction result is output. The present application improves the overall recognition efficiency and accuracy, and realizes the lightness of the system as a whole. It can be migrated and used on most mobile devices, expand the use range of the system, improve the use efficiency of the system, and minimize the physical configuration requirements of the system.

[0080] It should be noted that the method of the present application can be executed by a single device, such as a computer or a server. The method of the present application can also be applied to a distributed scenario, and completed by multiple devices cooperating with each other. In this distributed scenario, one of the multiple devices can only execute one or more steps of the method of the present application, and the multiple devices can interact with each other to complete the method.

[0081] It is to be understood that the foregoing description is directed to embodiments of the application. Various embodiments are described herein, including the best mode of the inventors. It will be apparent, however, to those skilled in the art having the benefit of this disclosure, that variations and / or modifications of these embodiments can be made without departing from the spirit and scope of the application. Accordingly, it is intended that such variations and / or modifications be included within the scope of the application. It is the intention, therefore, to be limited only as indicated by the scope of the claims appended hereto.

[0082] Referring to Figure 6 , based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the embodiments of the application also provide a road surface crack detection system, comprising:

[0083] A data preprocessing module 1 is configured to preprocess road surface image data, the preprocessing including image padding / cutting and image compression, and the preprocessed road surface image is normalized to obtain preprocessed data D.

[0084] A pruning processing module 2 is configured to perform pruning operation on the n-layer neural network of the trained supervised learning model by using a reinforcement learning automatic pruning model.

[0085] A preliminary prediction module 3 is configured to perform preliminary prediction on the preprocessed data D by using the pruned supervised learning model to obtain an initial result matrix M0.

[0086] A flip processing module 4 is configured to input the obtained preprocessed data D and initial result matrix M0 into a reinforcement learning auxiliary model Agent to judge the current situation and output a flip region matrix M s .

[0087] An update processing module 5 is configured to change the initial result matrix M0 according to the flip region matrix M s to obtain a first processed result matrix M1, and input the preprocessed data D and the first processed result matrix M1 into a new environment <D, M1>.

[0088] A result prediction module 6 is configured to end the prediction and output a final prediction result M s when the reinforcement learning auxiliary model Agent reaches a set maximum execution step number N, or the flip region matrix M n output by the reinforcement learning auxiliary model Agent is empty.

[0089] In the embodiment, the data preprocessing module 1 includes an image padding / cutting submodule 11.

[0090] The image padding / cutting submodule 11 is configured to read the target road surface image grayscale graph, and cut the target road surface image according to a preset size from the top left corner when the size of the target road surface image grayscale graph is greater than the preset size.

[0091] The image padding / cutting submodule 11 is further configured to pad the target road surface image grayscale graph on the right side and the lower side with 255 pixel values when the size of the target road surface image grayscale graph is less than the preset size, so as to obtain a road surface image grayscale graph with a uniform size.

[0092] In this embodiment, the data preprocessing module 1 further includes a normalization processing submodule 12.

[0093] The normalization processing submodule 12 is configured to obtain a pixel value p of a pixel point of the road surface image, calculate a pixel average value μ of the pixel point of the road surface image according to the pixel value p, and obtain normalized preprocessed data D by subtracting the pixel average value μ from the pixel value p of each pixel point and then dividing by 255.

[0094] In this embodiment, a supervised learning model training module 7 is further included.

[0095] In the supervised learning model training module 7, a down-sampling operation is performed on the road surface image data for a specified number of times 1 / 2 in a forward transmission process of the supervised learning model, and a result matrix M is output, wherein each value in the result matrix M represents a probability value of a specified pixel region position in the original road surface image region having a disease.

[0096] The supervised learning model training module 7 uses a Dice Loss loss function to supervise the training of the supervised learning model.

[0097]

[0098] In the formula, M represents the result matrix, and T represents the target matrix.

[0099] In the pruning processing module 2, the reinforcement learning automatic pruning model Agent sets the output action as simultaneously selecting a pruning ratio parameter for each layer of the neural network, and the pruning ratio parameter ranges from 0 to 50%.

[0100] A third aspect of the embodiments of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the road surface crack detection method of the first aspect or any possible implementation manner thereof when executing the program.

[0101] A fourth aspect of the present application provides a non-transitory computer readable storage medium, which stores computer instructions for causing a computer to execute the road surface crack detection method of the first aspect or any possible implementation manner thereof.

[0102] The device of the above embodiment is used to implement the corresponding road crack detection method in any of the preceding embodiments, and has the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0103] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the road crack detection method of any of the above embodiments when executing the program.

[0104] Figure 7 A more specific hardware structure schematic diagram of an electronic device provided by the embodiment is shown, which can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 for communication within the device.

[0105] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present specification.

[0106] The memory 1020 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 1020 and executed by the processor 1010.

[0107] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0108] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through wired mode (such as USB, network cable, etc.), or can realize communication through wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0109] The bus 1050 includes a path for transmitting information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0110] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary for the implementation of the embodiments of the present specification, and does not have to contain all the components shown in the figure.

[0111] The electronic device of the above embodiment is used to realize the corresponding road crack detection method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here.

[0112] Based on the same inventive concept, corresponding to any of the above embodiment methods, the present application also provides a non-transitory computer readable storage medium, which stores computer instructions for causing the computer to execute the road crack detection method according to any of the above embodiments.

[0113] The computer readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be realized by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0114] The storage medium of the above embodiments stores computer instructions for causing the computer to execute the road surface crack detection method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here again.

[0115] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to suggest that the scope of the application (including the claims) is limited to these examples; the above embodiments or technical features among different embodiments can be combined, steps can be implemented in any order, and there are many other variations of the aspects of the embodiments of the application as described above, which are not provided in detail in order to be brief. They are not intended to suggest that the scope of the application (including the claims) is limited to these examples.

[0116] In addition, in order to simplify the description and discussion, and so as not to make the embodiments of the application difficult to understand, the well-known power / ground connections of integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. In addition, devices can be shown in block diagram form in order to avoid making the embodiments of the application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the application (i.e., these details should be fully within the understanding of those skilled in the art). Where specific details (e.g., circuitry) are set forth in order to describe an exemplary embodiment of the application, it will be apparent to those skilled in the art that the embodiments of the application can be practiced without these specific details or with variations on these specific details. Therefore, these descriptions should be considered as illustrative rather than limiting.

[0117] Although the application has been described in conjunction with specific embodiments thereof, many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0118] The embodiments of the application are intended to cover all such alternatives, modifications and variations as falling within the broad scope of the appended claims. Accordingly, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the application should be included in the protection scope of the application.

Claims

1. A pavement crack detection method, comprising: preprocessing pavement image data, the preprocessing comprising image padding / cutting and image compression, and normalizing the pavement image after image padding / cutting and image compression to obtain preprocessed data D; performing pruning operation on n-layer neural networks of a supervised learning model that has been trained using a reinforcement learning automatic pruning model, and performing preliminary prediction on the preprocessed data D using the pruned supervised learning model to obtain an initial result matrix M0; The obtained preprocessed data D and the initial result matrix M0 are input into a reinforcement learning auxiliary model Agent to output a flip region matrix M by the reinforcement learning auxiliary model judging the current situation s ; According to the flipping region matrix M s The initial result matrix M0 is changed to obtain a first processed result matrix M1, and the preprocessed data D and the first processed result matrix M1 form a new environment <D, M1>. When the execution of the reinforcement learning auxiliary model Agent reaches the set maximum execution step number N, or the flip region matrix M output by the reinforcement learning auxiliary model Agent s is empty, end the prediction and output the final prediction result M n .

2. The method of claim 1, wherein, the step of image padding / cutting comprises: reading a pavement image grayscale image, and when the size of the target pavement image grayscale image is greater than a preset size, cutting the target pavement image from the top left corner according to the preset size; when the size of the target pavement image grayscale image is less than the preset size, padding the target pavement image grayscale image on the right and bottom sides with 255 pixel values to obtain a pavement image grayscale image with uniform size.

3. The method of claim 2, wherein, the step of normalization comprises: obtaining pixel values p of pavement image pixels, calculating pixel mean values μ of the pavement image pixels according to the pixel values p, and subtracting the pixel mean values μ from each pixel value p and then dividing by 255 to obtain the normalized preprocessed data D.

4. The method of claim 1, wherein, performing downsampling operation on the pavement image data a specified number of times 1 / 2 during forward transmission of the supervised learning model, and outputting a result matrix M, wherein each value in the result matrix M represents a probability value of the existence of a disease in a specified pixel region position in the original pavement image region; supervising and training the supervised learning model using a Dice Loss loss function; wherein M is a result matrix and T is a target matrix.

5. The method of claim 4, wherein, The reinforcement learning automatic pruning model Agent sets the output action as simultaneously selecting a pruning ratio parameter for each neural network, and the pruning ratio parameter ranges from 0 to 50%.

6. A road surface crack detection system, wherein, comprises: a data preprocessing module configured to preprocess pavement image data, the preprocessing comprising image padding / cutting and image compression, and normalizing the pavement image after image padding / cutting and image compression to obtain preprocessed data D; a pruning processing module configured to perform pruning operation on n-layer neural networks of a supervised learning model that has been trained using a reinforcement learning automatic pruning model; a preliminary prediction module configured to perform preliminary prediction on the preprocessed data D using the pruned supervised learning model to obtain an initial result matrix M0; The flip processing module is configured to input the obtained pre-processed data D and the initial result matrix M0 into the reinforcement learning auxiliary model Agent to output a flip region matrix M by judging the current situation through the reinforcement learning auxiliary model. s ; An updating processing module is configured to update the initial result matrix M0 according to the flipping region matrix M s The initial result matrix M0 is changed to obtain a first processed result matrix M1, and the preprocessed data D and the first processed result matrix M1 form a new environment <D, M1>. A result prediction module is configured to output a final prediction result M when the execution of the reinforcement learning auxiliary model Agent reaches a set maximum number of execution steps N, or the flip region matrix M output by the reinforcement learning auxiliary model Agent s is empty, the prediction is ended and the final prediction result M is output n .

7. The system for detecting cracks in a roadway surface of claim 6, wherein, the data preprocessing module comprises an image padding / cutting submodule: the image padding / cutting submodule is configured to read a pavement image grayscale image, and when the size of the target pavement image grayscale image is greater than a preset size, cut the target pavement image from the top left corner according to the preset size; the image padding / cutting submodule is also configured to, when the size of the target pavement image grayscale image is less than the preset size, pad the target pavement image grayscale image on the right and bottom sides with 255 pixel values to obtain a pavement image grayscale image with uniform size.

8. The system for detecting cracks in a roadway surface of claim 7, wherein, the data preprocessing module further comprises a normalization processing submodule: The normalization processing submodule is configured to obtain a pixel value p of a pixel point of a road surface image, calculate a pixel average value μ of the pixel point of the road surface image according to the pixel value p, and obtain normalized preprocessed data D by subtracting the pixel average value μ from the pixel value p of each pixel point and then dividing by 255.

9. The system for detecting cracks in a roadway surface of claim 8, wherein, Further comprising a supervised learning model training module: In the supervised learning model training module, a downsampling operation with a specified number of times 1 / 2 is performed on the road surface image data in a forward transmission process of the supervised learning model, and an output result matrix M is output, wherein each value in the result matrix M represents a probability value of a specified pixel region position in an original road surface image region having a disease; The supervised learning model training module uses a Dice Loss loss function to supervise training of the supervised learning model. In the formula, M is a result matrix, and T is a target matrix.

10. The system for detecting cracks in a roadway surface of claim 9, wherein, In the pruning processing module, the reinforcement learning automatic pruning model Agent sets an output action as simultaneously selecting a pruning ratio parameter for each layer of neural network, and the pruning ratio parameter ranges from 0 to 50%.

Images