Double-branch network-based pavement disease identification method and apparatus, and medium

Abstract

The present application provides a double-branch network-based pavement disease identification method and apparatus, a medium. The method comprises: carrying out iterative optimization on a preprocessed hyper-parameter of an image by means of a Bayesian optimizer to obtain an optimal preprocessed hyper-parameter of the image; applying the optimal preprocessed hyper-parameter of the image to the image to obtain an optimized image corresponding to the image, and the image and the optimized image corresponding to the image constituting a training image pair; on the basis of the training image pair, training an initial double-branch multi-task deep network model to obtain a double-branch multi-task deep network model meeting standards, wherein the initial double-branch multi-task deep network model comprises an image adaptive preprocessing network model and a freezing disease identification model; and inputting an image to be identified into the double-branch multi-task deep network model meeting the standards, so as to carry out disease identification to obtain a target disease prediction result. The present application aims at improving the accuracy and precision of identifying a disease area by an existing model.

Description

A pavement disease identification method, device and medium based on a dual-branch network

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to a Chinese patent application filed with the Patent Office of China on July 19, 2023, with application number 202310889997.X, entitled “A Method, Device and Medium for Identifying Pavement Defects Based on a Dual-Branch Network,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to the technical field of pavement defect identification, and in particular to a pavement defect identification method, device and medium based on a dual-branch network. Background Art

[0004] To achieve accurate and efficient scientific analysis of large-scale pavement images, there is an urgent need for image-based intelligent pavement defect recognition systems. Consequently, systems based on deep neural network models are developing rapidly, with recognition accuracy increasing year by year. However, in actual engineering applications, image acquisition is subject to numerous interfering factors, such as changes in external lighting direction and occlusions. These factors can cause uneven color distribution in the image, reducing the accuracy of identifying pavement defects such as cracks and repairs.

[0005] Summary of the Invention

[0006] In view of this, this application provides a road surface disease identification method based on a dual-branch network, aiming to improve the recognition accuracy and precision of existing models for diseased areas.

[0007] In a first aspect of an embodiment of the present application, a pavement defect identification method based on a dual-branch network is provided, the method comprising:

[0008] The image preprocessing hyperparameters are iteratively optimized using the Bayesian optimizer to obtain the optimal image preprocessing hyperparameters.

[0009] Apply the optimal preprocessing hyperparameters of the image to the image to obtain an optimized image corresponding to the image, and the image and the optimized image corresponding to the image constitute a training image pair;

[0010] Training an initial dual-branch multi-task deep network model through training image pairs to obtain a qualified dual-branch multi-task deep network model, wherein the initial dual-branch multi-task deep network model at least includes an image adaptive preprocessing network model and a frozen disease recognition model;

[0011] The image to be identified is input into a qualified dual-branch multi-task deep network model for disease recognition to obtain the target predicted disease result.

[0012] In some embodiments of the present application, iterative optimization of image preprocessing hyperparameters is performed using a Bayesian optimizer to obtain optimal image preprocessing hyperparameters, including:

[0013] Construct the image loss function based on the predicted disease results and the actual disease results of the image;

[0014] Based on the loss function of the image, the image preprocessing hyperparameters are iteratively optimized through the Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image.

[0015] In some embodiments of the present application, based on the loss function of the image, the image preprocessing hyperparameters are iteratively optimized by a Bayesian optimizer to obtain the optimal image preprocessing hyperparameters, including:

[0016] Preprocess the image according to the preprocessing algorithm;

[0017] The preprocessed image is input into the initial dual-branch multi-task deep network model to freeze the disease recognition model for disease recognition, and the predicted disease result of the preprocessed image is obtained;

[0018] According to the predicted disease result and the actual disease result of the preprocessed image, the loss function value of the preprocessed image is determined by the loss function;

[0019] Determine whether the current iterative optimization meets the preset conditions based on the loss function value of the preprocessed image and the preprocessing hyperparameters;

[0020] When it is determined that the current iterative optimization meets the preset conditions, the preprocessing hyperparameters of the preprocessed image are determined to be the optimal preprocessing hyperparameters of the image; when it is determined that the current iterative optimization does not meet the preset conditions, the image is preprocessed again according to the preprocessing algorithm, and the process returns to step: inputting the preprocessed image into the initial dual-branch multi-task deep network model to freeze the disease recognition model for disease recognition, and obtaining the predicted disease result of the preprocessed image.

[0021] In some embodiments of the present application, the image is preprocessed according to a preprocessing algorithm, including:

[0022] Cascading multiple preprocessing algorithms to obtain a target preprocessing algorithm, the multiple preprocessing algorithms at least including: an image contrast preprocessing algorithm, an image brightness preprocessing algorithm, and a local adaptive histogram equalization preprocessing algorithm;

[0023] The image is preprocessed using the target preprocessing algorithm.

[0024] In some embodiments of the present application, before iteratively optimizing the image preprocessing hyperparameters using a Bayesian optimizer based on the image loss function to obtain the optimal image preprocessing hyperparameters, the method further includes:

[0025] Determine and initialize the list of preprocessing hyperparameters;

[0026] Set the value range of each hyperparameter in the preprocessing hyperparameter list and set the preset conditions.

[0027] In some embodiments of the present application, an initial dual-branch multi-task deep network model is trained by training image pairs to obtain a qualified dual-branch multi-task deep network model, including:

[0028] The image adaptive preprocessing network model in the initial dual-branch multi-task deep network model is trained by training image pairs to obtain a qualified image adaptive preprocessing network model;

[0029] The qualified image adaptive preprocessing network model and frozen disease recognition model are determined to be qualified dual-branch multi-task deep network models.

[0030] In some embodiments of the present application, the image to be identified is input into a qualified dual-branch multi-task deep network model for disease identification to obtain a target disease prediction result, including:

[0031] Inputting the image to be identified into the image adaptive preprocessing network model and the frozen disease recognition model in the qualified dual-branch multi-task deep network model for processing, respectively, obtaining a feature map of the first target image output by the image adaptive preprocessing network model, and obtaining a feature map of the image to be identified after being processed by the frozen disease recognition model;

[0032] The feature map of the first target image is fused with the feature map of the image to be recognized through the attention mechanism module to obtain a fused feature map;

[0033] The frozen disease recognition model in the qualified dual-branch multi-task deep network model is used to identify diseases on the fused feature map to obtain the target predicted disease results.

[0034] In some embodiments of the present application, a qualified image adaptive preprocessing network model is obtained by training the image adaptive preprocessing network model, including:

[0035] Determine the loss function of the frozen disease recognition model, and determine the fusion loss function of the image adaptive preprocessing network model, the fusion loss function includes the generative adversarial loss function and the L1 loss function;

[0036] Construct the total loss function based on the loss function and the fusion loss function;

[0037] Based on the total loss function and the training image pairs, the image adaptive preprocessing network model is trained to obtain a qualified image adaptive preprocessing network model.

[0038] In a second aspect of an embodiment of the present application, a road surface defect identification device based on a dual-branch network is provided, the device comprising:

[0039] An optimal preprocessing hyperparameter determination unit, configured to iteratively optimize the image preprocessing hyperparameters using a Bayesian optimizer to obtain the optimal image preprocessing hyperparameters;

[0040] a clear image determination unit, configured to apply optimal preprocessing hyperparameters of the image to the image to obtain an optimized image corresponding to the image, wherein the image and the optimized image corresponding to the image constitute a training image pair;

[0041] A model training unit is used to train an initial dual-branch multi-task deep network model using training image pairs to obtain a qualified dual-branch multi-task deep network model, wherein the initial dual-branch multi-task deep network model includes at least an image adaptive preprocessing network model and a frozen disease recognition model;

[0042] The disease prediction unit is used to input the image to be identified into a qualified dual-branch multi-task deep network model for disease identification to obtain the target predicted disease result.

[0043] In a third aspect of an embodiment of the present application, a computer-readable storage medium is provided for storing a program, which, when executed by a processor, implements the steps of a pavement defect identification method based on a dual-branch network provided in the first aspect above.

[0044] Compared with the prior art, this application has the following advantages:

[0045] The embodiment of the present application provides a pavement disease recognition method based on a dual-branch network. First, the preprocessing hyperparameters of an image are iteratively optimized using a Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image. Then, the optimal preprocessing hyperparameters of the image are applied to the image to obtain an optimized image corresponding to the image. The image and the optimized image corresponding to the image together constitute a training image pair. An initial dual-branch multi-task deep network model is trained using a large number of training image pairs to obtain a qualified dual-branch multi-task deep network model. The initial dual-branch multi-task deep network model includes an image adaptive preprocessing network model and a frozen disease recognition model whose parameters have been frozen after training using large-scale data. Finally, the image to be recognized is input into the qualified dual-branch multi-task deep network model for disease recognition to obtain a target predicted disease result. In this way, the quality of the input image can be targetedly improved by optimizing the image preprocessing method without changing the disease recognition model and reducing the training cost of the disease recognition model. The quality-improved image is then integrated into the disease recognition model, thereby improving the recognition accuracy and precision of the disease recognition.

[0046] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0048] FIG1 is a flow chart of a pavement defect identification method based on a dual-branch network provided in an embodiment of the present application;

[0049] FIG2 is a flowchart of iterative optimization of a Bayesian optimizer in a pavement defect identification method based on a dual-branch network provided in an embodiment of the present application;

[0050] FIG3 is a structural diagram of a dual-branch multi-task deep network model in a pavement defect identification method based on a dual-branch network provided in an embodiment of the present application;

[0051] FIG4 is a structural diagram of an attention mechanism module in a pavement defect recognition method based on a dual-branch network provided in an embodiment of the present application;

[0052] FIG5 is a schematic diagram of obtaining a corresponding clear image after an image to be identified is processed by an image adaptive preprocessing network model in a pavement defect recognition method based on a dual-branch network provided by an embodiment of the present application;

[0053] FIG6 is a schematic diagram of obtaining a predicted disease result by performing disease identification on an image to be identified using a frozen disease identification model in a pavement disease identification method based on a dual-branch network provided by an embodiment of the present application;

[0054] FIG7 is a schematic diagram of a method for identifying road defects based on a dual-branch network provided by an embodiment of the present application, in which a qualified dual-branch multi-task deep network model is used to identify defects in an image to be identified, thereby obtaining a target predicted defect result;

[0055] FIG8 is a schematic diagram of a pavement defect identification device based on a dual-branch network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings.

[0057] Before explaining this application, the background of this application will be explained. When training a deep network model, random brightness, contrast and other operations are generally sampled to enhance or preprocess the input image, increase the number of difficult samples for training, and improve the robustness of the deep network model. However, experimental verification shows that the use of the above-mentioned image enhancement method does not improve the performance of the intelligent recognition system for pavement defects. The main reasons are: first, the scale of the model's training data has reached millions, which has covered a large variety of training samples, including a large number of difficult samples, and the model training tends to be saturated; second, random image enhancement cannot control the change pattern of image quality, which will cause the image clarity to deteriorate to a large extent, making it difficult for the model to converge; third, online image enhancement significantly increases the image preprocessing time. Due to the large scale of training, the accumulated preprocessing time causes the training cycle to be too long.

[0058] In view of this, the present application proposes a pavement defect identification method based on a dual-branch network. This method optimizes the image preprocessing method without making any changes to the existing defect recognition model that has been trained with large-scale data, specifically improves the input image quality, and integrates the quality-improved image into the above-mentioned existing defect recognition model, thereby improving the recognition accuracy and precision of defect recognition.

[0059] FIG1 is a flow chart of a pavement defect identification method based on a dual-branch network provided in an embodiment of the present application. As shown in FIG1 , the method includes:

[0060] Step S101: iteratively optimizing the image preprocessing hyperparameters using a Bayesian optimizer to obtain the optimal image preprocessing hyperparameters;

[0061] Step S102: applying the optimal preprocessing hyperparameters of the image to the image to obtain an optimized image corresponding to the image, and the image and the optimized image corresponding to the image constitute a training image pair;

[0062] Step S103: training an initial dual-branch multi-task deep network model using training image pairs to obtain a qualified dual-branch multi-task deep network model, wherein the initial dual-branch multi-task deep network model includes at least an image adaptive preprocessing network model and a frozen disease recognition model;

[0063] Step S104: Input the image to be identified into a qualified dual-branch multi-task deep network model for disease identification to obtain the target predicted disease result.

[0064] In an embodiment of the present application, a large amount of road surface image data is first collected to form an initial road surface image data set, and the actual diseased areas of each image in the initial road surface image data set are annotated by manual annotation, thereby forming a road surface image data set. The annotation method adopted in the present application is grid annotation, that is, it is determined whether there is a disease in a square image sub-block (non-overlapping division) with an actual size of 0.1 meters on a side in an image. The value of the annotation corresponding to each sub-block is 0 or 1, 0 indicates that the corresponding sub-image block does not have a disease, and 1 indicates that the corresponding sub-image block contains a disease (as shown in the square grid in Figure 2). It should be understood that the above-mentioned grid annotation is only a preferred embodiment of the present application for manually annotating image data. The annotation method of the present application for manually annotating image data also adopts other existing or future annotation methods for annotation, which is not specifically limited here.

[0065] After obtaining a dataset of pavement images with manually labeled real-world diseased areas, the preprocessing hyperparameters for each image in the dataset were iteratively optimized using a Bayesian optimizer to obtain the optimal preprocessing hyperparameters for each image. These optimal preprocessing hyperparameters were then applied to the corresponding image to obtain the optimized image. The image and its corresponding optimized image constituted a training image pair, which was then used to train the initial dual-branch multi-task deep network model.

[0066] For example, a road surface image dataset includes: images A1, A2, A3, ..., An; the preprocessing hyperparameters of one image in the road surface image dataset are iteratively optimized each time by a Bayesian optimizer, and finally each image in the road surface image dataset will obtain its corresponding optimal preprocessing hyperparameter, including: the optimal preprocessing hyperparameter a1 corresponding to image A1, the optimal preprocessing hyperparameter a2 corresponding to image A2, the optimal preprocessing hyperparameter a3 corresponding to image A3, ..., the optimal preprocessing hyperparameter an corresponding to image An; applying the obtained optimal preprocessing hyperparameters to the corresponding images will obtain the optimized images corresponding to the respective images, including: the corresponding optimized image A10 obtained by the optimal preprocessing hyperparameter a1 corresponding to image A1, the corresponding optimized image A20 obtained by the optimal preprocessing hyperparameter a2 corresponding to image A2, the corresponding optimized image A30 obtained by the optimal preprocessing hyperparameter a3 corresponding to image A3, ..., the corresponding optimized image An0 obtained by the optimal preprocessing hyperparameter an corresponding to image An; Each image and its corresponding optimized image together constitute a training image pair for subsequent training of the initial dual-branch multi-task deep network model, including: training image pair A1 and A10, training image pair A2 and A20, training image pair A3 and A30, ..., training image pair An and An0.

[0067] The initial dual-branch multi-task deep network model is trained using a large number of training image pairs constructed based on a road surface image dataset until a qualified dual-branch multi-task deep network model is obtained. The initial dual-branch multi-task deep network model includes an untrained image adaptive preprocessing network model and a frozen disease recognition model whose parameters have been frozen after training using large-scale data. The frozen disease recognition model represents an existing disease recognition model that has been trained using large-scale data. The parameters of the existing disease recognition model will be frozen, and when training the initial dual-branch multi-task deep network model, the parameters of the existing disease recognition model will not be trained. In other words, the existing disease recognition model will not be trained when training the initial dual-branch multi-task deep network model.

[0068] After obtaining a qualified dual-branch multi-task deep network model, the image to be identified is input into the qualified dual-branch multi-task deep network model for disease identification, and the target predicted disease result is obtained, that is, the area where the disease exists in the image to be identified is predicted.

[0069] The embodiment of the present application provides a pavement disease recognition method based on a dual-branch network. First, the preprocessing hyperparameters of an image are iteratively optimized using a Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image. Then, the optimal preprocessing hyperparameters of the image are applied to the image to obtain an optimized image corresponding to the image. The image and the optimized image corresponding to the image together constitute a training image pair. An initial dual-branch multi-task deep network model is trained using a large number of training image pairs to obtain a qualified dual-branch multi-task deep network model. The initial dual-branch multi-task deep network model includes an image adaptive preprocessing network model and a frozen disease recognition model whose parameters have been frozen after training using large-scale data. Finally, the image to be recognized is input into the qualified dual-branch multi-task deep network model for disease recognition to obtain a target predicted disease result. In this way, the quality of the input image can be targetedly improved by optimizing the image preprocessing method without changing the disease recognition model and reducing the training cost of the disease recognition model. The quality-improved image is then integrated into the disease recognition model, thereby improving the recognition accuracy and precision of the disease recognition.

[0070] In this application, the image preprocessing hyperparameters are iteratively optimized through a Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image, including: constructing a loss function of the image based on the predicted disease results and the actual disease results of the image; based on the loss function of the image, the image preprocessing hyperparameters are iteratively optimized through a Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image.

[0071] In an embodiment of the present application, an implementation method of iteratively optimizing the image preprocessing hyperparameters by a Bayesian optimizer to obtain the optimal image preprocessing hyperparameters is as follows: constructing an image loss function based on the predicted disease results and the actual disease results of the image, and the loss function is as follows:

[0072] Where X represents the predicted disease result obtained by performing disease recognition on the image using the frozen disease recognition model in the initial dual-branch multi-task deep network model; Y represents the actual disease result obtained by manually annotating the image; σ represents the smoothing factor added to avoid division by zero errors, which is a small positive real number of 10 -3 to 10 -4 .

[0073] The image loss function is used as a criterion for the iterative optimization of the Bayesian optimizer. This involves first identifying defects in the image using the frozen defect recognition model within the initial dual-branch multi-task deep network model to obtain the corresponding predicted defect results. Since the images used in the training phase are all from the aforementioned pavement image dataset, these images already have manually annotated real-world defect results. The current loss function value for the image can then be calculated using the defined loss function. If this value does not meet the set conditions, the Bayesian optimizer optimizes the image's preprocessing hyperparameters. These optimized preprocessing hyperparameters are then applied to the image to obtain a corresponding, somewhat optimized image.

[0074] This optimized image is then fed into the frozen disease recognition model within the initial dual-branch multi-task deep network model for disease identification, obtaining the corresponding predicted disease result. At this point, the defined loss function is used to calculate the current loss function value for the optimized image. If this value does not meet the set conditions, indicating poor optimization results, the Bayesian optimizer is then used to optimize the preprocessing hyperparameters for the image. These optimized preprocessing hyperparameters are then applied to the image to obtain another optimized image corresponding to the image. This cycle continues until the preprocessing hyperparameters obtained by optimizing the preprocessing hyperparameters for the image using the Bayesian optimizer meet the set conditions, resulting in a loss function value corresponding to the other optimized image. At this point, these preprocessing hyperparameters are considered the optimal preprocessing hyperparameters for the image.

[0075] In the embodiments of the present application, the setting conditions can be set according to the specific application scenarios. For example, the setting conditions can be that the loss function value corresponding to the image is less than 3, or less than 5, etc., or the difference between the previous loss function value of the image and the current loss function value is less than 0.3, or less than 0.5, etc., and no specific limitation is made here.

[0076] In the present application, based on the loss function of the image, the preprocessing hyperparameters of the image are iteratively optimized through a Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image, including: preprocessing the image according to a preprocessing algorithm; inputting the preprocessed image into the frozen disease recognition model in the initial dual-branch multi-task deep network model for disease recognition to obtain the predicted disease result of the preprocessed image; determining the loss function value of the preprocessed image through the loss function according to the predicted disease result and the actual disease result of the preprocessed image; determining whether the current iterative optimization meets the preset conditions according to the loss function value of the preprocessed image and the preprocessing hyperparameters; if it is determined that the current iterative optimization meets the preset conditions, determining the preprocessing hyperparameters of the preprocessed image as the optimal preprocessing hyperparameters of the image; if it is determined that the current iterative optimization does not meet the preset conditions, preprocessing the image again according to the preprocessing algorithm, and returning to the step: inputting the preprocessed image into the frozen disease recognition model in the initial dual-branch multi-task deep network model for disease recognition to obtain the predicted disease result of the preprocessed image.

[0077] In an embodiment of the present application, since each image in the road image dataset undergoes the same iterative optimization process through the Bayesian optimizer, an image is taken as an example to perform iterative optimization through the Bayesian optimizer to illustrate the image-based loss function in the present application, and the preprocessing hyperparameters of the image are iteratively optimized through the Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image.

[0078] Specifically, the image is first preprocessed using a preprocessing algorithm to obtain a first preprocessed image. The first preprocessed image is then input into the frozen disease recognition model in the initial dual-branch multi-task deep network model for disease recognition, thereby obtaining a predicted disease result for the first preprocessed image. Since the preprocessing only preprocesses the image's preprocessing hyperparameters, the true disease result of the first preprocessed image is the same as the true disease result of the original unprocessed image corresponding to the first preprocessed image. Furthermore, no matter how many times the image is subsequently preprocessed, the true disease result of the obtained preprocessed image and the true disease result of the original unprocessed image corresponding to the preprocessed image are also the same, which will not be explained further.

[0079] Based on the predicted disease results and actual disease results obtained for the first preprocessed image, a loss function value for the first preprocessed image is calculated using a defined loss function. Based on the loss function value and preprocessing hyperparameters for the first preprocessed image, a determination is made as to whether the loss function value and preprocessing hyperparameters for the first preprocessed image satisfy their respective preset conditions. If at least one of the loss function value and preprocessing hyperparameters for the first preprocessed image satisfies its respective preset conditions, the current iterative optimization is determined to satisfy the preset conditions. In this case, the preprocessing hyperparameters for the first preprocessed image are determined as the optimal preprocessing hyperparameters for the image, and the iterative optimization ends.

[0080] If it is determined that the loss function value and preprocessing hyperparameters of the first preprocessed image do not meet their respective preset conditions, indicating that the preprocessing hyperparameters of the first preprocessed image are not optimal for the image, the image is preprocessed a second time using the preprocessing algorithm to obtain a second preprocessed image. The second preprocessed image is input into the frozen disease recognition model of the initial dual-branch multi-task deep network model for disease recognition, thereby obtaining a predicted disease result for the second preprocessed image.

[0081] Based on the predicted disease results and actual disease results obtained for the secondary preprocessed image, a loss function value for the secondary preprocessed image is calculated using a defined loss function. Based on the loss function value and preprocessing hyperparameters for the secondary preprocessed image, a determination is made as to whether the loss function value and preprocessing hyperparameters for the secondary preprocessed image meet their respective preset conditions. When at least one of the loss function value and preprocessing hyperparameters for the secondary preprocessed image meets its respective preset conditions, the current iterative optimization is determined to meet the preset conditions. At this point, the preprocessing hyperparameters for the secondary preprocessed image are determined as the optimal preprocessing hyperparameters for the image, and the iterative optimization ends. It should be understood that each time an image is preprocessed, different preprocessing hyperparameters are obtained. After the image is preprocessed, a set of preprocessing hyperparameters corresponding to the preprocessing is obtained.

[0082] In the present application, before iteratively optimizing the preprocessing hyperparameters of the image through a Bayesian optimizer based on the image loss function to obtain the optimal preprocessing hyperparameters of the image, the method also includes: determining and initializing the preprocessing hyperparameter list; setting the value range of each hyperparameter in the preprocessing hyperparameter list and setting preset conditions.

[0083] In an embodiment of the present application, before executing the step of iteratively optimizing the preprocessing hyperparameters of the image using a Bayesian optimizer based on the loss function of the image and obtaining the optimal preprocessing hyperparameters of the image, it is also necessary to define a list of preprocessing hyperparameters that need to be preprocessed and initialize the list of preprocessing hyperparameters. The range of values ​​for each preprocessing hyperparameter in the list of preprocessing hyperparameters is then set to prevent excessive iterative optimization, and preset conditions are set to terminate the iterative optimization when the optimal preprocessing hyperparameters of the desired image are obtained.

[0084] Specifically, the preprocessing hyperparameter list defined in this application is θ = (a, b, c, m, l). Among them, a is the image contrast adjustment coefficient in the image contrast preprocessing algorithm, which is used to transform the pixels of the input image; b is the gain in the image brightness preprocessing algorithm, which is used to control the contrast of the image; c is the image brightness adjustment coefficient in the image brightness preprocessing algorithm, which is used to control the brightness of the image; m is the arithmetic square root of the number of image cut blocks. In the adaptive histogram equalization algorithm, a histogram distribution function is calculated for each image block to perform contrast limiting on each image block; l is the contrast limit parameter, which represents the maximum degree of contrast enhancement. In specific applications, all contrast amplitudes higher than l are set to l, and the parts with contrast amplitudes higher than l are evenly distributed below the entire contrast histogram, thereby obtaining the final contrast histogram. Then, each hyperparameter in the defined preprocessing hyperparameter list is initialized to θ0 = (a0, b0, c0, m0, l0). Among them, a preferred implementation method for initializing each hyperparameter in the defined preprocessing hyperparameter list is to initialize each hyperparameter in the preprocessing hyperparameter list to the lower limit of their respective value ranges. It should be understood that this is only a preferred implementation method for initialization, and each hyperparameter in the defined preprocessing hyperparameter list can also be initialized in other implementation methods, which are not specifically limited here. Then the value range of each preprocessing hyperparameter in the preprocessing hyperparameter list set in this application is Θ={a=[1,3],b=[0,3),c=[0,100],m= [8,16],l=[1,5]}. The preset conditions include two: and Where iter is the current iteration step, iter-1 is the iteration step before the current iteration step, and ∈ is the judgment threshold.

[0085] It should be understood that during the iterative optimization process, the current iteration is the current iteration step. As the iterative optimization progresses, the current iteration step changes. For example, if the second iteration is currently in progress, the current iteration step is the second iteration, and the previous iteration is the iteration step immediately preceding the current iteration. If the third iteration is currently in progress, the current iteration step is the third iteration, and the previous iteration is the iteration step immediately preceding the current iteration.

[0086] The relationship between the loss function value of the preprocessed image at the current iteration step and the loss function value of the preprocessed image at the previous iteration step of the current iteration step is satisfied. In the case of , each preprocessing hyperparameter in the preprocessing hyperparameter list of the image preprocessed at the current iteration step is determined as the optimal preprocessing hyperparameter corresponding to each preprocessing hyperparameter. The relationship between the preprocessing hyperparameter list of the image preprocessed at the current iteration step and the preprocessing hyperparameter list of the image preprocessed at the previous iteration step of the current iteration step is satisfied. In the case of , each preprocessing hyperparameter in the preprocessing hyperparameter list of the image preprocessed at the current iteration step is determined as the optimal preprocessing hyperparameter corresponding to each preprocessing hyperparameter. The loss function value of the image preprocessed at the current iteration step satisfies the relationship between the loss function value of the image preprocessed at the previous iteration step of the current iteration step. And, the relationship between the preprocessing hyperparameter list of the image preprocessed at the current iteration step and the preprocessing hyperparameter list of the image preprocessed at the previous iteration step of the current iteration step is satisfied In the case of , each preprocessing hyperparameter in the preprocessing hyperparameter list of the image preprocessed at the current iteration number is determined as the optimal preprocessing hyperparameter corresponding to each preprocessing hyperparameter.

[0087] In this application, the image is preprocessed according to the preprocessing algorithm, including: cascading multiple preprocessing algorithms to obtain a target preprocessing algorithm, the multiple preprocessing algorithms include at least: image contrast preprocessing algorithm, image brightness preprocessing algorithm and local adaptive histogram equalization preprocessing algorithm; the image is preprocessed by the target preprocessing algorithm.

[0088] In the embodiments of the present application, in order to apply the optimal preprocessing hyperparameters obtained based on preprocessing to the corresponding image to obtain a clearer image, thereby improving the accuracy of the final disease identification, the present application preprocesses the image by combining multiple preprocessing algorithms to obtain an overall preprocessing algorithm to preprocess the image.

[0089] Specifically, the present application cascades multiple preprocessing algorithms to obtain a target preprocessing algorithm. The preprocessing algorithms to be cascaded include an image contrast preprocessing algorithm, an image brightness preprocessing algorithm, and a local adaptive histogram equalization preprocessing algorithm. The target preprocessing algorithm obtained by cascading is used to preprocess the image. It should be understood that the type of preprocessing algorithm used in the above cascade is only a preferred implementation. Depending on the actual application scenario, other preprocessing algorithms can also be cascaded or the above-mentioned preprocessing algorithms can be removed from the cascade to form a target preprocessing algorithm. No specific limitation is made here.

[0090] In the embodiment of the present application, the expression of the image contrast preprocessing algorithm is: f1(I;a)=a×log(1+I)

[0091] Where I is the image to be preprocessed.

[0092] The expression of the image brightness preprocessing algorithm is: f2(I; b, c) = b × I + c

[0093] The expression of the local adaptive histogram equalization preprocessing algorithm is:

[0094] Among them, S max is the maximum slope of the local mapping function in image I.

[0095] In an embodiment of the present application, the process of iteratively optimizing the image preprocessing hyperparameters by using a Bayesian optimizer is as follows:

[0096] The first step is to perform initialization processing, including: defining the preprocessing hyperparameter list θ = (a, b, c, m, l), and then initializing each preprocessing hyperparameter in the preprocessing hyperparameter list a = a0, b = b0, c = c0, m = m0, l = l0, setting the value range of each preprocessing hyperparameter, and setting the number of loops #NUM and the judgment threshold ε in the termination condition.

[0097] After the initialization is completed, iterative optimization is performed. The image I is preprocessed according to the target preprocessing algorithm to obtain the preprocessed image The preprocessed image Input the frozen disease recognition model F in the initial dual-branch multi-task deep network model * Process and obtain predicted disease results Image-based The predicted disease results and the actual disease results are calculated by the loss function to obtain the corresponding loss function value; the preprocessing hyperparameter θ of the current iteration step after preprocessing is estimated by the Bayesian optimization algorithm. iter=(a iter ,b iter ,c iter ,m iter ,l iter ). The preprocessing hyperparameter θ at the current iteration step iter The preprocessing hyperparameter θ of the previous iteration step of the current iteration step iter-1 Meet the conditions Or the loss function value of the current iteration step is L iter The loss function value L of the previous iteration step is the same as the current iteration step iter-1 Meet the conditions In the case of the current iteration number of preprocessing hyperparameter θ iter =(a iter ,b iter ,c iter ,m iter ,l iter ) is determined as the optimal preprocessing hyperparameter θ for image I * =(a iter ,b iter ,c iter ,m iter ,l iter ); Under the above two conditions and If neither of the above two conditions is satisfied, the process returns to continue preprocessing the image I using the target preprocessing algorithm, and repeats this cycle until either of the above two conditions is satisfied or the maximum number of iterative optimization times #NUM is reached.

[0098] In the embodiment of the present application, the optimal preprocessing hyperparameter θ of image I is * The expression is:

[0099] Where f is the expression of the target preprocessing algorithm, f = f1*f2*f3, * represents cascade; Θ is the value range of the preprocessing hyperparameter list, Θ = {a∈[1,3],b∈[0,3),c∈[0,100],m∈[8,16],l∈[1,5]}.

[0100] In an embodiment of the present application, as shown in FIG2 , FIG2 shows a flowchart of iterative optimization of image preprocessing hyperparameters by a Bayesian optimizer in the present application. Each image in the road image dataset needs to be iteratively optimized to obtain the optimal preprocessing hyperparameters corresponding to each image. The iterative optimization process is illustrated below using an image as an example. First, the images in the road image dataset are preprocessed to obtain preprocessed images. The preprocessed images are then subjected to a frozen defect recognition model in an initial dual-branch multi-task deep network model for defect recognition to obtain corresponding predicted defect results. The similarity between the predicted defect results and the actual defect results of the images is determined using a loss function and the current preprocessing hyperparameters to determine whether the similarity meets the Bayesian optimizer's goal of maximizing similarity. If not, the preprocessing hyperparameters are adjusted and the image is processed again to obtain a new preprocessed image. The subsequent implementation is similar to the implementation of the first iterative optimization described above and will not be repeated here until the similarity of the current preprocessing meets the Bayesian optimizer's goal of maximizing similarity. At this time, the preprocessing hyperparameters corresponding to the current preprocessing are determined as the optimal preprocessing hyperparameters, and the iterative optimization ends.

[0101] In the present application, an initial dual-branch multi-task deep network model is trained by training image pairs to obtain a qualified dual-branch multi-task deep network model, including: training the image adaptive preprocessing network model in the initial dual-branch multi-task deep network model by training image pairs to obtain a qualified image adaptive preprocessing network model; and determining the qualified image adaptive preprocessing network model and the frozen disease recognition model as a qualified dual-branch multi-task deep network model.

[0102] In an embodiment of the present application, the frozen disease recognition model in the initial dual-branch multi-task deep network model of the present application is an existing disease recognition model that has been trained with large-scale data. The parameters of the existing disease recognition model are frozen, and in the process of training the initial dual-branch multi-task deep network model based on the training image pair, the frozen disease recognition model will not be trained, but only the image adaptive preprocessing network model will be trained in a targeted manner. In this way, the recognition accuracy and precision of disease recognition can be improved while reducing the training cost of the disease recognition model. This is because a qualified image adaptive preprocessing network model is obtained by optimizing the image preprocessing method and training the image adaptive preprocessing network model based on the training image pair consisting of the preprocessed optimized image and the original image. When performing disease recognition on the image to be recognized, the qualified image adaptive preprocessing network model can improve the input quality of the image to be recognized, and then be integrated into the disease recognition model, thereby improving the recognition accuracy and precision of disease recognition.

[0103] Specifically, the image adaptive preprocessing network model in the initial dual-branch multi-task deep network model is trained using a large number of training image pairs to obtain a qualified image adaptive preprocessing network model. If the image adaptive preprocessing network model in the initial dual-branch multi-task deep network model is trained to be qualified, the dual-branch multi-task deep network model composed of the trained image adaptive preprocessing network model and the frozen disease recognition model is a qualified dual-branch multi-task deep network model.

[0104] In the present application, the image to be identified is input into a qualified dual-branch multi-task deep network model for disease identification to obtain a target predicted disease result, including: inputting the image to be identified into the image adaptive preprocessing network model and the frozen disease recognition model in the qualified dual-branch multi-task deep network model for processing, obtaining a feature map of the first target image output by the image adaptive preprocessing network model, and obtaining a feature map of the image to be identified after being processed by the frozen disease recognition model; fusing the feature map of the first target image with the feature map of the image to be identified through an attention mechanism module to obtain a fused feature map; performing disease identification on the fused feature map through the frozen disease recognition model in the qualified dual-branch multi-task deep network model to obtain a target predicted disease result.

[0105] In an embodiment of the present application, an image to be identified is input into a qualified dual-branch multi-task deep network model for disease recognition, thereby obtaining a target disease prediction result. As shown in FIG3 , the same image to be identified is input into both an image adaptive preprocessing network model and a frozen disease recognition model within a qualified dual-branch multi-task deep network model. The image adaptive preprocessing network model processes the input image to be identified, obtaining a clear image corresponding to the image to be identified, and outputs a feature map of the clear image, which is also the feature map of the first target image. The frozen disease recognition model processes the input image to be identified, obtaining a feature map corresponding to the image to be identified. An attention mechanism is then used to fuse the feature map of the first target image output by the image adaptive preprocessing network model with the feature map corresponding to the image to be identified obtained by the frozen disease recognition model, resulting in a fused feature map. The frozen disease recognition model within the qualified dual-branch multi-task deep network model then performs disease recognition on this fused feature map, obtaining the final target disease prediction result. This improves the accuracy and precision of disease recognition while reducing the training cost of the disease recognition model.

[0106] In this application, a qualified image adaptive preprocessing network model is obtained by training the image adaptive preprocessing network model, including: determining the loss function of the frozen disease recognition model, and determining the fusion loss function of the image adaptive preprocessing network model, the fusion loss function includes the generative adversarial loss function and the L1 loss function; constructing a total loss function based on the loss function and the fusion loss function; training the image adaptive preprocessing network model based on the total loss function and the training image pairs to obtain a qualified image adaptive preprocessing network model.

[0107] In the embodiments of the present application, the loss function used in the model training process to evaluate the quality of the model is the core of the model. The present application defines the loss function of the image adaptive preprocessing network model as comprising two sections: one section is the loss function of the frozen disease recognition model, and the other section is the fusion loss function of the image adaptive preprocessing network model composed of the generative adversarial loss function and the L1 loss function. These two loss functions together constitute the total loss function used to evaluate the training results of the image adaptive preprocessing network model. The image adaptive preprocessing network model is then trained using a large number of constructed training image pair samples, and the training results are evaluated using the total loss function to obtain a qualified image adaptive preprocessing network model.

[0108] In the embodiment of the present application, the image adaptive preprocessing network model is essentially an image generation model. The present application uses a fusion of the generative adversarial loss function (GAN Loss) and the L1 loss function (L1 Loss) to drive the training of the image adaptive preprocessing network model. The expression is as follows:

[0109] Where N is the number of training image pairs, and D is the discriminator, which is implemented using a small convolutional neural network, such as the 5-layer AlexNet, and is mainly used to distinguish the authenticity of generated images. The clear image predicted by the image adaptive preprocessing network model for image I; is the real clear image corresponding to the input image I obtained through the above Bayesian iterative optimization, that is, the optimal preprocessing hyperparameter θ based on image I * =(a iter ,b iter ,c iter ,m iter ,l iter ) to obtain a clear image.

[0110] The total loss function expression is as follows:

[0111] Among them, λ1, λ2, and λ3 are weights used to balance the scales of each loss function. In this application, their values ​​are 0.4, 0.2, and 0.4 respectively.

[0112] In an embodiment of the present application, the network architecture of the image adaptive preprocessing network model is preferably the same network structure as the frozen disease recognition model, such as a U-Net model.

[0113] In an embodiment of the present application, as shown in FIG4 , FIG4 shows a structural diagram of the attention mechanism module for feature map fusion in the present application. The feature map output by the image adaptive preprocessing network model of the lower branch in the qualified dual-branch multi-task deep network model is processed by a global pooling layer and then processed by two fully connected layers. The result is fused with the feature map of the frozen disease recognition model of the upper branch in the qualified dual-branch multi-task deep network model through dot product.

[0114] In an embodiment of the present application, by comparing the positive sample similarity index of a pavement defect recognition method based on a dual-branch network provided by the present application with a defect recognition model trained using large-scale data and a qualified defect recognition model trained using a conventional preprocessing method after enhancing the training samples, the positive sample similarity index obtained by the pavement defect recognition method based on a dual-branch network provided by the present application is 0.722, the positive sample similarity index obtained by the disease recognition model trained using large-scale data is 0.652, and the positive sample similarity index obtained by the qualified defect recognition model trained using a conventional preprocessing method after enhancing the training samples is 0.657. This shows that the positive sample similarity index obtained by applying the pavement defect recognition method based on a dual-branch network provided by the present application has been significantly improved; the image adaptive preprocessing network model in the dual-branch multi-task deep network model of the present application learns image features that are adaptive to the defect image to improve the clarity of the defect area in the input image, thereby improving the recognition accuracy of the existing defect recognition model for the defect area and making the discrimination of pavement defects more accurate. As shown in Figures 5 to 7, Figures 5 to 7 respectively show the images obtained after the image to be identified is processed by the image adaptive preprocessing network model; the predicted disease results obtained by performing disease identification on the image to be identified by using the existing trained disease recognition model, which predicted disease results do not identify the diseased area in the image to be identified; and the target predicted disease results obtained by performing disease identification on the qualified trained dual-branch multi-task deep network model provided by this application. At the same time, since the qualified dual-branch multi-task deep network model in this application is not trained based on the recognition model itself for image recognition to make the recognition model have better recognition effect (accuracy, etc.), but the image adaptive preprocessing network model in the qualified dual-branch multi-task deep network model is used to improve the clarity and other related qualities of the input image, thereby indirectly improving the recognition effect (accuracy, etc.) of the recognition model. In this way, for other recognition models, it is only necessary to replace the frozen disease recognition model in the qualified dual-branch multi-task deep network model with other recognition models to improve the image recognition effect (accuracy, etc.), without the need to further train the other recognition models to improve the recognition effect (accuracy, etc.) of image recognition. This can effectively save the training cost of the recognition model while improving the recognition effect (accuracy, etc.) of various recognition models.

[0115] In a second aspect of the present application, a pavement defect identification device based on a dual-branch network is provided. As shown in FIG8 , the device 800 includes:

[0116] The optimal preprocessing hyperparameter determination unit 801 is used to iteratively optimize the image preprocessing hyperparameters using a Bayesian optimizer to obtain the optimal image preprocessing hyperparameters;

[0117] a clear image determining unit 802, configured to apply the optimal preprocessing hyperparameters of the image to the image to obtain an optimized image corresponding to the image, wherein the image and the optimized image corresponding to the image constitute a training image pair;

[0118] A model training unit 803 is configured to train an initial dual-branch multi-task deep network model using training image pairs to obtain a qualified dual-branch multi-task deep network model, wherein the initial dual-branch multi-task deep network model includes at least an image adaptive preprocessing network model and a frozen disease recognition model;

[0119] The disease prediction unit 804 is used to input the image to be identified into a qualified dual-branch multi-task deep network model for disease identification to obtain a target predicted disease result.

[0120] In some embodiments of the present application, the optimal preprocessing hyperparameter determination unit 801 includes:

[0121] A loss function construction unit, used to construct a loss function of an image based on the predicted disease results and the actual disease results of the image;

[0122] The first optimal preprocessing hyperparameter determination unit is used to iteratively optimize the image preprocessing hyperparameters based on the image loss function through a Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image.

[0123] In some embodiments of the present application, the first optimal preprocessing hyperparameter determination unit includes:

[0124] A preprocessing unit, configured to preprocess the image according to a preprocessing algorithm;

[0125] A predicted disease result determination unit is used to input the preprocessed image into the initial dual-branch multi-task deep network model to freeze the disease recognition model for disease recognition, thereby obtaining a predicted disease result of the preprocessed image;

[0126] A loss function value determination unit is used to determine the loss function value of the preprocessed image through the loss function according to the predicted disease result and the actual disease result of the preprocessed image;

[0127] A preset condition determination unit is used to determine whether the current iterative optimization satisfies the preset conditions based on the loss function value of the preprocessed image and the preprocessing hyperparameters;

[0128] The second optimal preprocessing hyperparameter unit is used to determine that the preprocessing hyperparameters of the preprocessed image are the optimal preprocessing hyperparameters of the image when it is determined that the current iterative optimization meets the preset conditions; and is used to preprocess the image again according to the preprocessing algorithm when it is determined that the current iterative optimization does not meet the preset conditions, and call the predicted disease result determination unit.

[0129] In some embodiments of the present application, the pre-processing unit includes:

[0130] a target preprocessing algorithm determination unit, configured to cascade multiple preprocessing algorithms to obtain a target preprocessing algorithm, wherein the multiple preprocessing algorithms include at least: an image contrast preprocessing algorithm, an image brightness preprocessing algorithm, and a local adaptive histogram equalization preprocessing algorithm;

[0131] The preprocessing subunit is used to preprocess the image using a target preprocessing algorithm.

[0132] In some embodiments of the present application, the apparatus further comprises:

[0133] A hyperparameter list determination unit, used to determine and initialize a preprocessing hyperparameter list;

[0134] The hyperparameter list value range determination unit is used to set the value range of each hyperparameter in the preprocessing hyperparameter list and set preset conditions.

[0135] The model training unit 803 includes:

[0136] A model training subunit is used to train the image adaptive preprocessing network model in the initial dual-branch multi-task deep network model through training image pairs to obtain a qualified image adaptive preprocessing network model;

[0137] The model determination unit is used to determine the qualified image adaptive preprocessing network model and frozen disease recognition model as a qualified dual-branch multi-task deep network model.

[0138] In some embodiments of the present application, the disease prediction unit 804 includes:

[0139] An image processing unit is used to input the image to be identified into the image adaptive preprocessing network model and the frozen disease recognition model in the qualified dual-branch multi-task deep network model for processing, thereby obtaining a feature map of the first target image output by the image adaptive preprocessing network model and a feature map of the image to be identified after being processed by the frozen disease recognition model;

[0140] A feature map fusion unit is used to fuse the feature map of the first target image with the feature map of the image to be recognized through an attention mechanism module to obtain a fused feature map;

[0141] The disease prediction subunit is used to identify diseases on the fused feature map through the frozen disease recognition model in the qualified dual-branch multi-task deep network model to obtain the target predicted disease result.

[0142] In some embodiments of the present application, the model training subunit includes:

[0143] A loss function determination unit is used to determine the loss function of the frozen disease recognition model and the fusion loss function of the image adaptive preprocessing network model, where the fusion loss function includes a generative adversarial loss function and an L1 loss function;

[0144] The total loss function construction unit is used to construct the total loss function based on the loss function and the fusion loss function;

[0145] The first model training subunit is used to train the image adaptive preprocessing network model based on the total loss function and the training image pairs to obtain a qualified image adaptive preprocessing network model.

[0146] The third aspect of the present application provides a computer-readable storage medium for storing a program, which, when executed by a processor, implements the steps of a pavement defect identification method based on a dual-branch network in the first aspect of the present application.

[0147] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0148] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0149] Each embodiment in this specification is described in a related manner. Similar portions between the embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so their description is relatively simple. For related portions, refer to the description of the method embodiments.

[0150] The above are only preferred embodiments of the present application and are not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A pavement disease identification method based on a double-branch network, characterized in that: The method comprises: Iteratively optimizing the preprocessing hyperparameters of the image through a Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image; Applying the optimal preprocessing hyperparameters of the image to the image to obtain an optimized image corresponding to the image, wherein the image and the optimized image corresponding to the image constitute a training image pair; Training an initial dual-branch multi-task deep network model through training image pairs to obtain a qualified dual-branch multi-task deep network model, wherein the initial dual-branch multi-task deep network model at least includes an image adaptive preprocessing network model and a freezing disease recognition model; The image to be identified is input into the qualified dual-branch multi-task deep network model for disease identification to obtain the target predicted disease result.

2. A pavement disease identification method based on a double-branch network according to claim 1, characterized in that: The iterative optimization of the preprocessing hyperparameters of the image by the Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image includes: Construct the loss function of the image based on the predicted disease results and the actual disease results of the image; Based on the loss function of the image, the preprocessing hyperparameters of the image are iteratively optimized through a Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image.

3. The pavement disease identification method based on a double-branch network according to claim 2 is characterized in that: The image-based loss function iteratively optimizes the image preprocessing hyperparameters through a Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image, including: Preprocess the image according to the preprocessing algorithm; Inputting the preprocessed image into the initial dual-branch multi-task deep network model to freeze the disease recognition model for disease recognition, and obtaining the predicted disease result of the preprocessed image; According to the predicted disease result and the actual disease result of the preprocessed image, the loss function value of the preprocessed image is determined by the loss function; Determining whether the current iterative optimization meets preset conditions according to the loss function value of the preprocessed image and the preprocessing hyperparameters; When it is determined that the current iterative optimization meets the preset conditions, the preprocessing hyperparameters of the preprocessed image are determined to be the optimal preprocessing hyperparameters of the image; when it is determined that the current iterative optimization does not meet the preset conditions, the image is preprocessed again according to the preprocessing algorithm, and the process returns to step: inputting the preprocessed image into the initial dual-branch multi-task deep network model to freeze the disease recognition model for disease recognition, and obtaining the predicted disease result of the preprocessed image.

4. The pavement disease identification method based on a double-branch network according to claim 3 is characterized in that: The image is preprocessed according to the preprocessing algorithm, including: Cascading multiple preprocessing algorithms to obtain a target preprocessing algorithm, wherein the multiple preprocessing algorithms at least include: an image contrast preprocessing algorithm, an image brightness preprocessing algorithm, and a local adaptive histogram equalization preprocessing algorithm; The image is preprocessed by the target preprocessing algorithm.

5. The pavement disease identification method based on a dual-branch network according to claim 3 is characterized in that: Before iteratively optimizing the preprocessing hyperparameters of the image by a Bayesian optimizer based on the loss function of the image to obtain the optimal preprocessing hyperparameters of the image, the method further includes: Determine and initialize the list of preprocessing hyperparameters; Set the value range of each hyperparameter in the preprocessing hyperparameter list and set the preset conditions.

6. The pavement disease identification method based on a double-branch network according to claim 1 is characterized in that: The method of training the initial dual-branch multi-task deep network model through training image pairs to obtain a qualified dual-branch multi-task deep network model comprises: training the image adaptive preprocessing network model in the initial dual-branch multi-task deep network model through training image pairs to obtain a qualified image adaptive preprocessing network model; The qualified image adaptive preprocessing network model and frozen disease recognition model are determined as qualified dual-branch multi-task deep network models.

7. The pavement disease identification method based on a double-branch network according to claim 1 is characterized in that: The step of inputting the image to be identified into the qualified dual-branch multi-task deep network model for disease identification to obtain a target disease prediction result includes: Inputting the image to be identified into the image adaptive preprocessing network model and the frozen disease recognition model in the qualified dual-branch multi-task deep network model for processing, respectively, obtaining a feature map of the first target image output by the image adaptive preprocessing network model, and obtaining a feature map of the image to be identified after being processed by the frozen disease recognition model; Fusing the feature map of the first target image with the feature map of the image to be recognized through an attention mechanism module to obtain a fused feature map; The frozen disease recognition model in the qualified dual-branch multi-task deep network model is used to perform disease recognition on the fused feature map to obtain a target predicted disease result.

8. The pavement disease identification method based on a double-branch network according to claim 6 is characterized in that: By training the image adaptive preprocessing network model, a qualified image adaptive preprocessing network model is obtained, including: Determine the loss function of the freezing disease recognition model, and determine the fusion loss function of the image adaptive preprocessing network model, wherein the fusion loss function includes a generative adversarial loss function and an L1 loss function; Constructing a total loss function according to the loss function and the fusion loss function; Based on the total loss function and the training image pairs, the image adaptive preprocessing network model is trained to obtain a qualified image adaptive preprocessing network model.

9. A road surface disease identification device based on a double-branch network, characterized in that: The device comprises: An optimal preprocessing hyperparameter determination unit, used for iteratively optimizing the preprocessing hyperparameters of the image through a Bayesian optimizer to obtain the optimal preprocessing hyperparameters of the image; a clear image determination unit, configured to apply the optimal preprocessing hyperparameters of the image to the image to obtain an optimized image corresponding to the image, wherein the image and the optimized image corresponding to the image constitute a training image pair; A model training unit, used to train an initial dual-branch multi-task deep network model through training image pairs to obtain a qualified dual-branch multi-task deep network model, wherein the initial dual-branch multi-task deep network model at least includes an image adaptive preprocessing network model and a freezing disease recognition model; The disease prediction unit is used to input the image to be identified into the qualified dual-branch multi-task deep network model for disease identification to obtain a target predicted disease result.

10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of a pavement damage identification method based on a dual-branch network as described in any one of claims 1 to 8 are implemented.

Images