Road surface disease identification method, model training method, electronic device and medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI ROAD & BRIDGE TESTING CO LTD
- Filing Date
- 2023-07-27
- Publication Date
- 2026-06-26
Smart Images
Figure CN116843983B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of road maintenance, and in particular to a method for identifying road defects, a model training method, electronic equipment, and media. Background Technology
[0002] Road surface defects typically include cracks, fractures, and potholes, and are common conditions encountered during road maintenance. The presence of road defects poses significant safety hazards to transportation, making accurate detection and timely repair crucial.
[0003] In related technologies, road surface image data is collected by inspection vehicles, and the collected image data is identified and detected based on a deep learning-based road surface defect detection model. After the detection is completed, the system automatically generates a road surface defect identification report, which includes a description of the crack type, degree of damage, and location information.
[0004] The detection schemes in related technologies have acceptable accuracy in identifying road surface defects in road surface images collected during the day when the light is good. However, the accuracy is low in identifying road surface defects in road surface images collected in poor lighting conditions, such as road surface images collected in rainy or foggy weather or at dusk.
[0005] Therefore, improving the accuracy of identifying road surface defects in road surface images acquired in low-light conditions is an urgent problem to be solved. Summary of the Invention
[0006] To improve the accuracy of identifying road surface defects in road surface images acquired in low-light conditions, this application provides a defect identification method, a model training method, an electronic device, and a medium.
[0007] Firstly, this application provides a method for identifying road surface defects, employing the following technical solution:
[0008] A method for identifying pavement defects includes:
[0009] Acquire road surface images;
[0010] The road surface image is input into the road defect identification model to obtain the defect type. The defect identification model is obtained by transfer learning of the first model based on the second model. The first model is obtained by training the initial network model based on the training sample set. The training sample set includes multiple initial images, each of which is an image taken of a road defect, and each initial image is associated with a defect type label. The second model is obtained by training the initial model based on the light and shadow training sample set. The light and shadow training sample set includes multiple light and shadow images, which are road surface images taken under a preset type of weather conditions or under conditions where the light intensity is less than a preset value.
[0011] By adopting the above technical solution, since the disease identification model is obtained by transferring learning from the first model to the second model, and the first model is obtained by training the initial network model based on the training sample set, and the second model is obtained based on the light and shadow training sample set of multiple images taken in preset weather conditions or in environments with light intensity less than preset values, the disease identification model obtained after transfer learning can not only identify road surface diseases, but also improve the ability to identify images in scenes with insufficient light and special weather conditions. After acquiring road surface images, the disease identification model is used to identify diseases, and the disease types obtained are relatively accurate.
[0012] Secondly, this application provides a model training method, which adopts the following technical solution:
[0013] A model training method, comprising:
[0014] Obtain a training sample set, which includes multiple initial images, each of which is an image of road surface defects, and each initial image is associated with a defect type label;
[0015] The initial network model is trained based on the training sample set to obtain the first model;
[0016] Based on the second model, the first model is transferred to obtain a disease identification model. The second model is trained on a light and shadow training sample set, which includes multiple light and shadow images. The light and shadow images are road surface images taken under a preset type of weather conditions or under conditions where the light intensity is less than a preset value.
[0017] By adopting the above technical solution, since the disease identification model is obtained by transferring learning from the first model to the second model, and the first model is obtained by training the initial network model based on the training sample set, and the second model is obtained based on the light and shadow training sample set of multiple images taken in preset weather conditions or in environments where the light intensity is less than a preset value, the disease identification model obtained after transfer learning can not only identify road surface diseases, but also improve the ability to identify images of scenes with insufficient light and special weather conditions.
[0018] In one possible implementation, training the initial network model based on the training sample set to obtain the first model includes:
[0019] The training sample set is subjected to feature enhancement preprocessing to obtain the training image corresponding to each of the initial images after preprocessing. The preprocessing includes multi-scale feature extraction, attention calculation processing, and dilated convolution processing.
[0020] The initial network model is trained based on each of the training images to obtain the first model.
[0021] In one possible implementation, the step of performing feature enhancement preprocessing on the training sample set to obtain a training image corresponding to each of the initial images after preprocessing includes:
[0022] Multi-scale feature extraction is performed on each initial image to obtain global features corresponding to each initial image. The global features include the low-level features corresponding to each initial image at multiple image scales. The low-level features include at least color features, texture features, edge features, and shape features.
[0023] Attention is calculated on the global features corresponding to each initial image to obtain the global features of each initial image after attention calculation.
[0024] Dilated convolution is performed on the global features of each initial image after attention calculation to obtain the training image corresponding to each initial image.
[0025] In one possible implementation, the attention calculation performed on the global features corresponding to each initial image to obtain the global features of each initial image after the attention calculation includes:
[0026] For each initial image, assign corresponding weight parameters to the underlying features at each image scale;
[0027] The weighted average of each of the underlying features of each initial image and the weight parameters corresponding to each underlying feature is calculated to obtain the global features of each initial image after attention calculation.
[0028] In one possible implementation, the step of performing feature enhancement preprocessing on the training sample set to obtain a training image corresponding to each of the initial images after preprocessing includes:
[0029] The GAN-based generator preprocesses the training sample set to obtain the training image corresponding to each of the initial images after preprocessing.
[0030] In one possible implementation, the transfer learning of the first model based on the second model to obtain the disease identification model includes:
[0031] The first feature extraction layer of the first model and the second feature extraction layer of the second model are fused to obtain a fused feature extraction layer. The first model includes the first feature extraction layer and the second model includes the second feature extraction layer.
[0032] A disease identification model is obtained based on the fused feature extraction layer and the first model.
[0033] Thirdly, this application provides a road surface defect identification device, which adopts the following technical solution:
[0034] A road surface defect identification device, comprising:
[0035] Road surface image acquisition module, used to acquire road surface images;
[0036] The road surface image is input into the road surface image recognition model to obtain the type of road surface defect.
[0037] Fourthly, this application provides a model training device, which adopts the following technical solution:
[0038] A model training device, comprising:
[0039] The training sample set acquisition module is used to acquire a training sample set, which includes multiple initial images, each of which is an image of road surface defects, and each of the initial images is associated with a defect type label;
[0040] The training module is used to train the initial network model based on the training sample set to obtain the first model;
[0041] The transfer learning module is used to perform transfer learning on the first model based on the second model to obtain a disease identification model. The second model is trained on a light and shadow training sample set, which includes multiple light and shadow images. The light and shadow images are road surface images taken under a preset type of weather conditions or under conditions where the light intensity is less than a preset value.
[0042] In one possible implementation, when the training module trains the initial network model based on the training sample set to obtain the first model, it is specifically used for:
[0043] The training sample set is subjected to feature enhancement preprocessing to obtain the training image corresponding to each of the initial images after preprocessing. The preprocessing includes multi-scale feature extraction, attention calculation processing, and dilated convolution processing.
[0044] The initial network model is trained based on each of the training images to obtain the first model.
[0045] In one possible implementation, when the training module performs feature enhancement preprocessing on the training sample set to obtain the training image corresponding to each preprocessed initial image, it specifically performs the following:
[0046] Multi-scale feature extraction is performed on each initial image to obtain global features corresponding to each initial image. The global features include the low-level features corresponding to each initial image at multiple image scales. The low-level features include at least color features, texture features, edge features, and shape features.
[0047] Attention is calculated on the global features corresponding to each initial image to obtain the global features of each initial image after attention calculation.
[0048] Dilated convolution is performed on the global features of each initial image after attention calculation to obtain the training image corresponding to each initial image.
[0049] In one possible implementation, when the training module performs attention calculations on the global features corresponding to each initial image to obtain the global features of each initial image after the attention calculation, it specifically performs the following:
[0050] For each initial image, assign corresponding weight parameters to the underlying features at each image scale;
[0051] The weighted average of each of the underlying features of each initial image and the weight parameters corresponding to each underlying feature is calculated to obtain the global features of each initial image after attention calculation.
[0052] In one possible implementation, when the training module performs feature enhancement preprocessing on the training sample set to obtain the training image corresponding to each preprocessed initial image, it specifically performs the following:
[0053] The GAN-based generator preprocesses the training sample set to obtain the training image corresponding to each of the initial images after preprocessing.
[0054] In one possible implementation, when the transfer learning module performs transfer learning on the first model based on the second model to obtain the disease identification model, it is specifically used for:
[0055] The first feature extraction layer of the first model and the second feature extraction layer of the second model are fused to obtain a fused feature extraction layer. The first model includes the first feature extraction layer and the second model includes the second feature extraction layer.
[0056] A disease identification model is obtained based on the fused feature extraction layer and the first model.
[0057] Fifthly, this application provides an electronic device that adopts the following technical solution:
[0058] An electronic device comprising:
[0059] At least one processor;
[0060] Memory;
[0061] At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: execute the pavement defect identification method described in the first aspect above and the model training method described in the second aspect above.
[0062] Sixthly, this application provides a computer-readable storage medium, which adopts the following technical solution:
[0063] A computer-readable storage medium includes: a computer program that can be loaded by a processor and execute the pavement defect identification method described in the first aspect and the model training method described in the second aspect.
[0064] In summary, this application includes at least one of the following beneficial technical effects:
[0065] Since the road damage identification model is obtained by transferring learning from the first model to the second model, and the first model is obtained by training the initial network model based on the training sample set, while the second model is obtained based on the light and shadow training sample set of multiple images taken in preset weather conditions or in environments with light intensity less than preset values, the road damage identification model obtained after transfer learning can not only identify road damage, but also improve the ability to identify images in low light and special weather conditions. After acquiring road images, the road damage identification model can be used to identify road damage, and the types of damage obtained are relatively accurate. Attached Figure Description
[0066] Figure 1 This is a flowchart illustrating the pavement defect identification method in the embodiments of this application;
[0067] Figure 2 This is a flowchart illustrating the model training method in an embodiment of this application;
[0068] Figure 3 This is a schematic diagram of the road surface defect identification device in the embodiments of this application;
[0069] Figure 4 This is a schematic diagram of the structure of the model training device in the embodiments of this application;
[0070] Figure 5 This is a schematic diagram of the structure of the electronic device according to an embodiment of this application. Detailed Implementation
[0071] The following is in conjunction with the appendix Figure 1 - Appendix Figure 5 This application will be described in further detail.
[0072] After reading this specification, those skilled in the art may make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.
[0073] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0074] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.
[0075] This application provides a method for identifying road surface defects, executed by an electronic device, as described above. Figure 1 The method includes steps S11 and S12, wherein:
[0076] Step S11: Acquire road surface image;
[0077] Step S12: Input the road surface image into the disease identification model to obtain the disease type.
[0078] In the embodiments of this application, the disease identification model is obtained by transferring learning from the first model to the second model. The first model is obtained by training the initial network model based on the training sample set. The training sample set includes multiple initial images, each of which is an image taken of road surface disease, and each initial image is associated with a disease type label. The second model is obtained by training the initial model based on the light and shadow training sample set, which includes multiple light and shadow images, which are road surface images taken under a preset type of weather conditions or under conditions where the light intensity is less than a preset value.
[0079] Since the road damage identification model is obtained by transferring learning from the first model to the second model, and the first model is obtained by training the initial network model based on the training sample set, while the second model is obtained based on the light and shadow training sample set of multiple images taken in preset weather conditions or in environments with light intensity less than preset values, the road damage identification model obtained after transfer learning can not only identify road damage, but also improve the ability to identify images in low light and special weather conditions. After acquiring road images, the road damage identification model can be used to identify road damage, and the types of damage obtained are relatively accurate.
[0080] The following section explains the methods for training disease identification models from the perspective of model training.
[0081] This application provides a model training method for training the pavement defect identification model used in the above-described pavement defect identification method embodiments. The model training method is executed by an electronic device, as described above. Figure 2 The method includes steps S21-S23, wherein:
[0082] Step S21: Obtain the training sample set;
[0083] In this embodiment of the application, the training sample set includes multiple training samples, which in turn include multiple initial images. Each initial image is an image taken of road surface defects, and each initial image is associated with a defect type label. Each initial image and its corresponding defect type label serve as a training sample. Some training samples are positive training samples, and some are negative training samples. The defect type label of a positive training sample corresponds to the actual defect type in the corresponding initial image, while the defect type label of a negative training sample differs from the actual defect type in the corresponding initial image.
[0084] Step S22: Train the initial network model based on the training sample set to obtain the first model.
[0085] Step S23: Transfer learning is performed on the first model based on the second model to obtain the disease identification model. The second model is trained on the light and shadow training sample set, which includes multiple light and shadow images. The light and shadow images are road surface images taken under a preset type of weather conditions or under conditions where the light intensity is less than a preset value.
[0086] In this embodiment, some light and shadow images are taken under preset weather conditions, while others are road surface images taken in environments where the light intensity is lower than a preset value, such as images taken at dawn and dusk. The preset weather types can include rain, fog, overcast skies, and other special weather types, which are not specifically limited in this embodiment. The preset brightness values are not specifically limited in this embodiment, and the dusk and dawn time periods can be selected based on the actual environmental conditions of the region.
[0087] Due to the complex and / or insufficient lighting conditions under specific weather conditions and when the light intensity is below a preset value, the colors, contours, and even edges and textures in the captured road surface images differ from those captured under abundant, single-light conditions during the day. Therefore, the second model, trained using multiple images with varying lighting conditions, demonstrates superior extraction and resolution capabilities for color, contour, edge, and texture features in road surface images compared to the first model. Consequently, the road surface defect recognition model, obtained by transferring learning from the first model to the second model, not only effectively identifies road surface defects but also enhances the extraction and resolution capabilities for color, contour, edge, and texture features in images taken under low light and special weather conditions.
[0088] Furthermore, the initial network model is trained based on the training sample set to obtain the first model. Specifically, this may include: performing feature enhancement preprocessing on the training sample set to obtain the training image corresponding to each preprocessed initial image, and then training the initial network model based on each training image to obtain the first model.
[0089] Specifically, the preprocessing includes multi-scale feature extraction, attention calculation, and dilated convolution. The training sample set is subjected to feature enhancement preprocessing to obtain the training image corresponding to each initial image after preprocessing. This includes: performing multi-scale feature extraction on each initial image to obtain the global features corresponding to each initial image; then performing attention calculation on the global features corresponding to each initial image to obtain the global features of each initial image after attention calculation; and finally performing dilated convolution on the global features of each initial image after attention calculation to obtain the training image corresponding to each initial image.
[0090] Specifically, multiple reference image scales are preset, and each initial image is transformed according to each reference image scale to obtain the corresponding reference images at each reference scale for each initial image. That is, one initial image corresponds to multiple reference images. Multi-scale feature extraction is performed on each initial image, that is, feature extraction is performed on each reference image corresponding to each initial image to obtain the underlying features corresponding to each reference image. The underlying features include at least color features, texture features, edge features, and shape features. Of course, it may also include the underlying features of other images. In this embodiment, no specific limitation is made.
[0091] When extracting features from an initial image at different scales, the level of detail of the features obtained varies. For example, in an image of fruit, determining whether it is an apple or a peach is sufficient to extract relevant features at a 30x30 image scale. However, to determine the ripeness of an apple or its specific variety, features need to be extracted at a 60x60 image scale for an accurate judgment. Similarly, road surface defects vary in size; some are large, others small. Therefore, at smaller image scales, extracted features can more accurately identify larger defects, but less accurately identify smaller defects, which may even be overlooked. Likewise, at larger image scales, extracted features can more accurately identify smaller defects, but less accurately identify larger defects. Therefore, multi-scale feature extraction from the initial image can increase the perceptual field of view.
[0092] Furthermore, weight parameters are assigned to the underlying features of each initial image at each image scale. These weight parameters can be preset based on the image scale. Then, attention is calculated on the global features of each initial image to obtain the global features of each initial image after attention calculation. This involves assigning weight parameters to the underlying features of each initial image at each image scale, and then performing a weighted average of the underlying features and their respective weight parameters to obtain the global features of each initial image after attention calculation.
[0093] Furthermore, dilated convolutions are performed on the global features corresponding to each initial image, and the initial images are further fused to obtain feature-enhanced images corresponding to the initial images, which are the training images used to train the initial network model. Each training image is associated with the disease type label corresponding to the initial image, and then the initial network model is trained based on each training image and its corresponding disease type label to obtain the first model.
[0094] Furthermore, the training sample set is preprocessed with feature enhancement to obtain the training image corresponding to each initial image after preprocessing. This can also be achieved using a generator from a Generative Adversarial Network (GAN). For example, using a generator from the U-Net model, the downsampling step of the generator is the multi-scale feature extraction of the initial image, and the attention calculation step and dilated convolution are the feature fusion in the upsampling step, thus obtaining the feature-enhanced image.
[0095] Furthermore, in the process of training the initial network model to obtain the second model using light and shadow training samples including multiple light and shadow images, feature enhancement preprocessing can be performed on each light and shadow image in the light and shadow training set. Then, the initial network model is trained based on the preprocessed light and shadow images to obtain the second model. Specifically, each light and shadow image should contain a corresponding type label, which can be a road surface defect type label, a weather type label, or other labels that can characterize the information of the light and shadow image. However, the label type of all light and shadow images should be uniform, but the specific label type is not specifically limited in this embodiment.
[0096] Of course, the second model can also be obtained by directly training the initial network model based on each light and shadow image and its corresponding label.
[0097] Furthermore, after obtaining the first model and the second model, a first feature extraction layer for feature extraction of the input image corresponding to the first model and a second feature extraction layer for feature extraction of the input image corresponding to the second model are extracted respectively. Then, the first feature extraction layer of the first model and the second feature extraction layer of the second model are fused to obtain the fused feature extraction layer.
[0098] Specifically, the difference parameters between the first feature extraction layer and the second feature extraction layer are compared, and then the difference parameters are added to the first feature extraction in the first model for fusion to obtain the disease identification model.
[0099] The above embodiments introduce a road surface defect identification method and a model training method from the perspective of method flow. The following embodiments introduce a road surface defect identification device and a model training device from the perspective of virtual modules or virtual units. For details, please refer to the following embodiments.
[0100] This application provides a road surface defect identification device, such as... Figure 3 As shown, the road surface defect identification device may specifically include a road surface image acquisition module 301 and a defect identification module 302, wherein:
[0101] Road surface image acquisition module 301 is used to acquire road surface images;
[0102] The disease identification module 302 is used to input the road surface image into the disease identification model to obtain the disease type.
[0103] This application provides a model training device, such as... Figure 4 As shown, the model training device may specifically include a training sample set acquisition module 401, a training module 402, and a transfer learning module 403, wherein:
[0104] The training sample set acquisition module 401 is used to acquire a training sample set, which includes multiple initial images. Each initial image is an image taken of road surface defects, and each initial image is associated with a defect type label.
[0105] Training module 402 is used to train the initial network model based on the training sample set to obtain the first model;
[0106] The transfer learning module 403 is used to perform transfer learning on the first model based on the second model to obtain a disease identification model. The second model is trained on a light and shadow training sample set, which includes multiple light and shadow images. The light and shadow images are road surface images taken under a preset type of weather conditions or under conditions where the light intensity is less than a preset value.
[0107] In one possible implementation, when training the initial network model based on the training sample set to obtain the first model, the training module 402 is specifically used for:
[0108] The training sample set is subjected to feature enhancement preprocessing to obtain the training image corresponding to each initial image after preprocessing. The preprocessing includes multi-scale feature extraction, attention calculation and dilated convolution.
[0109] The initial network model is trained based on each training image to obtain the first model.
[0110] In one possible implementation, when the training module 402 performs feature enhancement preprocessing on the training sample set to obtain the preprocessed training image corresponding to each initial image, it is specifically used for:
[0111] Multi-scale feature extraction is performed on each initial image to obtain the global features corresponding to each initial image. The global features include the low-level features corresponding to each initial image at multiple image scales. The low-level features include at least color features, texture features, edge features, and shape features.
[0112] Attention is calculated on the global features corresponding to each initial image to obtain the global features of each initial image after attention calculation.
[0113] Dilated convolution is performed on the global features of each initial image after attention calculation to obtain the training image corresponding to each initial image.
[0114] In one possible implementation, when the training module 402 performs attention calculation on the global features corresponding to each initial image to obtain the global features of each initial image after the attention calculation, it is specifically used for:
[0115] For each initial image, assign corresponding weight parameters to the underlying features at each image scale.
[0116] We calculate the global features of each initial image after attention calculation by weighting the low-level features and their corresponding weights for each initial image.
[0117] In one possible implementation, when the training module 402 performs feature enhancement preprocessing on the training sample set to obtain the preprocessed training image corresponding to each initial image, it is specifically used for:
[0118] The GAN-based generator preprocesses the training sample set to obtain the corresponding training image after preprocessing each initial image.
[0119] In one possible implementation, when the transfer learning module 403 performs transfer learning on the first model based on the second model to obtain the disease identification model, it is specifically used for:
[0120] The first feature extraction layer of the first model and the second feature extraction layer of the second model are fused to obtain the fused feature extraction layer. The first model includes the first feature extraction layer and the second model includes the second feature extraction layer.
[0121] The disease identification model is obtained based on the fused feature extraction layer and the first model.
[0122] This application provides an electronic device, such as... Figure 5 As shown, Figure 5 The illustrated electronic device 500 includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the electronic device 500 may also include a transceiver 504. It should be noted that in practical applications, the transceiver 504 is not limited to one type, and the structure of this electronic device 500 does not constitute a limitation on the embodiments of this application.
[0123] Processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 501 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.
[0124] Bus 502 may include a pathway for transmitting information between the aforementioned components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 502 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0125] The memory 503 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.
[0126] The memory 503 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 501. The processor 501 is used to execute the application code stored in the memory 503 to implement the content shown in the foregoing method embodiments.
[0127] Electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Servers can also be included. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0128] This application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content in the aforementioned road surface defect identification method embodiments and model training method embodiments.
[0129] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0130] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method for identifying road surface defects, characterized in that, include: Acquire road surface images; The road surface image is input into the road defect identification model to obtain the defect type. The defect identification model is obtained by transfer learning of the first model based on the second model. The first model is obtained by training the initial network model based on the training sample set. The training sample set includes multiple initial images, each of which is an image taken of a road defect, and each initial image is associated with a defect type label. The second model is obtained by training the initial model based on the light and shadow training sample set. The light and shadow training sample set includes multiple light and shadow images, which are road surface images taken under a preset type of weather conditions or under conditions where the light intensity is less than a preset value. The first model includes a first feature extraction layer, and the second model includes a second feature extraction layer. The transfer learning of the first model based on the second model includes: fusing the first feature extraction layer of the first model and the second feature extraction layer of the second model to obtain a fused feature extraction layer; and obtaining a disease identification model based on the fused feature extraction layer and the first model.
2. A model training method, characterized in that, include: Obtain a training sample set, which includes multiple initial images, each of which is an image of road surface defects, and each initial image is associated with a defect type label; The initial network model is trained based on the training sample set to obtain the first model; Based on the second model, the first model is transferred to obtain a disease identification model. The second model is obtained by training on a light and shadow training sample set, which includes multiple light and shadow images. The light and shadow images are road surface images taken under a preset type of weather conditions or under conditions where the light intensity is less than a preset value. The step of performing transfer learning on the first model based on the second model to obtain a disease identification model includes: The first feature extraction layer of the first model and the second feature extraction layer of the second model are fused to obtain a fused feature extraction layer. The first model includes the first feature extraction layer and the second model includes the second feature extraction layer. A disease identification model is obtained based on the fused feature extraction layer and the first model.
3. The model training method according to claim 2, characterized in that, The step of training the initial network model based on the training sample set to obtain the first model includes: The training sample set is subjected to feature enhancement preprocessing to obtain the training image corresponding to each of the initial images after preprocessing. The preprocessing includes multi-scale feature extraction, attention calculation processing, and dilated convolution processing. The initial network model is trained based on each of the training images to obtain the first model.
4. The model training method according to claim 3, characterized in that, The step of performing feature enhancement preprocessing on the training sample set to obtain the training image corresponding to each of the initial images after preprocessing includes: Multi-scale feature extraction is performed on each initial image to obtain global features corresponding to each initial image. The global features include the low-level features corresponding to each initial image at multiple image scales. The low-level features include at least color features, texture features, edge features, and shape features. Attention is calculated on the global features corresponding to each initial image to obtain the global features of each initial image after attention calculation. Dilated convolution is performed on the global features of each initial image after attention calculation to obtain the training image corresponding to each initial image.
5. The model training method according to claim 4, characterized in that, The step of performing attention calculation on the global features corresponding to each initial image to obtain the global features of each initial image after attention calculation includes: For each initial image, assign corresponding weight parameters to the underlying features at each image scale; The weighted average of each of the underlying features of each initial image and the weight parameters corresponding to each underlying feature is calculated to obtain the global features of each initial image after attention calculation.
6. The model training method according to claim 3, characterized in that, The step of performing feature enhancement preprocessing on the training sample set to obtain the training image corresponding to each of the initial images after preprocessing includes: The GAN-based generator preprocesses the training sample set to obtain the training image corresponding to each of the initial images after preprocessing.
7. A road surface defect identification device, characterized in that, include: Road surface image acquisition module, used to acquire road surface images; The road surface image is input into the road surface image recognition model to obtain the road surface image type. The road surface image recognition model is obtained by transfer learning of the first model based on the second model. The first model is trained on the initial network model based on the training sample set. The training sample set includes multiple initial images, each of which is an image of a road surface image, and each initial image is associated with a road surface image type label. The second model is trained on the initial model based on the light and shadow training sample set. The light and shadow training sample set includes multiple light and shadow images, which are road surface images taken under a preset weather condition or under an environment where the light intensity is less than a preset value. The first model includes a first feature extraction layer, and the second model includes a second feature extraction layer. The transfer learning of the first model based on the second model includes: fusing the first feature extraction layer of the first model and the second feature extraction layer of the second model to obtain a fused feature extraction layer; and obtaining a disease identification model based on the fused feature extraction layer and the first model.
8. An electronic device, characterized in that, The electronic device includes: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: perform the pavement distress identification method according to any one of claims 1 or 2-6.
9. A computer-readable storage medium, characterized in that, include: The system contains a computer program that can be loaded by a processor and executed as described in any one of claims 1 to 2-6 for identifying pavement defects.