Pavement disease identification method, device, equipment, storage medium and program product

Abstract

The present disclosure provides a road surface disease recognition method, device, equipment, storage medium and program product, including: extracting a road surface image to obtain a first feature map, extracting the first feature map based on a skip connection convolution module to obtain a second feature map; decoupling the second feature map based on a first prediction head to obtain a first decoupled feature map to predict grid-level road surface diseases; fusing the first decoupled feature map based on an attention head to obtain a fused feature, splicing the fused feature and the second feature map to obtain a third feature map; extracting the third feature map based on convolution kernels with different expansion rates to obtain features and splicing, extracting the spliced ​​features based on a convolution module to obtain a fourth feature map; decoupling the fourth feature map based on a second prediction head to obtain a second decoupled feature map to predict pixel-level road surface diseases; matching the grid-level road surface disease prediction result and the pixel-level road surface disease prediction result to obtain a road surface disease recognition result. The present disclosure improves the scope of application and efficiency of road surface disease recognition and reduces costs.

Description

Technical Field

[0001] The present disclosure relates to the technical field of pavement damage identification, and in particular to a pavement damage identification method, device, equipment, storage medium and program product. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] Pavement defect identification is an important part of ensuring road safety and extending road service life. With the development of technology, pavement defect identification has evolved from manual detection to automation and intelligence.

[0004] Different types of pavement and different types of pavement disease identification standards are also different. For example, for ordinary trunk roads and highway asphalt pavements, it is necessary to identify whether the pavement damage is included within the preset size of the pavement grid, such as cracks, repairs, potholes, etc.; for cement pavements, it is necessary to identify special disease types, such as exposed bones, broken boards, and board corner fractures; and for rural roads, it is necessary to identify the width of the cracks to define the severity of the cracks.

[0005] In the related art, one model is often used to identify one disease or one model is often used to identify several homogeneous diseases, such as diseases that are also defined using a grid.

[0006] However, this requires the design and deployment of multiple recognition models for a variety of damage types and identification criteria. On the one hand, this greatly reduces the system's efficiency in practical applications. On the other hand, it is impossible to establish effective associations between various diseases, reducing the recognition accuracy. Summary of the invention

[0007] In view of this, the purpose of the present disclosure is to propose a road surface disease identification method, device, equipment, storage medium and program product, which at least to a certain extent solve one of the technical problems in the related art.

[0008] Based on the above purpose, the first aspect of the exemplary embodiment of the present disclosure provides a method for identifying road damage, including:

[0009] Acquire a road image to be identified, perform a first feature extraction on the road image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, and perform a second feature extraction on the first feature map based on a second number of cascaded skip connection convolution modules to obtain a second feature map;

[0010] Based on the third number of first prediction heads in parallel, the second feature maps are respectively decoupled to obtain first decoupled feature maps corresponding to different pavement damage types; grid-level pavement damage prediction is performed based on the first decoupled feature maps to obtain a grid-level pavement damage prediction result corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement damage types;

[0011] Mapping different first decoupled feature maps to a fourth number of groups of attention heads respectively, performing feature fusion on the first decoupled feature maps based on the attention heads to obtain fused features, and concatenating the fused features with the second feature map to obtain a third feature map;

[0012] Performing a third feature extraction on the third feature map based on a fifth number of convolution kernels with different dilation rates to obtain a fifth number of groups of features at a fifth number of scales, splicing the fifth number of groups of features to obtain spliced ​​features, and performing a fourth feature extraction on the spliced ​​features based on a sixth number of cascaded convolution modules to obtain a fourth feature map;

[0013] Decoupling the fourth feature map based on a seventh number of second prediction heads in parallel, respectively, to obtain second decoupled feature maps corresponding to different pavement damage types, performing pixel-level pavement damage prediction based on the second decoupled feature maps, to obtain a pixel-level pavement damage prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement damage types;

[0014] The grid-level pavement damage prediction result and the pixel-level pavement damage prediction result are matched to obtain a pavement damage recognition result corresponding to the pavement image to be recognized.

[0015] Based on the same inventive concept, a second aspect of the exemplary embodiment of the present disclosure provides a road surface disease identification device, comprising:

[0016] The encoding module is configured to obtain a road surface image to be identified, perform a first feature extraction on the road surface image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, and perform a second feature extraction on the first feature map based on a second number of cascaded skip connection convolution modules to obtain a second feature map;

[0017] A grid-level pavement damage prediction module is configured to decouple the second feature map based on a third number of first prediction heads in parallel, respectively, to obtain first decoupled feature maps corresponding to different pavement damage types, and to perform grid-level pavement damage prediction based on the first decoupled feature maps to obtain a grid-level pavement damage prediction result corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement damage types;

[0018] a feature fusion module, configured to map different first decoupled feature maps to a fourth number of groups of attention heads, perform feature fusion on the first decoupled feature maps based on the attention heads to obtain fused features, and concatenate the fused features with the second feature map to obtain a third feature map;

[0019] a decoding module configured to perform a third feature extraction on the third feature map based on a fifth number of convolution kernels with different dilation rates to obtain a fifth number of groups of features at a fifth number of scales, splice the fifth number of groups of features to obtain spliced ​​features, and perform a fourth feature extraction on the spliced ​​features based on a sixth number of cascaded convolution modules to obtain a fourth feature map;

[0020] a pixel-level pavement damage prediction module, configured to decouple the fourth feature map based on a seventh number of second prediction heads in parallel, respectively, to obtain second decoupled feature maps corresponding to different pavement damage types, and to perform pixel-level pavement damage prediction based on the second decoupled feature maps, to obtain a pixel-level pavement damage prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement damage types;

[0021] The regularized matching module is configured to match the grid-level pavement damage prediction result and the pixel-level pavement damage prediction result to obtain a pavement damage recognition result corresponding to the pavement image to be recognized.

[0022] Based on the same inventive concept, a third aspect of the exemplary embodiment of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0023] Based on the same inventive concept, a fourth aspect of the exemplary embodiments of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method described in the first aspect.

[0024] Based on the same inventive concept, a fifth aspect of the exemplary embodiments of the present disclosure provides a computer program product, including computer program instructions. When the computer program instructions are executed on a computer, the computer executes the method described in the first aspect.

[0025] From the above, it can be seen that the pavement disease recognition method, device, equipment, storage medium and program product provided by the embodiments of the present disclosure include: obtaining a pavement image to be identified, performing a first feature extraction on the pavement image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, performing a second feature extraction on the first feature map based on a second number of cascaded jump connection convolution modules to obtain a second feature map; decoupling the second feature map based on a third number of parallel first prediction heads to obtain first decoupled feature maps corresponding to different pavement disease types, performing grid-level pavement disease prediction based on the first decoupled feature maps to obtain a grid-level pavement disease prediction result corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement disease types; mapping different first decoupled feature maps to a fourth number of groups of attention heads respectively, performing feature fusion on the first decoupled feature map based on the attention heads to obtain a Fusion features, splicing the fusion features with the second feature map to obtain a third feature map; extracting the third feature from the third feature map based on a fifth number of convolution kernels with different expansion rates to obtain a fifth number of groups of features at a fifth number of scales, splicing the fifth number of groups of features to obtain splicing features, extracting the fourth feature from the splicing features based on a sixth number of cascaded convolution modules to obtain a fourth feature map; decoupling the fourth feature map based on a seventh number of parallel second prediction heads to obtain second decoupled feature maps corresponding to different pavement disease types, performing pixel-level pavement disease prediction based on the second decoupled feature map to obtain a pixel-level pavement disease prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement disease types; matching the grid-level pavement disease prediction result with the pixel-level pavement disease prediction result to obtain a pavement disease recognition result corresponding to the pavement image to be recognized.

[0026] The present invention unifies the automatic recognition of various road surface damage types for various recognition standards under the same deep neural network framework, expands the scope of application of the model, improves the efficiency of the model in identifying damage in large-scale images, and reduces production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1a A schematic diagram of cracks in pavement damage provided by an exemplary embodiment of the present disclosure;

[0029] Figure 1b A schematic diagram of repairing a pavement defect provided by an exemplary embodiment of the present disclosure;

[0030] Figure 1c A schematic diagram of a road surface disease exposed in an exemplary embodiment of the present disclosure;

[0031] Figure 1d A schematic diagram of a pothole in a pavement defect provided by an exemplary embodiment of the present disclosure;

[0032] Figure 2a A schematic diagram of the width of cracks in pavement damage provided by an exemplary embodiment of the present disclosure;

[0033] Figure 2b A schematic diagram of the location of joints in pavement damage provided by an exemplary embodiment of the present disclosure;

[0034] Figure 3 A schematic diagram of an application scenario of a road surface disease identification method provided by an exemplary embodiment of the present disclosure;

[0035] Figure 4 A schematic diagram of a process flow of a road surface disease identification method provided by an exemplary embodiment of the present disclosure;

[0036] Figure 5 A schematic diagram of another application scenario of the road surface disease identification method provided by an exemplary embodiment of the present disclosure;

[0037] Figure 6 A schematic diagram of a pavement damage identification model provided by an exemplary embodiment of the present disclosure;

[0038] Figure 7 Another structural schematic diagram of a pavement disease identification model provided by an exemplary embodiment of the present disclosure;

[0039] Figure 8 A schematic diagram of a structure of a prediction head provided for an exemplary embodiment of the present disclosure;

[0040] Fig. 9 A schematic diagram of a structure of an attention head provided for an exemplary embodiment of the present disclosure;

[0041] Fig.10 Another structural schematic diagram of a prediction head provided for an exemplary embodiment of the present disclosure;

[0042] Fig.11 A schematic diagram of a structure of a road surface disease identification device provided by an exemplary embodiment of the present disclosure;

[0043] Fig.12A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, usage scenarios, etc. of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0045] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present application according to the prompt message.

[0046] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0047] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of the present application. Other methods that meet the relevant laws and regulations may also be applied to the implementation method of the present application.

[0048] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0049] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, the principles and spirit of the present disclosure will be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0050] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0051] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. The article "one" or "a" before an element does not exclude the existence of multiple such elements.

[0052] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

[0053] As described in the background art, road surface disease identification is an important part of ensuring road safety and extending road service life. With the development of technology, road surface disease identification has developed from the manual detection stage to the automation and intelligent stage.

[0054] With the establishment and improvement of the pavement technical condition evaluation standard system, the identification standards for different types of pavements and different types of pavement diseases are also different. For example, for ordinary trunk roads and asphalt pavements of highways, it is necessary to identify whether the pavement damage, such as cracks, repairs, potholes, etc., is included within the pavement grid range of a preset size (such as 10cm×10cm); for cement pavements, it is necessary to identify special disease types, such as exposed bones, broken plates, plate corner fractures, etc.; and for rural roads, it is necessary to identify the width of the cracks to define the severity of the cracks.

[0055] As a specific example:

[0056] refer to Figure 1a , showing cracks in typical grid-level pavement disease manifestations defined in highway technical condition assessment specifications;

[0057] refer to Figure 1b , showing the repairs in typical grid-level pavement disease manifestations defined in the highway technical condition assessment specifications;

[0058] refer to Figure 1c , showing the nakedness in the manifestation of typical grid-level pavement diseases defined in the highway technical condition assessment specifications;

[0059] refer to Figure 1d , showing potholes as a typical grid-level pavement disease manifestation as defined in the Highway Technical Condition Assessment Specifications.

[0060] refer to Figure 2a , showing that the width of the crack in the definition of the indicator for road condition evaluation requires semantic segmentation of the image.

[0061] refer to Figure 2b , showing the locations of seams in the definition of indicators for road condition evaluation that require semantic segmentation of images.

[0062] In the related art, one model is often used to identify one disease or one model is often used to identify several homogeneous diseases, such as diseases that are also defined using a grid.

[0063] However, the inventors of the present disclosure found that this requires the design and deployment of multiple recognition models for various damage types and discrimination criteria. On the one hand, this greatly reduces the system's working efficiency in practical applications, and on the other hand, it is impossible to establish effective associations between various diseases, thereby reducing the recognition accuracy.

[0064] In order to solve the above problems, the present disclosure provides a road surface disease recognition solution, which specifically includes: obtaining a road surface image to be identified, performing a first feature extraction on the road surface image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, performing a second feature extraction on the first feature map based on a second number of cascaded jump connection convolution modules to obtain a second feature map; decoupling the second feature map based on a third number of parallel first prediction heads to obtain first decoupled feature maps corresponding to different road surface disease types, performing grid-level road surface disease prediction based on the first decoupled feature maps to obtain a grid-level road surface disease prediction result corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different road surface disease types; mapping different first decoupled feature maps to a fourth number of groups of attention heads respectively, performing feature fusion on the first decoupled feature maps based on the attention heads to obtain fused features, and combining the fused features The combined features are spliced ​​with the second feature map to obtain a third feature map; based on a fifth number of convolution kernels with different expansion rates, a third feature is extracted from the third feature map to obtain a fifth number of groups of features at a fifth number of scales, the fifth number of groups of features are spliced ​​to obtain a spliced ​​feature, and a fourth feature is extracted from the spliced ​​feature based on a sixth number of cascaded convolution modules to obtain a fourth feature map; based on a seventh number of parallel second prediction heads, the fourth feature map is decoupled respectively to obtain second decoupled feature maps corresponding to different pavement disease types, pixel-level pavement disease prediction is performed based on the second decoupled feature map to obtain a pixel-level pavement disease prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement disease types; the grid-level pavement disease prediction result and the pixel-level pavement disease prediction result are matched to obtain a pavement disease recognition result corresponding to the pavement image to be recognized.

[0065] The present invention unifies the automatic recognition of various road surface damage types for various recognition standards under the same deep neural network framework, expands the scope of application of the model, improves the efficiency of the model in identifying damage in large-scale images, and reduces production costs.

[0066] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0067] refer to Figure 3 , which is a schematic diagram of an application scenario of the road surface disease identification method provided by an exemplary embodiment of the present disclosure.

[0068] This application scenario includes a terminal device 101, a server 102, and a data storage system 103. The terminal device 101, the server 102, and the data storage system 103 may be connected via a wired or wireless communication network to achieve data interaction.

[0069] The terminal device 101 may be an electronic device with data transmission and multimedia input / output functions close to the user side, including but not limited to a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a smart wearable device, a personal digital assistant (PDA) or other electronic devices capable of realizing the above functions. The electronic device may include a processor and a display screen with a touch input function, the display screen is used to present a graphical user interface, the graphical user interface can display an application interface, and the processor is used to process application data, generate a graphical user interface, and control the display of the graphical user interface on the display screen.

[0070] Server 102 and data storage system 103 can both be independent physical servers, or a server cluster or distributed system composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0071] In some exemplary embodiments, the pavement damage identification method may be run on the server 102 .

[0072] When the pavement defect recognition method is run on the server 102, the server 102 is used to provide a pavement defect recognition service to a user of the terminal device 101. The terminal device 101 is installed with a client that communicates with the server 102. The user can input the pavement image to be recognized through the client. The client sends the pavement image to be recognized to the server 102. The server 102 inputs the pavement image to be recognized into the trained pavement defect recognition model, obtains the pavement defect recognition result corresponding to the pavement image to be recognized output by the pavement defect recognition model, sends the pavement defect recognition result to the client, and the client displays the pavement defect recognition result to the user to help the user repair the pavement defect.

[0073] Specifically, the pavement damage recognition model in the server 102 recognizes pavement damage by using the pavement damage recognition method provided by the exemplary embodiment of the present disclosure:

[0074] Acquire a road image to be identified, perform a first feature extraction on the road image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, perform a second feature extraction on the first feature map based on a second number of cascaded jump connection convolution modules to obtain a second feature map; decouple the second feature map based on a third number of parallel first prediction heads to obtain first decoupled feature maps corresponding to different road damage types, perform grid-level road damage prediction based on the first decoupled feature maps to obtain a grid-level road damage prediction result corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different road damage types; map different first decoupled feature maps to a fourth number of groups of attention heads, perform feature fusion on the first decoupled feature map based on the attention heads to obtain fused features, and splice the fused features with the second feature map. A third feature map is obtained; based on a fifth number of convolution kernels with different expansion rates, a third feature is extracted from the third feature map to obtain a fifth number of groups of features at a fifth number of scales; the fifth number of groups of features are spliced ​​to obtain spliced ​​features; based on a sixth number of cascaded convolution modules, a fourth feature is extracted from the spliced ​​features to obtain a fourth feature map; based on a seventh number of parallel second prediction heads, the fourth feature map is decoupled to obtain second decoupled feature maps corresponding to different pavement damage types; based on the second decoupled feature maps, pixel-level pavement damage prediction is performed to obtain a pixel-level pavement damage prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement damage types; the grid-level pavement damage prediction result and the pixel-level pavement damage prediction result are matched to obtain a pavement damage recognition result corresponding to the pavement image to be recognized.

[0075] A large amount of training data is stored in the data storage system 103, each of which includes a road surface image to be identified and a road surface disease identification result corresponding to the road surface image to be identified. The server 102 can train the road surface disease identification model based on a large amount of training data, so that the road surface disease identification model can identify the input road surface image to be identified, such as identifying cracks, repairs, exposed bones, potholes, crack widths, joints, etc. in the road surface image to be identified. The source of the training data includes but is not limited to existing databases, data crawled from the Internet, or data uploaded by users when using the client. When the accuracy of the road surface disease identification model output reaches a certain requirement, the server 102 can provide the user with a road surface disease identification service based on the road surface disease identification model. At the same time, the server 102 can also continuously optimize the road surface disease identification model based on the newly added training data.

[0076] The pavement disease recognition model of the disclosed embodiment can be applied to scenes such as crack, repair, exposed bone, pothole, crack width, joint recognition, etc. The pavement disease recognition model can be trained based on the training data of different pavement diseases to obtain a pavement disease recognition model that can be applied to different pavement diseases at the same time.

[0077] Combine the following Figure 3 The training method of the pavement damage identification model and the pavement damage identification method according to the exemplary embodiment of the present disclosure are described by using the application scenario. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0078] refer to Figure 4 , which is a flow chart of a pavement damage identification method provided by an exemplary embodiment of the present disclosure.

[0079] In this exemplary embodiment, the road surface disease identification method includes the following steps:

[0080] Step S210: Acquire a road surface image to be identified, perform a first feature extraction on the road surface image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, and perform a second feature extraction on the first feature map based on a second number of cascaded skip connection convolution modules to obtain a second feature map.

[0081] refer to Figure 5 In a specific implementation, step S210 is implemented by an encoder, which includes a first number of cascaded convolution operations (as an input layer of the encoder) and a second number of cascaded skip-connected convolution modules.

[0082] In a specific implementation, the first number of convolution operations of the cascade refers to a first number of convolution operations arranged in sequence, where the output of one convolution operation serves as the input of the next convolution operation.

[0083] In a specific implementation, the second number of cascaded skip connection convolution modules refers to a second number of skip connection convolution modules arranged in sequence, wherein the output of one skip connection convolution module serves as the input of the next skip connection convolution module.

[0084] In specific implementation, a skip connection convolution module refers to a convolution module that connects the input to the output through a skip connection.

[0085] Next, we will refer to Figure 6 and Figure 7 , respectively introduce the sub-steps in step S210:

[0086] Step S310: Acquire a road surface image to be identified.

[0087] In specific implementation, there are many ways to obtain the road image to be identified, including using vehicle-mounted cameras, remote sensing satellite images, drone photography, etc.

[0088] Step S320: Perform a first feature extraction on the to-be-identified road surface image based on a first number of cascaded convolution operations to obtain a first feature map.

[0089] In a specific implementation, the first number of convolution operations in the cascade serves as the input layer of the encoder, and is used to map the road surface image to be identified into a high-dimensional feature space.

[0090] As a specific example, the first number is two.

[0091] As a specific example, in the first number of convolution operations in the cascade, the window size of each convolution operation is 3×3 and the sliding window is 2.

[0092] As a specific example, the size of the first feature map is 1 / 4 of the original image (ie, the road surface image to be identified).

[0093] Through the cascaded first number of convolution operations provided by the above exemplary embodiment, since each convolution operation can extract different features of the road surface image to be identified, the cascaded first number of convolution operations can help the network learn more complex feature representations. Since the cascaded first number of convolution operations increase the depth of the network, the learning ability of the model is improved, enabling it to capture more complex data patterns. Since the cascaded first number of convolution operations increase the depth of the network, it can help reduce overfitting because the deep network can control the complexity of the model by adding regularization terms (such as Dropout).

[0094] Step S330: Perform second feature extraction on the first feature map based on a second number of cascaded skip-connected convolutional modules to obtain a second feature map.

[0095] As a specific example, the second number is four.

[0096] In a specific implementation, each skip connection convolution module includes an eighth number of cascaded convolution operations.

[0097] As a specific example, the eighth number is three.

[0098] In specific implementation, each convolution operation includes a convolution layer, a group normalization layer, and a GELU activation layer.

[0099] In this exemplary embodiment, the second number of cascade-based skip connection convolution modules performs second feature extraction on the first feature map to obtain a second feature map, including:

[0100] Based on the cascaded second number of the skip connection convolution modules, different preset numbers of skip connection convolutions are performed on the first feature map, and the first feature map is downsampled by a preset multiple between every two skip connection convolution modules to obtain the second feature map.

[0101] As a specific example, take four skip-connected convolutional modules as an example:

[0102] Based on the first skip connection convolution module, the second skip connection convolution module, the third skip connection convolution module and the fourth skip connection convolution module, the skip connection convolution is repeated 3, 4, 24 and 3 times respectively. The feature map is downsampled by 2 times between every two skip connection convolution modules, so that the size of the feature map is reduced to 1 / 2 of the previous skip connection convolution module. Finally, the size of the feature map output by the encoder is 1 / 32 of the original image (i.e., the road image to be identified).

[0103] Through the above exemplary embodiment, each pixel point in the output second feature map represents the 32-pixel grid size of the original image. If the actual physical size of each pixel in the original image is 3.125 mm, the output grid covers a 10 cm × 10 cm road surface area, which meets the requirements of the standard specification and can be directly used to implement the grid-level disease recognition task.

[0104] The skip connection convolution module provided by the above exemplary embodiment allows information to flow more freely, and retains the details and semantic information in the original input data, making it easier for information to propagate to later layers and avoiding information loss. At the same time, it plays a key role in solving the gradient vanishing problem in deep neural network training, especially in very deep networks. This structure allows the network to be very deep because the shortcut connection provides a fast channel to propagate gradients and information, thereby reducing overfitting and making the network easier to train. At the same time, it improves the performance and stability of the network.

[0105] Step S220: Decouple the second feature map based on the third number of parallel first prediction heads to obtain first decoupled feature maps corresponding to different pavement damage types; perform grid-level pavement damage prediction based on the first decoupled feature maps to obtain grid-level pavement damage prediction results corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement damage types.

[0106] In this exemplary embodiment, the first prediction head is used for grid-level disease prediction.

[0107] In specific implementation, in the pavement defect identification solution provided by the present disclosure, there are grid-level defect prediction and pixel-level defect prediction, wherein the first prediction head here is used for grid-level defect prediction (converting the information obtained from the feature extraction layer into the final prediction result), and the subsequent second prediction head is used for pixel-level defect prediction.

[0108] In this exemplary embodiment, different from the aforementioned cascade mode, the third number of first prediction heads adopts a parallel mode.

[0109] In practice, parallelism processes multiple tasks or data streams simultaneously, which are merged at a certain point in the network to share information and improve performance. In parallel models, models for different tasks may share some underlying feature extractors while retaining some task-specific layers. This design can improve parameter efficiency because shared layers can learn common features while specific layers can capture task-specific information.

[0110] In specific implementation, based on the third number of first prediction heads in parallel mode, detection tasks such as cracks, repairs, exposed bones and potholes can be performed simultaneously, and different tasks can benefit from each other and improve the overall performance.

[0111] In the present exemplary embodiment, the third number is four.

[0112] It should be noted that the present disclosure uses four first prediction heads as an example for explanation. In actual implementation, the number of first prediction heads can be configured according to actual needs.

[0113] Next, step S220 will be introduced in conjunction with an exemplary structure of the first prediction head, specifically:

[0114] In this exemplary embodiment, the first prediction head includes a cascaded disease feature decoupling module, a convolution layer and a Sigmoid activation function, and the disease feature decoupling module includes a cascaded convolution operation, a GroupNorm group normalization layer and a SiLU activation function;

[0115] The third number of first prediction heads based on parallelism respectively decouple the second feature map to obtain first decoupled feature maps corresponding to different pavement damage types, and perform grid-level pavement damage prediction based on the first decoupled feature maps to obtain grid-level pavement damage prediction results corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement damage types, including:

[0116] Based on the cascaded convolution operation, the GroupNorm group normalization layer and the SiLU activation function, the second feature map is decoupled to obtain the first decoupled feature map corresponding to different pavement disease types;

[0117] Predicting the grid positions in the first decoupled feature map based on the convolution layer to obtain grid-level pavement disease prediction information corresponding to the grid positions;

[0118] The grid-level pavement damage prediction information is normalized to a probability value within a preset numerical range based on a Sigmoid activation function as the grid-level pavement damage prediction result.

[0119] In a specific implementation, the second feature map is decoupled based on the third number of first prediction heads in parallel, so as to obtain first decoupled feature maps corresponding to different road surface disease types, including:

[0120] By decoupling the second feature map, feature representations that can represent independent defects (such as cracks, repairs, exposed bones, and potholes, etc.) are learned from the second feature map to predict different defects separately.

[0121] refer to Figure 5 As a specific example, the output of the encoder (i.e., the second feature map) is input into multiple first prediction heads respectively. The multiple first prediction heads are parallel multi-branch structures, and each first prediction head corresponds to a type of road surface disease, such as cracks, repairs, exposed surfaces, and potholes.

[0122] refer to Figure 8 ,As a specific example, the first prediction head includes a disease feature decoupling module, a convolutional layer (with a window size of 1×1 and a sliding step of 1), and a Sigmoid activation function.

[0123] The disease feature decoupling module includes cascaded convolution operations (window size is 3×3, sliding step size is 1), GroupNorm group normalization layer and SiLU activation layer.

[0124] Among them, the disease feature decoupling module is used to realize the decoupling of disease feature decoupling, and then a convolutional layer (window size is 1×1, sliding step size is 1) is used to predict each grid position to determine whether the grid contains a specific disease, and finally a Sigmoid activation function is used to normalize the output to a probability value between 0 and 1.

[0125] Step S230: Map different first decoupled feature maps to a fourth number of groups of attention heads respectively, perform feature fusion on the first decoupled feature maps based on the attention heads to obtain fused features, and concatenate the fused features with the second feature map to obtain a third feature map.

[0126] In this exemplary embodiment, the step of performing feature fusion on the first decoupled feature map based on the attention head to obtain a fused feature includes:

[0127] Determine the target decoupled feature map and the decoupled feature map to be fused;

[0128] Determine the information requirement of the target decoupling feature graph, the information content of the target decoupling feature graph, the information index of the decoupling feature graph to be fused, and the information content of the decoupling feature graph to be fused;

[0129] Determine the correlation between the information requirement of the target decoupled feature graph and the information index of the decoupled feature graph to be fused;

[0130] Based on the correlation and the information content of the decoupled feature graph to be fused, obtaining a weighted summation result of the information of the decoupled feature graph to be fused;

[0131] The fusion feature is obtained based on the weighted summation result of the information content of the target decoupled feature map and the information of the decoupled feature map to be fused.

[0132] refer to Figure 7 and Fig. 9 , as a specific example:

[0133] Based on the last layer of features output by the encoder, the features of each disease are mapped to 4 groups of attention heads, and each attention head is responsible for fusing the corresponding disease. Taking attention head ij as an example, the features of the jth disease are fused to the features of the ith disease. For example, attention head 1-1 fuses the local and global features of the crack itself through the self-attention mechanism; attention head 1-2 fuses the local and global features of the repair into the crack features through the cross-semantic attention mechanism.

[0134] Among them, each attention head is mapped into three sets of output Key (K), Query (Q) and Value (V) by three fully connected layers.

[0135] Query (Q): represents the requirements or problems of the current element or the current processing step. When processing a sequence, Query usually represents the information that the current step needs to obtain from other elements.

[0136] Key (K): An index that represents the information that each element in the sequence can provide. The Key is used to match the Query to determine which elements are most relevant to the current step.

[0137] Value (V): represents the actual information content of each element in the sequence. Once the Key matches the Query, the corresponding Value will be selected and used to construct the output.

[0138] Taking the attention head ij as an example, first, calculate and The relevance of , and secondly, according to the above correlation Perform weighted summation , and finally get the output features of the first attention head ij Through this operation, the features of different diseases are fused into each disease feature, realizing feature fusion across disease semantics. It is concatenated with the output of the encoder and dimensionally adjusted through a 1x1 convolution before being used as the input feature of the decoder.

[0139] Step S240, performing a third feature extraction on the third feature map based on a fifth number of convolution kernels with different expansion rates to obtain a fifth number of groups of features at a fifth number of scales, splicing the fifth number of groups of features to obtain spliced ​​features, and performing a fourth feature extraction on the spliced ​​features based on a sixth number of cascaded convolution modules to obtain a fourth feature map.

[0140] In this exemplary embodiment, step S240 is implemented by a decoder, and a third feature is extracted from the third feature map based on a spatial pyramid pooling structure (ASPP, Atrous Spatial Pyramid Pooling) in the decoder, where the spatial pyramid pooling structure includes a fifth number of convolution kernels with different dilation rates.

[0141] In the present exemplary embodiment, the fifth number is four.

[0142] In the specific implementation, four different expansion rates are taken as examples for description, which are 1, 3, 5, and 7 respectively.

[0143] Through the above exemplary embodiments, the ASPP structure can effectively improve the model's ability to identify defects of different scales, for example, when both large cracks and small cracks exist in the road surface image to be identified, and help improve accuracy and robustness.

[0144] In this exemplary embodiment, the sixth number of cascade-based convolution modules performs fourth feature extraction on the splicing feature to obtain a fourth feature map, including:

[0145] The features extracted during the fourth feature extraction are concatenated with the features extracted during the first feature extraction and the features of the corresponding scales extracted during the second feature extraction step by step.

[0146] In this exemplary embodiment, after the fifth number of groups of features are concatenated to obtain concatenated features:

[0147] The concatenated features are adjusted in feature dimension through the convolution layer;

[0148] The concatenated features after adjusting the feature dimension are upsampled by 2 times;

[0149] Step by step, the features extracted during the fourth feature extraction are concatenated with the features extracted during the first feature extraction and the features of the corresponding scales among the features extracted during the second feature extraction;

[0150] The concatenated features are adjusted in feature dimension through the convolution layer;

[0151] The concatenated features after adjusting the feature dimension are upsampled by 2 times;

[0152] A feature map of the same size as the input image (i.e., the road image to be identified) is obtained for pixel-level target recognition.

[0153] In specific implementation, the convolution layer adjusts the feature dimension: At the beginning of the decoder, a convolution layer (or multiple convolution layers) is configured to reduce or increase the number of channels of the feature map to match the feature map of the corresponding scale of the encoder. This process can be a 1x1 convolution or a convolution kernel of other sizes, the purpose of which is to adjust the number of channels.

[0154] 2x upsampling operation: After the convolutional layer adjusts the dimension, the feature map will increase its spatial resolution through upsampling. After 2x upsampling, the width and height of the feature map will double, so that the resolution of the original image can be gradually restored. Upsampling can use different methods, such as nearest neighbor interpolation, bilinear interpolation, transposed convolution (also known as deconvolution), etc.

[0155] Concatenate features of the corresponding scale with the encoder level by level: After upsampling, concatenate the decoder's feature map with the encoder's feature map of the corresponding scale. This concatenation is performed on the channel dimension of the feature map, also known as feature fusion. This fusion can be a simple concatenation or a more complex fusion method, such as using convolutional layers to integrate features of different scales.

[0156] Through the above exemplary embodiments, it is possible to utilize feature information of different levels in the encoder, which contains semantic content of different scales. Low-level feature maps usually contain more positional information, while high-level feature maps contain more semantic information. Through this feature fusion, the decoder can generate a high-resolution output that retains detail information and has rich semantic information.

[0157] Step S250: Decouple the fourth feature map based on the seventh number of second prediction heads in parallel to obtain second decoupled feature maps corresponding to different pavement damage types; perform pixel-level pavement damage prediction based on the second decoupled feature maps to obtain pixel-level pavement damage prediction results corresponding to each of the second prediction heads, wherein different second prediction heads correspond to different pavement damage types.

[0158] In this exemplary embodiment, the second prediction head includes a cascaded disease feature decoupling module, a convolution layer and a Sigmoid activation function, and the disease feature decoupling module includes a cascaded convolution operation, a GroupNorm group normalization layer and a SiLU activation function;

[0159] The seventh number of second prediction heads based on parallelism respectively decouple the fourth feature map to obtain second decoupled feature maps corresponding to different pavement damage types, and perform pixel-level pavement damage prediction based on the second decoupled feature maps to obtain a pixel-level pavement damage prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement damage types, including:

[0160] Based on the cascaded convolution operation, the GroupNorm group normalization layer and the SiLU activation function, the fourth feature map is decoupled to obtain the second decoupled feature map corresponding to different pavement disease types;

[0161] Predicting the pixel position in the second decoupled feature map based on the convolution layer to obtain pixel-level pavement disease prediction information corresponding to the pixel position;

[0162] The pixel-level pavement damage prediction information is normalized to a probability value within a preset numerical range based on a Sigmoid activation function as the pixel-level pavement damage prediction result.

[0163] refer to Figure 5 As a specific example, the second feature map output by the encoder is input into multiple second prediction heads respectively. The multiple second prediction heads are parallel multi-branch structures, and each second prediction head corresponds to a road surface disease type.

[0164] In a specific implementation, the second prediction head includes a pixel-level defect prediction head.

[0165] refer to Fig.10 ,As a specific example, the second prediction head includes a disease feature decoupling module, a convolutional layer (with a window size of 3×3 and a sliding step of 1), and a Sigmoid activation function.

[0166] The disease feature decoupling module includes cascaded convolution operations (window size is 3×3, sliding step size is 1), GroupNorm group normalization layer and SiLU activation layer.

[0167] Among them, the disease feature decoupling module is used to realize the decoupling of disease feature decoupling, and then a convolution layer (window size is 3×3, sliding step size is 1) is used to predict each pixel position to determine whether the pixel contains a specific disease, and finally a Sigmoid activation function is used to normalize the output to a probability value between 0 and 1.

[0168] Step S260: Match the grid-level pavement damage prediction result and the pixel-level pavement damage prediction result to obtain a pavement damage recognition result corresponding to the pavement image to be recognized.

[0169] In specific implementation, based on preset rules, the grid-level pavement damage prediction results and the pixel-level pavement damage prediction results are matched to achieve secondary identification.

[0170] In this exemplary embodiment, the matching of the grid-level pavement damage prediction result and the pixel-level pavement damage prediction result to obtain the pavement damage recognition result corresponding to the pavement image to be recognized includes:

[0171] Determining straight line joints in the pixel-level pavement damage prediction results and cracks in the grid-level pavement damage prediction results;

[0172] In response to determining that the crack intersects with the two perpendicular straight seams and the distance between the intersection point of the crack and the straight seam and the intersection point of the two perpendicular straight seams is less than half the length of the straight seam, determining that a first closed area formed by the crack and the two perpendicular straight seams is a plate corner fracture area;

[0173] and / or,

[0174] In response to determining that at least two of the cracks exist in the second closed area formed by the four straight seams and the cracks divide the second closed area into at least three non-overlapping sub-areas, the second closed area is determined to be a crushing plate area.

[0175] In the above exemplary embodiment, a pavement disease recognition method implemented by a pavement disease recognition model is introduced. Next, a training method of the pavement disease recognition model will be introduced:

[0176] The structure of the pavement disease identification model is as follows: Figure 7 As shown, in the above exemplary embodiments, a detailed introduction has been made, so it will not be repeated here.

[0177] The training methods of the pavement disease recognition model include:

[0178] Constructing a sample set including a number of samples; wherein the samples include: sample data and label data; the sample data includes a road surface image to be identified; the label data includes a road surface disease identification result corresponding to the road surface image to be identified;

[0179] According to the sample set, a pavement disease recognition model is constructed and trained using the pavement disease recognition method provided by the present disclosure.

[0180] In specific implementation, the loss function used in training the pavement disease recognition model is:

[0181] The combined loss of cross entropy loss (CELoss) and Dice score loss (DiceLoss) is used to achieve:

[0182] ;

[0183] ;

[0184] in, and Respectively represent the predicted results and the actual labeled results The value at It is an adjustment number, usually 0.0001, to prevent the denominator from being 0.

[0185] The source of sample data includes but is not limited to existing databases, data crawled from the Internet, or data uploaded by users when using the client. When the accuracy of the pavement defect recognition model output reaches a certain requirement, the pavement defect recognition service can be provided to users based on the pavement defect recognition model. At the same time, the pavement defect recognition model can be continuously optimized based on the newly added training data.

[0186] It can be seen from the above that the pavement disease recognition method provided by the embodiment of the present disclosure includes: obtaining a pavement image to be identified, performing a first feature extraction on the pavement image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, performing a second feature extraction on the first feature map based on a second number of cascaded jump connection convolution modules to obtain a second feature map; decoupling the second feature map based on a third number of parallel first prediction heads to obtain first decoupled feature maps corresponding to different pavement disease types, performing grid-level pavement disease prediction based on the first decoupled feature maps to obtain a grid-level pavement disease prediction result corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement disease types; mapping different first decoupled feature maps to a fourth number of groups of attention heads respectively, performing feature fusion on the first decoupled feature maps based on the attention heads to obtain fused features, and combining the fused features The combined features are spliced ​​with the second feature map to obtain a third feature map; based on a fifth number of convolution kernels with different expansion rates, a third feature is extracted from the third feature map to obtain a fifth number of groups of features at a fifth number of scales, the fifth number of groups of features are spliced ​​to obtain a spliced ​​feature, and a fourth feature is extracted from the spliced ​​feature based on a sixth number of cascaded convolution modules to obtain a fourth feature map; based on a seventh number of parallel second prediction heads, the fourth feature map is decoupled respectively to obtain second decoupled feature maps corresponding to different pavement disease types, pixel-level pavement disease prediction is performed based on the second decoupled feature map to obtain a pixel-level pavement disease prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement disease types; the grid-level pavement disease prediction result and the pixel-level pavement disease prediction result are matched to obtain a pavement disease recognition result corresponding to the pavement image to be recognized.

[0187] The present invention unifies the automatic recognition of various road surface damage types for various recognition standards under the same deep neural network framework, expands the scope of application of the model, improves the efficiency of the model in identifying damage in large-scale images, and reduces production costs.

[0188] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0189] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0190] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a road surface disease identification device.

[0191] refer to Fig.11 , which is a structural schematic diagram of a road surface disease identification device provided by an exemplary embodiment of the present disclosure.

[0192] The pavement disease identification device provided by this exemplary embodiment includes the following modules:

[0193] The encoding module 910 is configured to obtain a road surface image to be identified, perform a first feature extraction on the road surface image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, and perform a second feature extraction on the first feature map based on a second number of cascaded skip connection convolution modules to obtain a second feature map;

[0194] The grid-level pavement damage prediction module 920 is configured to decouple the second feature map based on a third number of first prediction heads in parallel, respectively, to obtain first decoupled feature maps corresponding to different pavement damage types, and to perform grid-level pavement damage prediction based on the first decoupled feature maps to obtain a grid-level pavement damage prediction result corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement damage types;

[0195] A feature fusion module 930 is configured to map different first decoupled feature maps to a fourth number of groups of attention heads, perform feature fusion on the first decoupled feature maps based on the attention heads to obtain fused features, and concatenate the fused features with the second feature map to obtain a third feature map;

[0196] The decoding module 940 is configured to perform a third feature extraction on the third feature map based on a fifth number of convolution kernels with different dilation rates to obtain a fifth number of groups of features at a fifth number of scales, splice the fifth number of groups of features to obtain spliced ​​features, and perform a fourth feature extraction on the spliced ​​features based on a sixth number of cascaded convolution modules to obtain a fourth feature map;

[0197] The pixel-level pavement damage prediction module 950 is configured to decouple the fourth feature map based on a seventh number of second prediction heads in parallel, respectively, to obtain second decoupled feature maps corresponding to different pavement damage types, and to perform pixel-level pavement damage prediction based on the second decoupled feature maps to obtain a pixel-level pavement damage prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement damage types;

[0198] The regularized matching module 960 is configured to match the grid-level pavement damage prediction result and the pixel-level pavement damage prediction result to obtain a pavement damage recognition result corresponding to the pavement image to be recognized.

[0199] In some exemplary embodiments, the encoding module 910 is specifically configured to:

[0200] Based on the cascaded second number of the skip connection convolution modules, different preset numbers of skip connection convolutions are performed on the first feature map, and the first feature map is downsampled by a preset multiple between every two skip connection convolution modules to obtain the second feature map.

[0201] In some exemplary embodiments, the first prediction head includes a cascaded disease feature decoupling module, a convolution layer, and a Sigmoid activation function, and the disease feature decoupling module includes a cascaded convolution operation, a GroupNorm group normalization layer, and a SiLU activation function;

[0202] The grid-level pavement damage prediction module 920 is specifically configured as follows:

[0203] Based on the cascaded convolution operation, the GroupNorm group normalization layer and the SiLU activation function, the second feature map is decoupled to obtain the first decoupled feature map corresponding to different pavement disease types;

[0204] Predicting the grid positions in the first decoupled feature map based on the convolution layer to obtain grid-level pavement disease prediction information corresponding to the grid positions;

[0205] The grid-level pavement damage prediction information is normalized to a probability value within a preset numerical range based on a Sigmoid activation function as the grid-level pavement damage prediction result.

[0206] In some exemplary embodiments, the feature fusion module 930 is specifically configured to:

[0207] Determine the target decoupled feature map and the decoupled feature map to be fused;

[0208] Determine the information requirement of the target decoupling feature graph, the information content of the target decoupling feature graph, the information index of the decoupling feature graph to be fused, and the information content of the decoupling feature graph to be fused;

[0209] Determine the correlation between the information requirement of the target decoupled feature graph and the information index of the decoupled feature graph to be fused;

[0210] Based on the correlation and the information content of the decoupled feature graph to be fused, obtaining a weighted summation result of the information of the decoupled feature graph to be fused;

[0211] The fusion feature is obtained based on the weighted summation result of the information content of the target decoupled feature map and the information of the decoupled feature map to be fused.

[0212] In some exemplary embodiments, the decoding module 940 is specifically configured to:

[0213] The features extracted during the fourth feature extraction are concatenated with the features extracted during the first feature extraction and the features of the corresponding scales extracted during the second feature extraction step by step.

[0214] In some exemplary embodiments, the regularized matching module 960 is specifically configured to:

[0215] Determining straight line joints in the pixel-level pavement damage prediction results and cracks in the grid-level pavement damage prediction results;

[0216] In response to determining that the crack intersects with the two perpendicular straight seams and the distance between the intersection point of the crack and the straight seam and the intersection point of the two perpendicular straight seams is less than half the length of the straight seam, determining that a first closed area formed by the crack and the two perpendicular straight seams is a plate corner fracture area;

[0217] and / or,

[0218] In response to determining that at least two of the cracks exist in the second closed area formed by the four straight seams and the cracks divide the second closed area into at least three non-overlapping sub-areas, the second closed area is determined to be a crushing plate area.

[0219] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0220] The device of the above embodiment is used to implement the corresponding pavement disease identification method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0221] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the pavement disease identification method described in any of the above embodiments is implemented.

[0222] Fig.12 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0223] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0224] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0225] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0226] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0227] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0228] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0229] The electronic device of the above embodiment is used to implement the corresponding road surface disease identification method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0230] The memory 1020 stores machine-readable instructions executable by the processor 1010. When the electronic device is running, the processor 1010 communicates with the memory 1020 via the bus 1030, so that the processor 1010 executes the following instructions when running:

[0231] Acquire a road image to be identified, perform a first feature extraction on the road image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, and perform a second feature extraction on the first feature map based on a second number of cascaded skip connection convolution modules to obtain a second feature map;

[0232] Based on the third number of first prediction heads in parallel, the second feature maps are respectively decoupled to obtain first decoupled feature maps corresponding to different pavement damage types; grid-level pavement damage prediction is performed based on the first decoupled feature maps to obtain a grid-level pavement damage prediction result corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement damage types;

[0233] Mapping different first decoupled feature maps to a fourth number of groups of attention heads respectively, performing feature fusion on the first decoupled feature maps based on the attention heads to obtain fused features, and concatenating the fused features with the second feature map to obtain a third feature map;

[0234] Performing a third feature extraction on the third feature map based on a fifth number of convolution kernels with different dilation rates to obtain a fifth number of groups of features at a fifth number of scales, splicing the fifth number of groups of features to obtain spliced ​​features, and performing a fourth feature extraction on the spliced ​​features based on a sixth number of cascaded convolution modules to obtain a fourth feature map;

[0235] Decoupling the fourth feature map based on a seventh number of second prediction heads in parallel, respectively, to obtain second decoupled feature maps corresponding to different pavement damage types, performing pixel-level pavement damage prediction based on the second decoupled feature maps, to obtain a pixel-level pavement damage prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement damage types;

[0236] The grid-level pavement damage prediction result and the pixel-level pavement damage prediction result are matched to obtain a pavement damage recognition result corresponding to the pavement image to be recognized.

[0237] In a possible implementation, in the instruction executed by the processor 1010, the second feature extraction is performed on the first feature map based on the second number of cascaded skip connection convolution modules to obtain the second feature map, including:

[0238] Based on the cascaded second number of the skip connection convolution modules, different preset numbers of skip connection convolutions are performed on the first feature map, and the first feature map is downsampled by a preset multiple between every two skip connection convolution modules to obtain the second feature map.

[0239] In a possible implementation, in the instructions executed by the processor 1010, the first prediction head includes a cascaded disease feature decoupling module, a convolution layer, and a Sigmoid activation function, and the disease feature decoupling module includes a cascaded convolution operation, a GroupNorm group normalization layer, and a SiLU activation function;

[0240] The third number of first prediction heads based on parallelism respectively decouple the second feature map to obtain first decoupled feature maps corresponding to different pavement damage types, and perform grid-level pavement damage prediction based on the first decoupled feature maps to obtain grid-level pavement damage prediction results corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement damage types, including:

[0241] Based on the cascaded convolution operation, the GroupNorm group normalization layer and the SiLU activation function, the second feature map is decoupled to obtain the first decoupled feature map corresponding to different pavement disease types;

[0242] Predicting the grid positions in the first decoupled feature map based on the convolution layer to obtain grid-level pavement disease prediction information corresponding to the grid positions;

[0243] The grid-level pavement damage prediction information is normalized to a probability value within a preset numerical range based on a Sigmoid activation function as the grid-level pavement damage prediction result.

[0244] In a possible implementation, in the instructions executed by the processor 1010, the step of performing feature fusion on the first decoupled feature map based on the attention head to obtain a fused feature includes:

[0245] Determine the target decoupled feature map and the decoupled feature map to be fused;

[0246] Determine the information requirement of the target decoupling feature graph, the information content of the target decoupling feature graph, the information index of the decoupling feature graph to be fused, and the information content of the decoupling feature graph to be fused;

[0247] Determine the correlation between the information requirement of the target decoupled feature graph and the information index of the decoupled feature graph to be fused;

[0248] Based on the correlation and the information content of the decoupled feature graph to be fused, obtaining a weighted summation result of the information of the decoupled feature graph to be fused;

[0249] The fusion feature is obtained based on the weighted summation result of the information content of the target decoupled feature map and the information of the decoupled feature map to be fused.

[0250] In a possible implementation, in the instruction executed by the processor 1010, the sixth number of convolution modules based on the cascade performs a fourth feature extraction on the splicing feature to obtain a fourth feature map, including:

[0251] The features extracted during the fourth feature extraction are concatenated with the features extracted during the first feature extraction and the features of the corresponding scales extracted during the second feature extraction step by step.

[0252] In a possible implementation manner, in the instructions executed by the processor 1010, the matching of the grid-level pavement damage prediction result and the pixel-level pavement damage prediction result to obtain the pavement damage recognition result corresponding to the pavement image to be recognized includes:

[0253] Determining straight line joints in the pixel-level pavement damage prediction results and cracks in the grid-level pavement damage prediction results;

[0254] In response to determining that the crack intersects with the two perpendicular straight seams and the distance between the intersection point of the crack and the straight seam and the intersection point of the two perpendicular straight seams is less than half the length of the straight seam, determining that a first closed area formed by the crack and the two perpendicular straight seams is a plate corner fracture area;

[0255] and / or,

[0256] In response to determining that at least two of the cracks exist in the second closed area formed by the four straight seams and the cracks divide the second closed area into at least three non-overlapping sub-areas, the second closed area is determined to be a crushing plate area.

[0257] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the pavement disease identification method described in any of the above embodiments.

[0258] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0259] The above-mentioned non-transitory computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0260] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the pavement disease identification method described in any embodiment in the above exemplary method part, and have the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0261] Based on the same inventive concept, corresponding to the road surface disease identification method described in any of the above embodiments, the present disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the road surface disease identification method. Corresponding to the execution subject corresponding to each step in each embodiment of the road surface disease identification method, the processor that executes the corresponding step can belong to the corresponding execution subject.

[0262] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the pavement defect identification method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0263] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." In addition, in some embodiments, the present disclosure may also be implemented in the form of a computer program product in one or more computer-readable media, which contains computer-readable program code.

[0264] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive examples) of computer-readable storage media may include, for example: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device, or device.

[0265] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0266] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0267] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0268] It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, and these computer program instructions are executed by a computer or other programmable data processing device to produce a device that implements the functions / operations specified in the boxes in the flowchart and / or block diagram.

[0269] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable medium produce a product that includes an instruction device that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0270] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby enabling the instructions executed on the computer or other programmable device to provide a process for implementing the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0271] In addition, although the operations of the disclosed method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0272] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0273] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0274] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0275] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device may be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present application are to be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0276] Although the present application has been described in conjunction with specific embodiments of the present application, many alternatives, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the discussed embodiments.

[0277] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

[0278] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims. The scope of the attached claims conforms to the broadest interpretation, thereby including all such modifications and equivalent structures and functions.

Claims

1. A method for identifying road surface defects, characterized in that: include: Acquire a road image to be identified, perform a first feature extraction on the road image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, and perform a second feature extraction on the first feature map based on a second number of cascaded skip connection convolution modules to obtain a second feature map; Based on the third number of first prediction heads in parallel, the second feature maps are respectively decoupled to obtain first decoupled feature maps corresponding to different pavement damage types; grid-level pavement damage prediction is performed based on the first decoupled feature maps to obtain a grid-level pavement damage prediction result corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement damage types; Mapping different first decoupled feature maps to a fourth number of groups of attention heads respectively, performing feature fusion on the first decoupled feature maps based on the attention heads to obtain fused features, and concatenating the fused features with the second feature map to obtain a third feature map; Performing a third feature extraction on the third feature map based on a fifth number of convolution kernels with different dilation rates to obtain a fifth number of groups of features at a fifth number of scales, splicing the fifth number of groups of features to obtain spliced ​​features, and performing a fourth feature extraction on the spliced ​​features based on a sixth number of cascaded convolution modules to obtain a fourth feature map; Decoupling the fourth feature map based on a seventh number of second prediction heads in parallel, respectively, to obtain second decoupled feature maps corresponding to different pavement damage types, performing pixel-level pavement damage prediction based on the second decoupled feature maps, to obtain a pixel-level pavement damage prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement damage types; The grid-level pavement damage prediction result and the pixel-level pavement damage prediction result are matched to obtain a pavement damage recognition result corresponding to the pavement image to be recognized.

2. The method according to claim 1, characterized in that: The second number of cascade-based skip connection convolution modules performs second feature extraction on the first feature map to obtain a second feature map, including: Based on the cascaded second number of the skip connection convolution modules, different preset numbers of skip connection convolutions are performed on the first feature map, and the first feature map is downsampled by a preset multiple between every two skip connection convolution modules to obtain the second feature map.

3. The method according to claim 1, characterized in that The first prediction head includes a cascaded disease feature decoupling module, a convolution layer and a Sigmoid activation function, and the disease feature decoupling module includes a cascaded convolution operation, a GroupNorm group normalization layer and a SiLU activation function; The third number of first prediction heads based on parallelism respectively decouple the second feature map to obtain first decoupled feature maps corresponding to different pavement damage types, and perform grid-level pavement damage prediction based on the first decoupled feature maps to obtain grid-level pavement damage prediction results corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement damage types, including: Based on the cascaded convolution operation, the GroupNorm group normalization layer and the SiLU activation function, the second feature map is decoupled to obtain the first decoupled feature map corresponding to different pavement disease types; Predicting the grid positions in the first decoupled feature map based on the convolution layer to obtain grid-level pavement disease prediction information corresponding to the grid positions; The grid-level pavement damage prediction information is normalized to a probability value within a preset numerical range based on a Sigmoid activation function as the grid-level pavement damage prediction result.

4. The method according to claim 1, characterized in that: The performing feature fusion on the first decoupled feature map based on the attention head to obtain a fused feature includes: Determine the target decoupled feature map and the decoupled feature map to be fused; Determine the information requirement of the target decoupling feature graph, the information content of the target decoupling feature graph, the information index of the decoupling feature graph to be fused, and the information content of the decoupling feature graph to be fused; Determine the correlation between the information requirement of the target decoupled feature graph and the information index of the decoupled feature graph to be fused; Based on the correlation and the information content of the decoupled feature graph to be fused, obtaining a weighted summation result of the information of the decoupled feature graph to be fused; The fusion feature is obtained based on the weighted summation result of the information content of the target decoupled feature map and the information of the decoupled feature map to be fused.

5. The method according to claim 1, characterized in that The sixth number of cascade-based convolution modules performs fourth feature extraction on the spliced ​​features to obtain a fourth feature map, including: The features extracted during the fourth feature extraction are concatenated with the features extracted during the first feature extraction and the features of the corresponding scales extracted during the second feature extraction step by step.

6. The method according to claim 1, characterized in that The matching of the grid-level pavement damage prediction result and the pixel-level pavement damage prediction result to obtain a pavement damage recognition result corresponding to the pavement image to be recognized includes: Determining straight line joints in the pixel-level pavement damage prediction results and cracks in the grid-level pavement damage prediction results; In response to determining that the crack intersects with the two perpendicular straight seams and the distance between the intersection point of the crack and the straight seam and the intersection point of the two perpendicular straight seams is less than half the length of the straight seam, determining that a first closed area formed by the crack and the two perpendicular straight seams is a plate corner fracture area; and / or, In response to determining that at least two of the cracks exist in the second closed area formed by the four straight seams and the cracks divide the second closed area into at least three non-overlapping sub-areas, the second closed area is determined to be a crushing plate area.

7. A road surface disease identification device, characterized in that: include: The encoding module is configured to obtain a road surface image to be identified, perform a first feature extraction on the road surface image to be identified based on a first number of cascaded convolution operations to obtain a first feature map, and perform a second feature extraction on the first feature map based on a second number of cascaded skip connection convolution modules to obtain a second feature map; A grid-level pavement damage prediction module is configured to decouple the second feature map based on a third number of first prediction heads in parallel, respectively, to obtain first decoupled feature maps corresponding to different pavement damage types, and to perform grid-level pavement damage prediction based on the first decoupled feature maps to obtain a grid-level pavement damage prediction result corresponding to each of the first prediction heads, wherein different first prediction heads correspond to different pavement damage types; a feature fusion module, configured to map different first decoupled feature maps to a fourth number of groups of attention heads, perform feature fusion on the first decoupled feature maps based on the attention heads to obtain fused features, and concatenate the fused features with the second feature map to obtain a third feature map; a decoding module configured to perform a third feature extraction on the third feature map based on a fifth number of convolution kernels with different dilation rates to obtain a fifth number of groups of features at a fifth number of scales, splice the fifth number of groups of features to obtain spliced ​​features, and perform a fourth feature extraction on the spliced ​​features based on a sixth number of cascaded convolution modules to obtain a fourth feature map; a pixel-level pavement damage prediction module, configured to decouple the fourth feature map based on a seventh number of second prediction heads in parallel, respectively, to obtain second decoupled feature maps corresponding to different pavement damage types, and to perform pixel-level pavement damage prediction based on the second decoupled feature maps, to obtain a pixel-level pavement damage prediction result corresponding to each second prediction head, wherein different second prediction heads correspond to different pavement damage types; The regularized matching module is configured to match the grid-level pavement damage prediction result and the pixel-level pavement damage prediction result to obtain a pavement damage recognition result corresponding to the pavement image to be recognized.

8. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer program instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 6.

Images