Pavement disease identification method and device, equipment, storage medium and program product

Abstract

The invention provides a pavement disease recognition method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining an original image of a to-be-recognized pavement, carrying out the dichotomy segmentation of the original image, and obtaining a binary image and visual feature information of the to-be-recognized pavement; performing geometric feature extraction processing on the binary image to obtain geometric feature information of the pavement to be identified; performing fusion processing on the visual feature information and the geometric feature information based on an attention mechanism to obtain semantic fusion feature information of the to-be-recognized road surface; performing clustering segmentation processing based on the binary image and the semantic fusion feature information to obtain false label information of the to-be-identified road surface; and performing disease category identification processing based on the semantic fusion feature information and the pseudo label information to obtain disease category information of the to-be-identified pavement. According to the invention, the recognition precision of pavement diseases is improved, the cost of data annotation is reduced, and the complexity of post-processing is avoided.

Description

Technical Field

[0001] This disclosure relates to the fields of computer vision and image processing technology, and in particular to a method, apparatus, device, storage medium and program product for identifying road surface defects. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this disclosure as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] Automatic identification of road surface defects (such as cracks and repairs) is crucial for road maintenance and safety management.

[0004] However, the road surface defect identification solutions provided by related technologies have low identification accuracy and rely on pixel-level data annotation, which is costly and depends on complex post-processing. Summary of the Invention

[0005] In view of this, the purpose of this disclosure is to provide a method, apparatus, equipment, storage medium and program product for identifying road surface defects, which at least to some extent solves one of the technical problems in the related art.

[0006] To achieve the above objectives, the first aspect of this exemplary embodiment provides a method for identifying pavement defects, including: The original image of the road surface to be identified is obtained, and the original image is subjected to binary classification segmentation to obtain the binary image and visual feature information of the road surface to be identified. Geometric feature extraction is performed on the binary image to obtain the geometric feature information of the road surface to be identified; The visual feature information and the geometric feature information are fused based on an attention mechanism to obtain the semantic fusion feature information of the road surface to be identified; Clustering and segmentation processing is performed based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified; Based on the semantic fusion feature information and the pseudo-label information, the disease category identification process is performed to obtain the disease category information of the road surface to be identified.

[0007] In some exemplary embodiments, the step of performing binary classification segmentation on the original image to obtain a binary image and visual feature information of the road surface to be identified includes: The original image is input into a pre-trained binary classification segmentation model; The original image is segmented by the binary classification model to obtain a binary image, the resolution of which is the same as that of the original image. The visual feature information is obtained from the intermediate layer of the decoder of the binary classification segmentation model.

[0008] In some exemplary embodiments, the step of performing geometric feature extraction processing on the binary image to obtain the geometric feature information of the road surface to be identified includes: The binary image is subjected to connected component analysis to obtain connected components; After skeletonization of the connected components, topological feature extraction is performed to obtain the topological features. The connected components are subjected to region attribute extraction processing to obtain region attributes; The geometric feature information is obtained by comprehensively processing the topological features and the regional attributes.

[0009] In some exemplary embodiments, the step of performing attention-based fusion processing on the visual feature information and the geometric feature information to obtain semantic fusion feature information of the road surface to be identified includes: The visual feature information and the geometric feature information are processed by data splicing to obtain spliced ​​features; The splicing features are processed by global channel context information extraction and local channel context information extraction to obtain global channel context information and local channel context information. The global channel context information and the local channel context information are added together and then normalized to obtain the attention weight information; The attention weight information and the visual feature information are multiplied element-wise to obtain the semantic fusion feature information.

[0010] In some exemplary embodiments, the step of performing clustering and segmentation processing based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified includes: Based on the binary image, determine the set of pixels belonging to the same disease category; From the semantic fusion feature information, feature vectors corresponding to the pixel set are extracted, and semantic feature vectors corresponding to the disease category are generated through an aggregation function; The semantic feature vectors are grouped using a clustering algorithm, with each group corresponding to a disease subcategory, to generate the pseudo-label information.

[0011] In some exemplary embodiments, the step of performing disease category identification processing based on the semantic fusion feature information and the pseudo-label information to obtain disease category information of the pavement to be identified includes: Determine the disease category information provided by the pseudo-label information; Based on the disease category information, the semantic fusion feature information is processed for disease category identification to obtain the disease category information of the road surface to be identified.

[0012] Based on the same inventive concept, a second aspect of the exemplary embodiments of this disclosure provides a road surface defect identification device, comprising: The binary classification segmentation module is configured to acquire the original image of the road surface to be identified, perform binary classification segmentation on the original image, and obtain the binary image and visual feature information of the road surface to be identified. The geometric feature extraction module is configured to perform geometric feature extraction processing on the binary image to obtain the geometric feature information of the road surface to be identified; The attention mechanism feature fusion module is configured to perform attention mechanism-based fusion processing on the visual feature information and the geometric feature information to obtain the semantic fusion feature information of the road surface to be identified. The pseudo-label generation module is configured to perform clustering and segmentation processing based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified; The disease category identification module is configured to perform disease category identification processing based on the semantic fusion feature information and the pseudo-label information to obtain the disease category information of the road surface to be identified.

[0013] Based on the same inventive concept, a third aspect of the exemplary embodiments of this disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method as described in the first aspect.

[0014] Based on the same inventive concept, a fourth aspect of the exemplary embodiments of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method as described in the first aspect.

[0015] Based on the same inventive concept, a fifth aspect of the exemplary embodiments of this disclosure provides a computer program product including computer program instructions that, when run on a computer, cause the computer to perform the method as described in the first aspect.

[0016] As can be seen from the above description, the road surface defect identification method, apparatus, device, storage medium, and program product provided in this disclosure include: acquiring an original image of the road surface to be identified; performing binary classification segmentation on the original image to obtain a binary image and visual feature information of the road surface to be identified; performing geometric feature extraction on the binary image to obtain geometric feature information of the road surface to be identified; performing attention-based fusion processing on the visual feature information and the geometric feature information to obtain semantic fusion feature information of the road surface to be identified; performing clustering segmentation based on the binary image and the semantic fusion feature information to obtain pseudo-label information of the road surface to be identified; and performing defect category identification processing based on the semantic fusion feature information and the pseudo-label information to obtain defect category information of the road surface to be identified. This disclosure improves the accuracy of road surface defect identification, reduces the cost of data annotation, and avoids the complexity of post-processing. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram illustrating an application scenario of the pavement distress identification method provided in an exemplary embodiment of this disclosure; Figure 2 A schematic diagram of a framework structure for a road surface defect identification system provided as an exemplary embodiment of this disclosure; Figure 3 A schematic flowchart of a pavement distress identification method provided for an exemplary embodiment of this disclosure; Figure 4 A schematic flowchart illustrating a geometric feature extraction method provided for an exemplary embodiment of this disclosure; Figure 5 A flowchart illustrating an attention mechanism feature fusion method provided for an exemplary embodiment of this disclosure; Figure 6 A schematic flowchart illustrating a pseudo-tag generation method provided for an exemplary embodiment of this disclosure; Figure 7 A schematic diagram of a road surface defect identification device provided as an exemplary embodiment of the present disclosure; Figure 8 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present disclosure. Detailed Implementation

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0020] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this application's technical solution, based on the prompt message.

[0021] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0023] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0024] To make the objectives, technical solutions, and advantages of this disclosure clearer, the principles and spirit of this disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0025] In this article, it is important to understand that any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0026] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar words used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly. The article "a" or "an" preceding an element does not exclude the existence of multiple such elements.

[0027] The principles and spirit of this disclosure will be explained in detail below with reference to several representative embodiments.

[0028] As described in the background section, the automatic identification of pavement defects (such as cracks, repairs, etc.) is crucial for road maintenance and safety management.

[0029] However, the inventors of this disclosure have found that the road surface defect identification schemes provided by related technologies have low identification accuracy and rely on pixel-level data annotation, which is costly and depends on complex post-processing.

[0030] To address the aforementioned problems, this disclosure provides a pavement defect identification scheme, specifically comprising: acquiring an original image of the pavement to be identified; performing binary classification segmentation on the original image to obtain a binary image and visual feature information of the pavement to be identified; performing geometric feature extraction on the binary image to obtain geometric feature information of the pavement to be identified; performing attention-based fusion processing on the visual feature information and the geometric feature information to obtain semantic fusion feature information of the pavement to be identified; performing clustering segmentation based on the binary image and the semantic fusion feature information to obtain pseudo-label information of the pavement to be identified; and performing defect category identification processing based on the semantic fusion feature information and the pseudo-label information to obtain defect category information of the pavement to be identified.

[0031] This disclosure improves the accuracy of pavement distress identification, reduces the cost of data annotation, and avoids the complexity of post-processing.

[0032] After introducing the basic principles of this disclosure, various non-limiting embodiments of this disclosure will be described in detail below.

[0033] refer to Figure 1 This is a schematic diagram illustrating an application scenario of the road surface defect identification method provided in an exemplary embodiment of this disclosure.

[0034] This application scenario includes a terminal device 101, a server 102, and a data storage system 103. The terminal device 101, server 102, and data storage system 103 can all be connected via wired or wireless communication networks to achieve data interaction.

[0035] Terminal device 101 may be an electronic device located close to the user side, possessing data transmission and multimedia input / output functions, including but not limited to desktop computers, mobile phones, portable computers, tablet computers, media players, smart wearable devices, personal digital assistants (PDAs), or other electronic devices capable of performing the aforementioned functions. This electronic device may include a processor and a display screen with touch input functionality. The display screen is used to present a graphical user interface (GUI), which can display an application interface. The processor is used to process application data, generate the GUI, and control the display of the GUI on the screen.

[0036] Both server 102 and data storage system 103 can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0037] In some exemplary embodiments, the pavement defect identification method may run on terminal device 101 or server 102.

[0038] When the road surface defect identification method runs on server 102, server 102 provides road surface defect identification services to users of terminal device 101. Terminal device 101 has a client installed that communicates with server 102. Users can input the original image of the road surface to be identified through this client. The client sends the original image of the road surface to be identified to server 102. Server 102 obtains the original image of the road surface to be identified, performs binary classification segmentation on the original image to obtain a binary image and visual feature information of the road surface to be identified; performs geometric feature extraction on the binary image to obtain geometric feature information of the road surface to be identified; performs attention-based fusion processing on the visual feature information and the geometric feature information to obtain semantic fusion feature information of the road surface to be identified; performs clustering segmentation processing based on the binary image and the semantic fusion feature information to obtain pseudo-label information of the road surface to be identified; and performs defect category identification processing based on the semantic fusion feature information and the pseudo-label information to obtain defect category information of the road surface to be identified. Server 102 can send the road surface defect category information to the client, and the client can display the road surface defect category information to the user to help the user carry out road maintenance and safety management.

[0039] The data storage system 103 stores a large amount of training data, which may come from sources including but not limited to existing databases, data crawled from the Internet, or data uploaded by users when using the client.

[0040] The following is combined with Figure 1 The above application scenarios are used to describe the pavement distress identification method according to exemplary embodiments of this disclosure. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of this disclosure, and the embodiments of this disclosure are not limited in any way. Rather, the embodiments of this disclosure can be applied to any applicable scenario.

[0041] refer to Figure 2 This is a schematic diagram of a framework structure of a road surface defect identification system provided in an exemplary embodiment of this disclosure.

[0042] The system consists of the following main modules: 1) Binary Classification Segmentation Model: As the basic module of the system, this model is responsible for identifying coarse-grained (e.g., 10x10 pixel grid) crack and repair regions from the input raw road surface image. Its output is a coarse-grained binary grid map indicating whether each 10x10 region contains a crack or a repair.

[0043] 2) Geometric Feature Extraction Submodule: This is a non-learning module that receives a coarse-grained binary lattice image from the binary classification model. This submodule employs a series of mature traditional image processing algorithms to accurately extract various quantified geometric attributes, such as the area of ​​connected regions, bounding boxes, centroids, skeleton length, average width, principal direction, number of branch points, number of endpoints, and curvature. These features are considered "information compensation" for the coarse-grained output, transforming the ambiguous coarse-grained pattern into structured, interpretable semantic information.

[0044] 3) Attention Mechanism Feature Fusion Module: This module receives two main inputs: a visual feature map generated within the binary classification model (feature output from the decoder's intermediate layer) and a geometric feature vector output from the geometric feature extraction submodule. This method learns to deeply fuse heterogeneous information (fine-grained semantic information and image information) to generate an enhanced feature map containing rich semantic information. This fusion process is adaptive, allowing the model to learn how to best utilize geometric prior knowledge to guide fine-grained classification.

[0045] 4) Pseudo-label generator: The pseudo-label generator mainly receives two inputs: the coarse-grained result from the binary classification model and the semantic fusion feature map. It associates the geometric features with pre-constructed semantic information, thereby generating fine-grained pseudo-labels through clustering methods. It is worth noting that this method randomly generates class centers in the initial state of model training, and the class centers are dynamically updated and adjusted during training in conjunction with the output of the attention mechanism fusion module. The output pseudo-labels provide self-supervised signals for the fine-grained classifier.

[0046] 5) Fine-grained classifier: The fine-grained classifier is a convolutional network that receives the fused enhanced feature map as input. Since rule-based algorithms are insufficient in complex disease scenarios, this classifier aims to fine-tune the results of the geometric feature extraction submodule to improve the model's fine-grained classification accuracy. Based on these features that fuse visual and geometric semantic information, it directly outputs the fine-grained disease type for each 10x10 grid region, such as transverse cracks, longitudinal cracks, crazing, transverse repairs, and longitudinal repairs.

[0047] refer to Figure 3 This is a schematic flowchart of a road surface defect identification method provided by an exemplary embodiment of the present disclosure.

[0048] The method for identifying road surface defects includes the following steps: Step S210: Obtain the original image of the road surface to be identified, and perform binary classification segmentation on the original image to obtain the binary image and visual feature information of the road surface to be identified.

[0049] In some exemplary embodiments, the step of performing binary classification segmentation on the original image to obtain a binary image and visual feature information of the road surface to be identified includes: The original image is input into a pre-trained binary classification segmentation model; The original image is segmented by the binary classification model to obtain a binary image, the resolution of which is the same as that of the original image. The visual feature information is obtained from the intermediate layer of the decoder of the binary classification segmentation model.

[0050] The training method for the binary classification segmentation model will be introduced below: Image annotation: The first step of this invention is to prepare training data efficiently and economically, which is achieved through a block-level annotation method, aiming to overcome the cost bottleneck of pixel-level annotation. The preprocessed image is conceptually divided into a regular grid consisting of non-overlapping 10x10 pixel image blocks. If an image block contains any part of any "crack-type" defect, it is labeled "crack"; if it contains any part of any "patch-type" defect, it is labeled "patch"; and image blocks that do not contain any of the above defects are implicitly labeled "background." For images containing both cracks and patches, a multi-label annotation method is used. It is important to note that the image size is padded and rounded before division to ensure that the image can be completely divided into 10x10 grid regions.

[0051] Binary Classification Segmentation Model Training: A key feature of this invention is its backbone network independence. This means that the encoder part of the segmentation model can be any standard or state-of-the-art convolutional neural network (CNN) architecture (e.g., ResNet, VGG, EfficientNet, HRNet, etc.), or a Transformer-based architecture, which excels in feature extraction. This modular design ensures the method's broad applicability, flexibility, and ability to fully utilize the binary classification results generated by such networks to produce high-confidence feature maps. The preprocessed image in step S210 is fed into the model, which then generates a segmentation map corresponding to the block grid resolution. Each "pixel" in the output corresponds to a 10x10 image block in the original image, classified as one of "crack," "patch," or "background." Common loss functions are used during training. Since the binary classification network trained in this step needs to provide feature maps for subsequent processes, it is worth noting that feature maps of different depths should be extracted when selecting feature maps based on different network characteristics to ensure that image information can effectively guide fine-grained classification. The skeleton network used in this method is a skip connection network. Therefore, the feature maps used in this method are feature maps from the decoder that are close to the resolution of the original image.

[0052] Step S220: Perform geometric feature extraction processing on the binary image to obtain the geometric feature information of the road surface to be identified.

[0053] In some exemplary embodiments, the step of performing geometric feature extraction processing on the binary image to obtain the geometric feature information of the road surface to be identified includes: The binary image is subjected to connected component analysis to obtain connected components; After skeletonization of the connected components, topological feature extraction is performed to obtain the topological features. The connected components are subjected to region attribute extraction processing to obtain region attributes; The geometric feature information is obtained by comprehensively processing the topological features and the regional attributes.

[0054] refer to Figure 4 In specific implementation, for the geometric feature extraction module: This module receives a coarse-grained binary lattice image output from a binary classification model and performs a series of image processing operations on it to extract quantized geometric features. The coarse-grained binary lattice image output by the binary classification model is... A binary matrix, where Each element in the matrix represents a classification result for a 10x10 region; for example, 1 represents crack / repair and 0 represents background.

[0055] Specifically, for crack-type and repair-type crack-type cracks, separate binary images are generated and processed.

[0056] First, extract connectivity attributes: For each identified connected component, the following geometric attributes can be extracted: Area: The number of 10x10 cells contained in a connected component.

[0057] Bounding box: The coordinates of the smallest rectangular region that encloses a connected component. It can be used to calculate the spatial extent and location of a connected component.

[0058] Centroid: The geometric center of a connected component. It can be used to locate defects.

[0059] Aspect ratio: The ratio of the width to the height of the bounding box, or the ratio of the principal axis length to the minor axis length. For linear cracks (transverse and longitudinal cracks), the aspect ratio will deviate significantly from 1; for block cracks, the aspect ratio may be close to 1.

[0060] Roundness / Compactness: Reflects the compactness of the area's shape. The higher the roundness, the closer the shape is to a circle; the lower the roundness, the more irregular or elongated the shape. It is an important basis for identifying area-related and length-related diseases.

[0061] Second, extract the graph topology: For each identified connected component, the following geometric properties can be extracted: Skeleton length: The total length of the skeleton, i.e., the number of pixel blocks on the skeleton. This can more accurately reflect the actual length of the crack, especially for curved cracks, and is more representative of its true size than the length of the bounding box diagonal.

[0062] Branch point and right-angle point detection: Points in the skeleton that connect three or more pixels. The number of branch points is a key topological feature for distinguishing between transverse and longitudinal linear cracks.

[0063] Endpoint count: Points in the skeleton that are connected to only one pixel. The number of endpoints can help identify the start and end points of a crack, which is helpful in determining the continuity or interruption of a crack.

[0064] The final output of step S220 should be a unified geometric feature vector formed by integrating all extracted geometric features. , where N is the number of features. This vector will serve as one of the inputs to the learnable network fusion submodule. For each identified connected component, there will be a corresponding geometric feature vector. In order to fuse with the visual feature map, these discrete feature vectors need to be mapped back onto a grid with the same spatial resolution as the feature map, broadcasting the geometric features of each connected component to its corresponding 10x10 grid region.

[0065] Step S230: Perform attention-based fusion processing on the visual feature information and the geometric feature information to obtain the semantic fusion feature information of the road surface to be identified.

[0066] In some exemplary embodiments, the step of performing attention-based fusion processing on the visual feature information and the geometric feature information to obtain semantic fusion feature information of the road surface to be identified includes: The visual feature information and the geometric feature information are processed by data splicing to obtain spliced ​​features; The splicing features are processed by global channel context information extraction and local channel context information extraction to obtain global channel context information and local channel context information. The global channel context information and the local channel context information are added together and then normalized to obtain the attention weight information; The attention weight information and the visual feature information are multiplied element-wise to obtain the semantic fusion feature information.

[0067] refer to Figure 5 In specific implementation, for the attention mechanism fusion module: The attention mechanism fusion module is a core component of the semantic information injection module. Its main function is to deeply fuse the visual feature maps generated by the binary classification model with the geometric feature vectors output by the geometric feature extraction submodule. This module aims to generate corresponding attention maps based on the actual spatial dimensions (H×W) of the feature maps, regardless of any differences between these dimensions and the original image dimensions. It dynamically adjusts the importance of each channel at each spatial location by combining global context (capturing large-scale information) and local context (preserving subtle details), thereby effectively fusing and enhancing features of different scales and semantics. This design also enhances the adaptability of the enhancement to different backbone networks.

[0068] Assume the feature map output by step S210 is Attention weights generated by channel attention Then, X is refined through element-wise multiplication to obtain the enhanced feature map X′. The calculation method is as follows: ,in The calculation method is as follows Where g(X) is the global channel context, obtained through global average pooling (GAP), which compresses the feature map of each channel into a scalar representing the global information of that channel. The calculation method is as follows: During fusion, It will be broadcast to a C×H×W dimension. ⊕ denotes broadcast addition. Since g(X) is a C-dimensional vector, it will be broadcast to a C×H×W dimension for element-wise addition with L(X). σ is the Sigmoid activation function, which normalizes the attention weights to the (0,1) range. L(X) is the local channel context, aggregated through pointwise convolutions, designed to capture inter-channel interactions at each spatial location while preserving spatial details. It is calculated as follows: ,in It is 1 1. Convolution is used for dimensionality reduction. L(X) is the activation function, using ReLU activation, and B is batch normalization. The output shape of L(X) is the same as the input feature X (C×H×W), which allows it to preserve and highlight subtle details in low-level features. The final output of this step is a feature map with high-dimensional semantic information. .

[0069] Step S240: Perform clustering and segmentation processing based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified.

[0070] In some exemplary embodiments, the step of performing clustering and segmentation processing based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified includes: Based on the binary image, determine the set of pixels belonging to the same disease category; From the semantic fusion feature information, feature vectors corresponding to the pixel set are extracted, and semantic feature vectors corresponding to the disease category are generated through an aggregation function; The semantic feature vectors are grouped using a clustering algorithm, with each group corresponding to a disease subcategory, to generate the pseudo-label information.

[0071] refer to Figure 6 In practical implementation, for pseudo-tag generators: The pseudo-label generator receives the results from steps S210 and S230 as input, with step S210 providing the coarse-grained recognition results. This is an important basis for generating pseudo-tags. The output of step S230... This module provides the number of cluster points and semantic feature maps, whose extracted feature maps are generally more semantically robust and stable. The pseudo-labels generated based on these high-quality features have higher confidence and accuracy compared to the pseudo-labels generated during the initial training of the fine-grained classifier. This reduces noise in the pseudo-labels and provides a more reliable supervision signal for the training of the fine-grained classifier. In this module, our method designs a clustering algorithm to perform clustering on the coarse-grained segmentation results of the binary classification model, and the feature maps serve as semantic guidance to ensure that the pseudo-labels have high confidence from the initial state. By capturing the deep semantic information of an image, it can be viewed as a high-dimensional feature space, where each pixel or region is mapped to a feature vector. For Each coarse-grained value (background, crack, repair) identified in the data needs to be processed from... Extract the corresponding robust semantic feature vector. This feature vector will serve as the input data point for the clustering algorithm. Assume there are coarse-grained results. Corresponding image A set of pixels }, through the Chinese correspondence The characteristics are obtained through aggregation functions. single feature vector , Through semantic guidance, we obtain a set of feature vectors to be clustered. .

[0072] Clustering algorithms group these semantic feature vectors X into k clusters, each cluster representing a potential fine-grained subclass. In clustering algorithms, these semantic feature vectors... Grouped into Within each cluster, each cluster represents a potential fine-grained subclass. It is also worth noting that... Upsampling is required to restore the spatial dimensions to match the coarse-grained grid output.

[0073] Step S250: Based on the semantic fusion feature information and the pseudo-label information, perform disease category identification processing to obtain the disease category information of the road surface to be identified.

[0074] In some exemplary embodiments, the step of performing disease category identification processing based on the semantic fusion feature information and the pseudo-label information to obtain disease category information of the pavement to be identified includes: Determine the disease category information provided by the pseudo-label information; Based on the disease category information, the semantic fusion feature information is processed for disease category identification to obtain the disease category information of the road surface to be identified.

[0075] In practical implementation, for fine-grained classifiers: The fine-grained classifier receives the enhanced feature maps output by the learnable network fusion submodule. As input, this classifier aims to map features that fuse visual and geometric semantic information to a final fine-grained disease category. Its structure consists of a series of convolutional layers, activation functions, and a final classification layer.

[0076] As can be seen from the above, the road surface defect identification method provided in this disclosure includes: acquiring an original image of the road surface to be identified; performing binary classification segmentation on the original image to obtain a binary image and visual feature information of the road surface to be identified; performing geometric feature extraction on the binary image to obtain geometric feature information of the road surface to be identified; performing attention-based fusion processing on the visual feature information and the geometric feature information to obtain semantic fusion feature information of the road surface to be identified; performing clustering segmentation based on the binary image and the semantic fusion feature information to obtain pseudo-label information of the road surface to be identified; and performing defect category identification processing based on the semantic fusion feature information and the pseudo-label information to obtain defect category information of the road surface to be identified.

[0077] This disclosure improves the accuracy of pavement distress identification, reduces the cost of data annotation, and avoids the complexity of post-processing.

[0078] Specifically, this disclosure cleverly incorporates computationally achievable geometric features from traditional image processing as semantic information into the feature learning process of deep learning models. This design not only improves the model's recognition accuracy, but more importantly, it achieves fine-grained classification relying solely on coarse-grained labeled data, significantly reducing data labeling costs and avoiding the complexity of traditional post-processing.

[0079] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0080] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0081] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides a road surface defect identification device.

[0082] refer to Figure 7 The road surface defect identification device includes the following modules: The binary classification segmentation module 910 is configured to acquire the original image of the road surface to be identified, perform binary classification segmentation on the original image, and obtain the binary image and visual feature information of the road surface to be identified.

[0083] The geometric feature extraction module 920 is configured to perform geometric feature extraction processing on the binary image to obtain the geometric feature information of the road surface to be identified.

[0084] The attention mechanism feature fusion module 930 is configured to perform attention mechanism-based fusion processing on the visual feature information and the geometric feature information to obtain the semantic fusion feature information of the road surface to be identified.

[0085] The pseudo-label generation module 940 is configured to perform clustering and segmentation processing based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified.

[0086] The disease category identification module 950 is configured to perform disease category identification processing based on the semantic fusion feature information and the pseudo-label information to obtain the disease category information of the road surface to be identified.

[0087] In some exemplary embodiments, the binary classification segmentation module 910 is specifically configured as follows: The original image is input into a pre-trained binary classification segmentation model; The original image is segmented by the binary classification model to obtain a binary image, the resolution of which is the same as that of the original image. The visual feature information is obtained from the intermediate layer of the decoder of the binary classification segmentation model.

[0088] In some exemplary embodiments, the geometric feature extraction module 920 is specifically configured as follows: The binary image is subjected to connected component analysis to obtain connected components; After skeletonization of the connected components, topological feature extraction is performed to obtain the topological features. The connected components are subjected to region attribute extraction processing to obtain region attributes; The geometric feature information is obtained by comprehensively processing the topological features and the regional attributes.

[0089] In some exemplary embodiments, the attention mechanism feature fusion module 930 is specifically configured as follows: The visual feature information and the geometric feature information are processed by data splicing to obtain spliced ​​features; The splicing features are processed by global channel context information extraction and local channel context information extraction to obtain global channel context information and local channel context information. The global channel context information and the local channel context information are added together and then normalized to obtain the attention weight information; The attention weight information and the visual feature information are multiplied element-wise to obtain the semantic fusion feature information.

[0090] In some exemplary embodiments, the pseudo-tag generation module 940 is specifically configured as follows: Based on the binary image, determine the set of pixels belonging to the same disease category; From the semantic fusion feature information, feature vectors corresponding to the pixel set are extracted, and semantic feature vectors corresponding to the disease category are generated through an aggregation function; The semantic feature vectors are grouped using a clustering algorithm, with each group corresponding to a disease subcategory, to generate the pseudo-label information.

[0091] In some exemplary embodiments, the disease category identification module 950 is specifically configured as follows: Determine the disease category information provided by the pseudo-label information; Based on the disease category information, the semantic fusion feature information is processed for disease category identification to obtain the disease category information of the road surface to be identified.

[0092] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0093] The apparatus of the above embodiments is used to implement the corresponding pavement defect identification method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0094] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the road surface defect identification method described in any of the above embodiments.

[0095] Figure 8 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0096] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0097] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0098] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0099] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0100] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0101] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0102] The electronic devices described above are used to implement the corresponding pavement defect identification methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0103] The memory 1020 stores machine-readable instructions executable by the processor 1010. When the electronic device is running, the processor 1010 communicates with the memory 1020 via the bus 1030, causing the processor 1010 to execute the following instructions during operation: The original image of the road surface to be identified is obtained, and the original image is subjected to binary classification segmentation to obtain the binary image and visual feature information of the road surface to be identified. Geometric feature extraction is performed on the binary image to obtain the geometric feature information of the road surface to be identified; The visual feature information and the geometric feature information are fused based on an attention mechanism to obtain the semantic fusion feature information of the road surface to be identified; Clustering and segmentation processing is performed based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified; Based on the semantic fusion feature information and the pseudo-label information, the disease category identification process is performed to obtain the disease category information of the road surface to be identified.

[0104] In one possible implementation, the instructions executed by the processor 1010, which involve performing binary classification segmentation on the original image to obtain a binary image and visual feature information of the road surface to be identified, include: The original image is input into a pre-trained binary classification segmentation model; The original image is segmented by the binary classification model to obtain a binary image, the resolution of which is the same as that of the original image. The visual feature information is obtained from the intermediate layer of the decoder of the binary classification segmentation model.

[0105] In one possible implementation, the instructions executed by the processor 1010, which involve performing geometric feature extraction processing on the binary image to obtain the geometric feature information of the road surface to be identified, include: The binary image is subjected to connected component analysis to obtain connected components; After skeletonization of the connected components, topological feature extraction is performed to obtain the topological features. The connected components are subjected to region attribute extraction processing to obtain region attributes; The geometric feature information is obtained by comprehensively processing the topological features and the regional attributes.

[0106] In one possible implementation, the instructions executed by the processor 1010, which involve performing attention-based fusion processing on the visual feature information and the geometric feature information to obtain semantic fusion feature information of the road surface to be identified, include: The visual feature information and the geometric feature information are processed by data splicing to obtain spliced ​​features; The splicing features are processed by global channel context information extraction and local channel context information extraction to obtain global channel context information and local channel context information. The global channel context information and the local channel context information are added together and then normalized to obtain the attention weight information; The attention weight information and the visual feature information are multiplied element-wise to obtain the semantic fusion feature information.

[0107] In one possible implementation, the instructions executed by the processor 1010, which involve performing clustering and segmentation processing based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified, include: Based on the binary image, determine the set of pixels belonging to the same disease category; From the semantic fusion feature information, feature vectors corresponding to the pixel set are extracted, and semantic feature vectors corresponding to the disease category are generated through an aggregation function; The semantic feature vectors are grouped using a clustering algorithm, with each group corresponding to a disease subcategory, to generate the pseudo-label information.

[0108] In one possible implementation, the instructions executed by the processor 1010, which include performing disease category identification processing based on the semantic fusion feature information and the pseudo-label information to obtain disease category information of the road surface to be identified, include: Determine the disease category information provided by the pseudo-label information; Based on the disease category information, the semantic fusion feature information is processed for disease category identification to obtain the disease category information of the road surface to be identified.

[0109] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the road surface defect identification method as described in any of the above embodiments.

[0110] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0111] The aforementioned non-transitory computer-readable storage media can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0112] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the pavement defect identification method as described in any of the embodiments in the exemplary method section above, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0113] Based on the same inventive concept, corresponding to the pavement defect identification method described in any of the above embodiments, this disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to perform the pavement defect identification method. Corresponding to the execution entity for each step in each embodiment of the pavement defect identification method, the processor executing the corresponding step can belong to the corresponding execution entity.

[0114] The computer program product of the above embodiments is used to cause the computer and / or the processor to execute the pavement defect identification method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0115] Those skilled in the art will recognize that embodiments of this disclosure can be implemented as a system, method, or computer program product. Therefore, this disclosure can be implemented as entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this disclosure can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0116] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (not exhaustive) of a computer-readable storage medium may include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0117] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0118] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0119] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0120] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine that, when executed by a computer or other programmable data processing device, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0121] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce a product comprising an instruction apparatus that implements the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0122] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable apparatus can provide a process for implementing the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0123] Furthermore, although the operations of the methods of this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be executed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0125] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0126] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0127] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0128] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0129] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

[0130] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the appended claims is to be interpreted in the broadest sense, thereby encompassing all such modifications and equivalent structures and functions.

Claims

1. A method for identifying road surface defects, characterized in that, include: The original image of the road surface to be identified is obtained, and the original image is subjected to binary classification segmentation to obtain the binary image and visual feature information of the road surface to be identified. Geometric feature extraction is performed on the binary image to obtain the geometric feature information of the road surface to be identified; The visual feature information and the geometric feature information are fused based on an attention mechanism to obtain the semantic fusion feature information of the road surface to be identified; Clustering and segmentation processing is performed based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified; Based on the semantic fusion feature information and the pseudo-label information, the disease category identification process is performed to obtain the disease category information of the road surface to be identified.

2. The method according to claim 1, characterized in that, The step of performing binary classification segmentation on the original image to obtain the binary image and visual feature information of the road surface to be identified includes: The original image is input into a pre-trained binary classification segmentation model; The original image is segmented by the binary classification model to obtain a binary image, the resolution of which is the same as that of the original image. The visual feature information is obtained from the intermediate layer of the decoder of the binary classification segmentation model.

3. The method according to claim 1, characterized in that, The geometric feature extraction process performed on the binary image to obtain the geometric feature information of the road surface to be identified includes: The binary image is subjected to connected component analysis to obtain connected components; After skeletonization of the connected components, topological feature extraction is performed to obtain the topological features. The connected components are subjected to region attribute extraction processing to obtain region attributes; The geometric feature information is obtained by comprehensively processing the topological features and the regional attributes.

4. The method according to claim 1, characterized in that, The step of performing attention-based fusion processing on the visual feature information and the geometric feature information to obtain the semantic fusion feature information of the road surface to be identified includes: The visual feature information and the geometric feature information are processed by data splicing to obtain spliced ​​features; The splicing features are processed by global channel context information extraction and local channel context information extraction to obtain global channel context information and local channel context information. The global channel context information and the local channel context information are added together and then normalized to obtain the attention weight information; The attention weight information and the visual feature information are multiplied element-wise to obtain the semantic fusion feature information.

5. The method according to claim 1, characterized in that, The step of performing clustering and segmentation based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified includes: Based on the binary image, determine the set of pixels belonging to the same disease category; From the semantic fusion feature information, feature vectors corresponding to the pixel set are extracted, and semantic feature vectors corresponding to the disease category are generated through an aggregation function; The semantic feature vectors are grouped using a clustering algorithm, with each group corresponding to a disease subcategory, to generate the pseudo-label information.

6. The method according to claim 1, characterized in that, The process of identifying the disease category based on the semantic fusion feature information and the pseudo-label information to obtain the disease category information of the road surface to be identified includes: Determine the disease category information provided by the pseudo-label information; Based on the disease category information, the semantic fusion feature information is processed for disease category identification to obtain the disease category information of the road surface to be identified.

7. A road surface defect identification device, characterized in that, include: The binary classification segmentation module is configured to acquire the original image of the road surface to be identified, perform binary classification segmentation on the original image, and obtain the binary image and visual feature information of the road surface to be identified. The geometric feature extraction module is configured to perform geometric feature extraction processing on the binary image to obtain the geometric feature information of the road surface to be identified; The attention mechanism feature fusion module is configured to perform attention mechanism-based fusion processing on the visual feature information and the geometric feature information to obtain the semantic fusion feature information of the road surface to be identified. The pseudo-label generation module is configured to perform clustering and segmentation processing based on the binary image and the semantic fusion feature information to obtain the pseudo-label information of the road surface to be identified; The disease category identification module is configured to perform disease category identification processing based on the semantic fusion feature information and the pseudo-label information to obtain the disease category information of the road surface to be identified.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method of any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer program instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1 to 6.

Images