Robot inspection defect detection method based on cross attention and multi-dimensional measurement

Abstract

The invention discloses a robot inspection defect detection method based on cross attention and multi-dimensional measurement, and the method comprises the steps: designing a small sample target detection model, collecting a dam image through the autonomous inspection of a robot, and extracting a defect region in the image through a cross convolution attention method, and the defect areas are classified through multi-dimensional measurement learning, so that high-precision and high-efficiency defect detection work is realized. According to the method, a cross convolution module and a multi-dimensional measurement learning module are designed for solving the problem of disastrous forgetting and the problem that the quantity of model parameters is too large in an existing small sample target detection method, so that disastrous forgetting of the model is effectively relieved, and the model parameters are greatly reduced. In combination with the robot autonomous inspection technology, the method significantly improves the recognition efficiency and accuracy of the dam concrete surface defects, and realizes the intelligence and automation of dam safety monitoring.

Description

Technical Field

[0001] The invention relates to a robot inspection defect detection method based on cross attention and multi-dimensional measurement, belonging to the technical field of engineering defect image classification. Background Art

[0002] my country's reservoir and dam projects play an irreplaceable and important role in flood control, hydropower generation, irrigation, etc. These water conservancy infrastructures are not only key nodes for regulating the temporal and spatial distribution of water resources, but also important hubs for optimizing water resource allocation, preventing and controlling water disasters, and protecting the ecological environment. They are an indispensable part of maintaining the safety of river flood control systems and ensuring the safety of people's lives and property. In the reservoir and dam safety assurance system, regular inspections and safety monitoring have always been one of the core links.

[0003] However, although dam safety monitoring is crucial to maintaining the safety of the country's water conservancy infrastructure, existing dam defect monitoring methods are difficult to meet actual needs. Traditional monitoring methods require manual construction of observation platforms and rely on professional operators for monitoring, which is not only inefficient but also easily affected by subjective factors. To overcome these difficulties, the new method uses a deep learning model for object detection in the field of computer vision to identify concrete defects in dam monitoring video images.

[0004] However, existing target detection methods face many difficulties in obtaining samples of concrete surface defects. On the one hand, the dam structure is complex, and there are various types of defects such as cracks and water seepage. It takes a lot of manpower and material resources to comprehensively collect samples covering various defects. On the other hand, since the dam is located in the wild, it is difficult to obtain clear and accurately labeled defect image samples due to the influence of factors such as climate environment and engineering construction.

[0005] In order to solve the above problems, small sample learning methods have been introduced into the target detection model of dam safety monitoring. However, the current small sample target detection model faces the problem of catastrophic forgetting of the model due to overfitting of new class samples, resulting in a decrease in the detection accuracy of the model for the base class. At the same time, the parameter scale of the current small sample target detection model is very large, making it difficult to deploy the model on the edge device of the monitoring system. Summary of the invention

[0006] Purpose of the invention: In view of the catastrophic forgetting problem of base class memory and the large number of model parameters in the training process of small sample target detection models in the existing concrete dam surface defect recognition work, the present invention proposes a robot inspection defect detection method based on cross-attention and multi-dimensional quantity, and designs a small sample target detection model based on cross-attention and multi-dimensional quantity in the defect detection part, which is referred to as the target detection model. The model can identify defects in the query image with only a small number of supporting samples, while alleviating the catastrophic forgetting problem and improving the model's detection accuracy for defects.

[0007] At the same time, by designing cross-convolution modules and multi-dimensional measurement modules, the parameter scale of the target detection model is effectively reduced, and the reasoning speed of the model is improved, so that the model can better adapt to edge devices such as inspection robots in the dam monitoring system.

[0008] Technical solution: A robot inspection defect detection method based on cross-attention and multi-dimensional measurement, comprising the following steps:

[0009] Step 1) Using the inspection robot to obtain defect images of concrete surface cracks, water seepage, etc. during the dam inspection process, preprocess the defect images and annotate the defects, organize the defects of the same category into a support image set, and construct a concrete dam defect detection dataset;

[0010] Step 2) In real time, dam inspection images are collected as query images, and a target detection model based on cross attention and multi-dimensional quantity is designed to detect defects in the query images. The target detection model uses a pre-trained convolutional neural network as a backbone network to extract features from the support image set and the query image in step 1) to obtain feature maps of support features and query features, and integrates the features of the feature maps through a lightweight feature encoder;

[0011] Step 3) Calculate the weights of the integrated support features obtained in step 2) respectively, perform weighted summation on the defect features of each category, and obtain a support prototype, which includes the visual features of the support image;

[0012] Step 4) fills the query features obtained in step 2), uses the support prototype obtained in step 3) as a convolution kernel to perform cross convolution on the filled query features, obtains an attention map of the query features, uses the attention map to enhance the query features, and then inputs the enhanced query features into the region proposal network to generate candidate regions;

[0013] Step 5) inputting the candidate regions of the supporting prototypes and query images of each category obtained in step 2) into the measurement modules of three different dimensions: high-level semantic features, global information, and local information, to obtain the target classification results of the three dimensions;

[0014] Step 6) weights and sums the classification results of each dimension obtained in step 5) to obtain the final prediction result, including the location, type and confidence of the target defect.

[0015] Among them, the construction of the concrete dam defect detection dataset in step 1) includes the following processes:

[0016] 1-1) Collect different types of concrete surface defect images from the image data during the inspection of the concrete dam, and mark various types of defects in the defect images, including the location and defect type;

[0017] 1-2) Classify and screen the annotated defect images, exclude images with unclear defect features or unsatisfactory clarity, and ensure that the number of defect images in each category is as balanced as possible and not less than 30, to obtain a defect detection dataset;

[0018] 1-3) The defect image dataset is divided into support set, query set and test set by random allocation.

[0019] The feature extraction and feature integration process of step 2) includes the following steps:

[0020] 2-1) Use the residual network ResNet-50 as the backbone network for feature extraction, input the support image and query image, and obtain multi-scale support features and query features through the residual convolution of ResNet-50;

[0021] 2-2) For k supporting features S of the same category i ,i∈{1,2,…,k}, perform regional feature alignment operation to change its shape to s×s, and perform the same operation on the query feature to obtain Q;

[0022] 2-3) Use lightweight feature encoder to support feature S i And query feature Q to integrate the features of shape s×s to get S i ′ and Q ′ Next, S i ′ and Q ′ Transform it into the shape The purpose of feature integration is to deal with the possible changes in the posture of the detection object in the query image. By integrating the features of the pixels around the target, the influence of the object posture change on feature extraction is reduced. The feature encoder includes two 1×1 convolutional layers, one 1×5 and one 5×1 convolutional layer. The first two layers are used for adaptive feature space conversion of the target detection model, and the last two layers are used to integrate the information of the surrounding pixels and reduce the influence of noise.

[0023] Wherein, the support aggregation process of obtaining the support prototype in step 3) includes the following steps:

[0024] 3-1) The integrated support features obtained in step 2) are passed through two fully connected layers in turn for weight calculation. The input dimension of the first fully connected layer is rd and the output dimension is d; the input dimension of the second fully connected layer is d and the output dimension is 1. Then the calculated results are scaled by applying the Softmax function, limiting their sum to 1, and obtaining the weight vector of each support feature.

[0025] 3-2) The obtained weight vector And the corresponding support feature S i ′ Multiply them together to get the weighted support features;

[0026] 3-3) Sum multiple weighted support features to obtain the support prototype of the support category. The specific formula is as follows:

[0027]

[0028] The size of the supporting prototype P is w p ×h p ×d, where w p Represents the width of the supported prototype, h p represents the height of the support prototype, and d represents the number of channels of the support prototype. The support prototype includes the semantic features and morphological features of each support image of the category.

[0029] Among them, the cross convolution process of step 4) includes the following steps:

[0030] 4-1) To ensure the query feature Q before and after convolution ′ The size remains unchanged, according to the support prototype P obtained in step 3) for the query feature Q obtained in step 2) ′ Perform a filling operation. By default, the filling operation fills the query feature with 0s at the top, bottom, left, and right. The specific filling method is as follows:

[0031]

[0032]

[0033]

[0034]

[0035] Among them, pad top 、pad bottom 、pad left 、pad rightThey represent the filling scale of the query feature in the four directions of up, down, left, and right, respectively. w represents the width of the supporting prototype, and h represents the height of the supporting prototype.

[0036] 4-2) Use the support prototype P as the convolution kernel, perform a cross convolution operation on the padded query features to obtain the attention map. The cross convolution operation does not require additional training and memory parameters. The specific formula of the cross convolution is as follows:

[0037] Y j =∑W j ×X j +b j ,j∈{1,2,…,n}

[0038] where Y j represents the output feature of the jth class, W j represents the weight of the jth convolution kernel, X j Represents the feature map of the jth input, i.e., the query feature Q ′ The corresponding part in b j represents the bias term of the jth convolution operation, and n represents the category of the supported samples;

[0039] 4-3) Perform pixel-level multiplication of the attention map and the original query feature to obtain the enhanced query feature;

[0040] The final prediction result generation process of the multi-dimensional metric learning process in step 5) includes the following steps:

[0041] 5-1) Input the candidate regions of the support prototype and query image into a lightweight feature decoder. The decoder contains two 1×1 convolutional layers and a Sigmoid layer to obtain the decoded features of the support prototype and candidate regions;

[0042] 5-2) Will support prototypes and candidate region Q d The three heterogeneous lightweight measurement modules designed for the defect detection method are input into the three measurement modules, namely, the measurement module based on high-level semantic features, the measurement module based on global information, and the measurement module based on local information, to obtain classification probabilities in three dimensions. The multidimensional measurement module mainly converts the feature vectors in the feature space to the measurement space. Due to the large span between the feature space and the measurement space, the conversion process requires a lot of computing resources for encoding. Unlike the existing metric learning head, the feature encoder of the multidimensional measurement module is shallow in level, its performance consumption is extremely low, and it is targeted at features of different dimensions, extracting high-level semantic information, global information, and local information from the candidate area;

[0043] 5-3) In each measurement module, support prototype and candidate region Q dFirst input into the encoder of three dimensions of high-level semantics, global information and local information to obtain the specified encoding information and Q e , and then and Q e Perform Euclidean distance measurement to obtain the inter-class measurement result D c ,c∈{1,2,…,n}, and Q d Input pseudorange encoder to obtain background metric D 0 ;

[0044] 5-4) The inter-class measurement result D c And the background measurement result D 0 Merge and obtain the classification probability L of different dimensional measurement modules through the distance decoder i ,i∈{0,1,…,n};

[0045] 5-5) The three measurement modules extract information from features from different dimensions. The measurement module SMMod based on high-level semantic features, the measurement module GMMod based on global information, and the measurement module LMMod based on local information focus on high-level semantic information, global semantic information, and local information, respectively, so as to perform more comprehensive classification. Due to the different information they focus on, the feature encoder structures of the three heads are different.

[0046] SMMod is evaluated from the perspective of high-level semantics. Its feature encoder Semantic Encoder contains two 3×3 convolutional layers, one 1×1 convolutional layer, one BatchNorm layer, and one LeakyReLU layer. The two 3×3 convolutions are responsible for sorting out the features around the pixels and generating high-level semantic features. The 1×1 convolution is responsible for integrating the features, the BatchNorm layer scales the features, and the LeakyReLU layer activates the features and filters out irrelevant features.

[0047] GMMod is evaluated from a global dimension, and its feature encoder contains two fully connected layers and a LeakyReLU layer. The input of the first fully connected layer is s×s×d and the output is 1024, and the input of the second fully connected layer is 1024 and the output is 64. Finally, it is activated by the LeakyReLU layer.

[0048] LMMod evaluates from a local dimension, and its feature encoder contains three 1×1 convolutional layers, a BatchNorm layer, and a LeakyReLU layer. Similar to SMMod, the first two 1×1 convolutions are responsible for organizing the features of each pixel, the last 1×1 convolution is responsible for integrating the features, the BatchNorm layer scales the features, and the LeakyReLU layer activates the features.

[0049] 5-6) Set three learnable parameters, denoted as σ i ,i∈{1,2,3}, corresponds to the weights of the three metric modules. The Softmax function is used to convert the three learnable parameters into weight coefficients whose sum is 1. The weight coefficients are used to weight the classification results of the three metric modules to obtain the final classification probability;

[0050] 5-7) Combine the candidate region location information generated by the region proposal network to obtain the location, type and confidence of the target defect in the query image.

[0051] Among them, the method also includes a loss function for training and optimizing the target detection model. The loss function includes the following parts:

[0052] The loss function of the model is divided into four parts: auxiliary loss in feature extraction, regression loss in the cross convolution module and the classification loss in the multidimensional metric module and regression loss Among them, the regression loss of the two stages and Use smooth L1 loss calculation, the specific formula is as follows:

[0053]

[0054] Here, x represents the difference between the predicted value and the true value.

[0055] Auxiliary loss L aux It is mainly used to reduce the correlation of supporting prototypes and increase the distance between classes. The specific formula is as follows:

[0056]

[0057] in, represents the supporting feature vector of category i, C represents the category set, Denotes the support eigenvector and The distance between.

[0058] Classification loss of the multi-dimensional metric module The improved N-Pair Loss is adopted, and the specific formula is as follows:

[0059]

[0060] Where Q represents the query feature, represents the mean of the positive sample query features, Represents the mean of the query features of the i-th sample.

[0061] In classification loss In the calculation, N-Pair Loss needs to be calculated separately for each metric module, so it needs to be divided into three metric module losses. The specific formula is as follows:

[0062]

[0063] The total loss formula of the model is as follows:

[0064]

[0065] Beneficial effects: Compared with the prior art, the present invention designs a target detection model based on cross-convolution and multi-dimensional quantities to address the catastrophic forgetting of base class memory and the problem of excessive model parameters that occur during the training of small sample target detection models. The model first uses a robot to obtain the surface image of the dam, and then integrates the features of the defects in the support sample into a support prototype through the support aggregation module. The support prototype is used as a convolution kernel to perform cross-convolution on the query image to generate an attention map and obtain the defect area. Through a multi-dimensional metric learning module, the defect area is compared with the defect category in the support sample to determine the defect type. The present invention utilizes the class-independent feature extraction capability of cross-convolution and multi-dimensional metric learning to greatly alleviate the catastrophic forgetting problem and improve the model's recognition accuracy for concrete defects. At the same time, through the lightweight model structure design, the parameter scale of the model is effectively reduced, and the reasoning speed of the model is improved, making it more suitable for edge devices in the dam monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0067] Figure 2 This is an overall framework diagram of a small sample target detection model based on cross-attention and multi-dimensional metrics according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0069] like Figure 1 As shown, a robot inspection defect detection method based on cross attention and multi-dimensional measurement, the specific implementation process of which includes:

[0070] Step 1: Use inspection robots to collect defect images of the concrete surface during daily inspections of the dam. The types of defects include cracks, water seepage, landslides, concrete damage, subsidence, and deformation. For the collected defect images, mark the location and type information of the defects in each image. The annotation work can be done manually or with the assistance of automatic annotation tools. Organize the defect images of the same category into a supporting image set. For each defect category, ensure that the number of supporting images is not less than 30, and try to ensure that the number of images in each category is balanced. The collected defect images are preprocessed, including image scale normalization, color space conversion, data enhancement and other operations to improve image quality and enhance the generalization performance of the model.

[0071] The sorted defect detection dataset is divided into support set, query set and test set by random allocation. The support set is used for the feature aggregation module and cross convolution module in training, the query set is used for the metric learning module in training, and the test set is used for the final model performance evaluation. The specific dataset division ratio can be adjusted according to the actual situation, usually support set: query set: test set = 6:2:2 or 7:2:1.

[0072] Step 2: Design an object detection model based on cross attention and multi-dimensional metrics. The model uses the pre-trained ResNet-50 as the backbone network for feature extraction. The support image and the query image are input into ResNet-50 respectively. After a series of residual convolution layers, the multi-scale support features S can be obtained. i and query features Q.

[0073] In order to cope with the posture changes of the target in the query image, the target detection model designs a feature aggregation module. Feature aggregation is a process of aggregating k supporting features S of the same category. i ,i∈{1,2,…,k}, and use regional feature alignment to unify it into a feature map of shape s×s. The same operation is performed on the query feature Q. The supporting features and query features after regional features are input into the feature integration module. The feature integration module contains two 1×1 convolutions, one 1×5 and one 5×1 convolution. The first two layers are used for adaptive feature space transformation, and the last two layers are used to integrate surrounding pixel information to enhance the model's robustness to posture changes. Although the convolution operation is translation invariant, it lacks rotation invariance and it is difficult to make accurate judgments on targets with posture changes. The model introduces a feature integration module to integrate the features of surrounding pixels and reduce the impact of posture changes on feature extraction. After the feature integration module, the integrated support feature S is obtained. i ′ and query feature Q ′ , in preparation for subsequent support aggregation and cross convolution.

[0074] The integrated support feature S with shape s×si ′ Transformed into where s×s is the feature resolution and d is the number of feature channels. This deformation operation is to facilitate subsequent weight calculation and feature aggregation.

[0075] Step 3: Extract the distinguishing features from the k supporting features of the same category to form the supporting prototype of the category. The specific implementation process is as follows:

[0076] The integrated support feature S obtained in step 2 i ′ Input two fully connected layers in turn for weight calculation. The input dimension of the first fully connected layer is rd, and the output dimension is d; the input dimension of the second fully connected layer is d, and the output dimension is 1. The output result of the second fully connected layer is normalized by the Softmax function to obtain the weight vector of each supporting feature The sum of the weight vectors is 1.

[0077] The calculated weight vector The corresponding support feature S i ′ Multiply them together to get the weighted support features. Sum multiple weighted support features to get the support prototype P of the support category. Since the support prototype contains information such as posture and shape of the support category, the model uses the support prototype as a convolution kernel to replace the learning process of high-level semantic features, making the backbone network more focused on the extraction of common features, thereby reducing the class-related parameters of the backbone network. The size of the support prototype is w p ×h p ×d, where w p and h p Represent the width and height of the support prototype, and d represents the number of channels. The support prototype P contains the semantic features and morphological features of each support image of the category, and can be used as the visual feature prototype of the category.

[0078] This step includes an auxiliary loss function L for the feature extraction process aux The auxiliary loss function is mainly used to reduce the relevance of supporting prototypes and increase the distance between classes. The specific formula is as follows:

[0079]

[0080] in, represents the supporting feature vector of category i, C represents the category set, Denotes the support eigenvector and Step 4: Using the support prototypes generated in step 3, the query features are enhanced by cross convolution to improve the performance of target detection.

[0081] Specifically, step 4 includes:

[0082] In order to ensure that the query feature Q before and after the cross convolution ′ The size of the same, according to the support prototype P obtained in step 3 to Q ′ Fill in. The filling formula is Among them, pad top 、pad bottom 、pad left 、pad right They represent the filling scale of the query feature in the four directions of up, down, left and right.

[0083] After the filling is completed, the support prototype P obtained in step 3 is used as the convolution kernel to fill the query feature Q ′ Perform cross convolution to obtain the attention map. Cross convolution generates attention by performing convolution operations between feature maps. This process does not require additional training and does not involve parameters that need to be remembered. Compared with complex self-attention mechanisms such as Transformer, cross convolution significantly reduces the class-related parameters in the target detection model. Cross convolution enables the model to not only effectively alleviate the catastrophic forgetting problem caused by parameter conflicts in small sample target detection, but also enhances its adaptability to changes in target posture and shape. The cross convolution formula is: Y j =∑W j ×X j +b j ,j∈{1,2,…,n}. j represents the output feature of the jth class, W j represents the weight of the jth convolution kernel, X j Represents the feature map of the jth input, i.e., the query feature Q ′ The corresponding part in b j Represents the bias term of the j-th convolution operation.

[0084] The obtained attention map is combined with the original query feature Q ′ Perform pixel-level multiplication to obtain enhanced query features.

[0085] This step includes a regression loss function for the cross convolution process The regression loss function is mainly used to improve the ability of the cross convolution part to enhance the query features. The regression loss function is calculated using the smooth L1 loss, and the specific formula is as follows:

[0086]

[0087] Where x represents the difference between the predicted value and the true value. Step 5: Using the candidate regions of the supporting prototype and query images, classification is performed through multiple heterogeneous lightweight metric modules to further improve the target recognition ability of the target detection model. Finally, the classification results of the three metric modules are fused to obtain the final prediction result, including the location, type and confidence of the target defect.

[0088] Specifically, step 5 includes:

[0089] First, the support prototype generated in step 3 and the query image candidate region Q generated in step 4 d Input a lightweight feature decoder. The feature decoder contains two 1×1 convolutional layers and a Sigmoid layer. The purpose is to reduce the dimension and nonlinear transformation of the features to obtain the decoded support prototype. and candidate region Q e .

[0090] The decoded support prototype and candidate region Q e Input three heterogeneous lightweight metric modules: the metric module SMMod based on high-level semantic features, the metric module GMMod based on global information, and the metric module LMMod based on local information. These three metric modules evaluate and classify targets from different dimensions respectively. The feature encoder of SMMod contains two 3×3 convolutions, one 1×1 convolution, BatchNorm, and LeakyReLU to extract high-level semantic features. The feature encoder of GMMod contains two fully connected layers and LeakyReLU to extract global information features. The feature encoder of LMMod contains three 1×1 convolutions, BatchNorm, and LeakyReLU to extract local information features. For each metric module, the support prototype is calculated With the candidate region Q e The Euclidean distance measurement result between c ,c∈{1,2,…,n}, and calculate the distance measurement result D between the candidate region and the background 0 .

[0091] The inter-class distance D c Distance from background D 0 Transformed into classification probability L through distance decoder i,i∈{0,1,…,n}. The distance decoder contains a learnable temperature coefficient γ to convert the distance into probability. The three metric modules SMMod, GMMod and LMMod extract feature information from different dimensions respectively, and then obtain the classification probability through the distance decoder. This multi-dimensional metric method can capture the characteristics of the target more accurately, thereby improving the performance of detection and classification.

[0092] The multidimensional measurement module sets three learnable parameters σ i , i∈{1,2,3}, corresponding to the weights of the three metric modules SMMod, GMMod and LMMod respectively. The Softmax function is used to convert these three parameters into weight coefficients σ whose sum is 1. i . The classification probability L of the three metric modules in step 5 SMMod , L GMMod and L LMMod Multiply by the corresponding weight coefficient σ i , and then sum to get the final classification probability L.

[0093] By combining the candidate region location information generated in step 4 with the final classification probability L obtained in step 5, the location, type and confidence of the target defect in the query image can be obtained.

[0094] This step includes the regression loss function for the multidimensional measurement process And the classification loss function The regression loss function and classification loss function are mainly used to improve the ability of the multidimensional measurement process to classify the region of interest. Use smooth L1 loss calculation, the specific formula is as follows:

[0095]

[0096] Here, x represents the difference between the predicted value and the true value.

[0097] Classification loss function The improved N-Pair Loss is adopted, and the specific formula is as follows:

[0098]

[0099] Where Q represents the query feature, represents the mean of the positive sample query features, Represents the mean of the query features of the i-th sample.

[0100] In classification loss In the calculation, N-Pair Loss needs to be calculated separately for each metric module, so it needs to be divided into three metric module losses. The specific formula is as follows:

[0101]

[0102] The total loss formula of the target detection model is as follows:

[0103]

[0104] During the training process, the total loss L is minimized through the back propagation algorithm and the gradient descent algorithm, thereby optimizing the model parameters and improving the accuracy and robustness of defect detection.

[0105] Obviously, those skilled in the art should understand that the above-mentioned robot inspection defect detection method based on cross-attention and multi-dimensional measurement of the embodiment of the present invention can be implemented by a general computing device, which can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices. Optionally, they can be implemented with program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules respectively, or the multi-dimensional modules or steps therein can be made into a single inherited circuit module for implementation. In this way, the embodiments of the present invention are not limited to being combined with any specific hardware and software.

Claims

1. A robot inspection defect detection method based on cross-attention and multi-dimensional measurement, characterized in that: The steps include: Step 1) The defect images of the concrete surface during the dam inspection are obtained by the inspection robot, the defect images are preprocessed and the defects are annotated, the defects of the same category are sorted into a support image set, and a concrete dam defect detection dataset is constructed; Step 2) In real time, dam inspection images are collected as query images, and a small sample target detection model based on cross attention and multi-dimensional quantity is designed to perform defect detection on the query image, referred to as the target detection model. The target detection model uses a pre-trained convolutional neural network as the backbone network to extract features from the support image set and the query image, obtain feature maps of the support features and the query features, and integrate the features of the feature maps through a feature encoder to obtain integrated support features; Step 3) weighting the integrated support features respectively, performing weighted summation on the defect features of each category to obtain a support prototype, which includes the visual features of the support image; Step 4) fill the query features, use the support prototype as the convolution kernel to perform cross convolution on the filled query features, obtain the attention map of the query features, use the attention map to enhance the query features, and then input the enhanced query features into the region proposal network to generate candidate regions; Step 5) The candidate regions of the supporting prototypes and query images of each category are input into the measurement modules of three different dimensions: high-level semantic features, global information, and local information, to obtain the target classification results of the three dimensions; Step 6) The classification results of each dimension are weighted and summed to obtain the final prediction result, including the location, type and confidence of the target defect.

2. The robot inspection defect detection method based on cross-attention and multi-dimensional measurement according to claim 1 is characterized in that: The construction of the concrete dam defect detection data set in step 1) includes the following process: 1-1) Collect different types of concrete surface defect images from the image data during the inspection of the concrete dam, and mark various types of defects in the defect images, including the location and defect type; 1-2) Classify and screen the annotated defect images, exclude images with unclear defect features or unsatisfactory clarity, balance the number of defect images in each category, and obtain a defect detection dataset; 1-3) The defect image dataset is divided into support set, query set and test set by random allocation.

3. The robot inspection defect detection method based on cross-attention and multi-dimensional measurement according to claim 1 is characterized in that: The feature extraction and feature integration process of step 2) comprises the following steps: 2-1) Using the residual network as the backbone network for feature extraction, the support image and query image are input, and after residual convolution, multi-scale support features and query features are obtained; 2-2) Perform regional feature alignment on k supporting features and query features of the same category to change their shape to s×s; 2-3) Use feature encoder to integrate support features and query features.

4. The robot inspection defect detection method based on cross-attention and multi-dimensional measurement according to claim 1 is characterized in that: The support aggregation process of step 3) obtaining the support prototype includes the following steps: 3-1) Calculate the weights of the integrated supporting features, and then scale the calculated results to limit their sum to 1, and obtain the weight vector of each supporting feature; 3-2) Multiply the obtained weight vector by the corresponding support feature to obtain the weighted support feature; 3-3) Sum multiple weighted support features to obtain the support prototype of the support category; the support prototype includes the semantic features and morphological features of each support image of the category.

5. The robot inspection defect detection method based on cross-attention and multi-dimensional measurement according to claim 3 is characterized in that: The cross convolution process of step 4) comprises the following steps: 4-1) Perform a filling operation on the query feature according to the supported prototype. By default, the filling operation fills the query feature with 0s above, below, left, and right. 4-2) Use the support prototype as the convolution kernel, perform a cross convolution operation on the padded query features to obtain the attention map. The cross convolution operation does not require additional training and memory parameters; 4-3) Perform pixel-level multiplication of the attention map and the original query feature to obtain the enhanced query feature.

6. The robot inspection defect detection method based on cross-attention and multi-dimensional measurement according to claim 1 is characterized in that: The final prediction result generation process of the multi-dimensional metric learning process in step 5) comprises the following steps: 5-1) Input the candidate regions of the support prototype and query image into the feature decoder. The decoder contains two 1×1 convolutional layers and a Sigmoid layer to obtain the features of the decoded support prototype and candidate regions; 5-2) Input the supporting prototypes and candidate regions into three heterogeneous lightweight measurement modules, namely, a measurement module based on high-level semantic features, a measurement module based on global information, and a measurement module based on local information, to obtain classification probabilities in three dimensions; 5-3) In each measurement module, the prototype and candidate regions are first input into the encoders of high-level semantics, global information and local information to obtain high-dimensional features, and then the high-dimensional features are measured by Euclidean distance to obtain the inter-class measurement results. At the same time, the pseudo-range encoder is used to measure the background of the query features; 5-4) Combine the inter-class measurement results with the background measurement results, and obtain the classification probabilities of different dimensional measurement modules through the distance decoder; 5-5) Set three learnable parameters corresponding to the weights of the three metric modules, convert the three learnable parameters into weight coefficients whose sum is 1, and use the weight coefficients to weight the sum of the classification results of the three metric modules to obtain the final classification probability; 5-6) Combined with the candidate region location information generated by the region proposal network, the location, type and confidence information of the target defect in the query image is obtained.

7. The robot inspection defect detection method based on cross-attention and multi-dimensional measurement according to claim 1 is characterized in that: The method also includes a loss function for training and optimizing the target detection model. The loss function includes the following parts: Auxiliary loss L for feature extraction aux , the auxiliary loss calculates the sum of the distances between all classes; the auxiliary loss is used to reduce the relevance of supporting prototypes and increase the distance between classes; The loss function of the cross-convolution based region proposal network includes regression loss The loss function of the classification head based on multi-dimensional metrics includes classification loss and regression loss The regression loss of the cross-convolution-based region proposal network and the multi-dimensional metric-based classification head is calculated using smoothed F1 loss; The classification loss of the classification head based on multi-dimensional metrics calculates N-Pair Loss for each metric module separately and adds the three loss results.

8. The robot inspection defect detection method based on cross-attention and multi-dimensional measurement according to claim 6 is characterized in that: The measurement module SMMod based on high-level semantic features, the measurement module GMMod based on global information, and the measurement module LMMod based on local information focus on high-level semantic information, global semantic information, and local information respectively; The measurement module SMMod based on high-level semantic features is evaluated from the dimension of high-level semantics. Its feature encoder Semantic Encoder contains two 3×3 convolutional layers, one 1×1 convolutional layer, one BatchNorm layer, and one LeakyReLU layer. The two 3×3 convolutions are responsible for sorting out the features around the pixels and generating high-level semantic features. The 1×1 convolution is responsible for integrating the features, the BatchNorm layer scales the features, and the LeakyReLU layer activates the features and filters out irrelevant features. The global information-based measurement module GMMod evaluates from a global perspective. Its feature encoder contains two fully connected layers and one LeakyReLU layer; the input dimension of the first fully connected layer is s×s×d and the output dimension is 1024, and the input dimension of the second fully connected layer is 1024 and the output dimension is 64. Finally, it is activated by the LeakyReLU layer; The local information-based measurement module LMMod evaluates from the local dimension. Its feature encoder contains three 1×1 convolutional layers, a BatchNorm layer, and a LeakyReLU layer; similar to SMMod, the first two 1×1 convolutions are responsible for organizing the features of each pixel, and the last 1×1 convolution is responsible for integrating the features, the BatchNorm layer scales the features, and the LeakyReLU layer activates the features.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the robot inspection defect detection method based on cross-attention and multi-dimensional measurement as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the robot inspection defect detection method based on cross-attention and multi-dimensional measurement are implemented as described in any one of claims 1-8.

Images