Cross-scene belt defect recognition method based on adaptive typical sample learning technology

Abstract

The application provides a cross-scene belt defect recognition method and device based on an adaptive typical sample learning technology, and relates to the technical field of defect detection.The method comprises the following steps: constructing a support image dataset and a query image dataset; inputting the support image dataset and the query image dataset into a defect recognition basic model to obtain first support image features and first query image features; inputting the first support image features and the first query image features into a prototype adaptive model for feature enhancement; constructing a loss function according to the support image dataset, the query image dataset and a defect detection image dataset; optimizing the prototype adaptive model to obtain an optimized prototype adaptive model; and performing belt defect recognition based on the defect recognition basic model and the optimized prototype adaptive model according to a to-be-recognized belt image dataset.The application is a cross-scene belt defect recognition method based on typical sample learning, which is efficient and accurate and only needs a small amount of labeling.

Description

Technical Field

[0001] The present invention relates to the field of defect detection technology, and in particular to a cross-scene belt defect recognition method and device based on adaptive typical sample learning technology. Background Art

[0002] In industrial production, belt conveyor systems are a crucial component of material transportation, and their operating status directly impacts the efficiency and safety of the entire production line. After prolonged, high-intensity operation, belts may develop various defects, such as wear, tear, and scratches from foreign objects. If these problems are not detected and addressed promptly, they can not only impact production efficiency but can also lead to serious safety accidents or equipment damage. Therefore, belt defect detection is crucial in industrial production.

[0003] Traditional detection methods mainly rely on manual visual inspection, which is not only time-consuming and labor-intensive, but also prone to missed detections or false detections due to human negligence. In recent years, with the development of computer vision and deep learning technologies, image-based automatic defect detection methods have gradually become a research hotspot. These methods identify defects on belts by training deep neural networks, which has high accuracy and efficiency, but usually requires a large amount of labeled data to train the model. In actual industrial environments, it is extremely expensive to obtain a large amount of high-quality and diverse labeled data. Belt materials and operating conditions vary greatly in different industrial scenarios, and the background of belt images and defect types vary greatly in different scenarios. The complex types of defects and their uneven distribution make them difficult to effectively identify using traditional target detection algorithms.

[0004] While existing belt defect recognition models often utilize different deep learning algorithms (such as convolutional neural networks or Transformers), there's currently no universal migration method to efficiently apply these models to different scenarios. These models are typically designed for specific scenarios, requiring retraining or modification of the entire model structure once the environment or data distribution changes. This increases the time and computational cost of model migration while reducing the system's flexibility and scalability.

[0005] Existing technologies rely on full fine-tuning to adjust belt defect recognition models. This approach consumes excessive computing resources, making it difficult to effectively apply the model in industrial scenarios. Because full fine-tuning requires re-optimizing all model parameters, which are large in industrial scenarios and have limited hardware resources, full fine-tuning is not only computationally expensive but can also lead to overfitting when insufficient data is available for the new scenario, further reducing model stability.

[0006] Existing belt defect recognition technology suffers from model collapse, feature forgetting, and decreased detection accuracy when using small amounts of labeled data. This makes it difficult to deploy the model in new scenarios with only a small number of representative samples. Existing algorithms rely heavily on large-scale, high-quality labeled data. However, the sample size of industrial belt damage defect data is typically low, making it difficult to obtain sufficient data. Furthermore, small amounts of labeled data can easily lead to class imbalance, making it difficult for the algorithm to effectively learn the characteristics of new defect categories. It can also lead to forgetting knowledge of existing categories, limiting the model's generalization and practical application.

[0007] In the existing technology, there is a lack of an efficient and accurate cross-scene belt defect recognition method based on typical sample learning that only requires a small amount of annotation. Summary of the Invention

[0008] To address the technical issues of existing technologies in adapting to new scenarios, computing resource efficiency, and learning with a small number of samples, the present invention provides a cross-scenario belt defect recognition method and device based on adaptive typical sample learning technology. The technical solution is as follows:

[0009] On the one hand, a cross-scene belt defect recognition method based on adaptive typical sample learning technology is provided. The method is implemented by a cross-scene belt defect recognition device, and the method includes:

[0010] Use cameras to capture images of belts in various industrial scenarios, build supporting image datasets, and query image datasets;

[0011] Inputting the supporting image dataset and the query image dataset into a defect recognition basic model for feature extraction to obtain a first supporting image feature and a first query image feature;

[0012] Inputting the first supporting image feature and the first query image feature into a prototype adaptive model for feature enhancement to obtain a third supporting image feature and a third query image feature;

[0013] The step of inputting the first supporting image feature and the first query image feature into the prototype adaptive model for feature enhancement to obtain the third supporting image feature and the third query image feature includes:

[0014] Inputting the first supporting image feature and the first query image feature into a prototype enhancement module for feature enhancement to obtain a second supporting image feature and a second query image feature;

[0015] Inputting the second supporting image feature and the second query image feature into a learnable adaptive module for adaptive adjustment to obtain a third supporting image feature and a third query image feature;

[0016] Inputting the third supporting image features and the third query image features into a target detection module to perform target detection to obtain a detection defect image dataset; constructing a loss function based on the supporting image dataset, the query image dataset, and the detection defect image dataset;

[0017] Optimizing the prototype adaptive model according to the loss function to obtain an optimized prototype adaptive model;

[0018] Acquire a belt image dataset to be identified; and perform belt defect identification based on the defect identification basic model and the optimized prototype adaptive model according to the belt image dataset to be identified.

[0019] On the other hand, a cross-scene belt defect recognition device based on adaptive typical sample learning technology is provided. The device is applied to a cross-scene belt defect recognition method based on adaptive typical sample learning technology. The device includes:

[0020] The dataset construction module is used to capture images of belts in various industrial scenes using a camera, build supporting image datasets, and query image datasets;

[0021] A feature extraction module, configured to input the supporting image dataset and the query image dataset into a defect recognition basic model for feature extraction, thereby obtaining a first supporting image feature and a first query image feature;

[0022] a feature enhancement module, configured to input the first supporting image feature and the first query image feature into a prototype adaptive model for feature enhancement to obtain a third supporting image feature and a third query image feature;

[0023] Wherein, the feature enhancement module is further used to:

[0024] Inputting the first supporting image feature and the first query image feature into a prototype enhancement module for feature enhancement to obtain a second supporting image feature and a second query image feature;

[0025] Inputting the second supporting image feature and the second query image feature into a learnable adaptive module for adaptive adjustment to obtain a third supporting image feature and a third query image feature;

[0026] a loss function construction module, configured to input the third supporting image features and the third query image features into a target detection module for target detection to obtain a detection defect image dataset; and construct a loss function based on the supporting image dataset, the query image dataset, and the detection defect image dataset;

[0027] A model optimization module, configured to optimize the prototype adaptive model according to the loss function to obtain an optimized prototype adaptive model;

[0028] The belt defect recognition module is used to obtain a belt image data set to be recognized; and perform belt defect recognition based on the belt image data set to be recognized and the defect recognition basic model and the optimized prototype adaptive model.

[0029] On the other hand, a cross-scene belt defect recognition device is provided, which includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned cross-scene belt defect recognition methods based on adaptive typical sample learning technology is implemented.

[0030] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned cross-scene belt defect recognition methods based on adaptive typical sample learning technology.

[0031] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0032] The present invention proposes a cross-scenario belt defect recognition method based on adaptive typical sample learning technology. To address the problem that existing belt defect recognition basic models cannot be directly adapted to new scenarios due to structural differences, a plug-and-play modular fine-tuning method is proposed, which can flexibly adapt to belt defect recognition algorithms of various structures, significantly improving the versatility and migration capabilities of the model. To solve the problem of excessive computational resource consumption during full-scale fine-tuning of the belt defect recognition model, a learnable adaptive module is designed. By combining downsampling and upsampling projections of features, the fine-tuning computational effort is effectively reduced, while accurately capturing belt defect features in new scenarios. To address the limitations of belt defect recognition algorithms in training with a small amount of labeled data, a prototype enhancement module is proposed to construct a defect category prototype library. The module achieves efficient matching of feature extraction and category information through a similarity mechanism. While learning new defect category information, the feature information of the original category is fully retained, significantly improving detection performance in typical sample scenarios. The present invention is an efficient and accurate cross-scenario belt defect recognition method based on typical sample learning that requires only a small amount of annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

[0034] Figure 1 is a cross-scene belt defect recognition method flowchart based on an adaptive typical sample learning technology provided by the embodiments of the present application.

[0035] Figure 2 is a prototype enhancement module structure schematic diagram provided by the embodiments of the present application.

[0036] Figure 3 is a learnable adaptive module structure schematic diagram provided by the embodiments of the present application.

[0037] Figure 4 is a target detection module structure schematic diagram provided by the embodiments of the present application.

[0038] Figure 5 is a cross-scene belt defect recognition device block diagram based on an adaptive typical sample learning technology provided by the embodiments of the present application.

[0039] Figure 6 is a cross-scene belt defect recognition device structure schematic diagram provided by the embodiments of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the present application will be described below with reference to the drawings.

[0041] In the embodiments of the present application, the words such as "example", "for example" are used to represent as an example, illustration or description. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0042] In the embodiments of the present application, "image" and "picture" can be used interchangeably at times. It should be pointed out that when the distinction is not emphasized, the meanings expressed are consistent. "Of", "corresponding" and "corresponding" can be used interchangeably at times. It should be pointed out that when the distinction is not emphasized, the meanings expressed are consistent.

[0043] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0044] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0045] The embodiment of the present invention provides a cross-scene belt defect recognition method based on adaptive typical sample learning technology, which can be implemented by a cross-scene belt defect recognition device, which can be a terminal or a server. Figure 1 The flowchart of the cross-scenario belt defect recognition method based on the adaptive typical sample learning technology is shown. The processing flow of the method may include the following steps:

[0046] S1. Use cameras to capture images of belts in various industrial scenes, build support image datasets and query image datasets.

[0047] Optionally, use a camera to capture images of belts in various industrial scenes, build a supporting image dataset, and query an image dataset, including:

[0048] The camera is used to shoot belts in various industrial scenes to obtain a belt image dataset containing normal states and various defect states;

[0049] Label the belt image dataset to obtain a typical defect sample dataset;

[0050] The typical defect sample dataset is divided according to a preset ratio to obtain a supporting image dataset and a query image dataset.

[0051] In a feasible implementation, in a belt working scenario, a 2048-resolution linear array camera (with an image acquisition resolution of 2048×1) is used to photograph the running belt. 2000 consecutive frames are taken at a certain frequency to form a 2048×2000 high-resolution two-dimensional belt tear image. Defective and non-defective images are screened to construct a data set.

[0052] The defects in the above image are divided into C different defect categories according to different shape, texture features and severity. The belt image is annotated using the labelimg tool. The annotation format uses the YOLO target detection annotation format (c t ,x t ,y t ,h t ,w t), where subscript t represents the true value of the label, c represents the defect category number, and the range is (0, C), (x, y) represents the center coordinates of the target detection frame, (h, w) represents the height and width of the target detection frame, x±w / 2 should be in the range of 0 to 2048, and y±h / 2 should be in the range of 0 to 2000. The output is stored in txt format.

[0053] The typical defect sample dataset consists of a support image dataset and a query image dataset, each consisting of images and target detection labels. The support dataset provides feature information about target categories, helping the model identify similar samples during inference. The model extracts features from samples in the support set, which are used to construct feature representations of the categories and provide a comparison "template" for the model. The query dataset is the set of images that the model needs to detect, used to evaluate the model's ability to identify typical belt defect samples. The distribution of targets in the query set does not overlap with samples in the support set, in order to enhance the model's generalization capabilities. Finally, the data undergoes image enhancement, such as flipping and translation, and data normalization before being fed into the model for training.

[0054] S2. Input the supporting image dataset and the query image dataset into the defect recognition basic model for feature extraction to obtain a first supporting image feature and a first query image feature.

[0055] In a feasible implementation, the present invention uses an insert module to fine-tune the existing belt defect recognition basic model. This structural design makes the present invention flexible and applicable to belt defect recognition models of different structures in new scenarios. The input data is processed layer by layer by the trained backbone network basic model to extract the first support image feature and the first query image feature, which are denoted as F and F respectively. s_n and F q_n .

[0056] Among them, the basic model of defect recognition is a pre-trained deep neural network.

[0057] In a feasible implementation, the defect recognition basic model has been fully trained on a large-scale data set and has good feature extraction capabilities. This model is used as the basic model for the subsequent adaptive belt defect typical sample learning stage.

[0058] S3, inputting the first supporting image feature and the first query image feature into the prototype adaptive model for feature enhancement to obtain a third supporting image feature and a third query image feature;

[0059] The first supporting image feature and the first query image feature are input into the prototype adaptive model for feature enhancement to obtain the third supporting image feature and the third query image feature, including:

[0060] Inputting the first supporting image feature and the first query image feature into the prototype enhancement module for feature enhancement to obtain the second supporting image feature and the second query image feature;

[0061] The second supporting image features and the second query image features are input into a learnable adaptive module for adaptive adjustment to obtain third supporting image features and third query image features.

[0062] In a feasible implementation, the prototype adaptive model proposed in the present invention consists of a prototype enhancement module and a learnable adaptive module. The former aims to register the defect category prototypes in the support image and enhance the model's ability to express the features of the support image and the query image. The latter aims to enable the model to effectively adapt to the defect feature extraction in new scenarios through training.

[0063] Optionally, inputting the first supporting image feature and the first query image feature into a prototype enhancement module for feature enhancement to obtain a second supporting image feature and a second query image feature includes:

[0064] Based on the preset target detection frame, the defect local details are cropped according to the supporting image dataset to obtain the defect local image dataset;

[0065] Performing element-by-element multiplication on the first support image feature and the defect local image dataset to obtain a first category prototype;

[0066] Based on multiple category prototypes in the defect category prototype warehouse, similarity measurement is performed based on the first category prototype to obtain the second category prototype;

[0067] Perform weighted calculation based on the first category prototype and the second category prototype to obtain the third category prototype;

[0068] Based on the third category prototype, feature enhancement is performed according to the first supporting image feature and the first query image feature to obtain the second supporting image feature and the second query image feature.

[0069] In a feasible implementation, the prototype enhancement module is a key component of the prototype adaptive model. The module structure is as follows: Figure 2 As shown in the figure, the main function is to dynamically enhance the category features of the supporting samples so that they can more effectively represent the target category and improve the performance of identifying typical belt defect samples.

[0070] The input of the prototype enhancement module is the first support image feature F s_n and the first query image feature F q_n , by calculating the support image feature F s_n and support image defect cropping I s_cThe first category prototype P of the target class is obtained by multiplying the defect local detail image (cropped from the support image according to the target detection frame annotated in the support image) t .

[0071] Calculate P t and the existing defect category prototypes in the prototype warehouse (P1, P2, ..., P n ) and select the second category prototype P corresponding to the i-th category with the highest score i Used for subsequent enhancement of image features. During the training phase, P t and P i The weighted sum of the third category prototypes is used as the new P i Through continuous training, the category prototypes in the defect category prototype warehouse become increasingly accurate. This method dynamically updates the defect category prototype warehouse to support the model's ability to generalize to multiple categories in new scenarios. While retaining the original defect category feature information extraction capability, it also updates and registers new defect category information.

[0072] Optionally, based on the third category prototype, feature enhancement is performed according to the first supporting image feature and the first query image feature to obtain the second supporting image feature and the second query image feature, including:

[0073] Perform matrix multiplication based on the first support image feature and the third category prototype to obtain a support image similarity matrix; perform calculation based on the first support image feature and the support image similarity matrix to obtain an enhanced support image feature;

[0074] Performing matrix multiplication based on the first query image feature and the third category prototype to obtain a query image similarity matrix; performing calculation based on the first query image feature and the query image similarity matrix to obtain an enhanced query image feature;

[0075] Adding the first supporting image feature and the enhanced supporting image feature element by element to obtain a fused supporting image feature;

[0076] Adding the first query image feature and the enhanced query image feature element by element to obtain a fused query image feature;

[0077] The fused support image features and the fused query image features are input into the defect recognition basic model for high-dimensional feature extraction to obtain the second support image features and the second query image features.

[0078] In a feasible implementation, based on the third category prototype obtained in the above steps, the module measures the first supporting image feature F by similarity. s_n and the first query image feature F q_n , respectively with the category prototype P iPerform association calculations to generate prototype enhancement features corresponding to defect categories.

[0079] Image feature F s_n Or the image feature F q_n With the third category prototype P i Matrix multiplication is performed to obtain a similarity matrix, which is then used as a weight for dot multiplication with the image features to obtain image features enhanced by the category prototypes. This process guides feature adjustment by using the defect category prototype representations of the support samples, thereby improving the correlation between the query features and the support features.

[0080] The enhanced image features of the third category prototype are added to the original image features for feature fusion, generating support image features or query image features containing category prototype information. This layer-by-layer feature enhancement and fusion strategy fully exploits the defect characteristics of the support and query images, improving the accuracy and robustness of feature representation. The fused features are then fed into subsequent network layers to extract higher-level semantic information.

[0081] Optionally, inputting the second supporting image feature and the second query image feature into a learnable adaptive module for adaptive adjustment to obtain a third supporting image feature and a third query image feature includes:

[0082] According to the second supporting image features and the second query image features, a dimensionality reduction mapping is performed through a downsampling projection layer to obtain a reduced dimensionality supporting image feature and a reduced dimensionality query image feature;

[0083] According to the dimensionality reduction support image features and the dimensionality reduction query image features, the ReLU activation function is used to process them to obtain the activated support image features and the activated query image features;

[0084] According to the activated supporting image features and the activated query image features, high-dimensional mapping is performed through an upsampling projection layer to obtain a third supporting image feature and a third query image feature.

[0085] In a feasible implementation, the learnable adaptive module is another important component of the prototype adaptive model, which aims to further optimize the expressive power of the support image features and query image features, so that they can be adaptively adjusted in a variety of scenarios to enhance the defect recognition and detection effect. Figure 3 The module input is the second support image features and the second query image features containing the category prototype information. These features have been processed by the prototype enhancement module and have preliminary category relevance.

[0086] The feature input passes through the downsampling projection layer, which uses dimensionality reduction to map high-dimensional features into a more compact feature space, reducing the interference of redundant information and extracting more essential feature representations. Downsampling enhances feature focus, facilitating subsequent nonlinear activation. This module models the nonlinear relationships between features through the ReLU activation function. The ReLU activation function effectively enhances the model's expressive power while suppressing negative features, improving feature sparsity and robustness.

[0087] The activated features are restored through the upsampling projection layer, remapping the reduced features back to the original feature space dimensions. The upsampling projection layer introduces additional adaptive learning capabilities during feature recovery, enabling the output features to dynamically adjust while preserving the original feature structure.

[0088] The second support image feature and the second query image feature are adaptively adjusted by the module, and the enhanced features and the third support image feature F are output respectively. s_n * and the third query image feature F q_n * These features integrate category characteristics and context information at a higher level, further improving the accuracy of feature representation and classification ability.

[0089] S4. Input the third supporting image features and the third query image features into the target detection module for target detection to obtain a detection defect image dataset; and construct a loss function based on the supporting image dataset, the query image dataset, and the detection defect image dataset.

[0090] In a feasible embodiment, the main function of the target detection module in the present invention is to generate a detection result (c o ,x o ,y o ,h o ,w o ), including the category label c o 、Target center coordinates (x o ,y o ) and bounding box size (h o ,w o ), the module structure is as follows Figure 4 shown.

[0091] The input feature map is processed by 1×1 convolution to extract high-level semantic information and target location information. These features are processed by the category prediction branch and the bounding box regression branch to output the final output result. Among them, the category prediction branch is responsible for determining the type of target, using two 3×3 convolutions and one 1×1 convolution to extract the category information of the image, calculate the classification probability distribution of the target, and generate an H×W×C classification result; the bounding box regression branch generates the position and size parameters of the target through regression analysis to determine the precise position of the target in the query image; first, two 3×3 convolutions are used for feature screening, and then a 1×1 convolution is used to output H×W×4 position information and H×W×1 intersection-over-union score, which are organized as (c o ,x o ,y o ,h o ,w o )’s defect detection output.

[0092] S5. Optimize the prototype adaptive model according to the loss function to obtain an optimized prototype adaptive model.

[0093] In a feasible implementation, the loss function reflects the difference between the model's detection results for the query image and the support image and the corresponding target detection labels. The present invention splits the leather defect recognition task into two tasks: category prediction and bounding box regression. Therefore, the overall loss function of the present invention includes two parts: classification loss and regression loss, which are combined in a weighted manner. The classification loss function is a binary cross entropy loss function, and the regression loss function is a complete intersection-over-union loss function. In the typical belt defect sample recognition task, the present invention optimizes the back propagation of the prototype adaptive model based on the loss function, taking into account the accuracy of classification and positioning.

[0094] S6. Obtain a data set of belt images to be identified; perform belt defect identification based on the data set of belt images to be identified and the defect identification basic model and the optimized prototype adaptive model.

[0095] In one feasible implementation, the present invention leverages the synergy of the support and query sets during training, guiding the model to learn key features of the target category using a small number of labeled samples, thereby maximizing generalization capabilities for small sample categories. During the inference phase, the model leverages the learned class prototype features and the small amount of labeled support data to effectively identify defect categories in new scenarios.

[0096] The present invention proposes a cross-scenario belt defect recognition method based on adaptive typical sample learning technology. To address the problem that existing belt defect recognition basic models cannot be directly adapted to new scenarios due to structural differences, a plug-and-play modular fine-tuning method is proposed, which can flexibly adapt to belt defect recognition algorithms of various structures, greatly improving the versatility and migration capabilities of the model. To solve the problem of excessive computational resource consumption during full-scale fine-tuning of the belt defect recognition model, a learnable adaptive module is designed. By combining downsampling and upsampling projections of features, the fine-tuning computational effort is effectively reduced, while accurately capturing belt defect features in new scenarios. To address the limitations of belt defect recognition algorithms in training with a small amount of labeled data, a prototype enhancement module is proposed to construct a defect category prototype library. The module achieves efficient matching of feature extraction and category information through a similarity mechanism. While learning new defect category information, the feature information of the original category is fully retained, significantly improving detection performance in typical sample scenarios. The present invention is an efficient and accurate cross-scenario belt defect recognition method based on typical sample learning that requires only a small amount of annotation.

[0097] Figure 5 This is a block diagram of a cross-scene belt defect recognition device based on adaptive typical sample learning technology according to an exemplary embodiment. The device is used in a cross-scene belt defect recognition method based on adaptive typical sample learning technology. Figure 5 The device includes a data set construction module 510, a feature extraction module 520, a feature enhancement module 530, a loss function construction module 540, a model optimization module 550 and a belt defect recognition module 560.

[0098] A data set construction module 510 is used to capture images of belts in various industrial scenes using a camera, and to construct a supporting image data set and a query image data set;

[0099] A feature extraction module 520 is configured to input the supporting image dataset and the query image dataset into the defect recognition basic model for feature extraction to obtain a first supporting image feature and a first query image feature;

[0100] A feature enhancement module 530 is configured to input the first supporting image feature and the first query image feature into the prototype adaptive model for feature enhancement to obtain a third supporting image feature and a third query image feature;

[0101] The feature enhancement module is further used to:

[0102] Inputting the first supporting image feature and the first query image feature into the prototype enhancement module for feature enhancement to obtain the second supporting image feature and the second query image feature;

[0103] Inputting the second supporting image feature and the second query image feature into the learnable adaptive module for adaptive adjustment to obtain a third supporting image feature and a third query image feature;

[0104] A loss function construction module 540 is configured to input the third supporting image features and the third query image features into the target detection module for target detection to obtain a detection defect image dataset; and construct a loss function based on the supporting image dataset, the query image dataset, and the detection defect image dataset.

[0105] A model optimization module 550 is used to optimize the prototype adaptive model according to the loss function to obtain an optimized prototype adaptive model;

[0106] The belt defect recognition module 560 is used to obtain a belt image dataset to be recognized; and perform belt defect recognition based on the belt image dataset to be recognized based on a defect recognition basic model and an optimized prototype adaptive model.

[0107] Optionally, the data set construction module 510 is further configured to:

[0108] The camera is used to shoot belts in various industrial scenes to obtain a belt image dataset containing normal states and various defect states;

[0109] Label the belt image dataset to obtain a typical defect sample dataset;

[0110] The typical defect sample dataset is divided according to a preset ratio to obtain a supporting image dataset and a query image dataset.

[0111] Among them, the basic model of defect recognition is a pre-trained deep neural network.

[0112] Optionally, the feature enhancement module 530 is further configured to:

[0113] Based on the preset target detection frame, the defect local details are cropped according to the supporting image dataset to obtain the defect local image dataset;

[0114] Performing element-by-element multiplication on the first support image feature and the defect local image dataset to obtain a first category prototype;

[0115] Based on multiple category prototypes in the defect category prototype warehouse, similarity measurement is performed based on the first category prototype to obtain the second category prototype;

[0116] Perform weighted calculation based on the first category prototype and the second category prototype to obtain the third category prototype;

[0117] Based on the third category prototype, feature enhancement is performed according to the first supporting image feature and the first query image feature to obtain the second supporting image feature and the second query image feature.

[0118] Optionally, the feature enhancement module 530 is further configured to:

[0119] Perform matrix multiplication based on the first support image feature and the third category prototype to obtain a support image similarity matrix; perform calculation based on the first support image feature and the support image similarity matrix to obtain an enhanced support image feature;

[0120] Performing matrix multiplication based on the first query image feature and the third category prototype to obtain a query image similarity matrix; performing calculation based on the first query image feature and the query image similarity matrix to obtain an enhanced query image feature;

[0121] Adding the first supporting image feature and the enhanced supporting image feature element by element to obtain a fused supporting image feature;

[0122] Adding the first query image feature and the enhanced query image feature element by element to obtain a fused query image feature;

[0123] The fused support image features and the fused query image features are input into the defect recognition basic model for high-dimensional feature extraction to obtain the second support image features and the second query image features.

[0124] Optionally, the feature enhancement module 530 is further configured to:

[0125] According to the second supporting image features and the second query image features, a dimensionality reduction mapping is performed through a downsampling projection layer to obtain a reduced dimensionality supporting image feature and a reduced dimensionality query image feature;

[0126] According to the dimensionality reduction support image features and the dimensionality reduction query image features, the ReLU activation function is used to process them to obtain the activated support image features and the activated query image features;

[0127] According to the activated supporting image features and the activated query image features, high-dimensional mapping is performed through an upsampling projection layer to obtain a third supporting image feature and a third query image feature.

[0128] The present invention proposes a cross-scenario belt defect recognition method based on adaptive typical sample learning technology. To address the problem that existing belt defect recognition basic models cannot be directly adapted to new scenarios due to structural differences, a plug-and-play modular fine-tuning method is proposed, which can flexibly adapt to belt defect recognition algorithms of various structures, significantly improving the versatility and migration capabilities of the model. To solve the problem of excessive computational resource consumption during full-scale fine-tuning of the belt defect recognition model, a learnable adaptive module is designed. By combining downsampling and upsampling projections of features, the fine-tuning computational effort is effectively reduced, while accurately capturing belt defect features in new scenarios. To address the limitations of belt defect recognition algorithms in training with a small amount of labeled data, a prototype enhancement module is proposed to construct a defect category prototype library. The module achieves efficient matching of feature extraction and category information through a similarity mechanism. While learning new defect category information, the feature information of the original category is fully retained, significantly improving detection performance in typical sample scenarios. The present invention is an efficient and accurate cross-scenario belt defect recognition method based on typical sample learning that requires only a small amount of annotation.

[0129] Figure 6 This is a schematic diagram of the structure of a cross-scenario belt defect recognition device provided by an embodiment of the present invention. Figure 6 As shown, the cross-scene belt defect recognition device may include the above Figure 5 The cross-scene belt defect recognition device based on the adaptive typical sample learning technology is shown. Optionally, the cross-scene belt defect recognition device 610 may include a first processor 2001.

[0130] Optionally, the cross-scene belt defect identification device 610 may also include a memory 2002 and a transceiver 2003.

[0131] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0132] The following combination Figure 6 The following is a detailed introduction to the various components of the cross-scenario belt defect recognition device 610:

[0133] The first processor 2001 is the control center of the cross-scenario belt defect identification device 610 and can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), or application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0134] Optionally, the first processor 2001 can perform various functions of the cross-scene belt defect identification device 610 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0135] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 6 CPU0 and CPU1 are shown in FIG.

[0136] In a specific implementation, as an embodiment, the cross-scene belt defect recognition device 610 may also include multiple processors, such as Figure 6 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0137] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0138] Optionally, the memory 2002 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 can be integrated with the first processor 2001, or it can exist independently and be connected to the first processor 2001 through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0139] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0140] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 6 (not shown separately in the figure). The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0141] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or can exist independently and be connected to the first processor 2001 through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0142] It should be noted that Figure 6 The structure of the cross-scene belt defect identification device 610 shown in the figure does not constitute a limitation on the router. The actual knowledge structure identification device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0143] In addition, the technical effects of the cross-scene belt defect recognition device 610 can refer to the technical effects of the cross-scene belt defect recognition method based on adaptive typical sample learning technology described in the above method embodiment, and will not be repeated here.

[0144] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0145] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0146] The above embodiments can be implemented in whole or in part through software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired method (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, or magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0147] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0148] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0149] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0150] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0151] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0152] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0153] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0154] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0155] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0156] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A cross-scenario belt defect recognition method based on adaptive typical sample learning technology, characterized in that: The method comprises: Use cameras to capture images of belts in various industrial scenarios, build supporting image datasets, and query image datasets; Inputting the supporting image dataset and the query image dataset into a defect recognition basic model for feature extraction to obtain a first supporting image feature and a first query image feature; Inputting the first supporting image feature and the first query image feature into a prototype adaptive model for feature enhancement to obtain a third supporting image feature and a third query image feature; The step of inputting the first supporting image feature and the first query image feature into the prototype adaptive model for feature enhancement to obtain the third supporting image feature and the third query image feature includes: Based on a preset target detection frame, cropping the defect local details according to the supporting image dataset to obtain a defect local image dataset; Performing element-by-element multiplication on the first supporting image feature and the defect local image dataset to obtain a first category prototype; Based on multiple category prototypes in the defect category prototype warehouse, similarity measurement is performed according to the first category prototype to obtain a second category prototype; Perform weighted calculation based on the first category prototype and the second category prototype to obtain a third category prototype; Based on the third category prototype, performing feature enhancement according to the first supporting image feature and the first query image feature to obtain a second supporting image feature and a second query image feature; Inputting the second supporting image feature and the second query image feature into a learnable adaptive module for adaptive adjustment to obtain a third supporting image feature and a third query image feature; Inputting the third supporting image features and the third query image features into a target detection module to perform target detection to obtain a detection defect image dataset; constructing a loss function based on the supporting image dataset, the query image dataset, and the detection defect image dataset; Optimizing the prototype adaptive model according to the loss function to obtain an optimized prototype adaptive model; Acquire a belt image dataset to be identified; and perform belt defect identification based on the defect identification basic model and the optimized prototype adaptive model according to the belt image dataset to be identified.

2. The cross-scenario belt defect recognition method based on adaptive typical sample learning technology according to claim 1 is characterized in that: The method uses a camera to capture images of belts in various industrial scenes, builds a supporting image dataset, and constructs a query image dataset, including: The camera is used to shoot belts in various industrial scenes to obtain a belt image dataset containing normal states and various defect states; Annotating the belt image dataset to obtain a typical defect sample dataset; The typical defect sample dataset is divided according to a preset ratio to obtain a supporting image dataset and a query image dataset.

3. The cross-scenario belt defect recognition method based on adaptive typical sample learning technology according to claim 1 is characterized in that: The defect recognition basic model is a pre-trained deep neural network.

4. The cross-scenario belt defect recognition method based on adaptive typical sample learning technology according to claim 1 is characterized in that: The step of performing feature enhancement based on the third category prototype and the first supporting image feature and the first query image feature to obtain the second supporting image feature and the second query image feature includes: Perform matrix multiplication based on the first supporting image feature and the third category prototype to obtain a supporting image similarity matrix; perform calculation based on the first supporting image feature and the supporting image similarity matrix to obtain an enhanced supporting image feature; Performing matrix multiplication on the first query image feature and the third category prototype to obtain a query image similarity matrix; performing calculation on the first query image feature and the query image similarity matrix to obtain an enhanced query image feature; Adding the first supporting image feature and the enhanced supporting image feature element by element to obtain a fused supporting image feature; Adding the first query image feature and the enhanced query image feature element by element to obtain a fused query image feature; The fused support image features and the fused query image features are input into the defect recognition basic model for high-dimensional feature extraction to obtain second support image features and second query image features.

5. The cross-scenario belt defect recognition method based on adaptive typical sample learning technology according to claim 1 is characterized in that: The step of inputting the second supporting image feature and the second query image feature into a learnable adaptive module for adaptive adjustment to obtain a third supporting image feature and a third query image feature includes: According to the second supporting image features and the second query image features, performing dimensionality reduction mapping through a downsampling projection layer to obtain reduced-dimensionality supporting image features and reduced-dimensionality query image features; According to the reduced dimensionality support image features and the reduced dimensionality query image features, processing is performed through a ReLU activation function to obtain activated support image features and activated query image features; According to the activated supporting image features and the activated query image features, high-dimensional mapping is performed through an upsampling projection layer to obtain a third supporting image feature and a third query image feature.

6. A cross-scene belt defect recognition device based on adaptive typical sample learning technology, wherein the cross-scene belt defect recognition device based on adaptive typical sample learning technology is used to implement the cross-scene belt defect recognition method based on adaptive typical sample learning technology as described in any one of claims 1 to 5, characterized in that: The device comprises: The dataset construction module is used to capture images of belts in various industrial scenes using a camera, build supporting image datasets, and query image datasets; A feature extraction module, configured to input the supporting image dataset and the query image dataset into a defect recognition basic model for feature extraction, thereby obtaining a first supporting image feature and a first query image feature; a feature enhancement module, configured to input the first supporting image feature and the first query image feature into a prototype adaptive model for feature enhancement to obtain a third supporting image feature and a third query image feature; Wherein, the feature enhancement module is further used to: Inputting the first supporting image feature and the first query image feature into a prototype enhancement module for feature enhancement to obtain a second supporting image feature and a second query image feature; Inputting the second supporting image feature and the second query image feature into a learnable adaptive module for adaptive adjustment to obtain a third supporting image feature and a third query image feature; a loss function construction module, configured to input the third supporting image features and the third query image features into a target detection module for target detection to obtain a detection defect image dataset; and construct a loss function based on the supporting image dataset, the query image dataset, and the detection defect image dataset; A model optimization module, configured to optimize the prototype adaptive model according to the loss function to obtain an optimized prototype adaptive model; The belt defect recognition module is used to obtain a belt image data set to be recognized; and perform belt defect recognition based on the belt image data set to be recognized and the defect recognition basic model and the optimized prototype adaptive model.

7. A cross-scene belt defect recognition device, characterized in that: The cross-scene belt defect recognition device includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 5.

Images