An image processing method, system, device and medium based on small sample learning

Abstract

The application discloses an image processing method and system based on small sample learning, a device and a medium, relates to the technical fields of computer vision, meta-learning and industrial intelligent detection, and obtains a target image with a known defect label; obtains a query image for defect identification of an industrial device; encodes the target image based on the known defect label to obtain preliminary features through context-aware feature encoding; extracts features from the preliminary features to obtain known features; extracts features from the query image to obtain query features; and performs interactive weight deduction on the known features and the query features to obtain dynamic segmentation parameters; and processes the query features based on the dynamic segmentation parameters to generate a target defect label of the query image. In the application, more essential defect representations can be learned through context-aware feature encoding, and the accuracy of defect identification of the industrial device is improved; and the robustness of defect identification of the industrial device is improved through interactive weight deduction.

Description

Technical Field

[0001] This application relates to the fields of computer vision, meta-learning and industrial intelligent detection technology, and more specifically, to an image processing method, system, device and medium based on few-shot learning. Background Technology

[0002] In precision manufacturing fields such as lithium battery manufacturing and nuclear material monitoring, the accurate identification of microscopic defects is a core bottleneck in ensuring safety and performance. These defects exhibit a "long-tail distribution" characteristic, with numerous types but scarce samples of each type, making traditional deep learning methods, which rely on massive amounts of data, costly and difficult to apply. Therefore, "few-shot learning" technology, which can learn from a small number of samples, has become a key breakthrough direction.

[0003] However, existing methods have fundamental limitations. First, in order to focus on the target, they often crudely strip away its local context through hard masking, ignoring the intrinsic relationship between defects and the substrate material. This "context isolation" prevents the model from learning the relationship between the environment and the target, resulting in poor generalization ability when detecting scene changes and a high likelihood of misjudgment. Second, the parameter generation process is static and unidirectional, relying solely on samples and completely "blindly" ignoring the real-time features of the image to be detected. This "static reasoning" mechanism lacks dynamic adaptability to the scene and cannot adjust according to changes in lighting, noise, etc., resulting in severely insufficient model robustness and making it unsuitable for demanding industrial applications.

[0004] In conclusion, improving the accuracy and robustness of defect identification for industrial devices is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide an image processing method based on few-shot learning, which can, to some extent, address the technical problem of how to improve the accuracy and robustness of defect identification in industrial devices. This application also provides an image processing system, electronic device, and computer-readable storage medium based on few-shot learning.

[0006] To achieve the above objectives, this application provides the following technical solution: An image processing method based on few-shot learning includes: Obtain the target image with known defect labels; Obtain query images for defect identification of industrial devices; Based on the known defect labels, the target image is subjected to context-aware feature encoding to obtain preliminary features; Feature extraction is performed on the preliminary features to obtain known features; Feature extraction is performed on the query image to obtain query features; Interactive weight deduction is performed on the known features and the query features to obtain dynamic segmentation parameters; The query features are processed based on the dynamic segmentation parameters to generate target defect labels for the query image.

[0007] Preferably, the step of performing context-aware feature encoding on the target image based on the known defect labels to obtain preliminary features includes: Based on the known defect labels, determine the minimum bounding rectangle of the defects in the target image; The minimum bounding rectangle is adaptively expanded by a target number of pixels to obtain the context region; The context region is subjected to central difference convolution processing to obtain preliminary features; The kernel of the central difference convolution is larger than the set kernel.

[0008] Preferably, the step of interactively weighting the known features and the query features to obtain dynamic segmentation parameters includes: The known features and the query features are deeply fused to generate fused features; The fusion features are subjected to sequence scanning calculations to obtain dynamic segmentation parameters.

[0009] Preferably, the step of performing sequence scanning calculation on the fused features to obtain dynamic segmentation parameters includes: Dynamic segmentation parameters are obtained by sequentially scanning the fusion features using a selective state-space model.

[0010] Preferably, the step of processing the query features based on the dynamic segmentation parameters to generate the target defect label of the query image includes: Logistic regression is performed on the dynamic segmentation parameters and the query features to obtain probability values; The probability value is upsampled to generate the target defect label of the query image.

[0011] Preferably, the step of extracting features from the preliminary features to obtain known features includes: The initial features are extracted using a convolutional network to obtain known features.

[0012] Preferably, the step of extracting features from the preliminary features to obtain known features includes: The initial features are extracted using a self-attention mechanism to obtain known features.

[0013] An image processing system based on few-shot learning includes: The conditional branch input module is used to obtain the target image with known defect labels; The segmented branch input module is used to acquire query images for defect identification of industrial devices; The context-aware feature encoding module is used to perform context-aware feature encoding on the target image based on the known defect labels to obtain preliminary features; The first feature extraction module is used to extract features from the preliminary features to obtain known features; The second feature extraction module is used to extract features from the query image to obtain query features; An interactive weight deduction module is used to perform interactive weight deduction on the known features and the query features to obtain dynamic segmentation parameters; The defect label generation module is used to process the query features based on the dynamic segmentation parameters to generate target defect labels for the query image.

[0014] An electronic device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the image processing method based on few-shot learning as described above.

[0015] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the image processing methods based on few-shot learning described above.

[0016] This application provides an image processing method based on few-shot learning, which involves: acquiring a target image with known defect labels; acquiring a query image for defect identification of industrial devices; performing context-aware feature encoding on the target image based on the known defect labels to obtain preliminary features; extracting features from the preliminary features to obtain known features; extracting features from the query image to obtain query features; performing interactive weight inference on the known features and query features to obtain dynamic segmentation parameters; and processing the query features based on the dynamic segmentation parameters to generate target defect labels for the query image. In this application, processing the target image with known defects through context-aware feature encoding allows learning more fundamental defect representations in the target image that are independent of background changes. This provides high-precision data for subsequent defect quantification analysis, fundamentally solving the model failure problem caused by environmental changes and improving the accuracy of defect identification of industrial devices. Furthermore, interactive weight inference generates dynamic segmentation parameters that adapt to both query features and known features, overturning existing static parameter generation mechanisms. This allows for adaptation to various complex working conditions and improves the robustness of defect identification of industrial devices. This application provides an image processing system, electronic device, and computer-readable storage medium based on few-shot learning, which also solves the corresponding technical problems. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an image processing method based on few-shot learning provided in this application embodiment; Figure 2 A flowchart illustrating the process of context-aware feature encoding; Figure 3 A schematic diagram of the structure of an image processing system based on few-shot learning provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; Figure 5 This is another structural schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] Please see Figure 1 , Figure 1 This is a flowchart of an image processing method provided in an embodiment of this application.

[0021] This application provides an image processing method based on few-shot learning, which can be applied to computing devices such as servers and can be implemented using a neural network model. The method includes the following steps: Step S101: Obtain the target image of the known defect label.

[0022] In practical applications, target images with known defect labels can be acquired so that the known defects in the target images can be used to identify defects in industrial devices. The known defect labels are used to describe the known defects in the target images. The content and type of the known defect labels can be flexibly determined according to the application scenario. For example, the known defect labels can be labels reflecting scratches, etc.

[0023] Step S102: Obtain the query image for defect identification of industrial devices.

[0024] In practical applications, query images can be acquired for defect identification of industrial devices, enabling image processing to identify defects within these devices. Understandably, the type of industrial device can be flexibly determined based on the application scenario. For example, industrial devices could be precision-manufactured components such as lithium batteries and nuclear materials. Furthermore, known defects in the target image can also be known defects in the industrial device, and the industrial devices in the target image and query image can be the same or different. For instance, the target image could be a scratch defect image rarely seen in the lithium battery industry, while the query image could be a scratch defect image rarely seen in the nuclear industry. In other words, for the pain point of "high precision, few samples" in the detection of precision-manufactured components such as lithium batteries and nuclear materials, applying the solution of this application for defect identification can significantly shorten the commissioning cycle of new production lines, improve product yield, and ensure operational safety under extreme conditions, thanks to its superior performance and data efficiency.

[0025] Step S103: Based on the known defect labels, perform context-aware feature encoding on the target image to obtain preliminary features.

[0026] Step S104: Extract features from the preliminary features to obtain known features.

[0027] In practical applications, to focus on features in the target image that are beneficial for defect identification in the query image, a hard mask can be used to crudely strip away its local context. However, this method ignores the inherent relationship between the defect and the substrate material, creating "contextual isolation." This prevents the model from learning the relationship between the environment and the target, resulting in poor generalization ability when detecting scene changes and a high risk of misjudgment. To avoid this, this application constructs a context-aware feature encoding mechanism that can jointly model the target and its local environment, forcing the model to learn the inherent relationship between the defect and the background. This injects robustness to environmental changes at the feature level, fundamentally improving the model's generalization ability. In other words, based on known defect labels, context-aware feature encoding can be performed on the target image to obtain preliminary features, breaking the semantic isolation between the target and the background and establishing a deep contextual relationship. Then, feature extraction is performed on the preliminary features to obtain known features, which can be used to identify defects in the query image later.

[0028] In an exemplary embodiment, during the process of obtaining preliminary features by performing context-aware feature encoding on the target image based on known defect labels, such as... Figure 2 As shown, the minimum bounding rectangle of the defect in the target image can be determined based on the known defect label. The minimum bounding rectangle is then adaptively expanded by the number of target pixels to obtain a context region that includes both the complete target and its key local background. The number of targets can be flexibly determined according to the application scenario. Central difference convolution is performed on the context region to obtain preliminary features. The kernel of the central difference convolution is larger than a set kernel, such as 5x5, 7x7, or 9x9, meaning that a central difference convolution with a large kernel is required for the context region. In this way, the fine-grained gradient capture capability of the central difference operator can enhance the microscopic differences between the defect edge and the substrate material, achieving "information enhancement." Furthermore, the large convolution kernel ensures a broad receptive field to capture a wider range of background texture and structural information, ultimately obtaining a high-dimensional "context-aware" feature vector containing rich target-environment relationships, laying a solid foundation for subsequent accurate defect inference.

[0029] In an exemplary embodiment, during the process of extracting features from preliminary features to obtain known features, features can be extracted from preliminary features through a convolutional network to obtain known features, or features can be extracted from preliminary features through a self-attention mechanism to obtain known features. This application does not make any specific limitations here.

[0030] Step S105: Extract features from the query image to obtain the query features.

[0031] In practical applications, feature extraction can be performed on the query image to obtain query features, which can then be used to identify defects in the query image. During this process, feature extraction can be performed using a convolutional network or a self-attention mechanism; this application does not impose any specific limitations on this method.

[0032] Step S106: Perform interactive weight deduction on the known features and query features to obtain dynamic segmentation parameters.

[0033] Step S107: Process the query features based on dynamic segmentation parameters to generate target defect labels for the query image.

[0034] In practical applications, target images are often used as samples to generate static segmentation parameters, completely ignoring the real-time features of the query image. This "static inference" mechanism lacks dynamic adaptability to the scene and cannot adjust according to changes in lighting, noise, etc., resulting in severely insufficient model robustness. To avoid this, this application establishes a bidirectional information path between the query image and the target image, enabling the real-time features of the query image to "guide" the generation process of segmentation parameters, achieving dynamic adaptation of segmentation parameters during the inference process, thereby giving the model unprecedented scene robustness. That is, it can comprehensively perform interactive weighted inference on known features and query features to obtain dynamic segmentation parameters, giving the model the ability to "act according to circumstances" through an interactive dynamic inference framework, enabling it to adapt to various complex working conditions and improve robustness. Then, the query features are processed based on the dynamic segmentation parameters to generate target defect labels for the query image.

[0035] In an exemplary embodiment, during the interactive weight deduction of known features and query features to obtain dynamic segmentation parameters, deep fusion of the known and query features can be performed, such as matrix multiplication, to generate fused features. This deep fusion enables cross-modal information alignment between the known and query features, forcing the model to understand the correlation and differences between the "scene to be detected" and "sample knowledge." Subsequently, the fused features are used for sequence scanning calculation to obtain dynamic segmentation parameters. In this way, by organically combining fine local contextual information with a grand global scene understanding, sub-pixel-level precise delineation of defect boundaries can be achieved. This is not merely segmentation, but a high-fidelity "digital twin" of the true physical form of the defect, providing unprecedented data accuracy for subsequent defect quantitative analysis.

[0036] In specific application scenarios, during the process of performing sequence scanning calculations on fused features to obtain dynamic segmentation parameters, a selective state-space model can be used to perform sequence scanning calculations on the fused features to obtain dynamic segmentation parameters. That is, the fused features can be regarded as an information sequence, input into a selective state-space model. Leveraging its sequence scanning calculations and its exceptional ability to capture long-distance dependencies with linear complexity, the selective state-space model performs global sequence calculations on the fused features. This allows for in-depth deduction of the complex problem of "how to optimally utilize sample knowledge (target image) based on the current scenario (query image)" from a higher dimension and broader perspective. The final output of the selective state-space model is a set of optimal segmenter parameters "tailor-made" for the current query image, which may include weights and biases. The generation of this set of parameters is a dynamic deduction completed within an abstract parameter space, conditioned on the query image. This signifies that the model has evolved from a fixed "tool" into an "intelligent agent" with reasoning capabilities.

[0037] In an exemplary embodiment, during the process of processing query features based on dynamic segmentation parameters to generate target defect labels for the query image, the dynamic segmentation parameters can be deployed in the final classification layer of the segmentation branch. This classification layer acts on the feature map of the query image, thus outputting pixel-level segmentation results for the defect region with extremely high topological fidelity. That is, logistic regression calculation is performed on the dynamic segmentation parameters and query features to obtain probability values. The calculation formula for logistic regression can be... Where p represents probability, q represents query image, Fq represents feature map of query image, and m and n represent coordinate positions. Represents dynamic weights. Represents the softmax function. The weight is multiplied by the pixel value at coordinate (m,n) on the feature map; the probability value is upsampled to generate the target defect label of the query image.

[0038] This application provides an image processing method based on few-shot learning, which involves: acquiring a target image with known defect labels; acquiring a query image for defect identification of industrial devices; performing context-aware feature encoding on the target image based on the known defect labels to obtain preliminary features; extracting features from the preliminary features to obtain known features; extracting features from the query image to obtain query features; performing interactive weight inference on the known features and query features to obtain dynamic segmentation parameters; and processing the query features based on the dynamic segmentation parameters to generate target defect labels for the query image. In this application, processing the target image with known defects through context-aware feature encoding allows learning more fundamental defect representations in the target image that are independent of background changes. This provides high-precision data for subsequent defect quantification analysis, fundamentally solving the model failure problem caused by environmental changes and improving the accuracy of defect identification of industrial devices. Furthermore, interactive weight inference generates dynamic segmentation parameters that adapt to both query features and known features, overturning existing static parameter generation mechanisms. This allows for adaptation to various complex working conditions and improves the robustness of defect identification of industrial devices.

[0039] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an image processing system based on few-shot learning, provided in an embodiment of this application.

[0040] This application provides an image processing system based on few-shot learning, which can be applied to neural network models, including: The conditional branch input module 101 is used to acquire the target image with known defect labels; The segmented branch input module 102 is used to acquire query images for defect identification of industrial devices; The context-aware feature encoding module 103 is used to perform context-aware feature encoding on the target image based on known defect labels to obtain preliminary features; The first feature extraction module 104 is used to extract features from the preliminary features to obtain known features; The second feature extraction module 105 is used to extract features from the query image to obtain query features; The interactive weight deduction module 106 is used to perform interactive weight deduction on known features and query features to obtain dynamic segmentation parameters. The defect label generation module 107 is used to process the query features based on dynamic segmentation parameters and generate target defect labels for the query image.

[0041] To facilitate understanding of the application of the image processing system based on few-shot learning in this application, assume that the input to the conditional branch is a scratch image rarely seen in the lithium battery industry and its labeled image, and features are obtained through a series of operations in the conditional branch; the input to the segmentation branch is a scratch image rarely seen in the nuclear industry, and features are obtained through a series of operations in the segmentation branch. After processing the features of the two branches, the rare samples from the lithium battery industry can be used to perform defect detection processing on rare samples from the nuclear power industry.

[0042] This application provides an image processing system based on few-shot learning. The context-aware feature encoding module can be used to: determine the minimum bounding rectangle of the defect in the target image based on the known defect label; adaptively expand the minimum bounding rectangle by the target number of pixels to obtain the context region; perform center difference convolution processing on the context region to obtain preliminary features; wherein the target convolution kernel is larger than a set convolution kernel.

[0043] This application provides an image processing system based on few-shot learning. The interactive weight inference module can be used to: perform deep fusion of known features and query features to generate fused features; and perform sequence scanning calculation on the fused features to obtain dynamic segmentation parameters.

[0044] This application provides an image processing system based on few-shot learning. The interactive weight inference module can be used to: perform sequential scanning calculations on fused features through a selective state-space model to obtain dynamic segmentation parameters.

[0045] This application provides an image processing system based on few-shot learning. The defect label generation module may include: a logistic regression layer, used to perform logistic regression calculation on dynamic segmentation parameters and query features to obtain probability values; The upsampling layer is used to upsample the probability values ​​to generate target defect labels for the query image.

[0046] This application provides an image processing system based on few-shot learning, wherein the first feature extraction module includes: The first feature extraction unit is used to extract features from the preliminary features through a convolutional network to obtain known features.

[0047] This application provides an image processing system that performs feature extraction on preliminary features. The first feature extraction module includes: The second feature extraction unit is used to extract features from the preliminary features through a self-attention mechanism to obtain known features.

[0048] This application also provides an electronic device and a computer-readable storage medium, both of which have the corresponding effects of the image processing method based on few-shot learning provided in the embodiments of this application. Please refer to... Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0049] An electronic device provided in this application includes a memory 201 and a processor 202. The memory 201 stores a computer program, and when the processor 202 executes the computer program, it implements the steps of the image processing method based on few-shot learning as described in any of the above embodiments.

[0050] Please see Figure 5 Another electronic device provided in this application embodiment may further include: an input port 203 connected to the processor 202 for transmitting commands input from the outside to the processor 202; a display unit 204 connected to the processor 202 for displaying the processing results of the processor 202 to the outside; and a communication module 205 connected to the processor 202 for enabling communication between the electronic device and the outside. The display unit 204 may be a display panel, a laser scanning display, etc.; the communication method adopted by the communication module 205 includes, but is not limited to, Mobile High-Definition Link (MHL), Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), wireless connection: Wireless Fidelity (WiFi), Bluetooth communication technology, Bluetooth Low Energy communication technology, and communication technology based on IEEE 802.11s.

[0051] This application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the image processing method based on few-shot learning as described in any of the above embodiments.

[0052] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs (compact disc read-only memory), or any other form of storage media known in the art.

[0053] This application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the image processing method based on few-shot learning as described in any of the above embodiments.

[0054] For descriptions of relevant parts in the image processing system, electronic device, and computer-readable storage medium based on few-shot learning provided in this application, please refer to the detailed description of the corresponding parts in the image processing method based on few-shot learning provided in this application, which will not be repeated here. Furthermore, parts of the technical solutions provided in this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0055] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0056] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method based on few-shot learning, characterized in that, include: Obtain the target image with known defect labels; Obtain query images for defect identification of industrial devices; Based on the known defect labels, the target image is subjected to context-aware feature encoding to obtain preliminary features; Feature extraction is performed on the preliminary features to obtain known features; Feature extraction is performed on the query image to obtain the query features; Interactive weight deduction is performed on the known features and the query features to obtain dynamic segmentation parameters; The query features are processed based on the dynamic segmentation parameters to generate target defect labels for the query image.

2. The method according to claim 1, characterized in that, The step of performing context-aware feature encoding on the target image based on the known defect labels to obtain preliminary features includes: Based on the known defect labels, determine the minimum bounding rectangle of the defects in the target image; The minimum bounding rectangle is adaptively expanded by a target number of pixels to obtain the context region; The context region is subjected to central difference convolution to obtain preliminary features; The kernel of the central difference convolution is larger than the set kernel.

3. The method according to claim 1, characterized in that, The interactive weight deduction of the known features and the query features to obtain dynamic segmentation parameters includes: The known features and the query features are deeply fused to generate fused features; The fusion features are subjected to sequence scanning calculations to obtain dynamic segmentation parameters.

4. The method according to claim 3, characterized in that, The step of performing sequence scanning calculations on the fused features to obtain dynamic segmentation parameters includes: Dynamic segmentation parameters are obtained by sequentially scanning the fusion features using a selective state-space model.

5. The method according to claim 1, characterized in that, The step of processing the query features based on the dynamic segmentation parameters to generate target defect labels for the query image includes: Logistic regression is performed on the dynamic segmentation parameters and the query features to obtain probability values; The probability value is upsampled to generate the target defect label of the query image.

6. The method according to claim 1, characterized in that, The step of extracting features from the preliminary features to obtain known features includes: The initial features are extracted using a convolutional network to obtain known features.

7. The method according to claim 1, characterized in that, The step of extracting features from the preliminary features to obtain known features includes: The initial features are extracted using a self-attention mechanism to obtain known features.

8. An image processing system based on few-shot learning, characterized in that, include: The conditional branch input module is used to obtain the target image with known defect labels; The segmented branch input module is used to acquire query images for defect identification of industrial devices; The context-aware feature encoding module is used to perform context-aware feature encoding on the target image based on the known defect labels to obtain preliminary features; The first feature extraction module is used to extract features from the preliminary features to obtain known features; The second feature extraction module is used to extract features from the query image to obtain query features; An interactive weight deduction module is used to perform interactive weight deduction on the known features and the query features to obtain dynamic segmentation parameters; The defect label generation module is used to process the query features based on the dynamic segmentation parameters to generate target defect labels for the query image.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the image processing method based on few-shot learning as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image processing method based on few-shot learning as described in any one of claims 1 to 7.

Images