A detection method and device, program product, storage medium

Abstract

This application provides a detection method, device, program product, and storage medium. The method includes: constructing a first network model and a second network model, wherein the first network model and the second network model are used to detect defects in an object to be detected; constructing an image training dataset, wherein the image training dataset includes multiple sets of image training data, wherein each set of image training data includes a first image and a second image, the resolution of the first image being lower than the resolution of the second image, and the first image and the second image being images of the object to be detected; training the first network model using the image training dataset to obtain a trained first network model; training the second network model based on the image training dataset and the trained first network model to obtain a trained second network model; and using the trained second network model to detect defects in a first object to obtain a defect detection result for the first object, wherein the first object and the object to be detected have the same attributes.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a detection method and device, program product, and storage medium. Background Technology

[0002] Fabric is an essential material for ensuring people's quality of life. Due to current production environments and equipment limitations, fabrics are prone to various defects, affecting the yield rate of finished products. If defective fabrics are not removed before sale, their quality will be compromised, impacting their selling price and causing economic losses for manufacturers. Therefore, fabric defect detection is a crucial aspect of textile production and quality assurance; however, current fabric detection methods still have certain limitations. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this application provide a detection method and equipment, a program product, and a storage medium, which can improve the accuracy of detection.

[0004] The detection method provided in this application includes: Construct a first network model and a second network model, which are used to detect defects in the object to be detected; Construct an image training dataset, which includes multiple sets of image training data. Each set of image training data includes a first image and a second image. The resolution of the first image is lower than that of the second image. The first image and the second image are images of the object to be detected. The first network model is trained using the image training dataset to obtain the trained first network model; The second network model is trained based on the image training dataset and the trained first network model to obtain the trained second network model. The defects of the first object are detected using the trained second network model, and the defect detection results of the first object are obtained. The first object and the object to be detected have the same attributes.

[0005] The testing device provided in this application includes a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to perform the above-described testing method.

[0006] This application provides a computer program product, comprising: a computer program that, when executed by a processor, implements the above-described detection method.

[0007] The computer-readable storage medium provided in this application is used to store a computer program that causes a computer to perform the above-described detection method.

[0008] In the technical solution of this application, a first network model and a second network model are constructed, which are used to detect defects in the object to be detected. An image training dataset is constructed, comprising multiple sets of image training data, each set including a first image and a second image. The resolution of the first image is lower than that of the second image, and both the first and second images are images of the object to be detected. The first network model is trained using the image training dataset to obtain a trained first network model. The second network model is trained based on the image training dataset and the trained first network model to obtain a trained second network model. The trained second network model is used to detect defects in the first object, yielding a defect detection result for the first object. The first object and the object to be detected have the same attributes. Thus, by training the first network model using high-resolution and low-resolution images, a first network model capable of accurately detecting defects in both high- and low-resolution images is obtained. Then, the trained first network model, along with the high-resolution and low-resolution images, is used to train the second network model. This achieves the final target detection model through knowledge distillation, maintaining excellent defect detection performance while reducing image resolution and improving the accuracy of the defect detection results of the second network model. Attached Figure Description

[0009] The accompanying drawings, which are provided to further illustrate this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0010] Figure 1 This is a flowchart illustrating the detection method provided in the embodiments of this application. Figure 1 ; Figure 2 This is a flowchart illustrating the detection method provided in the embodiments of this application. Figure 2 ; Figure 3 This is a schematic diagram of the structure of the teacher network model and student network model provided in the embodiments of this application; Figure 4 This is a schematic diagram of the EMA module structure provided in the embodiments of this application; Figure 5 This is a schematic diagram of the training process of the teacher network model provided in the embodiments of this application; Figure 6 This is a schematic diagram of the feature fusion EF module provided in the embodiments of this application; Figure 7 This is a schematic diagram of the training process of the student network model provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the characteristic distillation module AFD module provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the FcsaNet attention module provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structural composition of the detection device provided in the embodiments of this application; Figure 11 This is a schematic structural diagram of a testing device provided in an embodiment of this application; Figure 12 This is a schematic structural diagram of the chip according to an embodiment of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0012] In the following description, the term "some embodiments" refers to a subset of all possible embodiments. However, it is understood that "some embodiments" can be the same or different subsets of all possible embodiments and can be combined with each other without conflict. It should also be noted that the terms "first," "second," and "third" used in the embodiments of this application are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first," "second," and "third" can be interchanged in a specific order or sequence where permissible, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein. Furthermore, the terms "system" and "network" are often used interchangeably herein. The term "and / or" in this document is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects are in an "or" relationship. It should also be understood that the "instruction" mentioned in the embodiments of this application can be a direct instruction, an indirect instruction, or an indication of an association relationship. For example, A instructing B can mean that A directly instructs B, for example, B can obtain information through A; it can also mean that A indirectly instructs B, for example, A instructs C, and B can obtain information through C; it can also mean that there is an association between A and B. It should also be understood that the term "correspondence" mentioned in the embodiments of this application can mean that there is a direct or indirect correspondence between the two, or that there is an association between the two, or that there is an instruction and being instructed, configuration and being configured, etc.

[0013] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and they all fall within the protection scope of the embodiments of this application.

[0014] Fabric is an essential material for ensuring people's quality of life. Due to current production environments and equipment limitations, fabrics are prone to various defects, affecting product yield. If defective fabrics are not removed before sale, their quality and selling price will be impacted, leading to economic losses for manufacturers. Therefore, fabric defect detection is a crucial link in textile production and quality assurance, significantly impacting production quality and efficiency. Currently, fabric defect detection is primarily done manually, using transmission devices and lighting to visually inspect moving fabrics for defects. However, workers experience visual fatigue and fatigue, leading to missed or false detections. Furthermore, the speed is limited, hindering improvements in detection efficiency. Prolonged manual inspection can also damage workers' eyesight. Despite this, it remains the mainstream method for fabric defect detection.

[0015] In recent years, with the development of computer technology, some automatic detection methods for fabric defects have been gradually studied. Statistical analysis methods mainly detect defects based on the statistical features of pixels. The most direct method is to detect defects by comparing image grayscale values, but this method is easily affected by noise and external conditions. Image detection methods based on texture structure parameters detect defect points by calculating the pixel mean and variance of normal fabric, but this method is easily affected by lighting or texture, resulting in poor detection results. Spectral methods convert images into spectrograms using methods such as Fourier transform, Gabor transform, and wavelet transform, and then identify fabric defects by image frequency, but their performance is poor in complex backgrounds. Deep learning involves building and training convolutional neural networks, and then using the trained model to detect defects in fabric images. Currently, there are many deep learning methods for fabric defect detection, but they still have certain limitations in scenarios with high image resolution, real-time performance, and accuracy requirements.

[0016] Currently, in industrial production, fabric images captured by industrial cameras on production lines typically have high resolution. Directly training and predicting from these images places high demands on hardware and results in poor real-time performance. Furthermore, downsampling the images directly leads to the loss of detail in small defects, thus reducing the model's detection accuracy. Therefore, a common approach is to crop the images and then leverage the parallel computing capabilities of graphics cards for defect prediction. However, in scenarios with higher real-time requirements, downsampling of the cropped images is necessary to improve real-time detection, but this also introduces a certain degree of performance degradation. Therefore, maximizing the accuracy of defect detection becomes a crucial consideration. To address this, the following technical solutions from embodiments of this application are proposed.

[0017] Figure 1 This is a flowchart illustrating the detection method provided in the embodiments of this application. Figure 1 ,like Figure 1 As shown, the detection method includes the following steps: Step 101: Construct a first network model and a second network model. The first network model and the second network model are used to detect defects in the object to be detected.

[0018] In some implementations, the first network model may also be referred to as the teacher network model, and the second network model may also be referred to as the student network model.

[0019] In some implementations, the first network model and the second network model have basically similar structures, only the parameters are different. For example, the first network model and the second network model can be constructed based on the YOLOv8 model.

[0020] In some implementations, the object to be detected is a piece of fabric, and the defect of the object to be detected is a defect of the fabric. For example, problems such as the current fabric color, texture, or pattern not matching the surroundings, or the presence of abnormal conditions such as loose threads, are all referred to as defects.

[0021] It is understood that the embodiments of this application can be applied to other defect detection methods without specific limitations. It should also be understood that these defects can be understood as non-standard parts of the object to be detected, such as paint peeling on the surface of a cup, unevenness on a wall, etc.

[0022] In some implementations, the first network model includes a first backbone network module, a first neck network module, a first prediction network module, a first multi-scale attention module, and a first feature fusion module. The second training network model includes a second backbone network module, a second neck network module, a second prediction network module, a second multi-scale attention module, and a first feature distillation module. The first and second multi-scale attention modules can simultaneously extract multi-frequency channel attention information and also extract rich spatial attention information. The first feature fusion module in the first training network model can focus on and fuse the spatial information of features across multiple channel dimensions, thereby enabling the fused features to more comprehensively reflect the spatial information between the various features.

[0023] Here, the YOLOv8 model sequentially comprises a backbone network, a neck network, and a prediction network. The original YOLOv8 only has a three-layer neck network and a corresponding prediction network. This design cannot meet the needs of scenarios where the image size corresponding to many defects is small. Therefore, a small object detection branch needs to be added to the original YOLOv8 model for small-sized detection. In addition, in this embodiment, to improve the model's ability to extract features from the image information corresponding to the object to be detected while minimizing the reduction in model real-time performance, an EMA (multi-scale attention) module is added at the beginning of each layer of the neck network, namely the aforementioned first and second multi-scale attention modules. Thus, a knowledge distillation low-resolution defect detection method combining EMA and object detection networks is realized, which better extracts spatial information of features without significantly increasing the number of model parameters, thereby improving the detection accuracy and knowledge distillation efficiency of the model.

[0024] It is understandable that each type of module in the network includes multiple sub-modules, and the newly added small target detection branch is a detection branch composed of these sub-modules, used to improve the detection effect of low-resolution images.

[0025] Step 102: Construct an image training dataset, which includes multiple sets of image training data.

[0026] Each set of image training data includes a first image and a second image, where the resolution of the first image is lower than that of the second image. Both images represent the objects to be detected. The dataset is obtained by cropping the original images based on the performance of the detection equipment and the characteristics of the defects to be detected. Two main criteria are used: First, the detection speed and the number of images detected in a batch are related to the image resolution. Higher resolution results in slower detection speed and fewer images detected in a batch; conversely, faster detection speed results in more images detected in a batch. Therefore, the resolution setting needs to consider production speed requirements. Second, the characteristics of the defects to be detected are also related to the image resolution. Normally, the original image has excessive pixels, requiring downsampling to improve detection performance, but the pixel features of the smallest defects must be clearly visible. This resolution is used as the minimum detection resolution. Considering these factors, the minimum detection resolution is set to low resolution, and twice the minimum detection resolution is set to high resolution. The original images are then cropped and downsampled to construct the training dataset. Each set of data contains one high-resolution image and one low-resolution image, i.e., the second image and the first image.

[0027] In some implementations, the image training dataset includes a low-resolution first image and a high-resolution second image. Each image includes a label; for example, the label may include information such as whether there is a defect, the type of defect, and the location of the defect. In other words, the image training dataset includes not only image data but also label information for each image.

[0028] In some implementations, a defect typically has five values: type c, coordinates x and y, width w and height h of the minimum bounding rectangle of the defect, and a txt file containing information about all defects for each image in the dataset.

[0029] It is understandable that the high-resolution and low-resolution images here are obtained by downsampling the cropped original images. This is because the original image has a high resolution, while the model's detection resolution is relatively low. Therefore, both the high-resolution and low-resolution images in the image training data are obtained by downsampling.

[0030] For example, when the object to be detected is cloth, the images in the constructed image training dataset are all images of cloth, and some of these images need to contain defective parts.

[0031] Step 103: Train the first network model using the image training dataset to obtain the trained first network model.

[0032] Here, the high-resolution and low-resolution images obtained above are input into the first network model to obtain the corresponding outputs and calculate the loss function, thereby realizing the training of the first network model.

[0033] In some implementations, training a first network model using an image training dataset to obtain a trained first network model includes: inputting a first image from each set of image training data into the first network model to obtain a first neck network feature set, a first prediction network feature set, and a first input loss; inputting a second image from each set of image training data into the first network model to obtain a second neck network feature set, a second prediction network feature set, and a second input loss; obtaining a third input loss based on the first neck network feature set and the second neck network feature set; determining a first total loss function for the first network model based on the first input loss, the second input loss, and the third input loss; and adjusting the parameters of the first network model so that the first total loss function reaches a first target loss value to obtain the trained first network model. Here, the low-resolution image, i.e., the first image, is input into the first network model to obtain the corresponding first input loss. The high-resolution image, i.e., the second image, is input into the first network model to obtain the corresponding second input loss. In addition, the first neck network feature set corresponding to the low-resolution image and the second neck network feature set corresponding to the high-resolution image are determined as fusion features and input into the prediction module to obtain the third input loss. Thus, the total loss of the first network model can be determined. That is, the total loss of the first network model considers multiple losses, so that the more important spatial and channel level information in each sub-feature can be calculated during distillation.

[0034] In some implementations, the first target loss value is the minimum value of the first total loss function.

[0035] In some implementations, inputting the first image from each set of image training data into a first network model to obtain a first neck network feature set, a first prediction network feature set, and a first input loss includes: inputting the first image from each set of image training data into a first backbone network module to obtain a first backbone network feature set; inputting the first backbone network feature set into a first multi-scale attention module to obtain a first attention feature set; inputting the first attention feature set into the first neck network module to obtain a first neck network feature set; inputting the first neck network feature set into the first prediction network module to obtain a first prediction network feature set; and determining the first input loss based on the first prediction network feature set.

[0036] In some implementations, the first backbone network model includes multiple backbone network sub-modules, the first multi-scale attention module includes multiple multi-scale attention sub-modules, the first neck network module includes multiple neck network sub-modules, and the first prediction network module includes multiple prediction network sub-modules. A first image is input into a backbone network sub-module. Multiple backbone network sub-modules obtain first backbone network features output by each of the aforementioned backbone network sub-modules based on their outputs. These backbone network features constitute a first backbone network feature set. The first backbone network features in the first backbone network feature set are input into the corresponding multi-scale attention sub-modules to obtain first attention features output by each multi-scale attention sub-module. These first attention features constitute a first attention feature set. The first attention features in the first attention feature set are then input into the corresponding neck network sub-modules to obtain first neck network features output by each neck network sub-module. These first neck network features constitute a first neck network feature set. Each neck network feature in the first neck network feature set is input into the corresponding prediction network sub-module to obtain first prediction network features output by each prediction network sub-module. These first prediction network features constitute a first prediction network feature set. The first input loss, also known as the low-resolution input loss, is obtained based on the feature set of the first prediction network and corresponds to the ground-resolution image.

[0037] In some implementations, inputting the second image from each set of image training data into a first network model to obtain a second neck network feature set, a second prediction network feature set, and a second input loss includes: inputting the second image from each set of image training data into a first backbone network module to obtain a second backbone network feature set; inputting the second backbone network feature set into a first multi-scale attention module to obtain a second attention feature set; inputting the second attention feature set into the first neck network module to obtain a second neck network feature set; inputting the second neck network feature set into the first prediction network module to obtain a second prediction network feature set; and determining the second input loss based on the second prediction network feature set.

[0038] In some implementations, a second image is input into a backbone network submodule. Multiple backbone network submodules obtain second backbone network features output by each of their respective submodules. These backbone network features constitute a second backbone network feature set. The second backbone network features in this set are then input into corresponding multi-scale attention submodules to obtain second attention features output by each multi-scale attention submodule. These second attention features constitute a second attention feature set. The second attention features in this set are then input into corresponding neck network submodules to obtain second neck network features output by each neck network submodule. These second neck network features constitute a second neck network feature set. Each neck network feature in this set is then input into a corresponding prediction network submodule to obtain second prediction network features output by each prediction network submodule. These second prediction network features constitute a second prediction network feature set. The second input loss, i.e., the high-resolution input loss, is obtained based on the second prediction network feature set and corresponds to the ground-resolution image.

[0039] In some implementations, obtaining the third input loss based on the first and second neck network feature sets includes: inputting the first and second neck network feature sets into a first feature fusion module to obtain a first fused feature set; inputting the first fused feature set into a first prediction network module to obtain a third prediction network feature set; and determining the third input loss based on the third prediction network feature set. Here, inputting the first and second neck network feature sets into the first fused feature set obtained by the first feature fusion module incorporates more crucial spatial and semantic information from the features. The third prediction network feature set is then input into the first prediction network module to obtain the third prediction network feature set, thereby determining the third input loss. In some implementations, the first feature fusion module includes multiple feature fusion sub-modules. Each first neck network feature from the first neck network feature set and each second neck network feature set from the second neck network feature set are input into the corresponding feature fusion sub-module to obtain the output first fused feature of each feature fusion sub-module. These first fused features constitute the first fused feature set. Here, the function of the first feature fusion module is to fuse neck network features of the same size. For example, the four-layer neck network features output at high resolution are 2×2, 4×4, 8×8, and 16×16, while the four-layer neck network features at low resolution are 1×1, 2×2, 4×4, and 8×8. Therefore, the difference in size needs to be matched to achieve the same size before concatenation. That is, the 2×2, 4×4, and 8×8 outputs at high resolution are fused with the 2×2, 4×4, and 8×8 outputs at low resolution, respectively.

[0040] In some implementations, each fusion feature in the first fusion feature set is input into the corresponding prediction network submodule, and each prediction network submodule outputs a third prediction network feature. These third prediction network features constitute a third prediction network feature set. A third input loss, i.e., the fusion feature input loss, is obtained based on the third prediction network feature set.

[0041] In some implementations, the first total loss function of the first network model is shown in equation (1): + + (1); in, For the first total loss function, This is the first input loss, which is the low-resolution input loss obtained by inputting the low-resolution image into the first network model. This is the second input loss, which is the high-resolution input loss obtained by inputting the high-resolution image into the first network model. This is the third input loss, also known as the feature fusion input loss. . To balance the parameters, the number of training iterations is gradually increased from 0 to 0.5.

[0042] In some embodiments, the method further includes: if the first image or the second image includes a defective portion, determining a first new image training set based on the first image or the second image, the first new image training set including a third image and a fourth image, the resolution of the third image being lower than the resolution of the fourth image; and updating the first network model using the first new image training set.

[0043] In some implementations, if the first or second image includes a defect, one defect is randomly selected, and then a rectangular region containing the defect in the original image is selected. The image is then cropped from this region and enlarged to the same size as the original image. The defect location in the new image is calculated based on the cropped coordinates and used as the image's label. This generates a new pair of high- and low-resolution images, known as the first new image training set. The first network model is trained using this first new image training set according to the aforementioned process, thus updating the first network model. Because the cropping is random, the possibility of overfitting is reduced to some extent. It also increases the amount of data in the dataset. Furthermore, since the defective target occupies a larger proportion of the image, it is beneficial for object detection and knowledge distillation to learn more accurate defect features. This method of using cropped images to retain the main defective region to generate new image pairs for object detection and knowledge distillation effectively reduces interference from background distillation, improves the ability of object detection and knowledge distillation to capture defect features, and enhances the model's defect detection capability.

[0044] In some implementations, the rectangular bounding box region can be required to have both its length and width being half of the original image. This application does not specifically limit the size of the rectangular bounding box region.

[0045] In some implementations, random selection involves randomly selecting a defect from the corresponding txt file of the image and cropping around that defect.

[0046] It is understandable that each set of image training data in the image training set is not input only once. Therefore, if it is selected for cropping during the training process, it needs to be cropped again each time.

[0047] It should be noted that during the selection of the bounding box containing the defect, if other defects are located in the area, they are retained; otherwise, only one defect is randomly selected. Furthermore, this random defect selection process is performed only once for each set of image training data. It should also be understood that the generation of the first new image training set is also random; it is not the case that a new image training set is generated for every set of image training data containing defects to update the first network model. In other words, the generation of new image training data is randomized during the training process.

[0048] It should also be noted that the cropping of the original image or the random selection of defects for cropping in the embodiments of this application are performed by a separate model or code execution, and are not the execution process of the first network model and the second network model.

[0049] Step 104: Train the second network model based on the image training data set and the trained first network model to obtain the trained second network model.

[0050] Here, after obtaining the trained first network model, the trained first network model is used to help train the second network model, thus obtaining the trained second network model.

[0051] In some implementations, training a second network model based on an image training dataset and a trained first network model includes: training the second network model based on a first image from each set of image training data in the image training dataset and the trained first network model. This improves the detection performance of the second network model on low-resolution images by utilizing the trained first network model.

[0052] In some implementations, training a second network model based on an image training dataset and a trained first network model to obtain a trained second network model includes: loading the parameters of the trained first network model into the second network model to obtain an intermediate second network model; inputting the first image of each set of image training data into the intermediate second network model to obtain a third neck network feature set, a third backbone network feature set, a fourth prediction network feature set, and a fourth input loss; inputting the first image of each set of image training data into the trained first network model to obtain a fourth neck network feature set; inputting the second image of each set of image training data into the trained first network model to obtain a fifth neck feature network set and a fourth backbone network feature set; obtaining a second fusion feature set based on the fourth and fifth neck network feature sets; determining a fifth input loss based on the third neck feature set and the second fusion feature set; determining a sixth input loss based on the third and fourth backbone network feature sets; determining a second total loss function for the second network model based on the fourth, fifth, and sixth input losses; and adjusting the parameters of the second network model so that the second total loss function reaches a second target loss value to obtain the trained second network model. Here, since the structures of the first and second network models are basically similar, the parameters of the corresponding modules in the trained first network model are loaded into the second network model to obtain an intermediate network model. The low-resolution image, i.e., the first image, is then input into the second network model to obtain the fourth input loss. The backbone network features and fusion features obtained from the trained first network model are then combined with the backbone network features and neck network features obtained from the intermediate second network model to determine the fifth and sixth input losses. The parameters of the intermediate second network model are then adjusted to obtain the trained second network model, which is the final detection model. Thus, by training the second network model using the trained first network model, the detection performance of the second network model for low-resolution images is improved, thereby increasing the accuracy of detecting defects in the object. In other words, the total loss of the second network model considers multiple losses, enabling the calculation of more important spatial and channel-level information in each sub-feature during distillation.

[0053] It is understandable that the loaded parameters are those of the backbone network module, the neck network module, and the prediction network module.

[0054] In some implementations, the second target loss value is the minimum value of the second total loss function.

[0055] In some implementations, the first image of each set of image training data is input into an intermediate second network model to obtain a third neck network feature set, a third backbone network feature set, a fourth prediction network feature set, and a fourth input loss. This includes: inputting the first image of each set of image training data into a second backbone network module to obtain a third backbone network feature set; inputting the third backbone network feature set into a second multi-scale attention module to obtain a third attention feature set; inputting the third attention feature set into a second neck network module to obtain a third neck network feature set; inputting the third neck network feature set into a second prediction network module to obtain a fourth prediction network feature set; and determining the fourth input loss based on the fourth prediction network feature set.

[0056] In some implementations, the second backbone network model includes multiple backbone network sub-modules, the second multi-scale attention module includes multiple multi-scale attention sub-modules, the second neck network module includes multiple neck network sub-modules, and the second prediction network module includes multiple prediction network sub-modules. A first image is input into a backbone network sub-module. Multiple backbone network sub-modules obtain third backbone network features output by each of the aforementioned backbone network sub-modules based on their outputs. These backbone network features constitute a third backbone network feature set. The third backbone network features in the third backbone network feature set are input into the corresponding multi-scale attention sub-modules to obtain third attention features output by each multi-scale attention sub-module. These third attention features constitute a third attention feature set. The third attention features in the third attention feature set are input into the corresponding neck network sub-modules to obtain third neck network features output by each neck network sub-module. These third neck network features constitute a third neck network feature set. Each neck network feature in the third neck network feature set is input into the corresponding prediction network sub-module to obtain fourth prediction network features output by each prediction network sub-module. These fourth prediction network features constitute a fourth prediction network feature set. The fourth input loss, also known as the low-resolution input loss, is obtained based on the feature set of the fourth prediction network and corresponds to the ground-resolution image.

[0057] In some implementations, the first image of each set of image training data is input into the trained first network model to obtain a fourth neck network feature set, including: inputting the first image of each set of image training data into the first backbone network module of the trained first network model to obtain a fifth backbone network feature set; inputting the fifth backbone network feature set into the first multi-scale attention network module to obtain a fourth attention feature set; and inputting the fourth attention feature set into the first neck network module to obtain a fourth neck network feature set.

[0058] In some implementations, the second image of each set of image training data is input into the trained first network model to obtain a fifth neck feature network set and a fourth backbone network feature set, including: inputting the second image of each set of image training data into the first backbone network feature module to obtain a fourth backbone network feature set; inputting the fourth backbone network feature set into the first multi-scale attention network module to obtain a fifth attention feature set; and inputting the fifth attention feature set into the first neck network module to obtain a fifth neck network feature set.

[0059] In some implementations, obtaining a second fused feature set based on a fourth neck network feature set and a fifth neck network feature set includes inputting the fourth neck network feature set and the fifth neck network feature set into a first feature fusion module to obtain the second fused feature set.

[0060] In some implementations, determining the fifth input loss based on the third neck feature set and the second fused feature set includes: inputting the third neck feature set and the second fused feature set into a first feature distillation module to obtain the fifth input loss. Here, the first feature distillation module performs knowledge distillation and calculates the distillation loss, which is also the fifth input loss.

[0061] In some implementations, the first feature distillation module includes multiple feature distillation sub-modules, which input each third neck feature in the third neck feature set and each second fusion feature in the second fusion feature set into the corresponding feature distillation sub-module to obtain the fifth input loss.

[0062] In some implementations, determining the sixth input loss based on the third and fourth backbone network feature sets includes: scaling the fourth backbone network features to the same size as the third backbone network feature set using mean pooling to obtain the sixth backbone network feature set; and determining the sixth input loss based on the sixth backbone network feature set and the third backbone network feature set. Here, the sixth input loss is the backbone network distillation loss.

[0063] In some implementations, the second total loss function of the second network model is shown in equation (2): (2); in, The second total loss function is denoted by , and the fourth input loss is denoted by , which is the low-resolution input loss obtained by inputting the low-resolution image into the first network model. This is the fifth input loss, also known as the neck network distillation loss. This is the sixth input loss, also known as the backbone network distillation loss. and All of these are equilibrium parameters. Used to balance the difference between distillation loss and target detection loss. Used to balance the distillation loss of neck network features and backbone network distillation losses The difference. For example, The value range is [0.5, 0.9], at the beginning of training. The value is 0.5, and as the number of training iterations increases, Gradually increase to 0.9, The value range is [0.2, 1]. In the early stages of training, and Occupying the same proportion, with a value of 1, as the number of training iterations increases, The weight gradually decreases, and eventually The value is 0.2. The specific value can be determined according to the actual situation, and this application does not impose any specific restrictions on it.

[0064] In some embodiments, the method further includes: if the first image or the second image includes a defective portion, determining a second new image training set based on the first image or the second image, the second new image training set including a fifth image and a sixth image, the fifth image having a lower resolution than the sixth image; and updating a second network model using the second new image training set.

[0065] In some implementations, during the training of the second network model, if either the first or second image contains a defect, one defect is randomly selected, and a rectangular bounding box containing the defect in the original image is then chosen. The image is then cropped from this bounding box and enlarged to the same size as the original image. The defect location in the new image is calculated based on the cropped coordinates and used as the image's label. This generates a new pair of high- and low-resolution images, known as the second new image training set. The second network model is then trained using this second new image training set, updating the second network model. Because the cropping is random, the possibility of overfitting is reduced to some extent. It also increases the amount of data in the dataset, and since the defective target occupies a larger proportion of the image, it helps object detection and knowledge distillation learn more accurate defect features. This method of using cropped images to retain the main defective region to generate new image pairs for object detection and knowledge distillation effectively reduces interference from background distillation, improves the ability of object detection and knowledge distillation to capture defect features, and enhances the model's defect detection capabilities.

[0066] In some implementations, the rectangular bounding box region can be required to have both its length and width being half of the original image. This application does not specifically limit the size of the rectangular bounding box region.

[0067] In some implementations, random selection involves randomly selecting a defect from the corresponding txt file of the image and cropping around that defect.

[0068] It is understandable that each set of image training data in the image training set is not input only once. Therefore, if it is selected for cropping during the training process, it needs to be cropped again each time.

[0069] It should be noted that the pruning method and training process here are the same as those for training the first network model, but the pruning will be repeated.

[0070] Step 105: Use the trained second network model to detect defects in the first object and obtain the defect detection result of the first object. The first object and the object to be detected have the same attributes.

[0071] In some implementations, the trained second network model is a detection network model, used to detect objects with the same attributes as the object to be detected obtained during the training process. For example, when the object to be detected is fabric, the trained first network model is used to detect defects in the fabric and obtain defect detection results.

[0072] In some implementations, the defect detection results include whether a defect exists, and if a defect exists, the defect detection results also include defect type information and defect coordinate information.

[0073] Specifically, since the original captured images are usually too large, they need to be cropped. After cropping, they are put into a batch and fed into a trained second network model for batch detection. Then the results are analyzed to determine whether there are defects in the region and to obtain the type and coordinates of the defects. Since the images are cropped, the defect coordinates are transformed to obtain the specific location of the defects in the original image.

[0074] It should be noted that image cropping and coordinate transformation are not part of the detection model's execution process; these require additional processes or code. It should also be noted that the first feature distillation module is not needed when using the trained second network model for detection.

[0075] The technical solution of this application embodiment constructs a first network model and a second network model, which are used to detect defects in the object to be detected. An image training dataset is constructed, comprising multiple sets of image training data, each set including a first image and a second image. The resolution of the first image is lower than that of the second image, and both the first and second images are images of the object to be detected. The first network model is trained using the image training dataset to obtain a trained first network model. The second network model is trained based on the image training dataset and the trained first network model to obtain a trained second network model. The trained second network model is used to detect defects in the first object, yielding a defect detection result for the first object. The first object and the object to be detected have the same attributes. Thus, by training the first network model with high-resolution and low-resolution images, a first network model capable of accurately detecting defects in both high- and low-resolution images is obtained. Then, the trained first network model, along with the high-resolution and low-resolution images, is used to train the second network model. This achieves the final target detection model through knowledge distillation, maintaining excellent defect detection performance while reducing image resolution and improving the accuracy of the defect detection results of the second network model.

[0076] Based on the foregoing embodiments, when the object to be detected is fabric, the detection method provided by the embodiments of this application will be further explained.

[0077] The technical solution of this application first optimizes the input for knowledge distillation. The input includes not only the entire region image of the original image but also a portion of the image of the defect region, enabling the model to learn the features of the defect more accurately. Here, since the original image is large and the model input resolution is small, the original image is first cropped and then downsampled to obtain the input. At the same time, for the defective part, the image containing the defect is also cropped as input. The feature distillation of knowledge distillation adopts the idea of ​​iterative distillation. In the early training process, more emphasis is placed on the distillation of shallow features, and in the later training process, more emphasis is placed on the distillation of deep features, so that the model can learn accurate features more accurately during the knowledge distillation learning process. A new feature distillation loss calculation method is proposed, which can calculate the more important spatial and channel level information of each feature during distillation. A new attention module is also proposed, which can extract multi-frequency channel attention information and rich spatial attention information simultaneously. A new feature fusion module is proposed, which can focus on and fuse the spatial information of multiple channel dimensions of features, so that the fused features can more comprehensively reflect the spatial information between each feature. Figure 2 This is a flowchart illustrating the detection method provided in the embodiments of this application. Figure 2 ,like Figure 2As shown, it includes the following steps: Step 201: Construct the teacher network model and the student network model.

[0078] The teacher network model is equivalent to the first network model mentioned above, and the student network model is equivalent to the second network model mentioned above. Figure 3 This is a schematic diagram of the structure of the teacher network model and student network model provided in the embodiments of this application, as shown below. Figure 3 As shown, the teacher network model and student network model include a backbone network (equivalent to the aforementioned first and second backbone network modules), an EMA module (equivalent to the aforementioned first and second multi-scale attention modules), a neck network (equivalent to the aforementioned first and second neck network modules), and a prediction network (equivalent to the aforementioned first and second prediction network modules). The backbone network includes multiple backbone network sub-modules, namely backbone network module 1, backbone network module 2, backbone network module 3, backbone network module 4, and backbone network module 5. The EMA module includes multiple EMA sub-modules, namely EMA module 1, EMA module 2, EMA module 3, and EMA module 4. The neck network includes multiple neck network sub-modules, namely neck network module 1, neck network module 2, neck network module 3, neck network module 4, neck network module 5, and neck network module 6. The prediction module includes multiple prediction sub-modules, namely prediction network module 1, prediction network module 2, prediction network module 3, and prediction network module 4. Figure 3 As shown, the aforementioned teacher network model and student network model have the same model structure, differing only in model parameters. Both are built upon the YOLOv8 model, which consists of a backbone network, a neck network, and a prediction network. The original YOLOv8 only has a three-layer neck network and a corresponding prediction network. This design cannot meet the needs of many scenarios where fabric defects are small. Therefore, a small target detection branch needs to be added to the original YOLOv8 model. Specifically, this is... Figure 3 The model is divided into three parts: EMA module 1, neck network module 1, and prediction network module 1, as well as EMA module 1, neck network module 1, and neck network module 4. Furthermore, fabric defect detection focuses more on pixel information such as shape and color in the image. To improve the model's ability to extract these features while minimizing the reduction in real-time performance, an EMA (multi-scale attention, EMA) module is added at the beginning of each neck network layer. Figure 4 This is a schematic diagram of the EMA module structure provided in an embodiment of this application, as shown below. Figure 4As shown, the EMA module can extract spatial information from multiple channel dimensions of features. Compared with other similar attention modules, the EMA module has advantages in parameter efficiency and better performance. Specifically, the input is divided into G groups according to the channel dimension. Each group is processed as follows: mean pooling is performed on the H dimension and mean pooling is performed on the W dimension. After passing through a convolutional layer, the pooled features are concatenated and convolutional. Then, after passing through a convolutional layer and a Sigmoid activation function, the features are weighted. The weighted features are then normalized. After that, mean pooling and softmax are performed on the normalized features. This feature is then multiplied by the features after the convolutional layer. The two features after the matrix multiplication are then multiplied by the Simmoid activation function and weighted by the Gaiter frame. Finally, the output is the aforementioned attention feature.

[0079] Here, it should be noted that, Figure 3 The teacher network model and student network model shown have the same main component structure. As explained in subsequent steps, the teacher network model includes an EF module, while the student network model includes an AFD module. (The last sentence appears to be incomplete and possibly refers to a different topic.) Figure 3 As shown in the image.

[0080] Step 202: Construct the training dataset.

[0081] The dataset is obtained by cropping the original images based on a comprehensive consideration of the detection equipment performance and the characteristics of the defects to be detected. There are two main criteria. First, the detection speed is related to the number of images detected in a batch and the image resolution. Higher resolution results in slower detection speed and fewer images detected in a batch; conversely, lower resolution results in faster detection speed and more images detected in a batch. Therefore, the resolution needs to be set considering production speed requirements. Second, the characteristics of the defects to be detected are also related to the image resolution. Normally, the original image has excessive pixels, requiring downsampling, which can improve detection performance while ensuring that the pixel features of the smallest defects are clearly visible. This resolution is used as the minimum detection resolution. Considering the above, the minimum detection resolution is set to low resolution, and twice the minimum detection resolution is set to high resolution. The original images are then cropped and downsampled to construct the training dataset. Each dataset contains one high-resolution image and one low-resolution image. That is, both the high-resolution and low-resolution images are obtained by downsampling the cropped original images. In addition, each image includes corresponding labels, as described above, and will not be repeated here.

[0082] Step 203: Train the teacher network model.

[0083] Figure 5 This is a schematic diagram of the training process of the teacher network model provided in the embodiments of this application, such as... Figure 5 As shown, the high-resolution and low-resolution images obtained in step 202 are input into the teacher network model to obtain the corresponding outputs, and the loss function is calculated for training. The detailed design of this step is as follows: Low-resolution images are fed into the teacher network model, and multi-layer neck network features are obtained through calculation. , , (Equivalent to the aforementioned first neck network features), save these features, and continue subsequent calculations to obtain the predicted network features. , , (Equivalent to the aforementioned first prediction network features), and the loss is calculated using the prediction network features to obtain the low-resolution input loss. (Equivalent to the aforementioned first input loss).

[0084] Similarly, the high-resolution image corresponding to the low-resolution image is fed into the teacher network model, and the multi-layer neck network features are obtained through calculation. , , (Equivalent to the aforementioned second neck network features), save these features, and continue subsequent calculations to obtain the predicted network features. , , (Equivalent to the aforementioned second prediction network feature), and the loss is calculated using the prediction network feature to obtain the high-resolution input loss. (Equivalent to the aforementioned second input loss).

[0085] The low-resolution input features of the multi-layer neck network obtained above , , Features of multilayer neck networks with high-resolution input , , The features are fed into the Feature Fusion EF module (equivalent to the aforementioned first feature fusion module) for feature fusion, and the corresponding relationship is as follows: and (i=5,4,3), obtain the fusion features , , (Equivalent to the aforementioned first fusion feature set). The fusion features are fed into the prediction network to obtain the corresponding prediction features (equivalent to the aforementioned third prediction network feature set), and then the feature loss is calculated to obtain the fusion feature input loss. (Equivalent to the aforementioned third input loss). Figure 6This is a schematic diagram of the feature fusion EF module provided in an embodiment of this application, as shown below. Figure 6 As shown, and Intermediate features are obtained by fusing along the channel direction. After passing through the EMA module, intermediate features are obtained. Then, segmentation is performed along the channel direction to obtain two intermediate features. , , respectively with and Perform matrix multiplication, then add the results to obtain the final fused features. It incorporates elements from and Features processed by the EMA attention mechanism contain more critical spatial and semantic information from the features.

[0086] Here, the feature sizes fed into the EF module are consistent. It can be understood that the EF module includes the EMA module.

[0087] The total loss function of the teacher network model is shown in equation (1), where, , All losses were calculated using the original YOLOv8 loss function. Figure 5 In and (i=5,4,3) have the same scale, but differ in the number of channels. Convolution is needed to reorganize the channels so that the scale and number of channels are consistent. and (i=5,4,3) Features of the same scale in all three are predicted using the same prediction network module and share weights. Integrating from and The feature information, after fusion It not only contains high-resolution feature information, but can also adapt to low-resolution features, thus better guiding students in training network models.

[0088] If a defect exists in the image, one defect is randomly selected, and a rectangular region containing that defect is randomly selected from the original images in the training dataset. This region must have a length and width equal to that of the original image. If other defects are located in the same region, they are retained. Then, the image is cropped from this region and enlarged to the same size as the original image. The defect location in the new image is calculated based on the cropping coordinates and used as the image's label. This generates a new pair of high- and low-resolution images. Because the cropping is random, it reduces the possibility of overfitting to some extent, increases the dataset size, and because the defective target occupies a larger proportion of the image, it is beneficial for object detection and knowledge distillation to learn more accurate defect features. The new image pair is then fed into the teacher network model for training using the aforementioned steps. Here, a defect typically has five values: category c, coordinates x, y, width w, and height h of the defect's minimum bounding rectangle. Each image in the dataset corresponds to a txt file containing information on all defects. Random selection involves randomly selecting a defect from the txt file and cropping around it. Cropping is done repeatedly; each image is only cropped once. That is, if the cropping region contains at least one defect, the remaining defects in the image are not cropped again. Object generation is performed randomly on a training image during training, equivalent to data augmentation. In other words, this process is not applied to all defective images; defective images are randomly selected for re-cropping.

[0089] After the above steps, the teacher network model is trained.

[0090] It should be noted that, Figure 5 The neck network features in this example are all neck network features obtained after being processed by the EMA module and then input into the neck network module. Here, neck network feature 1, neck network feature 2, neck network feature 3, and neck network feature 4 correspond to... Figure 3 For example, the inputs to the prediction network modules are: neck network feature 1 is the input to prediction network module 1, specifically obtained by passing the output of EMA module 1 to the output of neck network module 1; neck network feature 2 is equivalent to the output of neck network module 4 and the input to prediction network module 2; neck network feature 3 is equivalent to the output of neck network module 5 and the input to prediction network module 3; and neck network feature 4 is equivalent to the output of neck network module 6 and the input to prediction network module 4. Figure 5 Prediction network module 1, prediction network module 2, and prediction network module 3 in the text are equivalent to Figure 3 The prediction network modules are 2, 3, and 4.

[0091] Step 204: Train the student network model.

[0092] Although the student network directly loads the weights from the teacher network, the performance of both the student and teacher networks in detecting low-resolution images is still not optimal. Therefore, it is necessary to utilize high-resolution feature information from the teacher network to guide the training of the student network on low-resolution inputs. The high- and low-resolution datasets obtained through the aforementioned steps, along with the teacher network model, are used to distill and train the student network. Figure 7 This is a schematic diagram of the training process of the student network model provided in the embodiments of this application, such as... Figure 7 As shown, the detailed steps are as follows: The weight parameters of the teacher network model obtained in step 203 are loaded into the student network model. Here, the parameters of the corresponding modules are loaded, namely the backbone network module, neck network module and prediction network module in the teacher network model.

[0093] Low-resolution images are fed into the student network model, and multi-layer neck network features are obtained through calculation. , , (equivalent to the aforementioned third neck network feature set), and backbone network features. , , , , (Equivalent to the aforementioned third backbone network feature set), storing except This part of the feature is used to continue subsequent calculations, and the prediction network feature is obtained through this feature. , , , (Equivalent to the aforementioned fourth prediction network feature set), and the loss is calculated using the prediction network features to obtain the low-resolution input loss. (Equivalent to the aforementioned fourth input loss).

[0094] Low-resolution images are fed into the teacher network model, and multi-layer neck network features are obtained through calculation. , , (Equivalent to the aforementioned fourth neck network feature set), this part of the features is saved; the high-resolution image corresponding to the low-resolution image is fed into the teacher network model, and the multi-layer neck network features are obtained through calculation. , , (equivalent to the aforementioned fifth neck network feature set), and backbone network features. , , , , (Equivalent to the aforementioned fourth backbone network feature set), save this part of the features; input the low-resolution multilayer neck network features obtained above. , , Features of multilayer neck networks with high-resolution input , , The features are fed into the Feature Fusion EF module for feature fusion, and the corresponding relationship is as follows: and (i=5,4,3), obtain the fusion features , , (Equivalent to the aforementioned second set of fusion features).

[0095] Directly utilizing high-resolution features for knowledge distillation results in poor distillation performance due to scale discrepancies. Therefore, it is necessary to train the student network using fused features adapted to both high-resolution and low-resolution inputs. This can be achieved by leveraging the multi-layered neck network features of the student network. , , Features of integration with teacher networks , , The data is fed into the Feature Distillation Module (AFD) for knowledge distillation and the distillation loss is calculated. and (i=5,4,3) have the same size and number of channels, and the neck network feature distillation loss. (equivalent to the fifth input loss mentioned above), where L1 represents the L1 norm. Figure 8 This is a schematic diagram of the structure of the characteristic distillation module AFD module provided in the embodiments of this application, as shown below. Figure 8 As shown, and After FcsaNet attention processing, more significant information can be extracted from the features. Therefore, this information, as input for distillation, not only improves the purity of the distillation but also better reflects the differences in the main target information in the features. Figure 9 This is a schematic diagram of the structure of the FcsaNet attention module provided in the embodiments of this application, as shown below. Figure 9 As shown, the input features first need to be divided into N groups on an average basis according to the channel dimension. Each group has the same number of channels, and then their respective frequency components are calculated. , The dimension is 2-dimensional, and it is necessary to... Each value is calculated to obtain Calculate as follows: Where W and H are Width and height, w, h represent the current calculated values ​​at... The coordinate values ​​can be obtained by iterating through the coordinates in sequence. ; , The frequency component index is obtained based on optimal performance; for each group Its corresponding Multiply, then activate via the Sigmoid function, and then combine with the input features. Multiplication yields features containing diverse information from both high-frequency and low-frequency channels. Then respectively Mean pooling and max pooling are performed, and the pooling results are concatenated along the channel dimension. Then, they are activated by DSConv convolution and the Signoid activation function to obtain features that contain spatial attention information. ,Will and Multiply to obtain the final output . It extracts data from the input. The multi-frequency channel information was also extracted. The diverse spatial information allows the final features to contain richer channel information and more diverse spatial information, and the subsequent distillation loss calculation can better reflect the main information differences of each feature.

[0096] It is understandable that the AFD module is only used during the distillation training of the student network model, and not during the final prediction process.

[0097] During neural network training, shallow neural network features converge first and initially contain more shape and pixel information. As the number of training iterations increases, the speech information contained in deep neural network features gradually converges. Therefore, the backbone network features of the teacher network are used for distillation training of the student network. Because the backbone network features of the student network... (i=5,4,3,2,1) and the backbone network characteristics of the teacher network The number of channels is the same, but the size is different. yes of Therefore, it is necessary to Scaling to the mean pooling level Obtain the same size (Equivalent to the aforementioned sixth backbone network feature set). Then, the backbone network distillation loss is calculated. (Equivalent to the sixth input loss mentioned above), where The balancing parameter has a value range of [0, 0.5]. In the early stages of training, the shallow backbone network corresponds to... They occupy a larger proportion, and with the increase of training iterations, the deeper backbone network features correspond to... Gradually increase, until finally, all Both are 0.5.

[0098] Based on low-resolution input loss Neck network characteristic distillation loss and backbone network distillation losses The total loss function of the student network model is determined as shown in equation (2), where, and All of these are equilibrium parameters. Used to balance the difference between distillation loss and target detection loss. The value range is [0.5, 0.9], at the beginning of training. The value is 0.5, and as the number of training iterations increases, It gradually increased to 0.9; Used to balance the distillation loss of neck network features and backbone network distillation losses Differences The value range is [0.2, 1]. In the early stages of training, and Occupying the same proportion, with a value of 1, as the number of training iterations increases, The weight gradually decreases, and eventually The value is 0.2.

[0099] Similarly, if a defect exists in the image, one defect is randomly selected, and a rectangular region containing that defect in the original image is randomly selected, requiring that the length and width of this region be the same as those in the original image. If other defects are located in the region, they are retained. Then, the image is cropped from this region and enlarged to the same size as the original image. The defect location in the new image is calculated based on the cropping coordinates and used as the label for the new image. This generates a new pair of high- and low-resolution images. Because the cropping is random, it reduces the possibility of overfitting to some extent, increases the amount of data, and because the defective target occupies a larger proportion of the image, it is beneficial for object detection and knowledge distillation to learn more accurate defect features. The new image pair is then fed into the student network model for training using the aforementioned steps. The rules are the same as in step 203, but the images are obtained randomly. If, during training and iteration, eight images are taken from the dataset, then eight images are randomly cropped and labeled sequentially before being fed into the model for calculation and training. This is equivalent to replacing the original input with a new one.

[0100] Based on the total loss function The student network model is trained, and the completed student network model is the final fabric defect detection model.

[0101] It should be noted that, Figure 7 The neck network characteristics in the above Figure 4 Analogical understanding of network features in the middle neck Figure 7 The prediction network in the middle is equivalent to Figure 3 The prediction network modules 1, 2, 3, and 4 are used. Furthermore, in step 204, only prediction network modules 1, 2, 3, and 4 are trained. Figure 7 The parameters of the student network model in the lower half and the teacher network model in the upper half are no longer updated. In other words, the student network model in the lower half is the final fabric defect detection model.

[0102] Step 205: Use the fabric defect detection model to detect fabric defects.

[0103] Typically, due to the large size of the original acquired images, they need to be cropped. After cropping, they are batch-processed and sent to the inspection equipment for batch testing. The results are then analyzed to determine whether defects exist in the affected areas, and to obtain the type and coordinates of the defects. Because the images are cropped, the defect coordinates are transformed to pinpoint the exact location of the defects in the original image. Here, both the cropping and coordinate transformation processes are executed by additional models or auxiliary code, not by the fabric defect detection model itself.

[0104] This application proposes a low-resolution fabric defect detection method based on multi-scale iterative knowledge distillation of features. Key features include: a novel feature distillation loss calculation method that enables the calculation of more important spatial and channel-level information within each feature during distillation; a novel attention module that simultaneously extracts multi-frequency channel attention information and rich spatial attention information; a novel feature fusion module that focuses on and fuses spatial information across multiple channel dimensions of features, resulting in a more comprehensive reflection of the spatial information between features; a knowledge distillation method combining EMA and object detection networks for low-resolution fabric defect detection, which improves feature spatial information extraction without significantly increasing model parameters, thereby enhancing detection accuracy and knowledge distillation efficiency; and an iterative knowledge distillation loss function training method, including a distillation method that varies weight factors for various features such as the backbone and neck networks during training. This makes model training during knowledge distillation more consistent with deep learning feature distribution, leading to faster convergence and more accurate extraction of shape and pixel information. This paper proposes a method for generating diverse engineering datasets that facilitate knowledge distillation. By cropping images and preserving the main defect areas, new image pairs are generated for target detection and knowledge distillation. This method effectively reduces interference from background distillation, improves the ability of target detection and knowledge distillation to capture defect features, and enhances the model's defect detection capabilities.

[0105] In addition, different feature distillation loss functions can be used to calculate the feature distillation loss, such as calculating attention weights based on other dimensions or features, rather than focusing on the spatial or channel levels. For new attention modules, existing attention mechanisms such as the Squeeze-and-Excitation (SE) module or the CBAM module can be introduced, focusing on other dimensions (e.g., using only a global attention mechanism) without involving multi-frequency channel attention information extraction. For new feature fusion modules, different feature fusion methods can be used, such as directly performing feature fusion through element-wise addition, splicing, or simple linear transformations, without focusing on spatial information in multiple channel dimensions. The technical solution of this application can also be applied to the industrial production of fabrics. With the improvement of living standards, the demand for fabrics and related textile materials is increasing, and high-performance, low-resolution fabric defect detection methods have significant production value.

[0106] Figure 10 This is a schematic diagram of the structural composition of the detection device provided in the embodiments of this application, as shown below. Figure 10 As shown, the detection device includes: The construction unit 1001 is used to construct a first network model and a second network model, which are used to detect defects in the object to be detected; and to construct an image training dataset, which includes multiple sets of image training data, wherein each set of image training data includes a first image and a second image, the resolution of the first image is lower than the resolution of the second image, and the first image and the second image are images of the object to be detected. Training unit 1002 is used to train a first network model using an image training dataset to obtain a trained first network model; and to train a second network model based on the image training dataset and the trained first network model to obtain a trained second network model. The detection unit 1003 is used to detect defects in the first object using the trained second network model, and obtain the defect detection result of the first object. The first object and the object to be detected have the same attributes.

[0107] In some embodiments, the training unit 1002 is configured to input a first image from each set of image training data into a first network model to obtain a first neck network feature set, a first prediction network feature set, and a first input loss; input a second image from each set of image training data into the first network model to obtain a second neck network feature set, a second prediction network feature set, and a second input loss; obtain a third input loss based on the first neck network feature set and the second neck network feature set; determine a first total loss function of the first network model based on the first input loss, the second input loss, and the third input loss; and adjust the parameters of the first network model so that the first total loss function reaches a first target loss value to obtain the trained first network model.

[0108] In some implementations, the first network model includes a first backbone network module, a first neck network module, a first prediction network module, a first multi-scale attention module, and a first feature fusion module.

[0109] In some implementations, the training unit 1002 is configured to input the first image from each set of image training data into a first backbone network module to obtain a first backbone network feature set; input the first backbone network feature set into a first multi-scale attention module to obtain a first attention feature set; input the first attention feature set into a first neck network module to obtain a first neck network feature set; input the first neck network feature set into a first prediction network module to obtain a first prediction network feature set; and determine a first input loss based on the first prediction network feature set.

[0110] In some implementations, the training unit 1002 is configured to input the second image from each set of image training data into the first backbone network module to obtain a second backbone network feature set; input the second backbone network feature set into the first multi-scale attention module to obtain a second attention feature set; input the second attention feature set into the first neck network module to obtain a second neck network feature set; input the second neck network feature set into the first prediction network module to obtain a second prediction network feature set; and determine the second input loss based on the second prediction network feature set.

[0111] In some embodiments, the training unit 1002 is used to input a first neck network feature set and a second neck network feature set into a first feature fusion module to obtain a first fused feature set; input the first fused feature set into a first prediction network module to obtain a third prediction network feature set; and determine a third input loss based on the third prediction network feature set.

[0112] In some embodiments, the construction unit 1001 is used to determine a first new image training group based on the first image or the second image if the first image or the second image includes a defective portion. The first new image training group includes a third image and a fourth image, wherein the resolution of the third image is lower than the resolution of the fourth image. The training unit 1002 is used to update the first network model using the first new image training group.

[0113] In some embodiments, the training unit 1002 is used to load the parameters of the trained first network model into the second network model to obtain an intermediate second network model; input the first image of each set of image training data into the intermediate second network model to obtain a third neck network feature set, a third backbone network feature set, a fourth prediction network feature set, and a fourth input loss; input the first image of each set of image training data into the trained first network model to obtain a fourth neck network feature set; input the second image of each set of image training data into the trained first network model to obtain a fifth neck feature network set and a fourth backbone network feature set; obtain a second fusion feature set based on the fourth and fifth neck network feature sets; determine a fifth input loss based on the third neck feature set and the second fusion feature set; determine a sixth input loss based on the third and fourth backbone network feature sets; determine a second total loss function of the second network model based on the fourth, fifth, and sixth input losses; and adjust the parameters of the second network model so that the second total loss function reaches a second target loss value to obtain the trained second network model.

[0114] In some implementations, the intermediate second training network model includes a second backbone network module, a second neck network module, a second prediction network module, a second multi-scale attention module, and a first feature distillation module.

[0115] In some implementations, the training unit 1002 is configured to input the first image of each set of image training data into a second backbone network module to obtain a third backbone network feature set; input the third backbone network feature set into a second multi-scale attention module to obtain a third attention feature set; input the third attention feature set into a second neck network module to obtain a third neck network feature set; input the third neck network feature set into a second prediction network module to obtain a fourth prediction network feature set; and determine a fourth input loss based on the fourth prediction network feature set.

[0116] In some implementations, the training unit 1002 is used to input the third neck feature set and the second fused feature set into the first feature distillation module to obtain the fifth input loss.

[0117] In some embodiments, the construction unit 1001 is used to determine a second new image training group based on the first image or the second image if the first image or the second image includes a defective portion. The second new image training group includes a fifth image and a sixth image, wherein the resolution of the fifth image is lower than that of the sixth image. The training unit 1002 is used to update a second network model using the second new image training group.

[0118] Those skilled in the art should understand that Figure 10 The functions of each unit in the detection device shown can be understood by referring to the relevant description of the aforementioned method. Figure 10 The functions of each unit in the detection device shown can be implemented by a program running on a processor or by specific logic circuits.

[0119] Figure 11 This is a schematic structural diagram of a testing device 1100 provided in an embodiment of this application. Figure 11 The detection device 1100 shown includes a processor 1110, which can call and run computer programs from memory to implement the methods in the embodiments of this application.

[0120] Optionally, such as Figure 11 As shown, the detection device 1100 may further include a memory 1120. The processor 1110 can retrieve and run computer programs from the memory 1120 to implement the methods described in this embodiment.

[0121] The memory 1120 can be a separate device independent of the processor 1110, or it can be integrated into the processor 1110.

[0122] Optionally, such as Figure 11 As shown, the detection device 1100 may also include a transceiver 1130. The processor 1110 can control the transceiver 1130 to communicate with other devices. Specifically, it can send information or data to other devices or receive information or data sent by other devices.

[0123] The transceiver 1130 may include a transmitter and a receiver. The transceiver 1130 may further include an antenna, and the number of antennas may be one or more.

[0124] The detection device 1100 can implement the corresponding processes implemented by the detection device in the various methods of the embodiments of this application, which will not be described in detail here for the sake of brevity.

[0125] Figure 12 This is a schematic structural diagram of the chip according to an embodiment of this application. Figure 12 The chip 1200 shown includes a processor 1210, which can call and run computer programs from memory to implement the methods in the embodiments of this application.

[0126] Optionally, such as Figure 12 As shown, chip 1200 may further include memory 1220. Processor 1210 can retrieve and run computer programs from memory 1220 to implement the methods described in this embodiment.

[0127] The memory 1220 can be a separate device independent of the processor 1210, or it can be integrated into the processor 1210.

[0128] Optionally, the chip 1200 may also include an input interface 1230. The processor 1210 can control the input interface 1230 to communicate with other devices or chips; specifically, it can acquire information or data sent by other devices or chips.

[0129] Optionally, the chip 1200 may also include an output interface 1240. The processor 1210 can control the output interface 1240 to communicate with other devices or chips, specifically, to output information or data to other devices or chips.

[0130] This chip can implement the corresponding processes implemented by the detection device in the various methods of the embodiments of this application, which will not be described in detail here for the sake of brevity.

[0131] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0132] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0133] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0134] It should be understood that the above-described memory is exemplary and not a limiting description. For example, the memory in the embodiments of this application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM), etc. That is to say, the memory in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0135] This application also provides a computer program product, including a computer program.

[0136] When executed by the processor, the computer program implements the corresponding processes implemented by the detection device in the various methods of the embodiments of this application, which will not be described in detail here for the sake of brevity.

[0137] This application also provides a computer-readable storage medium for storing computer programs.

[0138] The computer program causes the computer to execute the corresponding processes implemented by the detection device in the various methods of the embodiments of this application, which will not be described in detail here for the sake of brevity.

[0139] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0140] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0144] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] The preferred embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application. Moreover, without conflict, the various embodiments and / or the technical features in the various embodiments described in this application can be arbitrarily combined with the prior art, and the resulting technical solutions should also fall within the protection scope of this application.

[0146] It should be understood that in the various method embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0147] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A detection method, characterized in that, The method includes: Construct a first network model and a second network model, which are used to detect defects in the object to be detected; Construct an image training dataset, which includes multiple sets of image training data. Each set of image training data includes a first image and a second image. The resolution of the first image is lower than that of the second image. The first image and the second image are images of the object to be detected. The first network model is trained using the image training dataset to obtain the trained first network model. The second network model is trained based on the image training dataset and the trained first network model to obtain the trained second network model. The trained second network model is used to detect defects in the first object, and the defect detection result of the first object is obtained. The first object and the object to be detected have the same attributes.

2. The method according to claim 1, characterized in that, The step of training the first network model using the image training dataset to obtain the trained first network model includes: The first image from each set of image training data is input into the first network model to obtain a first neck network feature set, a first prediction network feature set, and a first input loss. The second image from each set of image training data is input into the first network model to obtain a second neck network feature set, a second prediction network feature set, and a second input loss. The third input loss is obtained based on the first neck network feature set and the second neck network feature set; The first total loss function of the first network model is determined based on the first input loss, the second input loss, and the third input loss. The parameters of the first network model are adjusted so that the first total loss function reaches the first target loss value, thus obtaining the trained first network model.

3. The method according to claim 2, characterized in that, The first network model includes a first backbone network module, a first neck network module, a first prediction network module, a first multi-scale attention module, and a first feature fusion module; The step of inputting the first image from each set of image training data into the first network model to obtain a first neck network feature set, a first prediction network feature set, and a first input loss includes: The first image from each set of image training data is input into the first backbone network module to obtain the first backbone network feature set; The first backbone network feature set is input into the first multi-scale attention module to obtain the first attention feature set; The first attention feature set is input into the first neck network module to obtain the first neck network feature set; The first neck network feature set is input into the first prediction network module to obtain the first prediction network feature set; The first input loss is determined based on the first prediction network feature set; The step of inputting the second image from each set of image training data into the first network model to obtain a second neck network feature set, a second prediction network feature set, and a second input loss includes: The second image from each set of image training data is input into the first backbone network module to obtain the second backbone network feature set; The second backbone network feature set is input into the first multi-scale attention module to obtain the second attention feature set; The second attention feature set is input into the first neck network module to obtain the second neck network feature set; The second neck network feature set is input into the first prediction network module to obtain the second prediction network feature set; The second input loss is determined based on the second prediction network feature set; The step of obtaining the third input loss based on the first neck network feature set and the second neck network feature set includes: The first neck network feature set and the second neck network feature set are input into the first feature fusion module to obtain the first fused feature set; The first fused feature set is input into the first prediction network module to obtain the third prediction network feature set; The third input loss is determined based on the third prediction network feature set.

4. The method according to claim 2 or 3, characterized in that, The method further includes: If the first image or the second image includes a defective portion, a first new image training group is determined based on the first image or the second image. The first new image training group includes a third image and a fourth image, wherein the resolution of the third image is lower than the resolution of the fourth image. The first network model is updated using the first new image training set.

5. The method according to any one of claims 1 to 3, characterized in that, The step of training the second network model based on the image training data set and the trained first network model to obtain the trained second network model includes: The parameters of the trained first network model are loaded into the second network model to obtain the intermediate second network model; The first image of each set of image training data is input into the intermediate second network model to obtain the third neck network feature set, the third backbone network feature set, the fourth prediction network feature set, and the fourth input loss; The first image of each set of image training data is input into the first network model after training to obtain the fourth neck network feature set; The second image of each set of image training data is input into the trained first network model to obtain the fifth neck feature network set and the fourth backbone network feature set. A second fusion feature set is obtained based on the fourth neck network feature set and the fifth neck network feature set; The fifth input loss is determined based on the third neck feature set and the second fusion feature set; The sixth input loss is determined based on the third backbone network feature set and the fourth backbone network feature set; The second total loss function of the second network model is determined based on the fourth input loss, the fifth input loss, and the sixth input loss; The parameters of the second network model are adjusted so that the second total loss function reaches the second target loss value, thus obtaining the trained second network model.

6. The method according to claim 5, characterized in that, The intermediate second training network model includes a second backbone network module, a second neck network module, a second prediction network module, a second multi-scale attention module, and a first feature distillation module; The step of inputting the first image of each set of image training data into the intermediate second network model to obtain the third neck network feature set, the third backbone network feature set, the fourth prediction network feature set, and the fourth input loss includes: The first image of each set of image training data is input into the second backbone network module to obtain the third backbone network feature set; The third backbone network feature set is input into the second multi-scale attention module to obtain the third attention feature set; The third attention feature set is input into the second neck network module to obtain the third neck network feature set; The third neck network feature set is input into the second prediction network module to obtain the fourth prediction network feature set; The fourth input loss is determined based on the fourth prediction network feature set; The step of determining the fifth input loss based on the third neck feature set and the second fused feature set includes: The third neck feature set and the second fused feature set are input into the first feature distillation module to obtain the fifth input loss.

7. The method according to claim 5 or 6, characterized in that, The method further includes: If the first image or the second image includes a defective portion, a second new image training group is determined based on the first image or the second image. The second new image training group includes a fifth image and a sixth image, wherein the resolution of the fifth image is lower than the resolution of the sixth image. The second network model is updated using the second new image training group.

8. A testing device, characterized in that, include: A processor and a memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform the method as described in any one of claims 1 to 7.

9. A computer program product, characterized in that, include: A computer program that, when executed by a processor, implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 7.

Images