Training method and related device for regional detection model in spinning process

Abstract

The present disclosure provides a training method and related device for a region detection model in a spinning process. It relates to the field of artificial intelligence technology. The method comprises: sampling from a spinning task data set and a playback memory to obtain mixed data; inputting the mixed data set into a student model and a pre-trained teacher model respectively to obtain a first prediction result output by the teacher model and a second prediction result output by the student model; for a spinning image sample, the student model is used to detect a candidate fault region in the spinning image sample; cropping the candidate fault region from the spinning image sample to obtain a first sub-image; inputting the first sub-image into a yarn fault detection model to obtain a classification result of the first sub-image, the classification result including a floating yarn discrimination result and a broken yarn discrimination result; optimizing the parameters of the student model based on a first loss determined by the first prediction result and the second prediction result, and a second loss between the classification result and the classification label to obtain a region detection model.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to technical fields such as spinning technology and deep learning. Background Art

[0002] In the industrial spinning process, yarn quality directly determines the performance and quality of the final product. As the core output of the spinning process, the quality of yarn plays a decisive role in key performance indicators of downstream textiles, such as strength, toughness, and dyeing consistency. Summary of the Invention

[0003] The present disclosure provides a method for training a region detection model in a spinning process to solve or alleviate one or more technical problems in related technologies.

[0004] In a first aspect, the present disclosure provides a method for training a region detection model in a spinning process, comprising:

[0005] Sampling is performed from a spinning task data set and a playback memory to obtain mixed data; wherein the spinning task data set includes spinning image samples obtained by collecting images of a spinning box, and the playback memory includes historical sample images;

[0006] The mixed dataset is input into the student model and the pre-trained teacher model respectively, and a first prediction result output by the teacher model and a second prediction result output by the student model are obtained; for the spinning image sample, the student model is used to detect the candidate fault area in the spinning image sample;

[0007] cropping a candidate fault region from the spinning image sample to obtain a first sub-image;

[0008] Inputting the first sub-image into a yarn fault detection model to obtain a classification result of the first sub-image, the classification result including a floating yarn identification result and a broken yarn identification result;

[0009] Based on a first loss determined between the first prediction result and the second prediction result, and a second loss between the classification result and the classification label, parameters of the student model are optimized to obtain a region detection model.

[0010] In a second aspect, the present disclosure provides a training device for a region detection model in a spinning process, comprising:

[0011] A sampling module is used to sample from the spinning task data set and the playback memory to obtain mixed data; wherein the spinning task data set includes spinning image samples obtained by image acquisition of the spinning box, and the playback memory includes historical sample images;

[0012] A prediction module is configured to input the mixed dataset into a student model and a pre-trained teacher model, respectively, to obtain a first prediction result output by the teacher model and a second prediction result output by the student model; for a spinning image sample, the student model is configured to detect candidate fault regions in the spinning image sample;

[0013] a cropping module, configured to crop a candidate fault region from the spinning image sample to obtain a first sub-image;

[0014] a classification module, configured to input the first sub-image into a yarn fault detection model to obtain a classification result of the first sub-image, wherein the classification result includes a floating yarn identification result and a broken yarn identification result;

[0015] An optimization module is used to optimize parameters of the student model based on a first loss determined by the first prediction result and the second prediction result, and a second loss between the classification result and the classification label to obtain a region detection model.

[0016] In a third aspect, the present disclosure provides a method for detecting yarn faults in a spinning process, comprising:

[0017] Capturing images of the spinning beam to obtain images to be processed;

[0018] Inputting the image to be processed into the region detection model trained by the training method of the region detection model in the spinning process to obtain the fault candidate region in the image to be processed;

[0019] cropping a fault candidate region from the spinning image sample to obtain a second sub-image;

[0020] The second sub-image is input into the yarn fault detection model to obtain a classification result of the second sub-image, wherein the classification result includes a floating yarn identification result and a broken yarn identification result.

[0021] In a fourth aspect, the present disclosure provides a device for detecting yarn faults in a spinning process, comprising:

[0022] An acquisition module is used to acquire images of the spinning box to obtain images to be processed;

[0023] An input module is used to input the image to be processed into the region detection model trained by the training method of the region detection model in the spinning process, and obtain the fault candidate region in the image to be processed;

[0024] a processing module, configured to crop a candidate fault region from the spinning image sample to obtain a second sub-image;

[0025] The discrimination module is used to input the second sub-image into the yarn fault detection model to obtain a classification result of the second sub-image, wherein the classification result includes a floating yarn discrimination result and a broken yarn discrimination result.

[0026] According to a fifth aspect, an electronic device is provided, including:

[0027] at least one processor; and

[0028] a memory communicatively connected to the at least one processor; wherein,

[0029] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method in the embodiments of the present disclosure.

[0030] In a sixth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0031] In a seventh aspect, a computer program product is provided, comprising a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0032] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments provided in accordance with the present disclosure and should not be regarded as limiting the scope of the present disclosure.

[0034] Figure 1 is a flow chart of a method for training a region detection model in a spinning process according to an embodiment of the present disclosure;

[0035] Figure 2 is a schematic diagram of a process for determining a first loss according to an embodiment of the present disclosure;

[0036] Figure 3 is a schematic flow chart of a method for detecting yarn faults in a spinning process according to one embodiment of the present disclosure;

[0037] Figure 4 1 is a flow chart of a method for training a wire fault detection model according to an embodiment of the present disclosure;

[0038] Figure 5 is a schematic diagram of a process for obtaining a first global feature of a sample image according to an embodiment of the present disclosure;

[0039] Figure 6 is a schematic diagram of a process for obtaining intermediate semantic features of a sample image according to an embodiment of the present disclosure;

[0040] Figure 7 is an exemplary mask feature map according to an embodiment of the present disclosure;

[0041] Figure 8 is a schematic diagram of a process for obtaining fine-grained features of a sample image according to an embodiment of the present disclosure;

[0042] Figure 9 is a schematic diagram of a process for obtaining fusion features according to an embodiment of the present disclosure;

[0043] Figure 10 is a schematic diagram of a process for optimizing model parameters of a model to be trained according to an embodiment of the present disclosure;

[0044] Figure 11 1 is a schematic structural diagram of a training device for a region detection model in a spinning process according to an embodiment of the present disclosure;

[0045] Figure 12 is a structural schematic diagram of a yarn fault detection device in a spinning process according to an embodiment of the present disclosure;

[0046] Figure 13 It is a block diagram of an electronic device used to implement the training method of the area detection model in the spinning process and / or the yarn fault detection method in the spinning process according to the embodiments of the present disclosure. DETAILED DESCRIPTION

[0047] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0048] In addition, it should be noted that the terms "first," "second," and the like in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with certain aspects of the present application as detailed in the appended claims.

[0049] It should be noted that, unless it is explicitly stated that there is a sequence of execution between different operations shown in the flowchart in the embodiments of the present disclosure, or there is a sequence of execution between different operations in technical implementation, otherwise, the execution order between multiple operations may not be prioritized, and multiple operations may also be executed simultaneously.

[0050] In the chemical fiber industry, the spinning process is a critical step in yarn production, and its quality directly impacts the performance of the final product. However, during the spinning process, faults such as drifting and broken yarns often occur due to factors such as equipment wear, environmental changes, and operational errors. If these faults are not detected and addressed promptly, they can lead to product defects or even production stoppages.

[0051] However, traditional detection methods mostly rely on manual observation or rule-based image processing technology, which has disadvantages such as low detection efficiency and insufficient accuracy, and are difficult to meet the high efficiency and accuracy requirements in actual production.

[0052] In light of this, the present invention provides a method for training a region detection model in a spinning process and a method for detecting yarn faults in a spinning process. This method can train a region detection model, which can then be used to preliminarily screen areas of a spinning manifold image for possible drifting or broken yarn faults. The pre-trained yarn fault detection model is then used to further detect these areas to determine if drifting or broken yarn faults exist.

[0053] It should be noted that the main types of spun yarns involved in the embodiments of the present disclosure may include one or more of partially oriented yarns (POY), fully drawn yarns (FDY), polyester staple fibers, etc. For example, the types of yarns may specifically include polyester partially oriented yarns, polyester fully drawn yarns, polyester drawn yarns, polyester staple fibers, etc.

[0054] The following describes the training method of the regional detection model and the method for detecting yarn faults in the spinning process.

[0055] like Figure 1 FIG. 1 is a flow chart of a method for training a region detection model in a spinning process according to an embodiment of the present disclosure, including the following contents:

[0056] S101, sampling from a spinning task data set and a playback memory to obtain mixed data; wherein the spinning task data set includes spinning image samples obtained by collecting images of a spinning box, and the playback memory includes historical sample images.

[0057] The spinning task dataset is a dataset specifically designed for spinning tasks. It contains numerous spinning image samples captured by an image acquisition device of a spinning manifold. These images are annotated with bounding boxes and category labels for faulty areas within the spinning images, which are used for supervised learning of emerging fault features. In the disclosed embodiments, these emerging fault features specifically refer to features used to identify floating yarn faults and broken yarn faults.

[0058] The replay memory stores historical sample images, including high-quality data accumulated during the previous training of the teacher model. This includes labeled fault sample images, rare fault sample images that have not appeared for a long time, and sample images of similar task types that are not related to the spinning task and stored during previous training. This prevents the model from forgetting old knowledge due to focusing only on new data, thereby facilitating the transfer of knowledge learned by the teacher model to the student model.

[0059] During implementation, samples can be taken from the spinning task dataset and the playback memory respectively, and then they can be merged in proportion to form mixed data. The specific proportion needs to be adjusted according to the actual application scenario.

[0060] S102, inputting the mixed data set into the student model and the pre-trained teacher model respectively, obtaining a first prediction result output by the teacher model and a second prediction result output by the student model; for the spinning image sample, the student model is used to detect the fault candidate area in the spinning image sample.

[0061] The pre-trained teacher model is a fully trained, high-performance model, typically with a complex structure and large number of parameters, such as an advanced version of the YOLO (You Only Look Once) model. It is used to guide the student model. The student model is a lightweight model that learns both the teacher model's knowledge and true labels during training. For example, a simplified version of the teacher model can be used as the student model. Through knowledge distillation, the teacher model transfers its rich feature extraction capabilities and detection experience to the student model, guiding it to accurately detect potential fault areas in spinning images.

[0062] After the mixed dataset is fed into the student model and the pre-trained teacher model, the teacher model outputs a first prediction result, which includes the teacher model's probability distribution of the object categories in the image and the bounding box prediction of the potential fault area provided by the teacher model. Correspondingly, the student model outputs a second prediction result, which includes the student model's probability distribution of the object categories in the image and the bounding box prediction of the potential fault area provided by the student model.

[0063] S103 , cropping a candidate fault region from the spinning image sample to obtain a first sub-image.

[0064] Specifically, based on the candidate fault regions detected by the student model, the corresponding regions are cropped from the spinning image sample to obtain a first sub-image. To preserve contextual features, the cropped first sub-image can be slightly larger than the candidate fault regions, for example, by extending each side by a specified pixel value.

[0065] During implementation, if multiple predicted fault candidate regions are present, the cropped fault candidate regions can be independently used as the first sub-image. Alternatively, if the cropped fault candidate regions are adjacent to each other, these adjacent regions can be merged, and the merged image can be used as the first sub-image. For example, adjacent regions within a distance of n pixels can be considered adjacent, where n is a preset positive integer. Merging adjacent regions can thus maximize the preservation of correlations between multiple fault candidate regions and contextual features, thereby improving the accuracy of thread fault detection.

[0066] S104: Input the first sub-image into a yarn fault detection model to obtain a classification result of the first sub-image, wherein the classification result includes a floating yarn identification result and a broken yarn identification result.

[0067] These first sub-images are then input into the yarn fault detection model to obtain classification results. The classification results include the results of loose yarn identification and broken yarn identification, that is, a specific judgment is made on the fault type to clarify whether it is a loose yarn or broken yarn fault, or a fault with no loose yarn or broken yarn.

[0068] S105 , optimizing parameters of the student model based on a first loss determined between the first prediction result and the second prediction result, and a second loss between the classification result and the classification label, to obtain a region detection model.

[0069] The first loss reflects the distillation difference between the student model and the teacher model. By minimizing this loss, the student model can learn the knowledge of the teacher model and thus improve its own performance.

[0070] The second loss reflects the difference between the classification results of the wire fault detection model and the true classification label. By minimizing this loss, the student model can focus more on the characteristics of the wire fault area, improving the detection ability of wire faults.

[0071] In the disclosed embodiment, by sampling from the spinning task data set and the playback memory, the model can be prevented from forgetting old knowledge due to focusing only on new data. The teacher model is pre-trained and has certain prior knowledge and predictive capabilities. By inputting the mixed data set into the teacher model, the first prediction result of the teacher model can be obtained, which serves as a reference and guide for the student model. Comparing the output results of the teacher model with the output results of the student model can help the student model to learn and optimize better, so as to speed up the training speed of the student model and improve the training effect. Cropping the fault candidate area detected by the student model into an independent first sub-image can remove the part of the sample image that is not related to fault detection, reduce interference factors, and improve the subsequent student model's detection accuracy of the fault area. Based on the first loss determined by the first prediction result and the second prediction result, and the second loss between the classification result and the classification label, the parameters of the student model are optimized, which can enable the student model to imitate the detection behavior of the teacher model while ensuring that its classification output conforms to the actual label, thereby improving the accuracy of the student model in detecting the fault candidate area in the spinning image sample.

[0072] The wire fault detection model in the embodiment of the present disclosure may be a pre-trained model, or may be combined with the to-be-trained student model of the embodiment of the present disclosure to optimize model parameters.

[0073] In the embodiment of the present disclosure, sampling from the spinning task data set and the playback memory to obtain mixed data can be achieved based on the following steps:

[0074] Step A1: Determine the sampling ratio of various old task categories in the spinning task dataset and the playback memory.

[0075] In step A2, the spinning task dataset and the playback memory are sampled based on the sampling ratio.

[0076] During implementation, the sampling ratios for various categories of legacy tasks in the spinning task dataset and replay memory can be determined based on the needs of the actual business scenario. For example, if certain categories in the legacy tasks are more closely related to the spinning task, the sampling ratio of these related categories in the replay memory can be appropriately increased, while the sampling ratio of the spinning task dataset can be determined simultaneously. Alternatively, if the number of samples of certain categories in the legacy tasks in the replay memory is relatively small, the sampling ratio of these categories can be appropriately increased, while the sampling ratio of samples of other categories can be reduced.

[0077] After determining the sampling ratio of each old task category, random sampling can be performed from the spinning task dataset and the various old task categories in the playback memory according to the ratio, and the sample data obtained by sampling can be combined into mixed data for subsequent training process.

[0078] In the embodiment of the present disclosure, by determining the sampling ratio of various old task categories in the spinning task dataset and the replay memory, and sampling the spinning task dataset and the replay memory based on the sampling ratio, the number of samples of different task categories can be balanced, and the deviation of sample categories can be reduced as much as possible, so that samples of each category can be fully learned, thereby promoting the integration and transfer of new and old knowledge and enhancing the generalization ability of the model.

[0079] In the embodiment of the present disclosure, the first loss is determined based on the first prediction result and the second prediction result, such as Figure 2 As shown, it can be implemented as follows:

[0080] S201, determining a supervision loss based on the first prediction result and the second prediction result.

[0081] Supervision loss is a loss function that measures the difference between a model's predictions and the true labels. In the student-teacher model architecture, supervision loss uses the teacher model's predictions as a relatively more accurate reference (i.e., the true labels) to supervise the student model's output, facilitating the transfer of teacher model knowledge to the student model through knowledge distillation.

[0082] In practice, determining the supervision loss can be achieved based on the following steps:

[0083] Step B1: for an old sample of an old task category sampled from the playback memory, determine the first candidate frame with the highest intersection-over-union ratio with the true bounding box of the old sample from among multiple first candidate frames of the old sample in the second prediction result, and select it as the positive sample candidate frame;

[0084] A bounding box is a rectangular box used to calibrate the target position in target detection tasks. The position of the target in the image is usually defined by four parameters. These four parameters are usually the center coordinates (x, y), width w, and height h of the bounding box.

[0085] The student model and the teacher model often output feature maps of the same size. Each element in the feature map corresponds to a position point in the input original image, which can be regarded as an anchor point. Each anchor point can correspond to multiple predicted bounding boxes.

[0086] In the disclosed embodiments, a bounding box is used to clearly define the location range of the fault candidate region in the sample image, allowing the learning model to accurately predict the fault candidate region. The first candidate box is the fault candidate region predicted by the student model; the ground truth bounding box is the actual location range of the fault candidate region. This actual location range can be included in the first prediction result of the teacher model or determined through manual annotation results.

[0087] From the multiple first candidate boxes of the old sample in the second prediction result, the first candidate box with the highest intersection-over-union (IoU) ratio with the ground-truth bounding box of the old sample is selected as the positive candidate box. IoU is a metric that measures the degree of overlap between two bounding boxes. A higher IoU value indicates a closer match between the first candidate box and the ground-truth bounding box. Selecting the first candidate box with the highest IoU ratio as the positive candidate box allows the student model to focus on learning good predictions, reducing the student model's learning of noisy knowledge and improving distillation efficiency.

[0088] Step B2: Mark the positive sample candidate box as a foreground value, and mark the first candidate boxes other than the positive sample candidate box in the multiple first candidate boxes as background values to obtain a mask matrix;

[0089] During implementation, the positive sample candidate box can be marked as the foreground value, and its corresponding value is recorded as 1 to indicate that this area is a fault candidate area that needs to be detected; the first candidate box except the positive sample candidate box can be marked as the background value, and its corresponding value is recorded as 0 to indicate that this area is a fault candidate area that does not need to be detected. Through this marking method, the mask matrix is obtained.

[0090] Step B3: determining a bounding box regression loss, a confidence loss of the bounding box, and a classification loss of the bounding box between the first prediction result and the second prediction result based on the mask matrix;

[0091] Among them, the bounding box regression loss represents the deviation between the predicted position of the positive sample candidate box and the true bounding box position, and its calculation method is shown in formula (1):

[0092]

[0093] In formula (1), represents the bounding box regression loss; is the center coordinate of the true bounding box, which can be the bounding box predicted by the teacher model as the true bounding box in the disclosed embodiment; is the width and height of the ground-truth bounding box; is the center coordinate of the predicted bounding box; is the width and height of the predicted bounding box; N is the number of positive samples.

[0094] The confidence loss of the bounding box indicates the degree of deviation between the confidence of the judgment of whether the bounding box contains the fault candidate area and the actual situation. It is calculated as shown in formula (2):

[0095]

[0096] In formula (2), represents the confidence loss of the bounding box; is the true confidence (which can be understood as the confidence output by the teacher model); The confidence level of the student model prediction; BCE () is the binary cross entropy loss.

[0097] Among them, the classification loss of the bounding box represents the difference between the category predicted by the student model and the true category in the positive sample candidate box, and its calculation method is shown in formula (3):

[0098]

[0099] In formula (3), represents the classification loss of the bounding box; is the true category label (which can be understood as the category label output by the teacher model, or the annotated category label); The class labels predicted for the student model; CE () is the cross entropy loss.

[0100] In step B4, the bounding box regression loss, confidence loss, and classification loss are weighted summed to obtain the supervision loss.

[0101] During implementation, the process of obtaining the supervision loss can be expressed by formula (4):

[0102]

[0103] In formula (4), Indicates loss of supervision; is the weight coefficient of the bounding box regression loss; is the weight coefficient of the confidence loss of the bounding box; is the weight coefficient of the classification loss of the bounding box; other parameters have been explained above and will not be repeated here.

[0104] The mask matrix is used to calculate the supervision loss for the old samples, only for the positive sample candidate boxes in the old samples; the training labels of the old samples are determined based on the first prediction results output by the teacher model.

[0105] Among them, for the spinning image samples, the supervised loss is used to simulate the first prediction result of the teacher model.

[0106] That is, the student model needs to be as close as possible to the prediction results of the teacher model on the old task samples, so as to inherit the knowledge of the old task.

[0107] In the disclosed embodiment, the first candidate box with the highest intersection-over-union ratio is selected as the positive sample candidate box, which can ensure that the selected candidate box is closest to the real target and provide more accurate positive samples for subsequent training, thereby improving the model's ability to locate the fault area in the spinning image sample and reducing false detection and missed detection. The mask matrix can only focus on the positive sample candidate box, avoiding the interference of the other first candidates other than the positive sample candidate box on the loss calculation, so that the model can focus more on the learning of the real fault area. The bounding box regression loss is used to measure the difference between the predicted bounding box and the real bounding box, so that the model can learn more accurate fault area positioning information in the spinning image sample and improve the detection accuracy. The confidence loss of the bounding box reflects the model's confidence in whether there is a fault candidate area in the bounding box, which can enable the model to more accurately judge the existence of the fault candidate area and reduce false detection and missed detection. The classification loss of the bounding box is used to determine the fault category of the fault candidate area in the bounding box to ensure that the model can correctly identify different fault categories. Different types of losses may have different importance in model training. By weighted summation, different weights can be assigned to different losses according to the needs and characteristics of specific tasks, thereby balancing the impact of each loss on model training.

[0108] S202 : Determine a regression distillation loss based on a distribution of bounding box offsets between the first prediction result and the second prediction result.

[0109] The regression distillation loss is determined based on the following regression distillation loss determination formula (5):

[0110]

[0111]

[0112] Formula (5), is the regression distillation loss; Feature sizes of spinning image samples output for the student and teacher models; The characteristic size corresponds to Anchor points; k is the index of the anchor point; is the bounding box relative to the corresponding anchor point k The direction of the offset; The first k Anchor point i The target probability distribution of the offset, where the length of the target probability distribution is L , L is the default value; The first k Anchor point i The probability distribution of the offset to be optimized; The teacher model is trained on the spinning image sample k The classification confidence of the anchor points; For the teacher model k Anchor point j The log-odds of the predicted probabilities for each category; Indicates the total number of categories for classification; Represents an activation function, such as the sigmoid function, which is used to convert the original classification output into classification confidence.

[0113] Among them, the anchor point k No. i The target probability distribution of the offset It can be represented by Distribution FocalLoss (DFL). DFL uses the relative offset from the anchor point to the four sides of the bounding box as the regression target, and a single offset It can be represented by a general distribution. Specifically, assuming the range of label y has a minimum value y0 and a maximum value yn (y0 ≤ y ≤ yn), DFL discretizes the entire range [y0, yn] into a set {y0, y1, ..., yi, yi+1, ..., yn-1, yn}, increasing in steps of 1. In practice, reg_max can be set to 16, resulting in a discretized set of {0, 1, 2, ..., 7, 8, ..., 14, 15}. The model essentially predicts the probability of the offset falling within each integer between 0 and 15, with the sum of all probabilities equal to 1.

[0114] In the embodiment of the present disclosure, The acquisition process can be expressed by formula (6):

[0115]

[0116] In formula (6), The first k Anchor point i The target probability distribution of the offset; softmax () is an activation function used to normalize the original regression output of the teacher model; For the teacher model k Anchor point i The original regression output of the offset can be understood as the prediction i exist LThe log-odds of the probability of occurrence over length; It is the regression distribution temperature parameter, generally with a default parameter of 2.0, which is used to soften the distribution of the teacher model and prevent overconfident predictions from interfering with the student model.

[0117] Similarly, among them, The acquisition process can be expressed by formula (7):

[0118]

[0119] In formula (7), The first k Anchor point i The probability distribution of the offset to be optimized; softmax () is an activation function; is the original regression output of the student model, which can be understood as the student model's prediction i exist L The log-odds of the probability of occurrence over length; is the regression distribution temperature parameter, which is used to adjust the smoothness of the output distribution of the student model.

[0120] In the disclosed embodiment, regression distillation loss is used to take the target probability distribution of the teacher model as a reference, so that the student model is approximated to the distribution. This allows the student model to quickly learn the teacher model's experience and patterns in bounding box prediction, thereby improving its own prediction accuracy for fault areas in spinning image samples.

[0121] S203 : Determine a classification distillation loss based on a difference in classification scores for the same anchor point between the first prediction result and the second prediction result.

[0122] The classification distillation loss is determined based on the following classification distillation loss determination formula (8):

[0123]

[0124]

[0125] In formula (8), To classify distillation losses; Feature sizes of spinning image samples output for the student and teacher models; The characteristic size corresponds to Anchor points; k is the index of the anchor point; Indicates the total number of categories for classification; j is the category index; Anchor points for the teacher model k ,category The first log-odds of the predicted probability; Anchor points for student models k ,category j The second log-odds of the predicted probability; is the weight coefficient, is the first k The detection box output by the anchor point, For students to model k The detection box output by the anchor points; IoU () is used to determine the intersection-union ratio.

[0126] In the disclosed embodiment, classification distillation loss is used to make the classification score of the student model as close as possible to the classification score of the teacher model, so that the student model can improve its own accuracy in predicting the fault category in the spinning image samples by learning the classification score of the teacher model.

[0127] In step S204 , the supervision loss, the regression distillation loss, and the classification distillation loss are integrated according to preset weights to obtain a first loss.

[0128] During implementation, the process of obtaining the first loss can be expressed by formula (9):

[0129]

[0130] In formula (9), L Indicates the first loss obtained; To monitor losses; is the regression distillation loss; To classify distillation losses; is the weight coefficient of regression distillation loss; is the weight coefficient of classification distillation loss.

[0131] In the disclosed embodiments, the supervisory loss can guide the student model to better adapt to the requirements of new tasks. The regression distillation loss can improve the student model's accuracy in predicting fault regions in spinning image samples. The classification distillation loss also improves the student model's accuracy in predicting fault categories in spinning image samples. By fusing these losses according to preset weights, the impact of different losses on the student model can be comprehensively considered, resulting in balanced optimization of the student model across multiple aspects.

[0132] Based on the same concept, a method for detecting thread faults in a spinning process is proposed in an embodiment of the present disclosure. The method is implemented based on the regional detection model in the spinning process described above. Specifically, Figure 3 As shown, it can be implemented as:

[0133] S301, collecting images of the spinning beam to obtain images to be processed.

[0134] During implementation, suitable image acquisition equipment, such as a high-speed industrial camera, can be used to capture images of the yarn at specific locations on the spinning beam to serve as input data for a subsequent yarn fault detection model.

[0135] S302 : Inputting the image to be processed into a region detection model trained by a training method based on a region detection model in a spinning process to obtain a candidate fault region in the image to be processed.

[0136] S303 : cropping a candidate fault region from the spinning image sample to obtain a second sub-image.

[0137] S304: Input the second sub-image into a yarn fault detection model to obtain a classification result of the second sub-image, wherein the classification result includes a floating yarn identification result and a broken yarn identification result.

[0138] The yarn fault detection model is a model specifically used to determine whether there is a specific fault in the spinning image. The training method is as follows: Figure 4 As shown, it can be implemented as:

[0139] S401 , performing discrete cosine transform on a sample image to obtain a frequency domain representation of the sample image.

[0140] Sample images are acquired by capturing images of the spinning beam and represent the state of the yarn at different moments. After acquiring these sample images, to improve model training efficiency and test results, they are first standardized and resized to a uniform size. The processed sample images are then segmented into n×n pixel blocks to provide data for the subsequent discrete cosine transform (DCT), ensuring the standardization and consistency of the processing. Here, n is an integer greater than 1, and the specific value depends on the actual business needs and the requirements of the DCT transformation.

[0141] The discrete cosine transform (DCT) is a transform technique related to the Fourier transform that is used to convert signals from one domain to the frequency domain. In the disclosed embodiment, by performing a discrete cosine transform on a sample image, the sample image can be converted from the spatial domain to frequency domains at different levels, thereby obtaining a frequency domain representation of the sample image. For example, the zero-frequency component in the frequency domain representation can capture global information such as the overall brightness and contrast of the sample image; the low-frequency component in the frequency domain representation can reflect slowly changing areas in the sample image, such as portions of the sample image where the silk threads are more evenly distributed; and the high-frequency component in the frequency domain representation reflects rapidly changing details in the sample image, such as the edge contours of the silk threads and the interlaced texture of the silk threads in the sample image.

[0142] In specific implementation, the frequency domain representation of the sample image can be obtained by calculation using formula (10), as follows:

[0143]

[0144] In formula (10), Represents the frequency domain representation of the sample image, that is, c Channel, h block row, w DCT coefficients of block columns; Represents a sample image x Middle c Channel, nh+i The pixel value of row nw+j and column nw+j; It is the cosine basis function in the two-dimensional DCT transform, which is used to convert the pixel value in the spatial domain into the frequency domain coefficient. The cosine basis function is used to calculate the frequency component in the horizontal direction; Used to calculate the frequency component in the vertical direction, u,v are frequency indices, ranging from 0 to n-1; i,j are the pixel indices within the pixel block, ranging from 0 to n-1.

[0145] S402 : Input the zero-frequency component in the frequency domain representation and the sample image into the zero-frequency enhancer of the model to be trained to obtain the first global feature of the sample image.

[0146] In the frequency domain representation, the zero-frequency component usually refers to the frequency domain component corresponding to the frequency index u=0 and v=0. Therefore, the zero-frequency component can be obtained based on formula (11):

[0147]

[0148] In formula (11), represents the zero-frequency component. Other parameters are the same as those in formula (10) and will not be repeated here.

[0149] The zero-frequency component and the sample image in the frequency domain representation are input into the zero-frequency enhancer of the model to be trained. Based on the overall information of the sample image provided by the zero-frequency component and the information provided by the sample image, the zero-frequency enhancer generates a first global feature containing global information of multiple dimensions of the sample image. In other words, the zero-frequency enhancer is the first neural network module of the model to be trained, which is used to extract the first global feature from the zero-frequency component and the sample image.

[0150] S403: Input the frequency domain representation into the low-frequency reconstructor of the model to be trained to obtain the intermediate semantic features of the sample image.

[0151] Among them, the low-frequency regenerator is the second neural network module of the model to be trained, which is used to extract the features of the low-frequency components from the frequency domain representation to obtain the mid-level semantic features.

[0152] Alternatively, the frequency domain representation is fed into the low-frequency reconstruction of the model to be trained. This reconstruction processes the low-frequency components of the frequency domain representation and extracts mid-level semantic features of the sample image. These mid-level semantic features help the model to be trained understand the continuity characteristics of the silk threads in the sample image, such as the areas where the threads are evenly distributed.

[0153] S404: Input the frequency domain representation and the first global feature into the high-frequency refiner of the model to be trained to obtain fine-grained features of the sample image.

[0154] Among them, the high-frequency refiner is the third neural network module of the model to be trained, which is used to extract the features of the high-frequency components from the frequency domain representation and the first global features to obtain the detailed features of the high-frequency classification.

[0155] The frequency domain representation and the first global feature are fed into the high-frequency refiner of the model to be trained. The refiner combines the high-frequency components of the frequency domain representation with the first global feature to further extract detailed information from the sample image. These fine-grained features help the model to be trained capture the high-frequency vibration trajectory characteristics of the thread, such as the jittery edges of the thread and the specific form of motion blur.

[0156] S405: Input the first global feature, the intermediate semantic feature, and the fine-grained feature into the fusion device of the model to be trained to obtain a fusion feature.

[0157] The first global feature, intermediate semantic feature and fine-grained feature are input into the fusion device of the model to be trained. The fusion device can combine the first global feature, intermediate semantic feature and fine-grained feature, make full use of the advantages of features at each level, and generate a fusion feature with multi-semantic level feature representation.

[0158] S406: Input the fused features into the classifier of the model to be trained to obtain a classification result for the sample image, wherein the classification result includes a floating silk discrimination result and a broken silk discrimination result.

[0159] The classifier can determine whether there are floating or broken silk threads in the sample image by analyzing the fusion features, and output the corresponding judgment results based on this, that is, clearly indicating whether the silk thread in the current sample image is floating, broken, or in a normal state.

[0160] S407 , based on the classification result, optimizing the model parameters of the model to be trained to obtain a wire fault detection model.

[0161] In an embodiment of the present disclosure, the zero-frequency enhancer includes a first global average pooling layer and a convolutional layer.

[0162] The global average pooling layer is a pooling operation commonly used in deep learning. It performs a global average pooling operation on the input feature map, averaging all elements in each feature channel to obtain a value that represents the overall information of the feature channel. In the disclosed embodiment, the first global average pooling layer receives a sample image as input and outputs the globally averaged pooled features of the sample image.

[0163] The convolutional layer is an important structure for extracting features in deep learning. By sliding the convolution kernel over the input data and performing convolution operations, it can automatically learn local features and patterns in the data. In the disclosed embodiment, the convolutional layer receives the spliced feature as input and outputs the first global feature.

[0164] During implementation, the zero-frequency component in the frequency domain representation and the sample image are input into the zero-frequency enhancer of the model to be trained to obtain the first global feature of the sample image, such as Figure 5 As shown, it can be implemented as follows:

[0165] S501: Input the sample image into the first global average pooling layer to obtain the global average pooling features of the sample image.

[0166] The global average pooling feature is to average the spatial dimensions (usually height and width) of each channel of the sample image. The calculation method is shown in formula (12):

[0167]

[0168] In formula (12), Represents the global average pooling features of the sample image; Represents a sample image x In the channel dimension c , spatial dimension i (height direction) and j Elements in the width direction; h and w Represents the size of the tensor in height and width dimensions respectively.

[0169] S502: Concatenate the zero-frequency component and the global average pooling feature to obtain a concatenated feature.

[0170] During implementation, the zero-frequency component in the frequency domain representation and the global average pooling feature obtained in the previous step can be spliced in the channel dimension to obtain the spliced feature.

[0171] S503: Input the concatenated features into the convolution layer to obtain the first global features.

[0172] The spliced features are input into the convolution layer for convolution operation, that is, the final first global feature is generated by fusing and reducing the dimensions of the spliced features between channels.

[0173] During implementation, the process of obtaining the first global feature can be expressed by formula (13):

[0174]

[0175] In formula (13), Represents the global average pooling features of the sample image; Represents the zero-frequency component in the frequency domain representation; Represents the global average pooling features of the sample image; Represents the splicing operation of the channel dimension, which is used to splice the zero-frequency component in the frequency domain representation and the global average pooling feature of the sample image; Represents a 1×1 convolution operation, which is used to extract the first global feature of the sample image without changing the spatial size of the feature map.

[0176] In the disclosed embodiment, the low frequency regenerator includes a learnable frequency domain selector and a normalization layer.

[0177] The learnable frequency domain selector learns the correlation patterns between different frequency bands in the frequency domain representation, thereby generating a frequency weight matrix for frequency selection. This matrix automatically determines which frequency bands' features are most conducive to extracting appropriate mid-level semantic features. The learnable frequency domain selector is a key component for filtering and integrating frequency domain information. Through training and optimization, it can learn patterns for classifying and identifying thread faults, thereby improving the accuracy of thread fault detection.

[0178] The normalization layer normalizes the intermediate features to make their data distribution more stable, which helps the subsequent feature extraction and model training process. It can eliminate the differences in dimensions and numerical ranges between different features and accelerate the convergence of the model.

[0179] During implementation, the frequency domain representation is input into the low-frequency reconstructor of the model to be trained to obtain the intermediate semantic features of the sample image, such as Figure 6 As shown, it can be implemented as follows:

[0180] S601 , learning a correlation pattern between different frequency bands in a frequency domain representation based on a learnable frequency domain selector, and generating a frequency weight matrix for selecting frequencies based on the correlation pattern.

[0181] The frequency domain representation is fed into a learnable frequency selector. This selector uses a learning algorithm to analyze the correlation patterns between different frequency bands in the frequency domain representation. For example, in the frequency domain representation of a silk thread sample image, low-frequency bands may be associated with the thread's continuity characteristics, while high-frequency bands may be associated with local texture. Based on these correlation patterns, the learnable frequency selector calculates the importance weight of each frequency component and generates a frequency weight matrix.

[0182] In implementation, generating a frequency weight matrix for selecting frequencies based on the correlation pattern can be achieved based on the following steps:

[0183] Step C1: input the frequency domain representation into the first activation layer of the learnable frequency domain selector to obtain activation features; the activation features are used to represent the correlation pattern between different frequency bands in the frequency domain representation;

[0184] The frequency domain representation is input into the first activation layer of the learnable frequency domain selector. The first activation layer performs a nonlinear transformation on the frequency domain representation through the activation function to obtain activation features. The activation features can reflect the correlation pattern between different frequency bands in the frequency domain representation.

[0185] In step C2, the activation features are input into the second activation layer of the learnable frequency domain selector to obtain a frequency weight matrix.

[0186] The activation features obtained above are input into the second activation layer of the learnable frequency domain selector. The second activation layer undergoes another nonlinear transformation using the activation function, ultimately generating a frequency weight matrix. Each element in the frequency weight matrix corresponds to a frequency component in the frequency domain representation, and the value in the frequency weight matrix represents the importance of that frequency component in constructing the mid-level semantic features. Frequency components with higher weights play a greater role in generating mid-level semantic features, while frequency components with lower weights may be suppressed or ignored.

[0187] In the embodiment of the present disclosure, the learnable frequency domain selector can be expressed as shown in formula (14):

[0188]

[0189] In formula (14), represents the first activation layer, is the weight of the first activation layer, is the bias of the first activation layer. The first activation layer is represented by the frequency domain of the input Projected into a new feature space, and then learned parameters to capture the correlation patterns between different frequency bands; The bias term is used to adjust the distribution of features to help the model better fit the data; It is a nonlinear activation function that can introduce nonlinear characteristics, enabling the model to learn complex feature relationships. Represents the second activation layer, which is used to learn weights based on the activation features obtained based on the first activation layer The parameters of the second activation layer generate a frequency selection weight matrix, which is used to weight different frequency components; The bias term is used to adjust the distribution of the second layer features.

[0190] During the training phase, the weights and biases of the first and second activation layers in the learnable frequency domain selector are continuously optimized so that the learnable frequency domain selector can learn to capture important low-frequency component information. After the training, the weights and biases of the first and second activation layers no longer change, and the frequency domain representation can be adaptively adapted to screen out appropriate low-frequency components for constructing intermediate semantic features.

[0191] S602: Perform an inverse discrete cosine transform on the frequency weight matrix to obtain a first spatial domain feature.

[0192] That is, the generated frequency weight matrix is subjected to an inverse discrete cosine transform, which is converted from the frequency domain back to the spatial domain to obtain the first spatial domain feature.

[0193] S603: Determine intermediate features based on the spatial domain features and the mask features; the mask features include frequency band masks corresponding to each frequency in the frequency domain representation.

[0194] Based on the first spatial domain feature obtained above and the pre-set mask feature, the intermediate feature is determined. The mask feature includes the frequency band mask corresponding to each frequency in the frequency domain representation, such as Figure 7 As shown in FIG, an exemplary mask matrix (i.e., mask feature) provided for the implementation of the present disclosure is shown, wherein the upper left corner area can be used as a frequency band mask corresponding to low frequencies to capture macro features such as silk thread outlines in the sample image; the middle area can be used as a frequency band mask corresponding to medium frequencies to extract scale texture information of silk threads in the sample image; the lower left corner area can be used as a frequency band mask corresponding to high frequencies to retain finer image detail information in the sample image. The frequency band mask can be used to further refine the screening of frequency components, clarifying which frequency components should be retained or highlighted and which should be suppressed or discarded. For example, retaining the front part of the frequency domain representation × The low-frequency components are detected while other high-frequency components are shielded.

[0195] S604: Input the intermediate features into the normalization layer to obtain intermediate semantic features.

[0196] The determined intermediate features are input into the normalization layer for normalization. The normalization layer normalizes the intermediate feature data to meet specific distribution requirements, thereby obtaining intermediate semantic features.

[0197] In specific implementation, the process of obtaining intermediate semantic features can be described by formula (15), as follows:

[0198]

[0199] In formula (15), Represents the mid-level semantic features of the sample image; Represents the front of the frequency domain representation k × k Mask features of low-frequency components; k Is a positive integer used to control the range of retained low-frequency components; represents the obtained frequency domain representation; represents the frequency weight matrix for selecting frequencies based on the correlation pattern; represents the inverse discrete cosine transform, which is used to convert the frequency weight matrix from the frequency domain representation back to the spatial domain features; Represents element-by-element multiplication (Hadamard product), which multiplies the elements of corresponding positions of two matrices; LayerNorm () represents the normalization layer, which is used to process the intermediate features and obtain the intermediate semantic features.

[0200] In the disclosed embodiment, the high-frequency refiner includes a dynamic convolution layer. Unlike traditional convolution layers, the convolution kernel weights in a dynamic convolution layer are not fixed but are dynamically generated based on input features. This enables the model to adaptively adjust the convolution operation based on the characteristics of different sample images, thereby more accurately extracting fine-grained features.

[0201] During implementation, the frequency domain representation and the first global feature are input into the high-frequency refiner of the model to be trained to obtain the fine-grained features of the sample image, such as Figure 8 As shown, it can be implemented as follows:

[0202] S801, extracting high-frequency components from the frequency domain representation.

[0203] Based on the content described in the previous step S603, the mask feature retains the front × The low-frequency components of the image are masked out, while other high-frequency components are masked out. Therefore, the high-frequency components can be extracted from the frequency domain representation by selecting the complement of the mask feature to mask out the low-frequency components and retain only the high-frequency components.

[0204] S802: Convert the high-frequency component to the spatial domain to obtain a second spatial domain feature.

[0205] The high-frequency components to be extracted are converted to the spatial domain through inverse discrete cosine transform to obtain the second spatial domain features.

[0206] S803: Determine a convolutional layer weight of the dynamic convolutional layer based on the first global feature.

[0207] The first global feature contains the overall information of the sample image. By using this global information to dynamically generate convolutional layer weights, the convolution operation can be made more targeted. For example, if the first global feature indicates that a certain area in the sample image may contain thread fault characteristics, the generated convolutional layer weights may focus more on extracting details in this area.

[0208] That is, in order to enable the dynamic convolution layer to adaptively adjust according to the overall information of the sample image, so as to better capture the context and semantic information of the sample image, the convolution layer weights of the dynamic convolution layer can be determined based on the global features. The processing process is shown in formula (16):

[0209]

[0210] In formula (16), Represents the convolution layer weights of the dynamic convolution layer; represents the first global feature; Represents the weight matrix, which is used to perform dot product operation with the first global feature; Softmax () represents a function that converts the input vector into a probability distribution (such as an activation layer) and is used to generate the convolutional layer weights.

[0211] S804: Input the second spatial domain feature into a dynamic convolution layer using convolution layer weights to obtain fine-grained features.

[0212] That is, the second spatial domain feature is input into the dynamic convolution layer using the convolution layer weights generated above, and a convolution operation is performed to finally obtain fine-grained features.

[0213] In specific implementation, the process of obtaining fine-grained features can be described by formula (17), as follows:

[0214]

[0215] In formula (17), Represents fine-grained features of sample images; Represents the calculation of the complement of the mask feature, which is used to mask the low-frequency component and extract the high-frequency component; Used to indicate that the extracted k The high-frequency components covered by the frequency channels are extracted The high frequency components in It means multiplying the elements of the corresponding positions of the two matrices; DyConv The parentheses ( ) indicate a dynamic convolution operation, an adaptive convolution operation that dynamically adjusts the kernel weights based on the input features. The other parameters have been explained above and will not be repeated here.

[0216] In the embodiment of the present disclosure, in order to integrate feature information at different levels to obtain more comprehensive and richer fusion features, the first global feature, the intermediate semantic feature and the fine-grained feature can be input into the fusion device of the model to be trained to obtain the fusion feature, such as Figure 9 As shown, it can be implemented as follows:

[0217] S901: Input the first global feature, the intermediate semantic feature, and the fine-grained feature into a weight coefficient generator to obtain weight coefficients corresponding to the first global feature, the intermediate semantic feature, and the fine-grained feature, respectively.

[0218] During the training phase, the aggregator includes a weight coefficient generator, which is built based on the attention mechanism and consists of a second global average pooling layer, a third global average pooling layer, a multilayer perceptron, and a third activation layer. The first global features, mid-level semantic features, and fine-grained features are input into the weight coefficient generator. Based on the attention mechanism, the weight coefficient generator calculates the importance weight of each feature. Specifically, by focusing on the contribution of each feature to the final classification result, it assigns higher weights to features with greater contributions and lower weights to features with lesser contributions, thereby obtaining the corresponding weight coefficients.

[0219] It's important to note that during the training phase, the weight coefficient generator dynamically generates weight coefficients based on the input features at different levels, allowing the trained model to automatically learn how to fuse features at different levels to optimize the objective function. After training, the trained model has learned the optimal weights for fusing features at different levels. Therefore, the weight coefficient generator branch can be removed after training, allowing the trained model to be simplified and optimized while maintaining performance, meeting the efficiency requirements of the subsequent inference phase.

[0220] During implementation, the first global feature, the intermediate semantic feature, and the fine-grained feature are input into a weight coefficient generator to obtain the weight coefficients corresponding to the first global feature, the intermediate semantic feature, and the fine-grained feature, respectively. This can be achieved based on the following steps:

[0221] Step D1, using a second global average pooling layer to process the intermediate semantic features to obtain a second global feature;

[0222] The mid-level semantic features are fed into the second global average pooling layer, which performs a global average pooling operation on the mid-level semantic features to generate the second global features. The second global features represent a global overview of the mid-level semantic features, which helps to grasp the mid-level semantic features from a holistic perspective when subsequently considering features at different levels.

[0223] Step D2, using a third global average pooling layer to process the fine-grained features to obtain a third global feature;

[0224] That is, the fine-grained features are input to the third global average pooling layer, which performs a global average pooling operation on the fine-grained features to obtain the third global feature. The third global feature reflects the overall characteristics of the fine-grained features.

[0225] Step D3, inputting the first global feature, the second global feature, and the third global feature into a multilayer perceptron to obtain a multilayer perceptron feature;

[0226] The first, second, and third global features are input into a multilayer perceptron. The multilayer perceptron, composed of multiple neural network layers, performs nonlinear transformations and fusion on the input features, ultimately outputting multilayer perceptual features. These features combine information from three levels of features and can further reflect the inherent connections between different features.

[0227] In step D4, the multi-layer perception features are input into the third activation layer to obtain weight coefficients.

[0228] That is, the multi-layer perception features are input into the third activation layer, which performs nonlinear transformation through the activation function to obtain the weight coefficient.

[0229] In specific implementation, the process of obtaining the weight coefficient can be described by formula (18), as follows:

[0230]

[0231] In formula (18), are the weight coefficients corresponding to the first global feature, intermediate semantic feature and fine-grained feature respectively; MLP is represented by a multi-layer perceptron (i.e., multi-layer perceptron), which is used to map input features to output space. Softmax () function, which is used to convert the input vector into a probability distribution, that is, the weight coefficients corresponding to the first global feature, the intermediate semantic feature and the fine-grained feature in the implementation of the present disclosure, and the output value range is between [0,1] and the sum is 1. Indicates that a global average pooling operation is performed on the input; other parameters have been explained above and will not be repeated here.

[0232] S902: Perform weighted summation on the first global feature, the intermediate semantic feature, and the fine-grained feature based on the fuser to obtain a fused feature.

[0233] That is, the first global feature, the intermediate semantic feature, and the fine-grained feature are multiplied by their corresponding weight coefficients and then added together to obtain the fused feature.

[0234] In the specific implementation, fine-grained features are obtained, and the implementation process of the fusion device can be described by formula (19), as follows:

[0235]

[0236] In formula (19), represents fusion features; up () is an upsampling operation, which is used to adjust the size of the first global feature to the same size as the mid-level semantic features and fine-grained features; They represent the weight coefficients corresponding to the first global feature, intermediate semantic feature and fine-grained feature respectively.

[0237] In the embodiment of the present disclosure, in order to further improve the accuracy of the wire fault detection model in detecting wire faults, the model parameters of the to-be-trained model may be optimized based on the classification results, such as Figure 10 As shown, it can be implemented as follows:

[0238] S1001: Determine the detection loss based on the classification result.

[0239] The detection loss includes at least one of the following:

[0240] (1) A global loss between the image to be optimized and the sample image generated based on the first global feature;

[0241] During implementation, a diffusion model may be used to process the first global feature, with the goal of restoring a sample image based on the first global feature to obtain an image to be optimized.

[0242] This global loss focuses on the differences at the overall image level and is used to measure the degree of closeness between the image to be optimized generated by the model and the sample image in terms of global features.

[0243] In the embodiment of the present disclosure, the global loss can be expressed as shown in formula (20):

[0244]

[0245] In formula (20), Represents the global loss; ||()||2 is represented by norm (Euclidean norm); represents the image to be optimized restored based on the first global feature; yRepresents a sample image.

[0246] (2) Intermediate semantic loss; the intermediate semantic loss includes the accumulated value of the Hadamard product of each band mask and the corresponding residual in the mask feature; the residual includes the difference between the intermediate semantic feature and the target intermediate layer feature of the teacher model;

[0247] The mid-level semantic loss is used for knowledge distillation to transfer the knowledge of the intermediate-level features of the teacher model to the intermediate-level features of the current model.

[0248] In the embodiment of the present disclosure, the mid-level semantic loss can be expressed as shown in formula (21):

[0249]

[0250] In formula (21), represents mid-level semantic loss; is the mask feature; Represents mid-level semantic features; Represents the characteristics of the target middle layer of the teacher model; represents element-wise multiplication (Hadamard product); ||()||1 represents the L1 norm.

[0251] (3) Detail loss; Detail loss is determined based on the total variation regularization term of fine-grained features; The total variation regularization term is used to suppress high-frequency noise in fine-grained features;

[0252] That is, detail loss focuses more on the detail level of the sample image, which is used to reduce noise in fine-grained features while retaining edge information.

[0253] In the embodiment of the present disclosure, the mid-level semantic loss can be expressed as shown in formula (22):

[0254]

[0255] In formula (22), Indicates loss of detail; TV () represents the total variation regularization term, which is used to reduce noise and incoherence in detail features; Represents fine-grained features.

[0256] (4) Classification loss: Classification loss represents the loss value between the classification result and the target classification result.

[0257] S1002: Optimize model parameters of the model to be trained based on the detection loss.

[0258] During implementation, the calculated detection loss can be used to update the parameters of the model to be trained through the back-propagation algorithm, thereby reducing the loss value and improving the performance of the model to be trained.

[0259] During implementation, the classification results obtained based on the above classifier can be compared with the corresponding true labels, and indicators such as loss values can be calculated. Then, the model parameters of the model to be trained can be optimized based on this, so that the model to be trained can continuously improve the accuracy of the reasoning process, and finally an effective silk thread fault detection model is obtained, which can be used for silk thread fault detection on newly collected sample images in the actual spinning process.

[0260] After the yarn fault detection model is obtained, the second sub-image is input into the yarn fault detection model to detect whether the yarn has floating yarn or broken yarn problems.

[0261] The second sub-image is input into the yarn fault detection model. This model extracts multi-level features from the input image, including global features, mid-level semantic features, and fine-grained features. It then fuses these features at different levels using methods such as gated fusion to generate fused features. Finally, the model's classifier classifies the fused features to determine whether the image contains loose or broken yarns. Based on this classification, the model outputs a classification result. This classification result is the classification result for the second sub-image, including the identification of loose and broken yarns.

[0262] In the embodiment of the present disclosure, by inputting the second sub-image into the yarn fault detection model, automated and precise yarn fault detection of the spinning process can be achieved, which helps to improve production efficiency, reduce production costs, and improve product quality.

[0263] Based on the same technical concept, the present disclosure proposes a training device 1100 for a region detection model in a spinning process. Figure 11 Shown, including:

[0264] The sampling module 1101 is used to sample from the spinning task data set and the playback memory to obtain mixed data; wherein the spinning task data set includes spinning image samples obtained by image acquisition of the spinning box, and the playback memory includes historical sample images;

[0265] Prediction module 1102 is configured to input the mixed dataset into a student model and a pre-trained teacher model, respectively, to obtain a first prediction result output by the teacher model and a second prediction result output by the student model; for a spinning image sample, the student model is configured to detect a candidate fault region in the spinning image sample;

[0266] A cropping module 1103 is configured to crop a candidate fault region from the spinning image sample to obtain a first sub-image;

[0267] The classification module 1104 is configured to input the first sub-image into a yarn fault detection model to obtain a classification result of the first sub-image, wherein the classification result includes a floating yarn identification result and a broken yarn identification result;

[0268] The optimization module 1105 is used to optimize the parameters of the student model based on the first loss determined by the first prediction result and the second prediction result, and the second loss between the classification result and the classification label to obtain a region detection model.

[0269] In some embodiments, the optimization module includes:

[0270] a supervision loss unit, configured to determine a supervision loss based on the first prediction result and the second prediction result;

[0271] a regression distillation loss unit, configured to determine a regression distillation loss based on a distribution of bounding box offsets between the first prediction result and the second prediction result;

[0272] a classification distillation loss unit, configured to determine a classification distillation loss based on a difference in classification scores for the same anchor point between the first prediction result and the second prediction result;

[0273] The fusion unit is used to fuse the supervision loss, regression distillation loss, and classification distillation loss according to preset weights to obtain the first loss.

[0274] In some embodiments, the spinning task dataset includes new task category samples, and the playback memory includes old task category samples;

[0275] Monitor loss units, including:

[0276] For an old sample of an old task category sampled from the replay memory, determine, from among multiple first candidate frames of the old sample in the second prediction result, a first candidate frame having the highest intersection-over-union ratio with the true bounding box of the old sample as a positive sample candidate frame;

[0277] Mark the positive sample as a foreground value, and mark the first candidate boxes other than the positive sample candidate box in the multiple first candidate boxes as background values to obtain a mask matrix;

[0278] Determine, based on the mask matrix, a bounding box regression loss between the first prediction result and the second prediction result, a confidence loss of the bounding box, and a classification loss of the bounding box;

[0279] The bounding box regression loss, confidence loss, and classification loss are weighted summed to obtain the supervision loss;

[0280] The mask matrix is used to calculate the supervision loss for the old samples, only for the positive sample candidate boxes in the old samples; the training labels of the old samples are determined based on the first prediction results output by the teacher model;

[0281] Among them, for the spinning image samples, the supervised loss is used to simulate the first prediction result of the teacher model.

[0282] In some embodiments, the regression distillation loss unit comprises:

[0283] The regression distillation loss is determined based on the following regression distillation loss determination formula (5):

[0284]

[0285]

[0286] in, is the regression distillation loss; Feature sizes of spinning image samples output for the student and teacher models; The characteristic size corresponds to Anchor points; k is the index of the anchor point; i is the bounding box relative to the corresponding anchor point k The direction of the offset; The first k Anchor point i The target probability distribution of the offset, where the length of the target probability distribution is L , L is the default value; The first k Anchor point i The probability distribution of the offset to be optimized; The teacher model is trained on the spinning image sample k The classification confidence of each anchor point; For the teacher model k Anchor point j The log-odds of the predicted probabilities for each category; Indicates the total number of categories for classification; Represents the activation function.

[0287] In some embodiments, wherein the classified distillation loss unit comprises:

[0288] The classification distillation loss is determined based on the following classification distillation loss determination formula (8):

[0289]

[0290]

[0291] in, To classify distillation losses; Feature sizes of spinning image samples output for the student and teacher models; The characteristic size corresponds to Anchor points; k is the index of the anchor point; Indicates the total number of categories for classification j is the category index; Anchor points for the teacher model k ,category j The first log-odds of the predicted probability; Anchor points for student models k ,category The second log-odds of the predicted probability; is the weight coefficient, is the first k The detection box output by the anchor point, For students to model k The detection box output by the anchor points; IoU () is used to determine the intersection-union ratio.

[0292] In some embodiments, the sampling module includes:

[0293] a determination unit for determining the sampling ratios of various old task categories in the spinning task dataset and the playback memory;

[0294] The sampling unit is used to sample the spinning task dataset and playback memory based on the sampling ratio.

[0295] Based on the same technical concept, the present disclosure provides a device 1200 for detecting yarn faults in a spinning process. Figure 12 Shown, including:

[0296] The acquisition module 1201 is used to acquire images of the spinning beam to obtain images to be processed;

[0297] An input module 1202 is configured to input an image to be processed into a region detection model trained using a training method for a region detection model in a spinning process, and obtain a candidate fault region in the image to be processed;

[0298] A processing module 1203 is configured to crop a candidate fault region from the spinning image sample to obtain a second sub-image;

[0299] The discrimination module 1204 is configured to input the second sub-image into the yarn fault detection model to obtain a classification result of the second sub-image, wherein the classification result includes a floating yarn discrimination result and a broken yarn discrimination result.

[0300] For the description of specific functions and examples of each module, sub-module\unit of the device in the embodiment of the present disclosure, please refer to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0301] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0302] Figure 13 FIG. 1 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Figure 13 As shown, the electronic device includes: a memory 1310 and a processor 1320. The memory 1310 stores a computer program that can be executed on the processor 1320. The number of memory 1310 and processor 1320 can be one or more. The memory 1310 can store one or more computer programs. When the one or more computer programs are executed by the electronic device, the electronic device performs the method provided by the above method embodiment. The electronic device may also include: a communication interface 1330 for communicating with external devices and performing data exchange.

[0303] If the memory 1310, the processor 1320, and the communication interface 1330 are implemented independently, the memory 1310, the processor 1320, and the communication interface 1330 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 13 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0304] Optionally, in a specific implementation, if the memory 1310, the processor 1320 and the communication interface 1330 are integrated on a chip, the memory 1310, the processor 1320 and the communication interface 1330 can communicate with each other through an internal interface.

[0305] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0306] Furthermore, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0307] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, data subscriber line (DSL)) or wireless (e.g., infrared, Bluetooth, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid-state drive (SSD)). It is worth noting that the computer-readable storage medium mentioned in the present disclosure may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0308] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0309] In the description of the embodiments of the present disclosure, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials, or characteristics described may be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless they are mutually inconsistent.

[0310] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means or. For example, A / B can mean A or B. "And / or" in this document is only a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0311] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

[0312] The above description is merely an exemplary embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A method for training a region detection model in a spinning process, comprising: Sampling from a spinning task data set and a playback memory to obtain mixed data; wherein the spinning task data set includes spinning image samples obtained by collecting images of a spinning manifold, and the playback memory includes historical sample images; inputting the mixed data set into a student model and a pre-trained teacher model respectively, obtaining a first prediction result output by the teacher model and a second prediction result output by the student model; for the spinning image sample, the student model is used to detect a candidate fault area in the spinning image sample; cropping the fault candidate region from the spinning image sample to obtain a first sub-image; inputting the first sub-image into a yarn fault detection model to obtain a classification result of the first sub-image, wherein the classification result includes a floating yarn identification result and a broken yarn identification result; Based on a first loss determined by the first prediction result and the second prediction result, and a second loss between the classification result and the classification label, the parameters of the student model are optimized to obtain a region detection model.

2. The method according to claim 1, wherein Determining a first loss based on the first prediction result and the second prediction result includes: determining a supervision loss based on the first prediction result and the second prediction result; determining a regression distillation loss based on a distribution of bounding box offsets between the first prediction result and the second prediction result; determining a classification distillation loss based on a difference in classification scores for the same anchor point between the first prediction result and the second prediction result; The supervision loss, the regression distillation loss, and the classification distillation loss are fused according to preset weights to obtain the first loss.

3. The method according to claim 2, wherein: The spinning task data set includes new task category samples, and the playback memory includes old task category samples; The determining of the supervision loss based on the first prediction result and the second prediction result includes: For an old sample of the old task category sampled from the playback memory, determine, from among multiple first candidate frames of the old sample in the second prediction result, a first candidate frame having a highest intersection-over-union ratio with a true bounding box of the old sample as a positive sample candidate frame; Marking the positive sample as a foreground value, and marking the first candidate frames other than the positive sample candidate frame in the multiple first candidate frames as background values, to obtain a mask matrix; Determining a bounding box regression loss, a bounding box confidence loss, and a bounding box classification loss between the first prediction result and the second prediction result based on the mask matrix; Performing a weighted summation on the bounding box regression loss, the confidence loss, and the classification loss to obtain the supervision loss; The mask matrix is used to calculate the supervision loss for the old sample and only for the positive sample candidate box in the old sample; the training label of the old sample is determined based on the first prediction result output by the teacher model; Wherein, for the spinning image sample, the supervised loss is used to simulate the first prediction result of the teacher model.

4. The method according to claim 2, wherein: The determining the regression distillation loss based on the bounding box offset distribution between the first prediction result and the second prediction result includes: The regression distillation loss is determined based on the following regression distillation loss determination formula: in, is the regression distillation loss; characteristic sizes of the spinning image samples output by the student model and the teacher model; The characteristic size corresponds to Anchor points; k is the index of the anchor point; i is the bounding box relative to the corresponding anchor point k The direction of the offset; is the first k Anchor point i The target probability distribution of the offset, where the length of the target probability distribution is L , L is the default value; The first k Anchor point i The probability distribution of the offset to be optimized; The teacher model is used to calculate the spinning image sample k The classification confidence of the anchor points; For the teacher model k Anchor point j The log-odds of the predicted probabilities for each category; Indicates the total number of categories for classification; Represents the activation function.

5. The method according to claim 2, wherein: The determining of the classification distillation loss based on the difference in classification scores for the same anchor point between the first prediction result and the second prediction result includes: The classification distillation loss is determined based on the following classification distillation loss determination formula: in, Distillation loss for said classification; characteristic sizes of the spinning image samples output by the student model and the teacher model; The characteristic size corresponds to Anchor points; k is the index of the anchor point; Indicates the total number of categories for classification j is the category index; The teacher model is the anchor point k ,category j The first log-odds of the predicted probability; The anchor point for the student model k ,category The second log-odds of the predicted probability; is the weight coefficient, is the first k The detection box output by the anchor point, For the student model k The detection box output by the anchor points; IoU () is used to determine the intersection-union ratio.

6. The method according to claim 1, wherein The sampling from the spinning task dataset and playback memory includes: Determining the sampling ratio of the spinning task data set and various old task categories in the playback memory; The spinning task data set and the playback memory are sampled based on the sampling ratio.

7. A method for detecting yarn faults in a spinning process, comprising: Capturing images of the spinning beam to obtain images to be processed; Inputting the image to be processed into a region detection model trained by the method according to any one of claims 1 to 6 to obtain a candidate fault region in the image to be processed; cropping the fault candidate region from the spinning image sample to obtain a second sub-image; The second sub-image is input into a yarn fault detection model to obtain a classification result of the second sub-image, wherein the classification result includes a floating yarn identification result and a broken yarn identification result.

8. A training device for a region detection model in a spinning process, comprising: A sampling module is used to sample from a spinning task data set and a playback memory to obtain mixed data; wherein the spinning task data set includes spinning image samples obtained by collecting images of the spinning manifold, and the playback memory includes historical sample images; a prediction module, configured to input the mixed data set into a student model and a pre-trained teacher model, respectively, to obtain a first prediction result output by the teacher model and a second prediction result output by the student model; and for the spinning image sample, the student model is configured to detect a candidate fault area in the spinning image sample; a cropping module, configured to crop the fault candidate area from the spinning image sample to obtain a first sub-image; a classification module, configured to input the first sub-image into a yarn fault detection model to obtain a classification result of the first sub-image, wherein the classification result includes a floating yarn identification result and a broken yarn identification result; An optimization module is used to optimize the parameters of the student model based on a first loss determined by the first prediction result and the second prediction result, and a second loss between the classification result and the classification label to obtain a region detection model.

9. The device according to claim 8, wherein The optimization module comprises: a supervision loss unit, configured to determine a supervision loss based on the first prediction result and the second prediction result; a regression distillation loss unit, configured to determine a regression distillation loss based on a distribution of bounding box offsets between the first prediction result and the second prediction result; a classification distillation loss unit, configured to determine a classification distillation loss based on a difference in classification scores for the same anchor point between the first prediction result and the second prediction result; A fusion unit is used to fuse the supervision loss, the regression distillation loss, and the classification distillation loss according to preset weights to obtain the first loss.

10. The device according to claim 9, wherein The spinning task data set includes new task category samples, and the playback memory includes old task category samples; The supervision loss unit includes: For an old sample of the old task category sampled from the playback memory, determine, from among multiple first candidate frames of the old sample in the second prediction result, a first candidate frame having a highest intersection-over-union ratio with a true bounding box of the old sample as a positive sample candidate frame; Marking the positive sample as a foreground value, and marking the first candidate frames other than the positive sample candidate frame in the multiple first candidate frames as background values, to obtain a mask matrix; Determining a bounding box regression loss, a bounding box confidence loss, and a bounding box classification loss between the first prediction result and the second prediction result based on the mask matrix; Performing a weighted summation on the bounding box regression loss, the confidence loss, and the classification loss to obtain the supervision loss; The mask matrix is used to calculate the supervision loss for the old sample and only for the positive sample candidate box in the old sample; the training label of the old sample is determined based on the first prediction result output by the teacher model; Wherein, for the spinning image sample, the supervised loss is used to simulate the first prediction result of the teacher model.

11. The device according to claim 9, wherein The regression distillation loss unit comprises: The regression distillation loss is determined based on the following regression distillation loss determination formula: in, is the regression distillation loss; characteristic sizes of the spinning image samples output by the student model and the teacher model; The characteristic size corresponds to Anchor points; k is the index of the anchor point; i is the bounding box relative to the corresponding anchor point k The direction of the offset; is the first k Anchor point i The target probability distribution of the offset, where the length of the target probability distribution is L , L is the default value; The first k Anchor point i The probability distribution of the offset to be optimized; The teacher model is used to calculate the spinning image sample k The classification confidence of the anchor points; For the teacher model k Anchor point j The log-odds of the predicted probabilities for each category; Indicates the total number of categories for classification; Represents the activation function.

12. The device according to claim 9, wherein The classification distillation loss unit comprises: The classification distillation loss is determined based on the following classification distillation loss determination formula: in, Distillation loss for said classification; characteristic sizes of the spinning image samples output by the student model and the teacher model; The characteristic size corresponds to Anchor points; k is the index of the anchor point; Indicates the total number of categories for classification j is the category index; The teacher model is the anchor point k ,category j The first log-odds of the predicted probability; The anchor point for the student model k ,category The second log-odds of the predicted probability; is the weight coefficient, is the first k The detection box output by the anchor point, For the student model k The detection box output by the anchor point; IoU() is to determine the intersection over union ratio.

13. The device according to claim 8, wherein The sampling module includes: a determination unit, configured to determine a sampling ratio of the spinning task dataset and various old task categories in the playback memory; A sampling unit is used to sample the spinning task data set and the playback memory based on the sampling ratio.

14. A device for detecting yarn faults in a spinning process, comprising: An acquisition module is used to acquire images of the spinning box to obtain images to be processed; An input module, configured to input the image to be processed into a region detection model trained by the apparatus according to any one of claims 8 to 13, to obtain a candidate fault region in the image to be processed; a processing module, configured to crop the fault candidate area from the spinning image sample to obtain a second sub-image; The discrimination module is used to input the second sub-image into the yarn fault detection model to obtain a classification result of the second sub-image, wherein the classification result includes a floating yarn discrimination result and a broken yarn discrimination result.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images