Cloth defect online detection method based on multispectral imaging and AI fusion

Abstract

The invention relates to the technical field of cloth detection, in particular to a cloth defect online detection method based on multispectral imaging and AI fusion, which comprises the following steps: carrying out positioning interception and image correction on cloth in an image to be detected, and outputting a new cloth test set and a corrected image set; training a cloth surface flaw recognition model through the corrected image set, and judging the surface flaw state of the cloth image; feeding back the flaw position to a to-be-detected image in the cloth test set in a coordinate transformation form; and judging the surface defect type of the cloth, performing data summarization, verifying the recognition capability of the cloth surface defect recognition model, and determining that the recognition capability of the cloth surface defect recognition model meets the standard, or determining that the recognition capability does not meet the standard and analyzing the reason for the non-conforming to the standard, or performing secondary analysis on the recognition capability. According to the method, the adaptability to different types of cloth surface flaws on the cloth surface is improved by adjusting the detection parameters according to the flaw types.

Description

Technical Field

[0001] This invention relates to the field of fabric inspection technology, and in particular to an online method for detecting fabric defects based on multispectral imaging and AI fusion. Background Technology

[0002] During fabric production, various defects inevitably occur due to factors such as raw materials, equipment, processes, and environment, including broken warp and weft threads, holes, stains, color differences, and knots. These defects severely affect the quality and value of the fabric. Therefore, efficient and accurate defect detection during production is a crucial step for the textile industry to improve product quality and reduce production costs. Existing online defect detection methods for fabrics can be broadly categorized into three types: manual inspection, traditional machine vision-based automatic inspection, and artificial intelligence-based automatic inspection.

[0003] Traditional machine vision automatic inspection is currently the mainstream technology; however, this method has inherent limitations: First, its core algorithm relies on modeling specific textured backgrounds, resulting in extremely poor generalization ability. When the type, color, or texture of the fabric is changed, professional personnel must readjust the algorithm parameters, or even redevelop the algorithm, which is time-consuming, labor-intensive, and lacks adaptability. Second, for complex defects with low contrast to the background, similar colors, or varied shapes (such as minor oil stains, certain weft marks, or abrasions), the recognition accuracy (especially recall) drops significantly, with serious false positives and false negatives. Third, the system's performance is highly susceptible to changes in lighting, workshop environmental interference, and mechanical vibration, resulting in poor robustness.

[0004] Chinese Patent Publication No. CN112270687A discloses a training method for a fabric defect recognition model and a method for detecting fabric defects. The method includes: providing basic data for multiple samples, where the basic data corresponds to the contours of known fabric defects in the samples; acquiring multiple training data based on the basic data for each sample; and training a neural network model based on the samples containing the multiple training data, so that the neural network model can output multiple contour recognition data corresponding to the multiple training data based on the samples. Since the basic data of the samples corresponds to the contours of known fabric defects in the samples, and the multiple training data correspond to multiple training contours obtained based on the contours of known fabric defects in the samples, one sample carries multiple training samples. By training the neural network model based on samples containing multiple training data, this training method enables the contour recognition model to have the ability to output multiple contour recognition data corresponding to multiple training data based on a single sample. Therefore, the training method for the fabric defect recognition model and the method for detecting fabric defects have the following problems:

[0005] It has poor adaptability to different types of surface defects present on the fabric surface during production. Summary of the Invention

[0006] To address this, the present invention provides an online fabric defect detection method based on multispectral imaging and AI fusion, which overcomes the problem of poor adaptability to different types of fabric surface defects present on the fabric surface in the prior art.

[0007] To achieve the above objectives, this invention provides an online fabric defect detection method based on multispectral imaging and AI fusion, comprising:

[0008] Target detection is performed on several images to be detected contained in the fabric test set. The fabric in the images to be detected is located, cropped and corrected. Several fabric images are output to form a new fabric test set. Corrected images are output and summarized into a corrected image set.

[0009] By training a fabric surface defect recognition model using a calibrated image set, the model automatically identifies surface defects in several fabric images in the fabric test set according to the initial detection cycle, and determines the surface defect status of the fabric images.

[0010] Based on the surface defect status of the fabric image, the location of the fabric defect is output in the fabric test set, and the defect location is fed back to the image to be detected in the fabric test set in the form of coordinate transformation.

[0011] Determine the type of surface defects in the fabric and summarize the data. Verify the recognition capability of the fabric surface defect recognition model. Determine if the recognition capability of the fabric surface defect recognition model meets the standard, or determine if the recognition capability does not meet the standard and analyze the reasons for non-compliance, or conduct a secondary analysis of the recognition capability.

[0012] Issue a correction notice or revise the judgment criteria for the reasons why the identification capability does not meet the standard;

[0013] The detection rate of any surface defect is tested, and the recognition ability for any surface defect is judged based on the difference in detection rate and the corresponding weight of the surface defect. The distribution and total number of fabric images with different surface defects in the fabric test set are adjusted.

[0014] Furthermore, the process of locating, cropping, and correcting the fabric in the image to be detected includes,

[0015] The first training set is generated based on the initial number of cloth images to be detected. The YOLOv8 object detection algorithm is trained using the first cloth training set to build a cloth detection model.

[0016] The initial number of fabric surface images to be detected, acquired by a high-precision vision camera, are used as the first test set of the fabric. They are then input into the fabric detection model according to the initial detection cycle for detection, and the fabric in the fabric surface images to be detected is identified and cut.

[0017] Image correction is performed on the identified and cut fabric surface images. The corrected fabric surface images are output as the second fabric dataset, which includes a second fabric training set and a second fabric test set. The YOLO v8 instance segmentation algorithm is trained using the second fabric training set to build a fabric surface defect recognition model. Fabric surface defects in the second fabric test set are automatically recognized according to the initial detection cycle.

[0018] Furthermore, the process of determining the state and type of surface defects in the fabric includes,

[0019] The surface defect status of the fabric is determined based on the identification results of the fabric surface defect identification model.

[0020] Based on the surface defect status and the types of surface defects detected in the identification results, the type of surface defect in the fabric is determined, and a corresponding processing method is generated for the type of surface defect in the fabric.

[0021] Furthermore, the process of validating the recognition capability of the fabric surface defect recognition model includes,

[0022] The validation set is input into the fabric surface defect recognition model for detection. Based on the defect annotations in the validation set and the surface defect status identified by the fabric surface defect recognition model, the number of true negative cases, false positive cases, and false positive cases are divided.

[0023] Furthermore, the actual recall rate is calculated based on the number of true negative cases and the number of false negative cases. Based on the comparison process between the actual recall rate and the preset recall rate, it is determined whether the model's ability to identify positive cases meets the standard, whether a correction notice for the model needs to be issued, or whether the precision rate is determined, or the reason why the identification ability does not meet the standard is determined.

[0024] Furthermore, the process of determining accuracy includes,

[0025] Calculate the precision based on the number of true positives and false positives. Based on the precision result, re-evaluate whether the model's ability to identify positives meets the standard, and whether it is necessary to determine why the recognition ability does not meet the standard.

[0026] Furthermore, the process of determining why the recognition ability does not meet the standard includes,

[0027] The reasons why the identification ability does not meet the standard are determined based on the type and condition of the fabric surface defects;

[0028] Calculate the proportion of fabric images in the validation set that are in the second surface defect state and the first surface defect type in the dataset, representing the proportion of defect states and defect types.

[0029] The reason why the recognition ability does not meet the standard is determined by comparing the proportion of defective states and the proportion of defective types with the ratio of standard states and the ratio of standard types. This is because the number of samples in the dataset is too small, or there are errors or biases in the labeling of fabric defects in the dataset, or the ratio of defective samples to normal samples in the dataset is unbalanced.

[0030] A correction notice will be issued or training parameters will be adjusted accordingly based on the reason why the recognition ability does not meet the standard.

[0031] Furthermore, the process of issuing correction notices or adjusting training parameters accordingly based on the reasons why the recognition ability does not meet the standards includes,

[0032] When the number of samples in the dataset is small, a correction notification is issued to increase the amount of data, and the initial number of the first training set and the first test set, as well as the initial detection cycle for detecting and identifying defects on the test set, are called back. The initial detection cycle is reduced according to the ratio of the proportion of defective states to the proportion of standard states, and the initial number is increased according to the ratio of the proportion of standard states to the proportion of defective states.

[0033] When there are errors or biases in the labeling of fabric defects in the dataset, a correction notification is issued for the labeling of fabric defects in the images to be detected in the validation set.

[0034] When the ratio of defective samples to normal samples in the dataset is unbalanced, a correction notification is issued to adjust the ratio of training set to validation set in the dataset.

[0035] Furthermore, the difference between the detection rate of any surface defect in the validation set and the detection rate of 100% is calculated. Based on the comparison between the detection rate difference and the difference evaluation value and the corresponding weight, the ability of the fabric surface defect recognition model to recognize any corresponding surface defect is judged.

[0036] The proportion of fabric images corresponding to any surface defect type in the training set and the number of fabric images with corresponding surface defects in the test set are adjusted according to the ratio of the difference in detection rate of any surface defect type to the product of the difference evaluation value and the corresponding weight.

[0037] Furthermore, based on the surface defect type, the ability of the fabric surface defect recognition model to identify any corresponding surface defect does not meet the standard judgment criteria. The first surface defect type is set with a first weight, and the second surface defect type is set with a second weight. The first weight is greater than the second weight.

[0038] Compared with existing technologies, the advantages of this invention are that fabrics are oriented differently for visual cameras, and in order to comprehensively detect any area of ​​the fabric, the fabric surface images captured by the visual camera inevitably contain background or fabric skew. This method, based on the YOLO v8 target recognition and instance segmentation algorithm, proposes a two-stage cascaded fabric surface defect detection method. The first stage is fabric detection, which uses the fabric surface image to be detected to train a fabric detection model, realizing the recognition and positioning of the fabric in the global image. Then, the target fabric is cut and corrected. The second stage is fabric surface defect recognition, which uses the corrected image set output from the first stage to train a fabric surface defect recognition model, realizing the automatic recognition of fabric surface defects, including five types of defects such as broken warp, holes, stains, color difference, and oil stains. The defect position is fed back to the fabric surface image to be detected in the form of coordinate transformation, which is convenient for backtracking and maintenance.

[0039] Furthermore, the images of the fabric surface to be detected captured by the visual camera are affected by factors such as the shooting angle, fabric transport status, and illumination. Directly inputting these images into the fabric surface defect recognition model leads to extremely high false detection and false negative rates. Therefore, for the two-stage cascaded defect detection method adopted in this invention, in the first stage, the YOLO target recognition algorithm is used to identify the fabric and locate its position in the fabric surface to be detected image. The fabric surface in the original image is then cropped, and the cropped image is corrected by the algorithm to ensure the fabric presents a normal posture in the image. This reduces the false detection and false negative rates caused by direct image recognition, increases the uniformity of the fabric surface to be detected images in the test set, and improves the accuracy of defect detection for the fabric surface to be detected.

[0040] Furthermore, among the surface defects contained in the fabric surface, since warp breaks, holes, and color differences all cause relatively obvious damage to the fabric, they belong to the same type of surface defect. Stains and oil stains, however, do not cause significant damage to the fabric and belong to another type of surface defect. This method classifies surface defects into two types and generates different processing methods based on these types. This not only improves the flexibility and adaptability of the method for detecting defects in the prepared fabric images but also reduces the processing time for personnel to select processing methods for surface defects, saving labor and improving work efficiency.

[0041] Furthermore, recall measures the proportion of actual positive examples correctly identified by the model; a higher value indicates a stronger ability to identify actual targets. However, high recall is not always optimal, as it may lead to more false positives, thus affecting the overall performance of the model. This method uses a preset recall rate to determine whether the model's ability to identify positive examples meets the standard, avoiding false positive errors in fabric defect detection that exceed the normal range. Simultaneously, recall and precision are often contradictory; since both are difficult to improve simultaneously, a balance needs to be found in practical applications. In fabric defect detection, precision is more important; excessively high recall leads to false positives and wasted manpower. Therefore, this method limits recall based on precision. Judging whether the recall is too high based on precision requires identifying the reasons why the recognition ability does not meet the standard and whether the model's recognition ability meets the standard, further improving the applicability of this method for fabric surface defect detection.

[0042] Furthermore, the proportion of fabric images in the second surface defect state in the dataset is equivalent to the proportion of fabric images with surface defects identified by the model in the validation set. Similarly, the proportion of fabric images in the first surface defect type in the dataset is equivalent to the proportion of fabric images with one or more defects such as broken warp, holes, and color difference identified by the model in the validation set. When the proportion of defect states is less than the standard, it indicates that the model cannot completely identify fabric images with surface defects, and the model's recognition probability needs to be increased. Since the proportion of defect states includes the proportion of defect types, Therefore, if the percentage of defective states is not less than the standard but the percentage of defective types is less than the standard, it indicates that there may be errors or biases in the labeling of fabric defects in the dataset, which need to be corrected. When both the percentage of defective states and the percentage of defective types are less than the standard, the ratio of defective samples to normal samples in the dataset may be unbalanced. Too many normal samples cause the model to be biased towards recognizing fabric rather than detecting surface defects, which requires adjusting the ratio of training set to validation set in the dataset. This method analyzes the reasons why the recognition ability does not meet the standard by using the percentage of defective states and the percentage of defective types, and issues corresponding correction notices, which increases the adaptability and adjustment flexibility of defect detection.

[0043] Furthermore, since broken warp threads, holes, and color differences all cause significant damage to the fabric, the severity of the first surface defect type is greater than that of the second surface defect type. At the same time, compared to the first surface defect type, the second surface defect type often leads to more obvious differences in the fabric image, making it relatively easier for the model to detect. This method sets different weights for surface defect types to establish a criterion for judging the model's ability to identify any surface defect. Furthermore, it refines and distinguishes different detection criteria based on the application scenarios of fabric defect detection and the differences in existing detection types.

[0044] Furthermore, in the process of determining why the recognition ability does not meet the standard, the recall rate of fabric images with different surface defects jointly determines the actual recall rate of the model. This method judges the model's recognition ability for any surface defect by testing the detection rate of any surface defect, and increases the number and proportion of fabric images with the corresponding surface defect in the test set accordingly. By improving the detection rate of the corresponding surface defect, the actual recall rate of the model is optimized, the model's recognition ability is improved, the false alarm error of the model in detecting and recognizing fabric defects is reduced, and the accuracy and precision of defect detection are increased.

[0045] Furthermore, the coordinate transformation diagram is shown in the attached figure. The left figure shows the position of the defect anchor frame of the fabric cut in the first stage in the second stage, and the right figure shows the position of the defect anchor frame in the original figure after coordinate transformation. After the coordinate transformation of the position of the defect anchor frame of the fabric cut in the first stage in the second stage, the position of the defect anchor frame marked in the original figure can be seen from the result. It can be seen that the fabric defect anchor frame is correctly restored, which provides a basis and guidance for inspection and maintenance, and reduces the workload of subsequent inspection and maintenance. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the online fabric defect detection method based on multispectral imaging and AI fusion in an embodiment of the present invention.

[0047] Figure 2 This is a schematic diagram of a two-stage cascaded fabric surface defect detection method in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram illustrating the first stage of cropping and correcting the fabric in the image to be detected in an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of anchor frame regression of the original image to be detected in an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of a method for coordinate transformation of a defective anchor frame in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0052] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0053] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0054] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0055] Please see Figures 1-5 As shown, Figure 1 This is a flowchart illustrating the online fabric defect detection method based on multispectral imaging and AI fusion in an embodiment of the present invention. Figure 2 This is a schematic diagram of a two-stage cascaded fabric surface defect detection method in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the first stage of cropping and correcting the fabric in the image to be detected in an embodiment of the present invention; Figure 4 This is a schematic diagram of anchor frame regression of the original image to be detected in an embodiment of the present invention; Figure 5 This is a schematic diagram of a method for coordinate transformation of a defective anchor frame in an embodiment of the present invention.

[0056] This invention provides an online fabric defect detection method based on multispectral imaging and AI fusion, comprising:

[0057] Step S1: Target detection is performed on several images to be detected contained in the fabric test set. The fabric in the images to be detected is located, cropped and corrected. Several fabric images are output to form a new fabric test set, and corrected images are output to summarize them into a corrected image set.

[0058] Step S2: Train the fabric surface defect recognition model by correcting the image set, and automatically identify the surface defects of several fabric images in the fabric test set according to the initial detection cycle to determine the surface defect status of the fabric images.

[0059] Step S3: Output the fabric defect location in the fabric test set according to the surface defect status of the fabric image, and feed back the defect location to the image to be detected in the fabric test set in the form of coordinate transformation.

[0060] Step S4: Determine the type of surface defects of the fabric and summarize the data. Verify the recognition ability of the fabric surface defect recognition model. Determine whether the recognition ability of the fabric surface defect recognition model meets the standard, or determine whether the recognition ability does not meet the standard and analyze the reasons for not meeting the standard, or conduct a secondary analysis of the recognition ability.

[0061] Step S5: Issue a correction notice or revise the judgment criteria for the determined reasons why the recognition capability does not meet the standard.

[0062] Step S6: The detection rate of any surface defect is tested. Based on the difference in detection rate and the corresponding weight of the surface defect, the recognition ability for any surface defect is judged. The distribution and total number of fabric images with different surface defects in the fabric test set are adjusted.

[0063] In this embodiment, fabric surface defects include five types of defects: broken warp, holes, stains, color difference, and oil stains. It is understood that the fabric surface defects included in this embodiment are not limited to the above five types of defects. The above five types of defects are only preferred embodiments of the present invention and are not intended to limit the present invention.

[0064] Application scenario: A movable, high-precision vision camera is used to take several pictures of the prepared fabric to obtain several high-precision images of the fabric surface to be inspected.

[0065] This invention employs a two-stage cascaded fabric defect detection method based on multispectral imaging and AI fusion. The first stage is fabric detection, and the second stage is fabric surface defect recognition. In the first stage of the fabric defect detection method, the YOLO target detection algorithm is used to identify the fabric in the test set and locate its position in the fabric image to be detected. The fabric in the fabric image to be detected is cropped and corrected, and a corrected image set is output. In the second stage of the fabric defect detection method, the fabric surface defect recognition model is trained using the corrected image set output from the first stage. The model automatically identifies the fabric surface defects in the test set according to the initial detection cycle and determines the surface defect status of the fabric.

[0066] The process of positioning, cropping, and image correction of the fabric includes...

[0067] The first training set is generated based on the initial number of cloth images to be detected. The YOLOv8 object detection algorithm is trained using the first cloth training set to build a cloth detection model.

[0068] The initial number of fabric surface images to be detected, acquired by a high-precision vision camera, are used as the first test set of the fabric. They are then input into the fabric detection model according to the initial detection cycle for detection, and the fabric in the fabric surface images to be detected is identified and cut.

[0069] Image correction is performed on the identified and cut fabric surface images. The corrected fabric surface images are output as the second fabric dataset, which includes a second fabric training set and a second fabric test set. The YOLO v8 instance segmentation algorithm is trained using the second fabric training set to build a fabric surface defect recognition model. Fabric surface defects in the second fabric test set are automatically recognized according to the initial detection cycle.

[0070] Specifically, fabric surfaces are oriented differently for visual cameras, and to comprehensively detect any area of ​​the fabric, the images of the fabric surface captured by the visual camera inevitably contain background or fabric skew. This method, based on the YOLOv8 target recognition and instance segmentation algorithm, proposes a two-stage cascaded fabric surface defect detection method. The first stage is fabric detection, which uses the fabric surface image to be detected to train a fabric detection model, realizing the recognition and localization of the fabric in the global image. Then, the target fabric is cut and corrected. The second stage is fabric surface defect recognition, which uses the corrected image set output from the first stage to train a fabric surface defect recognition model, realizing the automatic recognition of fabric surface defects, including five types of defects such as broken warp, holes, stains, color difference, and oil stains. The defect location is fed back to the fabric surface image to be detected in the form of coordinate transformation, which is convenient for backtracking and maintenance.

[0071] The correction process uses the Hough transform algorithm. First, the Canny algorithm is used to extract the pixels on the edge of the fabric surface. Then, the pixels are input into the Hough transform algorithm to transform the line detection problem in the image space into the parameter space for solution.

[0072]

[0073] Where ρ is the perpendicular distance from the origin to the line, and θ is the angle between the perpendicular line and the positive direction of the axis.

[0074] For each edge point in the image, its corresponding curve in the ρ-θ parameter space is calculated. The intersection of multiple curves is the corresponding straight line in the image. Straight lines that may exist on the edge of the fabric surface are detected, and the average tilt angle of each straight line is calculated. Then, the image is rotated and corrected based on the average tilt angle to make the fabric present a normal posture in the image.

[0075] Specifically, the images of the fabric surface to be detected captured by the visual camera are affected by factors such as the shooting angle, the fabric's transport status, and the illumination. Directly inputting these images into the fabric surface defect recognition model leads to extremely high false detection and false negative rates. Therefore, the two-stage cascaded defect detection method adopted in this invention, in the first stage, uses the YOLO target recognition algorithm to identify the fabric and locate its position in the fabric surface image to be detected. The fabric surface in the original image is then cropped, and the cropped image is corrected by the algorithm to ensure the fabric presents a normal posture in the image. This reduces the false detection and false negative rates caused by direct image recognition, increases the uniformity of the fabric surface images to be detected in the test set, and improves the accuracy of defect detection for the fabric surface to be detected.

[0076] The process of determining the surface defect status of fabric includes automatically identifying surface defects in the second test set of fabrics.

[0077] If the fabric surface defect recognition model does not detect fabric surface defects, the fabric is determined to be in the first surface defect state.

[0078] If the fabric surface defect recognition model detects fabric surface defects, it determines that the fabric is in the second surface defect state.

[0079] When the fabric is in the second surface defect state, the location of the fabric defect is output to the fabric image in the second test set that is in the second surface defect state.

[0080] The process of determining the type of surface defect in a fabric includes determining the type of surface defect when the fabric is in a second-degree surface defect state.

[0081] If the fabric surface defects include any one or more of the following defects: broken warp, hole, color difference, then the fabric is judged to be in the first type of surface defect.

[0082] If the fabric surface defects do not include any of the defects such as broken warp, holes, or color difference, then the fabric is judged to be in the second type of surface defect.

[0083] Generate corresponding treatment methods for the types of defects on the fabric surface, including cleaning and replacement;

[0084] When the fabric is classified as having the first type of surface defect, the processing method is to cut off the corresponding fabric area.

[0085] When the fabric is classified as a second-level surface defect, the processing method is to clean the corresponding fabric area.

[0086] Specifically, among the surface defects contained in the fabric surface, broken warp threads, holes, and color differences all cause relatively obvious damage to the fabric, and therefore belong to the same type of surface defect. Stains and oil stains, however, do not cause significant damage to the fabric and belong to another type of surface defect. This method classifies surface defects into two types and generates different processing methods based on these types. This not only improves the flexibility and adaptability of the method for defect detection on the prepared fabric images but also reduces the processing time for personnel to select processing methods for surface defects, saving labor and improving work efficiency.

[0087] The process of validating the ability of the fabric surface defect recognition model includes inputting the validation set into the fabric surface defect recognition model for detection.

[0088] Fabrics that are identified as having a second surface defect state by the fabric surface defect recognition model and have defect markings in the validation set are classified as true cases.

[0089] Fabrics that are identified as having a second surface defect state by the fabric surface defect recognition model and for which there are no defect labels in the validation set are classified as false negatives.

[0090] Fabrics that are identified as having the first surface defect state by the fabric surface defect recognition model and have defect labels in the validation set are classified as false positives.

[0091] The actual recall rate is calculated based on the number of true positives and the number of false negatives. The actual recall rate is equal to the ratio of the number of true positives to the sum of the number of true positives and the number of false negatives. The higher the actual recall rate, the stronger the model's ability to identify positives, but it may lead to false positives.

[0092] In this embodiment, the preset recall rate includes a first preset recall rate and a second preset recall rate. The first preset recall rate is greater than the second preset recall rate. In this embodiment, the first preset recall rate and the second preset recall rate are assigned values ​​of 93% and 88%, respectively.

[0093] The first and second preset recall rates can be determined based on the actual application. The comparison process between the actual recall rate and the preset recall rate is as follows:

[0094] If the actual recall rate is less than or equal to the first preset recall rate and greater than the second preset recall rate, then the model is determined to meet the standard in terms of its ability to identify positive examples, and no correction notice or correction benchmark needs to be issued for the model.

[0095] If the actual recall rate is greater than the first preset recall rate, the precision rate is judged, and the model's ability to identify positive examples is re-evaluated based on the precision rate judgment result to determine whether it meets the standard and whether it is necessary to judge the reason why the recognition ability does not meet the standard.

[0096] If the actual recall rate is less than or equal to the first preset recall rate, it is determined that the model's ability to identify positive examples does not meet the standard, and it is necessary to determine the reason why the recognition ability does not meet the standard.

[0097] The process of determining accuracy includes,

[0098] Precision is equal to the ratio of the number of true positives to the sum of the number of true positives and false positives. The higher the value, the higher the accuracy of the model's predictions.

[0099] If the accuracy rate is greater than the preset accuracy rate, it is determined that the false alarm error of the model in detecting and identifying fabric defects is within the normal range, the model's recognition ability meets the standard, and there is no need to determine the reason why the recognition ability does not meet the standard.

[0100] If the accuracy rate is less than the preset accuracy rate, it is determined that the false alarm error of the model in detecting and identifying fabric defects exceeds the normal range, and the model's recognition ability does not meet the standard. It is necessary to determine the reason why the recognition ability does not meet the standard.

[0101] The preset accuracy rate is a preset value set according to the requirements for identifying defects.

[0102] Specifically, recall measures the proportion of actual positive examples correctly identified by the model; a higher value indicates a stronger ability to identify actual targets. However, a high recall is not always optimal, as it may lead to more false positives, thus affecting the overall performance of the model. This method uses a preset recall rate to determine whether the model's ability to identify positive examples meets the standard, avoiding false positive errors in detecting fabric defects that exceed the normal range. Furthermore, recall and precision are often contradictory; since both are difficult to improve simultaneously, a balance needs to be found in practical applications. In fabric defect detection, precision is more important; excessively high recall leads to false positives and wasted effort. Therefore, this method limits recall based on precision. Judging whether the recall is too high based on precision requires identifying the reasons why the recognition ability does not meet the standard and whether the model's recognition ability meets the standard, further improving the applicability of this method for detecting defects on fabric surfaces.

[0103] The process of determining why recognition capabilities do not meet standards includes,

[0104] The reasons why the identification ability does not meet the standard are determined based on the type and condition of the fabric surface defects;

[0105] Calculate the percentage of fabric images in the validation set that are in the second surface defect state and the percentage of defect types in the dataset, representing the proportion of defect states and defect types.

[0106] If the proportion of defective states is less than the proportion of standard states, it is determined that the number of samples in the dataset is too small. A correction notification is issued to increase the amount of data, and the initial number of the first training set and the first test set, as well as the initial detection cycle for detecting and identifying defects on the test set, are called back.

[0107] During implementation, the initial inspection cycle is reduced based on the ratio of the percentage of defective states to the percentage of standard states, and the initial quantity is increased based on the ratio of the percentage of standard states to the percentage of defective states.

[0108] If the proportion of defect types is less than the standard type ratio, it is determined that the labeling of fabric defects in the dataset may be incorrect or biased, and a correction notification is issued for the fabric defect labeling of the fabric images to be detected in the validation set.

[0109] If the proportion of defective states is less than the proportion of standard states, and the proportion of defective types is less than the proportion of standard types, it is determined that the ratio of defective samples to normal samples in the dataset may be severely imbalanced. The model tends to identify the dominant class and issues a correction notice to adjust the proportion of training and validation sets in the dataset.

[0110] The standard state ratio is the actual percentage of fabrics with defects in the verification set, and the standard type ratio is the actual percentage of fabrics in the verification set whose surface defects include any one or more of the defects such as broken warp, holes, and color difference.

[0111] Specifically, the proportion of fabric images in the dataset exhibiting the second surface defect state is equivalent to the proportion of fabric images with surface defects identified by the model in the validation set. Similarly, the proportion of fabric images exhibiting the first surface defect type is equivalent to the proportion of fabric images with one or more defects such as broken warp, holes, and color difference identified by the model in the validation set. A defect state proportion lower than the standard indicates that the model cannot completely identify fabric images with surface defects, requiring an increase in the model's recognition probability. Since the defect state proportion includes the defect type proportion... Therefore, if the percentage of defective states is not less than the standard but the percentage of defective types is less than the standard, it indicates that there may be errors or biases in the labeling of fabric defects in the dataset, which need to be corrected. When both the percentage of defective states and the percentage of defective types are less than the standard, the ratio of defective samples to normal samples in the dataset may be unbalanced. Too many normal samples cause the model to be biased towards recognizing fabric rather than detecting surface defects, which requires adjusting the ratio of training set to validation set in the dataset. This method analyzes the reasons why the recognition ability does not meet the standard by using the percentage of defective states and the percentage of defective types, and issues corresponding correction notices, which increases the adaptability and adjustment flexibility of defect detection.

[0112] In determining why the recognition capability does not meet the standard, the recall rate of fabric images with different surface defects jointly determines the actual recall rate. The detection rate of any surface defect is then tested.

[0113] Calculate the detection rate of any surface defect in the validation set. The detection rate is equal to the ratio of the number of fabrics in the validation set with any surface defect identified by the fabric surface defect recognition model to the number of fabrics in the validation set that are labeled with any surface defect.

[0114] Calculate the difference between the detection rate and the 100% detection rate, and compare this difference with the difference evaluation value and its corresponding weight.

[0115] If the difference in detection rate for any type of surface defect is less than the product of the difference evaluation value and the corresponding weight, then the fabric surface defect recognition model is judged to have a recognition capability that meets the standard for any surface defect.

[0116] If the difference in detection rate for any surface defect type is greater than the product of the difference evaluation value and the corresponding weight, it is determined that the fabric surface defect recognition model does not meet the standard for recognizing any surface defect, and the proportion of the fabric image corresponding to any surface defect type in the training set is adjusted.

[0117] In practice, the proportion of fabric images corresponding to any surface defect type in the training set is increased by the ratio of the difference in detection rate for any surface defect type to the product of the difference evaluation value and the corresponding weight.

[0118] If the difference in detection rates between two or more surface defects is greater than the product of the difference evaluation value and the corresponding weight, then the recognition ability of the fabric surface defect recognition model is deemed not to meet the standard, and the number of fabric images with corresponding surface defects in the test set is increased.

[0119] In practice, the number of fabric images with corresponding surface defects in the test set is increased by the ratio of the difference in detection rate for any type of surface defect to the product of the difference evaluation value and the corresponding weight.

[0120] Specifically, the judgment criteria for determining whether the fabric surface defect recognition model's ability to identify any surface defect does not meet the standard are adjusted based on the first surface defect type and the second surface defect type. Weights are set according to the first surface defect type and the second surface defect type, with the first surface defect type having a first weight and the second surface defect type having a second weight, and the first weight being greater than the second weight.

[0121] Specifically, since broken warp, holes, and color differences all cause significant damage to the fabric, the severity of the first type of surface defect is greater than that of the second type. At the same time, compared to the first type of surface defect, the second type of surface defect often leads to more obvious differences in the fabric image, making it easier for the model to detect. This method sets different weights for different types of surface defects to establish a criterion for judging the model's ability to identify any surface defect. Furthermore, it refines and distinguishes different detection criteria based on the application scenarios of fabric defect detection and the differences in existing detection types.

[0122] Meanwhile, in determining why the recognition ability does not meet the standard, the recall rate of fabric images with different surface defects jointly determines the actual recall rate of the model. This method examines the detection rate of any surface defect to judge the model's ability to recognize any surface defect. Based on any surface defect, the number and proportion of fabric images with the corresponding surface defect in the test set are increased accordingly. By improving the detection rate of the corresponding surface defect, the actual recall rate of the model is optimized, the model's recognition ability is improved, the false alarm error of the model in detecting and recognizing fabric defects is reduced, and the accuracy and precision of defect detection are increased.

[0123] The process of feeding back the location of defects to the fabric image to be inspected in the test set in the form of coordinate transformation includes:

[0124] The YOLO v8 instance segmentation algorithm was trained using the dataset after the first stage correction to obtain a fabric surface defect recognition model, which identifies five types of fabric surface defects and maps the defect coordinates to the fabric image to be detected using the homography matrix.

[0125] The algorithm detects feature points in the cropped image of the fabric and the original fabric image, calculates the descriptors of the feature points, matches the feature point descriptors, extracts the coordinates of the source and target points from the matched feature points, estimates the homography matrix of the original image, performs coordinate transformation on the homography matrix, and draws the anchor box.

[0126] Specifically, the coordinate transformation diagram is attached. Figure 4 As shown, the left image shows the position of the defect anchor frame in the second stage of fabric identification after the first stage of cutting, and the right image shows the position of the defect anchor frame in the original image after coordinate transformation. After the coordinate transformation of the position of the defect anchor frame in the second stage of fabric identification, the position of the defect anchor frame in the original image is marked by the defect anchor frame. The results show that the fabric defect anchor frame is correctly restored, providing a basis and guidance for inspection and maintenance, and reducing the workload of subsequent inspection and maintenance.

[0127] The detection results on the statistical test set are shown in Table 1. The lowest recognition accuracy for the five types of defects was for cracks at 76.4%, while the other four categories all exceeded 92%, with an average of 92.5%. The other evaluation indicators showed similar performance.

[0128] Table 1 Model Detection Results

[0129] Defect Category Dataset Number of defects Precision (P) Recall rate (R) <![CDATA[mAP 50 ]]> amenorrhea 57 66 98.4% 93.5% 98.9% Hole 62 62 96.6% 90.3% 97.8% stains 75 88 76.4% 75% 76.9% Color difference 56 56 98.2% 95.9% 98.9% oil stains 53 120 92.7% 98.3% 98.8% total 303 392 92.5% 90.6% 94.3%

[0130] The fabric defect detection method proposed in this paper is compared with the mainstream YOLO algorithm and the Faster R-CNN+YOLO v5s defect detection method. The results are shown in Table 2. The YOLO v8 fabric defect detection algorithm with a two-stage cascade strategy has the best performance in terms of precision, recall and mAP50, and can better meet the actual detection needs.

[0131] Table 2. Experimental results of fabric defect detection using different algorithms.

[0132] algorithm Accuracy (%) Recall rate (%) <![CDATA[mAP 50 (%)]]> Faster R-CNN+YOLO v5s 86.2% 90.2% 89.1% YOLO v9 87.4% 85.8% 86.5% YOLO 11 91.5% 86.3% 92.2% Ours 92.5% 90.6% 94.3%

[0133] (1) The present invention selects 1007 fabric images as the test dataset, including five types of defects such as broken warp, holes, stains, color difference, and oil stains.

[0134] (2) The data obtained from the experimental dataset were labeled using Labelme software. The labeled dataset was in JSON format. The format conversion code was used to convert it into a txt file. To ensure reasonable training and testing of the network model, the dataset was divided into training set and validation set in a 7:3 ratio.

[0135] (3) Input the above dataset into the YOLO v8 network model for training, output the fabric detection module, input the output fabric detection results into the YOLO v8 network model for training, and finally output the fabric surface defect detection results.

[0136] (4) The YOLOv8 model was trained for a total of 500 iterations with an initial learning rate of 0.01. Stochastic gradient descent (SGD) was used to update the network parameters with a learning momentum of 0.9 and a training batch size of 16. Table 3 shows the hardware and software configuration of the experimental environment.

[0137] Table 3 Hardware and software environment configuration table

[0138] Configuration Name Version parameters operating system Windows 10, 64-bit operating system CPU 12th Gen Intel(R) Core(TM) i7-12700KF 3.60 GHz GPU NVIDIA GeForce RTX 4090 D CUDA V12.3 CUDNN V9.4 Deep learning framework PyTorch 2.1

[0139] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0140] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A fabric defect online detection method based on multispectral imaging and AI fusion, characterized in that, The method comprises the following steps: target detection is performed on a plurality of to-be-detected images included in a cloth test set, positioning and image correction are performed on the cloth in the to-be-detected images, a plurality of cloth images are output to form a new cloth test set, and the corrected images are output and collected into a corrected image set; a cloth surface defect recognition model is trained through the corrected image set, and the surface defects of the plurality of cloth images in the cloth test set are automatically recognized according to an initial detection period to determine the surface defect state of the cloth images; the surface defect positions of the cloth in the cloth test set are output according to the surface defect state of the cloth images, and the defect positions are fed back to the to-be-detected images in the cloth test set in the form of coordinate transformation; the surface defect type of the cloth is determined and data is collected, the recognition ability of the cloth surface defect recognition model is verified, and it is determined whether the recognition ability of the cloth surface defect recognition model meets the standard, or it is determined that the recognition ability does not meet the standard and the reason is analyzed, or the recognition ability is analyzed again; a correction notice or correction determination criterion is issued according to the reason why the determined recognition ability does not meet the standard; the detection rate of any surface defect is tested, and the recognition ability for any surface defect is determined based on the detection rate difference and the corresponding weight of the surface defect, and the distribution and total number of cloth images with different surface defects in the cloth test set are adjusted.

2. The method according to claim 1, wherein, The process of positioning, cutting and image correction for the cloth in the to-be-detected image comprises: a first training set is generated according to an initial number of cloth to-be-detected images, a YOLO v8 target detection algorithm is trained using the cloth first training set, and a cloth detection model is built; an initial number of cloth surface to-be-detected images collected by a high-precision visual camera are used as a cloth first test set, and the cloth detection model is input for detection according to an initial detection period to recognize and cut the cloth in the cloth surface to-be-detected images; image correction is performed on the cloth surface images after recognition and cutting, the corrected cloth surface images are output as a cloth second data set, the cloth second data set includes a cloth second training set and a cloth second test set, a YOLO v8 instance segmentation algorithm is trained using the cloth second training set, a cloth surface defect recognition model is built, and the cloth surface defects in the cloth second test set are automatically recognized according to an initial detection period.

3. The method according to claim 2, wherein, The process of determining the surface defect state and the surface defect type of the cloth comprises: the surface defect state of the cloth is determined based on the recognition result of the cloth surface defect recognition model; the surface defect type of the cloth is determined based on the surface defect state and the surface defect type detected in the recognition result, and a corresponding processing method is generated for the surface defect type of the cloth.

4. The method according to claim 3, wherein, The process of verifying the recognition ability of the cloth surface defect recognition model comprises: a verification set is input into the cloth surface defect recognition model for detection, and the number of true positives, false negatives and false positives is determined based on the defect annotations in the verification set and the surface defect state recognized by the cloth surface defect recognition model.

5. The cloth defect online detection method based on multispectral imaging and AI fusion according to claim 4, characterized in that, The actual recall rate is calculated according to the number of true cases and false negative cases, and the identification ability of the model for the positive cases is determined based on the comparison between the actual recall rate and the preset recall rate, whether the identification ability meets the standard, whether a correction notice needs to be issued for the model, or the precision rate is determined, or the reason why the identification ability does not meet the standard is determined.

6. The method according to claim 5, wherein, The process of determining the precision rate comprises: The precision rate is calculated according to the number of true cases and false positive cases, and the identification ability of the model for the positive cases is re-determined according to the determination result of the precision rate, whether the identification ability meets the standard, and whether the reason why the identification ability does not meet the standard needs to be determined.

7. The method according to claim 6, wherein, The process of determining the reason why the identification ability does not meet the standard comprises: The reason why the identification ability does not meet the standard is determined according to the type and state of the surface defects of the cloth; The proportion of the state of the defects and the proportion of the type of the defects of the cloth images in the second surface defect state and the first surface defect type in the data set are calculated; The reason why the identification ability does not meet the standard is determined according to the comparison between the proportions of the defect state and the defect type and the standard state ratio and the standard type ratio; The correction notice is issued or the training parameters are adjusted according to the reason why the identification ability does not meet the standard.

8. The method according to claim 7, wherein, The process of issuing the correction notice or adjusting the training parameters according to the reason why the identification ability does not meet the standard comprises: When the number of samples in the data set is small, a correction notice is issued to increase the data quantity, the initial number of the first training set and the first test set is recalled, and the initial detection period for detecting and identifying defects in the test set is recalled, the initial detection period is reduced according to the ratio of the proportion of the defect state to the standard state ratio, and the initial number is increased according to the ratio of the standard state ratio to the proportion of the defect state; When the labeling of the defects of the cloth in the data set is wrong or biased, a correction notice is issued for the labeling of the defects of the cloth in the images to be detected in the verification set; When the proportion of the defect samples and the normal samples in the data set is imbalanced, a correction notice is issued to adjust the proportion of the training set and the verification set in the data set.

9. The method according to claim 8, wherein, The detection rate difference of any surface defect in the verification set is calculated; The identification ability of the cloth surface defect identification model for the corresponding any surface defect is determined based on the comparison between the detection rate difference and the difference evaluation value and the corresponding weight; And the proportion of the cloth images of any surface defect type in the training set and the number of cloth images with the corresponding surface defect in the test set are adjusted according to the ratio of the product of the detection rate difference of any surface defect type to the difference evaluation value and the corresponding weight.

10. The method according to claim 9, wherein, The judgment standard for determining whether the identification ability of the cloth surface defect identification model for the corresponding any surface defect meets the standard is adjusted according to the type of the surface defect, the first surface defect type is set with a first weight, the second surface defect type is set with a second weight, and the first weight is greater than the second weight.

Images