Cloth texture defect detection method, device, equipment and medium

Abstract

The invention relates to a cloth texture defect detection method and device, equipment and a medium, and the method comprises the steps: employing quaternion Hamiltonian transformation to process defect candidate frame features, so as to promote channel interaction between different channels in the defect candidate frame features, and to determine aggregated defect candidate frame features; performing structural reasoning on the defect candidate frame features and the background features by adopting a bidirectional long-short-term memory network to capture context dependency relationships between the defect candidate frame features and between the defect candidate frame features and the background features so as to determine the defect candidate frame features after structural reasoning; and inputting the aggregated defect candidate frame features and the defect candidate frame features after structure reasoning into a preset full-connection neural network to determine a cloth texture defect type corresponding to each defect candidate frame in the to-be-detected cloth texture image. According to the method, the detection precision of the cloth texture defects can be remarkably improved in a complex cloth texture environment.

Description

Technical Field

[0001] The present application relates to the field of cloth texture detection, and in particular to a cloth texture defect detection method, a corresponding device, an electronic device and a computer-readable storage medium. Background Art

[0002] Texture defect detection is an important part of quality control in industrial automation production. Its core goal is to quickly and accurately identify and locate potential defects on the surface of industrial products, thereby ensuring product quality consistency and reliability. In actual industrial applications, texture defect detection faces a variety of complex challenges, such as diverse texture characteristics, lighting changes, background interference, and highly variable surface morphology. These factors have put forward higher requirements on existing detection technologies.

[0003] Traditional texture defect detection methods are usually based on manually designed feature extraction techniques, such as SIFT, LBP and Gabor filtering. These methods mainly rely on the extraction and description of local texture features, and are suitable for defect detection with strong regularity and homogeneous textures. However, for surfaces with complex and uneven textures, such as cloth textures, traditional methods cannot fully capture contextual information and global correlations between textures, and are prone to insufficient detection accuracy and robustness. In addition, such methods rely heavily on manual features and are difficult to adapt to non-ideal conditions such as lighting, texture direction or scale changes in complex scenes.

[0004] With the rapid development of deep learning technology, convolutional neural networks (CNN) have become a major breakthrough in the field of texture defect detection. By utilizing large-scale annotated data, CNN can extract deep features in texture images and significantly improve detection accuracy. However, deep learning methods still have significant limitations in actual industrial scenarios. First, CNN relies on a large amount of annotated data, especially the collection and annotation of high-quality abnormal samples, which is extremely difficult, which becomes a major bottleneck in industrial applications where abnormal samples are scarce. Secondly, the feature extraction of existing convolutional neural network (CNN) methods is mostly concentrated in local areas, lacking effective modeling of image background semantic information and context associations, which limits its detection capabilities in highly complex or inhomogeneous texture scenes. In addition, the static feature expression of CNN is difficult to adjust dynamically, and fails to fully utilize the inter-regional relationship in texture images.

[0005] In recent years, graph neural networks (GNNs) have gradually gained attention in the field of texture detection. Their powerful graph structure modeling ability enables them to represent texture images as graph structures composed of nodes and edges, thereby capturing the complex associations between textures. By propagating information between nodes, GNNs can effectively extract global and local joint features, making up for the insufficient utilization of context information in CNN methods. However, current GNN-based detection methods still face challenges in practical applications, such as insufficient fusion mechanisms for local and background features, low graph inference efficiency, and limited adaptability to complex industrial scenarios.

[0006] In summary, for surfaces with complex and unevenly distributed textures in the prior art, traditional methods cannot fully capture context information and global associations between textures, and are prone to deficiencies in detection accuracy and robustness. Additionally, feature extraction in existing convolutional neural network methods mostly focuses on local regions, lacking effective modeling of image background semantic information and context associations, which limits their detection capabilities in high-complexity or non-uniform texture scenarios. In consideration of solving these problems, the applicant has made corresponding explorations. Summary of the Invention

[0007] The purpose of this application is to solve the above problems by providing a method for detecting fabric texture defects, a corresponding device, an electronic device, and a computer-readable storage medium.

[0008] To achieve the various objectives of this application, the following technical solutions are adopted:

[0009] A method for detecting fabric texture defects proposed to achieve one of the objectives of this application includes:

[0010] Obtain a fabric texture image to be detected containing texture defects, and use a preset object detection model to perform object detection on the fabric texture image to be detected to determine defect candidate boxes in the fabric texture image to be detected, so as to extract defect candidate box features and background features corresponding to the defect candidate boxes, where the defect candidate boxes represent regions where texture defects may occur;

[0011] Call a preset quaternion graph neural network, use the defect candidate box features corresponding to the defect candidate boxes as nodes in the quaternion graph neural network, and establish edge connections between its respective nodes through graph convolution operations to extract the mutual relationships between different defect candidate boxes;

[0012] Use quaternion Hamilton transformation to process the defect candidate box features to promote channel interaction between different channels in the defect candidate box features to determine the aggregated defect candidate box features;

[0013] Use a bidirectional long short-term memory network to perform structural inference on the defective candidate box features and the background features, so as to capture the context dependencies between the defective candidate box features and between the defective candidate box features and the background features, and to determine the defective candidate box features after structural inference;

[0014] Input the aggregated defective candidate box features and the defective candidate box features after structural inference into a preset fully connected neural network to determine the types of fabric texture defects corresponding to each defective candidate box in the fabric texture image to be detected, so as to complete the defect detection of the fabric texture.

[0015] Optionally, each defective candidate box feature is used as a node of the graph neural network, where l represents the layer where it is located, the number of layers of the graph neural network is 2, N represents the Nth candidate box, the set of k neighbors of each defective candidate box is N(i), and an edge is added for each j element belonging to N(i);

[0016] Construct a graph G(X,E), where X is the set of nodes and E is the set of edges. Then the expression of the inference process of the quaternion graph neural network is:

[0017]

[0018] where W is the weight, is the graph convolution function;

[0019] The expression of the maximum relative convolution of the graph convolution function is:

[0020]

[0021] where max() is the maximum value function.

[0022] Optionally, W and in the maximum relative convolution of the graph convolution function are respectively expressed as their corresponding quaternions. The expression of the quaternion corresponding to W is:

[0023] W = W r + W i i + W j j + W k k,

[0024] where W r represents the first part of the quaternion corresponding to W, W i represents the second part of the quaternion corresponding to W, W j represents the third part of the quaternion corresponding to W, W k represents the fourth part of the quaternion corresponding to W;

[0025] The expression of the corresponding quaternion is represented as:

[0026]

[0027] Wherein, represents the first part of the corresponding quaternion, represents the second part of the corresponding quaternion, represents the third part of the corresponding quaternion, represents the fourth part of the corresponding quaternion;

[0028] For the defect candidate box feature perform quaternion Hamiltonian transformation, and its expression is represented as:

[0029]

[0030] Use quaternion Hamiltonian transformation to process the defect candidate box feature to promote channel interaction between different channels in the defect candidate box feature, and then determine the aggregated defect candidate box feature

[0031] Optionally, the step of using a bidirectional long short-term memory network to perform structural reasoning on the defect candidate box feature and the background feature to capture the context dependence between each defect candidate box feature and between the defect candidate box feature and the background feature, and then determine the defect candidate box feature after structural reasoning includes:

[0032] Take each defect candidate box feature as the input at time step t of the bidirectional long short-term memory network, and the background feature x s as the input of the initial hidden state of the bidirectional long short-term memory network. For each defect candidate box feature x t in terms of, its expression of structural reasoning includes:

[0033]

[0034] Wherein, h t is the hidden state at time step t, and the initial hidden state h 0 = x s , x s represents the background feature, C t represents the cell state at time step t, represents the candidate cell state at time step t, f t represents the activation value of the forgetting gate, it represents the activation value of the input gate, o t represents the activation value of the output gate, σ(.) represents the sigmoid activation function, W represents the weight matrix, b represents the bias, and [.] represents the concatenation operation of vectors;

[0035] For the bidirectional long short-term memory network, the output at time step t is expressed as:

[0036]

[0037] where represents the output of the forward long short-term memory network, represents the output of the backward long short-term memory network, and the initial hidden states of the forward and backward long short-term memory networks and are both the background feature x s , represents the initial hidden state of the forward long short-term memory network, represents the initial hidden state of the backward long short-term memory network.

[0038] Optionally, perform three-layer feedforward neural network for gradual feature dimensionality reduction on the o t to keep the o t and the aggregated defect candidate box features in the same dimension without losing the original feature information, and its expression is:

[0039]

[0040] where represents the defect candidate box features after structure inference, and f(.) represents the RuLU activation function.

[0041] Optionally, the step of inputting the aggregated defect candidate box features and the defect candidate box features after structure inference into a preset fully connected neural network to determine the types of fabric texture defects corresponding to each defect candidate box in the fabric texture image to be detected includes:

[0042] Fuse the aggregated defect candidate box features and the defect candidate box features after structure inference to determine the fused defect candidate box feature x i , and its expression includes:

[0043]

[0044] where x i represents the fused defect candidate box feature;

[0045] Input the fused defect candidate box feature x i into a preset fully-connected neural network to determine the types of fabric texture defects corresponding to each defect candidate box in the fabric texture image to be detected.

[0046] Optionally, the basic network architecture of the object detection model includes the YOLOv5 object detection model. The types of fabric texture defects include one or any combination of stripe defects, breakage defects, yarn defects, color defects, and weaving defects. The stripe defects include one or any combination of spot defects, stripe defects, and uneven texture defects. The breakage defects include one or any combination of broken-end defects, flyaway defects, and penetration hole defects. The color defects include one or any combination of color difference defects, color spot defects, and uneven dyeing defects. The weaving defects include one or any combination of floating yarn defects, unclear weave defects, or indentation defects.

[0047] A fabric texture defect detection device provided to meet another object of the present application includes:

[0048] A candidate box feature extraction module configured to obtain a fabric texture image to be detected containing texture defects, perform object detection on the fabric texture image to be detected using a preset object detection model, determine defect candidate boxes in the fabric texture image to be detected, and extract defect candidate box features and background features corresponding to the defect candidate boxes, where the defect candidate boxes represent regions where texture defects may occur;

[0049] A graph convolution operation module configured to call a preset quaternion graph neural network, use the defect candidate box features corresponding to the defect candidate boxes as nodes in the quaternion graph neural network, and establish edge connections between its respective nodes through graph convolution operations to extract the mutual relationships between different defect candidate boxes;

[0050] A candidate box feature aggregation module configured to process the defect candidate box features using quaternion Hamilton transformation to promote channel interaction between different channels in the defect candidate box features to determine the aggregated defect candidate box features;

[0051] A structure inference module configured to perform structure inference on the defect candidate box features and the background features using a bidirectional long short-term memory network to capture the context dependence relationships between the respective defect candidate box features and between the defect candidate box features and the background features to determine the defect candidate box features after structure inference;

[0052] The texture defect detection module is configured to input the aggregated defect candidate box features and the defect candidate box features after structure inference into a preset fully connected neural network to determine the types of cloth texture defects corresponding to each defect candidate box in the cloth texture image to be detected, so as to complete the defect detection of the cloth texture.

[0053] An electronic device provided to meet another object of the present application includes a central processing unit and a memory. The central processing unit is configured to call and run a computer program stored in the memory to execute the steps of the cloth texture defect detection method of the present application.

[0054] A computer-readable storage medium provided to meet another object of the present application stores a computer program implemented according to the cloth texture defect detection method in the form of computer-readable instructions. When the computer program is called and run by a computer, it executes the steps included in the corresponding method.

[0055] Compared with the prior art, for the surface with complex and unevenly distributed texture in the prior art, traditional methods cannot fully capture the context information and global associations between textures, and are prone to deficiencies in detection accuracy and robustness. In addition, the feature extraction of existing convolutional neural network methods mostly focuses on local regions, lacking effective modeling of the semantic information of the image background and context associations, which limits its detection ability in high-complexity or heterogeneous texture scenarios. The present application includes but is not limited to the following beneficial effects:

[0056] First, the cloth texture defect detection method of the present application can significantly improve the accuracy and robustness of defect detection. By accurately characterizing the features of defect candidate boxes through a quaternion graph neural network, the feature expression ability between candidate boxes can be enhanced. In this network, the candidate box features can not only accurately represent their own information, but also effectively interact with surrounding candidate boxes, background information, and other contexts. This information fusion and interaction helps to improve the detection accuracy of cloth texture defects, especially in a complex cloth texture environment. Through graph convolution operations, the quaternion graph neural network can understand the relationships between defects from multiple perspectives, thereby improving the detection robustness;

[0057] Second, the cloth texture defect detection method of the present application can dynamically capture context dependencies. The present application dynamically captures the structural relationship between candidate boxes and the background through a structural inference process based on a bidirectional long short-term memory network (Bi-LSTM). This process can consider the front and back context information of candidate boxes simultaneously, and capture the complex dependencies between each defect candidate box and the background features. This helps to overcome the deficiencies of traditional methods that ignore global information or only rely on local features, enabling the model to understand texture defects in the image from an overall perspective, thereby improving the detection effect.

[0058] Thirdly, the cloth texture defect detection method of the present application can enhance the feature expression and information interaction. For the features after multi-channel interaction and aggregation, through the quaternion Hamilton transform, the features of the defect candidate boxes are optimized, which can promote the channel interaction between different channels and further improve the feature expression ability. The aggregated features can not only reflect the details of local defects, but also describe the overall characteristics of defects from multiple dimensions (such as texture, shape, background, etc.). This enhanced information interaction and expression ability greatly improves the accuracy and reliability of texture defect detection.

[0059] Fourthly, the detection framework proposed in the present application has strong generality and adaptability, can be applied in a variety of complex cloth texture scenarios, and can handle different types of texture defects. Traditional detection methods may perform poorly in specific scenarios, while the present invention can adapt to various different types of texture defects by improving information interaction and feature expression, and improves the generalization ability and application scope of the algorithm in different application scenarios.

[0060] Fifthly, by integrating technologies such as quaternion graph neural network, bidirectional long short-term memory network, and quaternion Hamilton transform, the present invention not only improves the performance of the model in specific scenarios, but also enhances its ability to handle unknown or complex defect patterns. Compared with traditional methods, this method can effectively avoid overfitting when dealing with variable texture defect types and complex backgrounds, thus having stronger generalization ability, making the cloth texture defect detection technology more intelligent, accurate and efficient, effectively reducing the burden of manual detection, and improving the production efficiency and quality control level. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0062] Figure 1 is a schematic flow chart of the cloth texture defect detection method in the embodiment of the present application;

[0063] Figure 2 is a schematic diagram of performing quaternion Hamilton transform in the embodiment of the present application;

[0064] Figure 3 is a schematic diagram of structure inference based on bidirectional long short-term memory network in the embodiment of the present application;

[0065] Figure 4 is a schematic block diagram of the cloth texture defect detection device in the embodiment of the present application;

[0066] Figure 5 is a schematic diagram of the structure of the computer device in the embodiment of the present application. Detailed Implementation Modes

[0067] The embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application and should not be construed as a limitation to the present application.

[0068] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the", and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application means the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or their groups. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more related listed items.

[0069] Those skilled in the art of the present technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the field to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0070] Those skilled in the art can understand that the "client", "terminal", and "terminal device" used herein include both devices with a wireless signal receiver that only has the ability to receive and no ability to transmit, and devices with receiving and transmitting hardware that can perform two-way communication on a two-way communication link. Such devices can include: cellular or other communication devices such as personal computers and tablet computers, which have a single-line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which can include a radio frequency receiver, a pager, Internet / intranet access, a web browser, a notepad, a calendar, and / or a GPS (Global Positioning System) receiver; conventional laptop and / or palm-held computers or other devices, which are conventional laptop and / or palm-held computers or other devices with and / or including a radio frequency receiver. The "client", "terminal", and "terminal device" used herein can be portable, transportable, installed in a vehicle (air, sea, and / or land), or suitable for and / or configured to run locally, and / or run in a distributed manner at any other location on the earth and / or in space. The "client", "terminal", and "terminal device" used herein can also be a communication terminal, an Internet access terminal, a music / video playback terminal. For example, it can be a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback function, or it can also be a smart TV, a set-top box, and other devices.

[0071] The hardware referred to by names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer, and is a hardware device with the necessary components disclosed by the von Neumann principle, including a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit loads the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input / output devices to complete specific functions.

[0072] It should be noted that the concept of "server" in this application can similarly be extended to apply to server clusters. According to the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can either be independent of each other but can be invoked through interfaces, or integrated into a single physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility and should not be restricted by this in the implementation of the network deployment method of this application.

[0073] One or several technical features of this application, unless expressly specified, can either be deployed on the server and accessed by the client remotely invoking the online service interface provided by the server, or directly deployed and run on the client for access.

[0074] The neural network models cited or possibly cited in this application, unless expressly specified, can either be deployed on a remote server and remotely invoked on the client, or deployed on a client with sufficient device capabilities for direct invocation. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0075] All kinds of data involved in this application, unless expressly specified, can either be remotely stored on the server or stored on the local terminal device, as long as it is suitable for being invoked by the technical solution of this application.

[0076] Those skilled in the art should be aware that although the various methods of this application are described based on the same concept and thus show commonality with each other, unless otherwise specified, these methods can be executed independently. Similarly, for the various embodiments disclosed in this application, they are all proposed based on the same inventive concept. Therefore, for concepts with the same expression, as well as concepts that are only appropriately transformed for convenience although the concept expressions are different, they should be equivalently understood.

[0077] For the various embodiments to be disclosed in this application, unless expressly stated to be mutually exclusive, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct new embodiments, as long as this combination does not deviate from the creative spirit of this application and can meet the requirements in the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0078] Please refer to Figure 1 , in one embodiment of the cloth texture defect detection method of this application, it includes:

[0079] Step S10: Obtain the texture image of the cloth to be detected that contains texture defects, use a preset object detection model to perform object detection on the texture image of the cloth to be detected, determine the defect candidate boxes in the texture image of the cloth to be detected, so as to extract the defect candidate box features and background features corresponding to the defect candidate boxes, where the defect candidate boxes represent areas where texture defects may occur;

[0080] The cloth texture defect detection system in the terminal device can respond to the cloth texture defect detection instruction, perform texture detection on the cloth in the cloth production equipment, obtain the texture image of the cloth to be detected that contains texture defects, use a preset object detection model to perform object detection on the texture image of the cloth to be detected, determine the defect candidate boxes in the texture image of the cloth to be detected, so as to extract the defect candidate box features and background features corresponding to the defect candidate boxes, where the defect candidate boxes represent areas where texture defects may occur;

[0081] In some embodiments, the basic network architecture of the object detection model includes the YOLOv5 object detection model, and the texture image of the cloth to be detected refers to the cloth image to be detected, which may contain different types of texture defects, such as stripe defects, breakage defects, yarn defects, color defects, and weaving defects, etc.

[0082] In a further embodiment, for a texture image I of a cloth to be detected, in this embodiment, the YOLOv5 object detection model is first used to generate defect candidate boxes r i ∈{r 1 ,r 2 ,r 3 ,...,r N}, and the defect candidate boxes and the texture image of the cloth to be detected are respectively extracted with defect candidate box features x i ∈{x 1 ,x 2 ,x 3 ,...,x N} and background features x s through the RoI Align operation and a layer of feed-forward neural network (FFN).

[0083] In a further embodiment, the types of fabric texture defects include one or any combination of stripe defects, breakage defects, yarn defects, color defects, and weaving defects. The stripe defects include one or any combination of spot defects, stripe defects, and non-uniform texture defects. The breakage defects include one or any combination of broken-end defects, floating-yarn defects, and penetration-hole defects. The color defects include one or any combination of color difference defects, color spot defects, and non-uniform dyeing defects. The weaving defects include one or any combination of floating-yarn defects, unclear weave defects, or indentation defects.

[0084] In a further embodiment, the spot defects refer to stains, color spots, etc. appearing on the fabric surface, which are usually caused by uneven dyeing or contamination. The stripe defects refer to irregular stripes appearing on the fabric, which may be caused by uneven tension of warp or weft yarns during the weaving process, uneven dyeing, etc. The non-uniform texture defects refer to inconsistent texture fluctuations on the fabric surface, which may be caused by uneven raw materials or weaving equipment problems. The broken-end defects refer to the breakage of yarns in the fabric, which may cause holes or gaps on the fabric surface. The floating-yarn defects refer to the lack of warp or weft yarns on the fabric surface, usually manifested as missing lines in certain areas of the fabric, forming blank areas. The penetration-hole defects refer to penetrating holes appearing on the fabric, which may be caused by excessive stretching of the fabric or damage to the fabric during production. The color difference defects refer to different colors appearing on the fabric, which may be caused by unevenness during the dyeing process or the use of different batches of dyes. The color spot defects refer to uneven color spots or dyeing defects appearing on the fabric, which are usually caused by improper dye treatment or dyeing equipment problems. The non-uniform dyeing defects refer to uneven dyeing of some parts of the fabric during the dyeing process, forming obvious color differences. The floating-yarn defects refer to warp or weft yarns not being fully embedded in the fabric structure during the weaving process, resulting in floating yarns on the surface, which may affect the flatness and appearance of the fabric. The unclear weave defects refer to the distortion or irregularity of the texture pattern of the fabric, which is usually caused by improper control of the weaving process. The indentation defects refer to obvious creases or indentations appearing on the fabric, which are usually caused by the fabric being pressed, stretched, or folded during the processing.

[0085] Step S20: Invoke a preset quaternion graph neural network, use the defect candidate box features corresponding to the defect candidate boxes as nodes in the quaternion graph neural network, and establish edge connections between its respective nodes through graph convolution operations to extract the mutual relationships between different defect candidate boxes;

[0086] Obtain the texture image of the cloth to be detected containing texture defects, use a preset object detection model to perform object detection on the texture image of the cloth to be detected, determine the defect candidate boxes in the texture image of the cloth to be detected, and after extracting the defect candidate box features and background features corresponding to the defect candidate boxes, call a preset quaternion graph neural network, use the defect candidate box features corresponding to the defect candidate boxes as the nodes in the quaternion graph neural network, and establish edge connections between its respective nodes through graph convolution operations to extract the mutual relationships between different defect candidate boxes;

[0087] In some embodiments, in the quaternion graph neural network, a quaternion is a mathematical representation form, which is often used to process multi-channel data. A graph neural network (GNN) is a neural network that learns on graph-structured data. Through the quaternion graph neural network, connections can be established between candidate boxes, the relationships between each defect candidate box can be captured, and the spatial and semantic relationships of these candidate boxes can be helped to be understood. This is crucial for the correlation analysis of defects.

[0088] In a specific embodiment, in order to promote the information interaction between texture defect candidate boxes and mine more connection information between candidate box feature channels in this process, the present application uses a quaternion graph neural network, and each defect candidate box feature is used as a node of the graph neural network, where l represents the layer where it is located, the number of layers of the graph neural network is 2, N represents the Nth candidate box, the k neighbor sets of each defect candidate box are N(i), and an edge is added for each j element belonging to N(i);

[0089] Construct a graph G(X, E), where X is the set of nodes and E is the set of edges, then the expression of the inference process of the quaternion graph neural network is:

[0090]

[0091] Among them, W is the weight, is the graph convolution function;

[0092] In order to ensure the efficiency of the algorithm, the present application uses the computationally simpler maximum relative convolution, and the expression of the maximum relative convolution of the graph convolution function is:

[0093]

[0094] Among them, max() is the maximum value function.

[0095] Step S30: Process the defect candidate box features by using quaternion Hamilton transformation to promote the channel interaction between different channels in the defect candidate box features to determine the aggregated defect candidate box features;

[0096] Take the defect candidate box features corresponding to the defect candidate boxes as the nodes in the quaternion graph neural network. An edge connection is established between each of these nodes through a graph convolution operation to extract the mutual relationship between different defect candidate boxes. After that, a quaternion Hamiltonian transform is used to process the defect candidate box features to promote the channel interaction between different channels in the defect candidate box features, so as to determine the aggregated defect candidate box features. Among them, the aggregated defect candidate box features represent the comprehensive information that combines local features, channel interaction, and context dependence after being processed by the quaternion Hamiltonian transform, and can accurately reflect the multi-dimensional information of the defect area in the cloth texture image, including its shape, position, type and other features. These aggregated features play a key role in subsequent classification and detection tasks.

[0097] In some embodiments, refer to Figure 2 , in order to explore the inter-channel connection of the defect candidate box features and promote more optimized feature expression, this application adopts a quaternion Hamiltonian transform. Its characteristic of promoting feature channel interaction, compared with real-valued transformation, the Hamiltonian transform can explore the inter-channel connection of candidate box features by virtue of the characteristic of promoting element sharing through the Hamiltonian product.

[0098] Specifically, express the W and in the maximum relative convolution of the graph convolution function as their corresponding quaternions respectively. The expression of the quaternion corresponding to W is:

[0099] W = W r + W i i + W j j + W k k,

[0100] Among them, W r represents the first part of the quaternion corresponding to W, W i represents the second part of the quaternion corresponding to W, W j represents the third part of the quaternion corresponding to W, W k represents the fourth part of the quaternion corresponding to W;

[0101] The expression of the corresponding quaternion is expressed as:

[0102]

[0103] Among them, represents the first part of the corresponding quaternion, represents the second part of the corresponding quaternion, represents the third component of the corresponding quaternion denote the fourth component of the corresponding quaternion;

[0104] For the defect candidate box feature perform quaternion Hamiltonian transform, and its expression is:

[0105]

[0106] Use quaternion Hamiltonian transform to process the defect candidate box feature to promote the channel interaction between different channels in the defect candidate box feature, and then determine the aggregated defect candidate box feature

[0107] wherein the each part of shares W each part of, for W making a small adjustment to may result in big changes. This feature enables the Hamiltonian transform to explore the connections between different channels of features and promote higher-quality feature expressions. After two layers of graph neural networks, the features of aggregating candidate box contexts through the quaternion graph neural network are obtained

[0108] Step S40: Use a bidirectional long short-term memory network to perform structural inference on the defect candidate box feature and the background feature to capture the context dependencies between each defect candidate box feature and between the defect candidate box feature and the background feature, so as to determine the defect candidate box feature after structural inference;

[0109] Use quaternion Hamiltonian transform to process the defect candidate box feature to promote the channel interaction between different channels in the defect candidate box feature, and then determine the aggregated defect candidate box feature. After that, use a bidirectional long short-term memory network to perform structural inference on the defect candidate box feature and the background feature to capture the context dependencies between each defect candidate box feature and between the defect candidate box feature and the background feature, so as to determine the defect candidate box feature after structural inference; wherein, the defect candidate box feature after structural inference is the defect candidate box feature processed by the bidirectional long short-term memory network, which can characterize the context dependencies between defect candidate boxes and between the defect candidate box and the background. It provides richer and more accurate semantic information, which helps the model to finally perform accurate defect classification and detection. These features are no longer just simple features such as local shape, color, and texture, but integrate the context information of the entire image to help the model understand the essence and type of fabric texture defects.

[0110] To promote the interaction between the texture defect candidate box features and the background features, the present application designs a structural inference based on a recurrent neural network. The recurrent neural network (RNN) enables the candidate box features to add background-related information and ignore background-unrelated information while retaining their original information. Among them, the recurrent neural network includes a bidirectional long short-term memory network (Bi-LSTM), etc.

[0111] To better promote the interaction between the background features and the candidate box features and to mine the context dependencies between the candidate boxes, the present invention selects an RNN with a more complex structure, that is, a bidirectional long short-term memory network (Bi-LSTM), as the medium for the interaction between the candidate box features and between the candidate boxes and the background information in the present application.

[0112] In a specific embodiment, a bidirectional long short-term memory network is used to perform structural inference on the defect candidate box features and the background features to capture the context dependencies between the individual defect candidate box features and between the defect candidate box features and the background features, and the steps to determine the defect candidate box features after structural inference include:

[0113] Please refer to Figure 3 , in the task of texture defect detection, the feature representation of each candidate box should not be affected by the order of the candidate boxes. To reduce the influence brought by the order of the candidate boxes, the present application uses a bidirectional long short-term memory network to perform structural inference on the defect candidate box features and the background features to capture the context dependencies between the individual defect candidate box features and between the defect candidate box features and the background features. Taking each defect candidate box feature as the input at time step t of the bidirectional long short-term memory network, and the background feature x s as the input of the initial hidden state of the bidirectional long short-term memory network. For each defect candidate box feature x t , its expression for structural inference includes:

[0114]

[0115] Among them, h t is the hidden state at time step t, and the initial hidden state h 0 =x s x s represents the background feature, C t represents the cell state at time step t, represents the candidate cell state at time step t, f t represents the activation value of the forget gate, i t represents the activation value of the input gate, o trepresents the activation value of the output gate, σ(.) represents the sigmoid activation function, W represents the weight matrix, b represents the bias, and [.] represents the concatenation operation of vectors;

[0116] For the bidirectional long short-term memory network, the output at time step t is expressed as:

[0117]

[0118] where, represents the output of the forward long short-term memory network, represents the output of the backward long short-term memory network, and the initial hidden states of the forward and backward long short-term memory networks and are both the background feature x s , represents the initial hidden state of the forward long short-term memory network, represents the initial hidden state of the backward long short-term memory network.

[0119] In a further embodiment, a three-layer feedforward neural network is used to gradually reduce the dimensionality of the o t so as to keep the o t and the aggregated defect candidate box features in the same dimension without losing the original feature information, and its expression is:

[0120]

[0121] where, represents the defect candidate box features after structural inference, and f(.) represents the RuLU activation function.

[0122] Step S50: Input the aggregated defect candidate box features and the defect candidate box features after structural inference into a preset fully connected neural network to determine the types of fabric texture defects corresponding to each defect candidate box in the fabric texture image to be detected, so as to complete the defect detection of the fabric texture.

[0123] Use a bidirectional long short-term memory network to perform structural inference on the defect candidate box features and the background features to capture the context dependence relationships between the defect candidate box features and between the defect candidate box features and the background features. After determining the defect candidate box features after structural inference, input the aggregated defect candidate box features and the defect candidate box features after structural inference into a preset fully connected neural network to determine the types of fabric texture defects corresponding to each defect candidate box in the fabric texture image to be detected, so as to complete the defect detection of the fabric texture.

[0124] In a specific embodiment, in order to integrate the defect candidate box features obtained from the graph neural network and the structural inference, the steps of inputting the aggregated defect candidate box features and the defect candidate box features after the structural inference into a preset fully connected neural network to determine the types of fabric texture defects corresponding to each defect candidate box in the fabric texture image to be detected include:

[0125] Aggregate the defect candidate box features And the defect candidate box features after the structural inference Are fused to determine the fused defect candidate box feature x i , and its expression includes:

[0126]

[0127] Where x i Represents the fused defect candidate box feature;

[0128] Input the fused defect candidate box feature x i Into a preset fully connected neural network to determine the types of fabric texture defects corresponding to each defect candidate box in the fabric texture image to be detected.

[0129] As can be seen from the above embodiments, compared with the prior art, for the surface with complex and unevenly distributed textures in the prior art, the traditional method cannot fully capture the context information and the global correlation between textures, and is prone to insufficient performance in terms of detection accuracy and robustness. In addition, the feature extraction of the existing convolutional neural network methods mostly focuses on local regions, lacking effective modeling of the image background semantic information and context correlation, which limits its detection ability in high-complexity or non-homogeneous texture scenarios. The present application includes, but is not limited to, the following beneficial effects:

[0130] First, the fabric texture defect detection method of the present application can significantly improve the accuracy and robustness of defect detection. By accurately characterizing the features of defect candidate boxes through the quaternion graph neural network, the feature expression ability between candidate boxes can be enhanced. In this network, the candidate box features can not only accurately represent their own information, but also effectively interact with surrounding candidate boxes, background information, and other contexts. This fusion and interaction of information helps to improve the detection accuracy of fabric texture defects, especially in complex fabric texture environments. Through graph convolution operations, the quaternion graph neural network can understand the relationships between defects from multiple perspectives, thereby improving the detection robustness;

[0131] Second, the fabric texture defect detection method of the present application can dynamically capture context dependencies. The present application dynamically captures the structural relationship between the candidate box and the background through a structural inference process based on a bidirectional long short-term memory network (Bi-LSTM). This process can consider the front and back context information of the candidate box simultaneously, and capture the complex dependencies between each defect candidate box and the background features. This helps to overcome the deficiencies of traditional methods that ignore global information or rely only on local features, enabling the model to understand texture defects in the image from an overall perspective, thereby improving the detection effect.

[0132] Third, the fabric texture defect detection method of the present application can enhance feature expression and information interaction. After multi-channel interaction and aggregation, the features are optimized through quaternion Hamilton transformation, which can promote channel interaction between different channels and further improve the feature expression ability. The aggregated features can not only reflect the details of local defects, but also describe the overall characteristics of defects from multiple dimensions (such as texture, shape, background, etc.). This enhanced information interaction and expression ability greatly improves the accuracy and reliability of texture defect detection.

[0133] Fourth, the detection framework proposed in the present application has strong generality and adaptability, can be applied in a variety of complex fabric texture scenarios, and can handle different types of texture defects. Traditional detection methods may perform poorly in specific scenarios, while the present invention can adapt to various different types of texture defects by improving information interaction and feature expression, improving the generalization ability and application scope of the algorithm in different application scenarios.

[0134] Fifth, by integrating technologies such as quaternion graph neural network, bidirectional long short-term memory network, and quaternion Hamilton transformation, the present invention not only improves the performance of the model in specific scenarios, but also enhances its ability to handle unknown or complex defect patterns. Compared with traditional methods, this method can effectively avoid overfitting when dealing with variable texture defect types and complex backgrounds, thus having stronger generalization ability, making the fabric texture defect detection technology more intelligent, accurate and efficient, effectively reducing the burden of manual detection, and improving production efficiency and quality control level.

[0135] Please refer to Figure 4, A cloth texture defect detection device provided to meet one of the purposes of the present application, including a candidate box feature extraction module 1100, a graph convolution operation module 1200, a candidate box feature aggregation module 1300, a structure inference module 1400, and a texture defect detection module 1500. Among them, the candidate box feature extraction module 1100 is configured to obtain a cloth texture image to be detected containing texture defects, perform object detection on the cloth texture image to be detected using a preset object detection model, determine defect candidate boxes in the cloth texture image to be detected, and extract defect candidate box features and background features corresponding to the defect candidate boxes, where the defect candidate boxes represent areas where texture defects may occur; the graph convolution operation module 1200 is configured to call a preset quaternion graph neural network, use the defect candidate box features corresponding to the defect candidate boxes as nodes in the quaternion graph neural network, and establish edge connections between its respective nodes through graph convolution operations to extract the mutual relationships between different defect candidate boxes; the candidate box feature aggregation module 1300 is configured to process the defect candidate box features using quaternion Hamilton transformation to promote channel interaction between different channels in the defect candidate box features to determine the aggregated defect candidate box features; the structure inference module 1400 is configured to perform structure inference on the defect candidate box features and the background features using a bidirectional long short-term memory network to capture the context dependence relationships between the respective defect candidate box features and between the defect candidate box features and the background features to determine the defect candidate box features after structure inference; the texture defect detection module 1500 is configured to input the aggregated defect candidate box features and the defect candidate box features after structure inference into a preset fully connected neural network to determine the cloth texture defect types corresponding to each defect candidate box in the cloth texture image to be detected, so as to complete the defect detection of the cloth texture.

[0136] Based on any embodiment of the present application, please refer to Figure 5 , Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Figure 5As shown, it is a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected through a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database can store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a method for detecting fabric texture defects. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device can store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the fabric texture defect detection method of this application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand, Figure 5 The structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0137] In this embodiment, the processor is used to execute Figure 4 the specific functions of each module and its sub-modules in the figure. The memory stores the program code and various types of data required to execute the above modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. The memory in this embodiment stores the program code and data required to execute all modules / sub-modules in the fabric texture defect detection device of this application. The server can call the program code and data of the server to execute the functions of all sub-modules.

[0138] This application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the steps of the fabric texture defect detection method described in any embodiment of this application.

[0139] This application also provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by one or more processors, the steps of the fabric texture defect detection method described in any embodiment of this application are implemented.

[0140] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments of the method of this application can be completed by instructing relevant hardware through a computer program. This computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned various methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0141] The above are only some embodiments of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. A method for detecting cloth texture defects, characterized in that: include: Acquire a texture image of a cloth to be detected containing texture defects, perform target detection on the cloth texture image to be detected using a preset target detection model, determine a defect candidate frame in the cloth texture image to be detected, and extract defect candidate frame features and background features corresponding to the defect candidate frame, wherein the defect candidate frame represents an area where texture defects may occur; Calling a preset quaternion graph neural network, taking the defect candidate box features corresponding to the defect candidate box as nodes in the quaternion graph neural network, and establishing edge connections between the nodes through graph convolution operations to extract the mutual relationships between different defect candidate boxes; The defect candidate frame features are processed by using quaternion Hamiltonian transformation to promote channel interaction between different channels in the defect candidate frame features, so as to determine aggregated defect candidate frame features; Using a bidirectional long short-term memory network to perform structural reasoning on the defect candidate box features and the background features, so as to capture the contextual dependency between the defect candidate box features and between the defect candidate box features and the background features, so as to determine the defect candidate box features after structural reasoning; The aggregated defect candidate frame features and the structurally inferred defect candidate frame features are input into a preset fully connected neural network to determine the cloth texture defect type corresponding to each defect candidate frame in the cloth texture image to be detected, so as to complete the cloth texture defect detection.

2. The cloth texture defect detection method according to claim 1, characterized in that: Each defect candidate box feature As a node of the graph neural network, l represents the number of layers, the number of layers of the graph neural network is 2, N represents the Nth candidate box, the set of k neighbors of each defect candidate box is N(i), and an edge is added for each j element belonging to N(i); Construct a graph G(X,E), where X is a set of nodes and E is a set of edges. The inference process of the quaternion graph neural network is expressed as: Among them, W is the weight, is the graph convolution function; The expression of the maximum relative convolution of the graph convolution function is expressed as: Among them, max() is the maximum value function.

3. The cloth texture defect detection method according to claim 2, characterized in that: The maximum relative convolution of the graph convolution function W and They are respectively expressed as their corresponding quaternions, and the expression of the quaternion corresponding to W is: W=W r +W i i+W j j+W k k, Among them, W r Represents the first part of the quaternion corresponding to W, W i The second part of the quaternion corresponding to W, W j Represents the third part of the quaternion corresponding to W, W k Represents the fourth division of the quaternion corresponding to W; The corresponding quaternion expression is expressed as: in, express The corresponding first division of the quaternion, express The corresponding second division of the quaternion, express The corresponding third division of the quaternion is, express The fourth division of the corresponding quaternion; For the defect candidate frame feature Perform quaternion Hamiltonian transformation, the expression is expressed as: The defect candidate frame features are processed by using quaternion Hamiltonian transformation to promote channel interaction between different channels in the defect candidate frame features to determine the aggregated defect candidate frame features.

4. The cloth texture defect detection method according to claim 3, characterized in that: The step of using a bidirectional long short-term memory network to perform structural reasoning on the defect candidate box features and the background features to capture contextual dependencies between the defect candidate box features and between the defect candidate box features and the background features to determine the defect candidate box features after structural reasoning includes: Each defect candidate box feature As the input of the bidirectional LSTM network at time step t, the background feature x s As the initial hidden state input of the bidirectional long short-term memory network, for each defect candidate box feature x t For example, the expressions of structural reasoning include: Among them, h t is the hidden state at time step t, and the initial hidden state h0 = x s , x s represents the background features, C t represents the cell state at time step t, represents the candidate cell state at time step t, f t represents the activation value of the forget gate, i t represents the activation value of the input gate, o t represents the activation value of the output gate, σ(.) represents the sigmoid activation function, W represents the weight matrix, b represents the bias, and [.] represents the concatenation operation of the vector; For the bidirectional long short-term memory network, the output at time step t is expressed as: in, represents the output of the forward long short-term memory network, represents the output of the backward LSTM network, and the initial hidden state of the forward and backward LSTM network as well as All are background features x s , represents the initial hidden state of the forward LSTM network, Represents the initial hidden state of the backward LSTM network.

5. The cloth texture defect detection method according to claim 4, characterized in that: Regarding the o t A three-layer feedforward neural network is used to gradually reduce the feature dimension to convert the o t and the defect candidate frame features after aggregation While maintaining the same dimension without losing the original feature information, its expression is expressed as: in, represents the defect candidate box feature after structural reasoning, and f(.) represents the RuLU activation function.

6. The cloth texture defect detection method according to any one of claims 1 to 5, characterized in that: The step of inputting the aggregated defect candidate frame features and the structurally inferred defect candidate frame features into a preset fully connected neural network to determine the cloth texture defect type corresponding to each defect candidate frame in the cloth texture image to be detected includes: The aggregated defect candidate frame features And the defect candidate frame features after the structural reasoning Fusion is performed to determine the fused defect candidate frame feature x i , whose expressions include: Among them, x i Represents the fused defect candidate frame features; The fused defect candidate frame feature x i The data is input into a preset fully connected neural network to determine the cloth texture defect type corresponding to each defect candidate frame in the cloth texture image to be detected.

7. The cloth texture defect detection method according to any one of claims 1 to 5, characterized in that: The basic network architecture of the target detection model includes the YOLOv5 target detection model, the cloth texture defect types include one or any multiple of stripe defects, damage defects, yarn defects, color defects and weaving defects, the stripe defects include one or any multiple of spot defects, stripe defects and uneven texture defects, the damage defects include one or any multiple of broken end defects, yarn skipping defects and penetration hole defects, the color defects include one or any multiple of color difference defects, color spot defects and uneven dyeing defects, and the weaving defects include one or any multiple of floating yarn defects, unclear weaving defects or indentation defects.

8. A cloth texture defect detection device, characterized in that: include: a candidate frame feature extraction module, configured to obtain a texture image of a cloth to be detected containing texture defects, perform target detection on the cloth texture image to be detected using a preset target detection model, determine a defect candidate frame in the cloth texture image to be detected, and extract defect candidate frame features and background features corresponding to the defect candidate frame, wherein the defect candidate frame represents an area where texture defects may occur; A graph convolution operation module is configured to call a preset quaternion graph neural network, and use defect candidate box features corresponding to the defect candidate boxes as nodes in the quaternion graph neural network, and establish edge connections between the nodes through graph convolution operations to extract the relationships between different defect candidate boxes; a candidate frame feature aggregation module, configured to process the defect candidate frame features by using a quaternion Hamiltonian transformation to promote channel interaction between different channels in the defect candidate frame features, so as to determine aggregated defect candidate frame features; A structural reasoning module is configured to use a bidirectional long short-term memory network to perform structural reasoning on the defect candidate box features and the background features, so as to capture contextual dependencies between the defect candidate box features and between the defect candidate box features and the background features, so as to determine the defect candidate box features after structural reasoning; The texture defect detection module is configured to input the aggregated defect candidate frame features and the structurally inferred defect candidate frame features into a preset fully connected neural network to determine the cloth texture defect type corresponding to each defect candidate frame in the cloth texture image to be detected, so as to complete the cloth texture defect detection.

9. An electronic device, comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images