Fault detection method, device, apparatus and storage medium

Abstract

The disclosure provides a fault detection method, device, equipment and storage medium. It relates to the technical field of computers. The method comprises: image acquisition of a false twist component of a texturing machine to obtain a target image; construction of first fusion features based on at least one attention module in an attention network; for each attention module, the attention module comprises a first submodule and a second submodule, the first submodule constructs first sub-features for fine-grained features within a first field of view and coarse-grained features within a second field of view, and the second submodule obtains a mask image based on residual errors constructed based on second input information; the first fusion features are input into an encoder network to obtain encoding features; the encoding features are decoded based on a decoder network to obtain decoded features; second fusion features are obtained based on the decoded features; and the second fusion features are input into a multilayer perceptron to obtain a fault detection result of the false twist component. The disclosure can automatically detect the false twist component.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of artificial intelligence, computer vision, and image processing. Background Technology

[0002] In industrial applications of texturing processes, texturing machines play a crucial role in the spinning process. The yarns involved in texturing workshops include POY (Pre-Oriented Yarn), also known as POY raw yarn, and DTY (Draw Textured Yarn).

[0003] Texturing machines process POY raw yarn to obtain DTY yarn. The texturing machine includes a false twisting component, which performs the false twisting operation on the POY raw yarn. Under frequent use, the false twisting component of the texturing machine may wear down, thus affecting the texturing process. Therefore, how to automatically monitor the false twisting component of the texturing machine is a problem faced by existing technologies. Summary of the Invention

[0004] This disclosure provides a fault detection method, apparatus, device, and storage medium for monitoring false twist components during the texturing process.

[0005] In a first aspect, this disclosure provides a fault detection method, characterized in that it includes:

[0006] Image acquisition is performed on the false twist component of the texturing machine to obtain the target image;

[0007] A first fusion feature is constructed based on at least one attention module in an attention network; wherein, for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature based on fine-grained features within a first field of view and coarse-grained features within a second field of view in the first input information, and the second sub-module uses a residual constructed based on the second input information to obtain a mask image; the mask image is multiplied by the first sub-feature to obtain the output feature of the attention module; the first field of view is smaller than the second field of view;

[0008] The first fused feature is input into the encoder network to obtain the encoded feature; wherein, the encoder network includes multiple encoders, each encoder outputs a corresponding encoded sub-feature, and the encoded feature includes the encoded sub-feature of at least one encoder;

[0009] The encoded features are decoded using a decoder network to obtain decoded features; wherein the decoder network includes multiple decoders, each decoder outputs a corresponding decoded sub-feature, and the decoded features include the decoded sub-features of multiple decoders;

[0010] The decoded sub-features in the decoded features are fused to obtain the second fused feature;

[0011] The second fusion feature is input into a multilayer perceptron to obtain the fault detection result of the false twist component of the texturing machine.

[0012] Secondly, this disclosure provides a fault detection device, characterized in that it includes:

[0013] The acquisition unit is used to acquire images of the false twist component of the texturing machine to obtain the target image;

[0014] A construction unit is used to construct a first fusion feature based on at least one attention module in an attention network; wherein, for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature based on fine-grained features within a first field of view and coarse-grained features within a second field of view in the first input information, and the second sub-module uses a residual constructed based on the second input information to obtain a mask image; the mask image is multiplied by the first sub-feature to obtain the output feature of the attention module; the first field of view is smaller than the second field of view;

[0015] An encoding unit is used to input the first fused feature into an encoder network to obtain an encoded feature; wherein the encoder network includes multiple encoders, each encoder outputs a corresponding encoded sub-feature, and the encoded feature includes the encoded sub-feature of at least one encoder;

[0016] A decoding unit is used to perform a decoding operation on the encoded features based on a decoder network to obtain decoded features; wherein, the decoder network includes multiple decoders, each decoder outputs a corresponding decoded sub-feature, and the decoded features include the decoded sub-features of multiple decoders;

[0017] A fusion unit is used to fuse the decoded sub-features in the decoded features to obtain a second fused feature;

[0018] The prediction unit is used to input the second fused feature into the multilayer perceptron to obtain the fault detection result of the false twist component of the texturing machine.

[0019] Thirdly, an electronic device is provided, comprising:

[0020] At least one processor; and

[0021] The memory is communicatively connected to the at least one processor; wherein,

[0022] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0023] Fourthly, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of the present disclosure.

[0024] Fifthly, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of the present disclosure.

[0025] Based on the method proposed in the embodiments of this disclosure, an automatic detection process for the false twist component of the texturing machine is realized.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0027] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments provided according to this disclosure and should not be construed as limiting the scope of this disclosure.

[0028] Figure 1 This is a schematic diagram of a texturing machine according to an embodiment of the present disclosure;

[0029] Figure 2 This is a schematic diagram of a false-twist component according to an embodiment of the present disclosure;

[0030] Figure 3 This is a flowchart illustrating a fault detection method according to an embodiment of the present disclosure;

[0031] Figure 4 This is a schematic diagram of the architecture of the entire neural network model according to an embodiment of the present disclosure;

[0032] Figure 5 This is a schematic diagram of the first sub-module of the attention module according to an embodiment of the present disclosure;

[0033] Figure 6 This is a schematic diagram of a second sub-module of an attention network according to an embodiment of the present disclosure;

[0034] Figure 7 This is a schematic diagram of an encoder network and a decoder network according to an embodiment of the present disclosure;

[0035] Figure 8 This is a schematic diagram of an encoder according to an embodiment of the present disclosure;

[0036] Figure 9 This is a schematic diagram of the structure of a fault detection device according to an embodiment of the present disclosure;

[0037] Figure 10 A block diagram of an electronic device used to implement the fault detection method of the embodiments of this disclosure. Detailed Implementation

[0038] The present disclosure will now be described in further detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0039] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0040] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "multiple" means two or more, unless otherwise explicitly specified.

[0041] In industrial settings involving texturing processes, the false twist components of the texturing machine may wear out, necessitating monitoring of these components.

[0042] One possible schematic diagram of the key structure of the texturing machine is shown below. Figure 1 As shown, the key structures of the texturing machine include: a raw yarn frame 101, a yarn cutter 102, a first roller 103, a first heating box 104, a cooling plate 105, a false twisting component 106, a nozzle 107, a second roller 108, a second heating box 109, a third roller 110, a yarn breakage detection device 111, and a winding component 112.

[0043] The first roller 103, the second roller 108, and the third roller 110 are used to ensure that the silk threads are processed along a predetermined path. In the texturing process, the speeds of the first roller 103, the second roller 108, and the third roller 110 are matched to ensure that the silk fabric is not broken or piled up.

[0044] Depending on the product form requirements, the nozzle 107 can be used to process the yarn that has been treated by the false twisting component into a network form of yarn product, so as to ensure that different yarn products have the required feel and form.

[0045] When producing high-elasticity yarn, the second heating box 109 can be omitted. When producing medium-elasticity yarn, the temperature of the second heating box 109 can be adjusted to the first preset temperature, for example, around 140°C. When producing low-elasticity yarn, the temperature of the second heating box 109 can be adjusted to the second preset temperature, for example, between 165°C and 195°C.

[0046] If the breakage detection device 111 detects a breakage, it will trigger the filament cutter 102 to cut the filament, thus preventing the POY filament from accumulating in subsequent processes.

[0047] When processing single strands of yarn, the corresponding false twist components, such as Figure 1 As shown in 106, two rubber rollers are mainly used to rub the silk fabric. In the case of processing multi-strand yarns, the false-twist component 106 processes the multi-strand yarns into a twisted yarn. A possible example is... Figure 2 As shown, the first thread passes through from the upper left, and the second thread passes through from the upper right. The two threads are then merged to form a single thread.

[0048] To automatically and accurately detect the false twist components of a texturing machine, this disclosure proposes a fault detection method, such as... Figure 3 As shown, it includes the following:

[0049] S301: Image acquisition is performed on the false twist component of the texturing machine to obtain the target image.

[0050] Drones can be used to periodically inspect the false twist components of the texturing machine, or images of the false twist components can be acquired based on monitoring. Any method that can acquire images of the false twist components can be applied to the embodiments of this disclosure.

[0051] S302, a first fusion feature is constructed based on at least one attention module in the attention network; wherein, for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature for fine-grained features in a first field of view and coarse-grained features in a second field of view in the first input information, the second sub-module uses the residual constructed based on the second input information to obtain a mask image; the mask image is multiplied by the first sub-feature to obtain the output feature of the attention module; the first field of view is smaller than the second field of view.

[0052] S303, the first fused feature is input into the encoder network to obtain the encoded feature; wherein, the encoder network includes multiple encoders, each encoder outputs a corresponding encoded sub-feature, and the encoded feature includes the encoded sub-feature of at least one encoder.

[0053] S304, Decoding features are obtained by decoding the encoded features based on the decoder network; wherein, the decoder network includes multiple decoders, each decoder outputs a corresponding decoded sub-feature, and the decoded features include the decoded sub-features of multiple decoders.

[0054] S305, fuse the various decoded sub-features in the decoded features to obtain the second fused feature.

[0055] S306, input the second fusion feature into the multilayer perceptron to obtain the fault detection result of the false twist component of the texturing machine.

[0056] The attention network, encoder network, decoder network, and multilayer perceptron can use real images of historical false-twist components as sample images to train the parameters of these models (including the attention network, encoder network, decoder network, and multilayer perceptron). Specifically, the fault location and fault type in the sample images are labeled, with the fault type being wear level. The sample images are sequentially input into the attention network, encoder network, decoder network, and multilayer perceptron to obtain the predicted fault location and predicted wear level. A first loss value is obtained based on the loss between the predicted fault location and the actual fault location, and a second loss value is obtained based on the loss between the predicted wear level and the actual wear level. The first and second loss values ​​are weighted and summed to obtain the target loss. The parameters of the attention network, encoder network, decoder network, and multilayer perceptron to be trained are adjusted based on the target loss. When the convergence condition is met, the attention network, encoder network, decoder network, and multilayer perceptron are obtained and used to detect faults in the false-twist components of the texturing machine in the target image.

[0057] In this embodiment, the attention network includes at least one attention module. Fine-grained features represent the local details of interest, while coarse-grained features represent the surrounding features of those local details. By using both fine-grained and coarse-grained features, information conforming to visual characteristics can be captured. Features within different granularity ranges can be fused to improve the expressive power of the extracted features, thereby aiding in the accurate detection of false-twist components. Furthermore, the residual constructed from the second input information of the false-twist component is also fused into the same attention module, further incorporating soft attention features into the output features of the attention module. This allows the same attention model to extract features of the false-twist component from multiple perspectives, improving feature expressive power and thus enhancing the accuracy of the detection results. Moreover, the more layers of the attention module in the attention network, the stronger the expressive power of the output features of the extracted target image. This output feature is then input into the subsequent encoder and decoder. Each encoder in the encoder network has a corresponding decoder in the decoder network. The decoded sub-features output by multiple decoders are fused to obtain a second fused feature. This second fused feature is then input into a multilayer perceptron to obtain the fault detection result of the false-twist component of the texturing machine, realizing an automatic detection process for the false-twist component of the texturing machine.

[0058] The architecture of the entire neural network model proposed in this embodiment is as follows: Figure 4 As shown, this model includes an attention network, an encoder network, a decoder network, and a multilayer perceptron. The specific structure of each network is described in detail below:

[0059] 1) Attention Network

[0060] In some embodiments, attention networks such as Figure 4 As shown, it can include N attention modules (including Figure 4 The first attention module, the second attention module, ..., the Nth attention module (where N is a positive integer) are defined. The specific input and output information for each attention module can be described as follows:

[0061] 1) For the first attention module in the attention network:

[0062] The first input information of the first submodule of the first attention module is the target image.

[0063] The second input information of the second submodule of the first attention module includes the target image and at least one historical image of the false twist component, and the acquisition time interval between the target image and any historical image is less than a preset duration.

[0064] The historical acquired images can be m images sampled at equal intervals within a preset time period before the target image is acquired. For example, if the target image is sampled at time t, and the preset time period is a, and one image is acquired at every interval b, then the first historical acquired image is acquired at time ta, the second historical acquired image is acquired at time t-a+b, and so on. Of course, historical acquired images can also be obtained by sampling at unequal intervals within the preset time period. Any method that can obtain historical acquired images is applicable to the embodiments of this disclosure.

[0065] 2) For any attention module other than the first attention module in the attention network:

[0066] The first input information of the first sub-module of any attention module is the output feature of the previous attention module of any attention module;

[0067] The second input information of the second submodule of any attention module is the residual output of the second submodule of the previous attention module of any attention module.

[0068] like Figure 4 As shown, the input of the first sub-module of the first attention module is the target image, and the output is the first sub-feature. The input of the second sub-module of the first attention module is the historically acquired image, and the output is the first residual. Multiplying the first sub-feature and the first residual yields the output of the first attention module, which is the first output feature. The first output feature is then used as the input of the first sub-module of the second attention module, and the output is the second sub-feature. The first residual is then used as the input of the second attention module, and the output is the second residual. This process is repeated sequentially until all N attention modules have been executed, resulting in the output of the attention network, which is the first fused feature.

[0069] In this embodiment, the target image can be used to detect the instantaneous state of the false-twist component, thereby achieving fault detection. The historical images of the target image can be used to determine the changes in the false-twist component within a preset time period. The instantaneous state and the historical state of the target image are comprehensively considered to extract the feature representation of the false-twist component, so as to facilitate the detection of the wear degree of the false-twist component.

[0070] In some embodiments, constructing the first sub-feature based on the fine-grained features within the first field of view and the coarse-grained features within the second field of view in the first input information can be implemented as follows:

[0071] Step A1: Determine multiple target points from the first input information, and perform the following operations for each target point:

[0072] Step A11: Extract fine-grained features of the target point within the first field of view centered on the target point.

[0073] In some embodiments, extracting fine-grained features of the target point within the first field of view centered on the target point can be implemented as follows:

[0074] Step B1: Determine the feature value of the target point in the first input information, and use it as the first query feature of the target point.

[0075] In this process, the false twist component of the texturing machine is cropped from the target image, and the pixel containing the false twist component is taken as the target point. Figure 5 The black rectangle in the middle represents the target point.

[0076] Step B2: Determine the first field of view of the target point with the position coordinates of the target point in the first input information as the center.

[0077] The first field of view can be 3×3 or 5×5, and can be determined based on the actual situation. This disclosure does not limit this aspect. The first field of view of this disclosure embodiment is as follows: Figure 5 The gray rectangle surrounding the target point.

[0078] Step B3: Based on the feature values ​​of feature points outside the target point within the first field of view, construct the first key feature and the first value feature of the target point.

[0079] Step B4: Based on the first query feature, the first key feature, and the first value feature, determine the fine-grained features of the target point.

[0080] In one possible implementation, the set of pixels in a sliding window centered at (i,j) on the feature map constructed from the first input information is defined as ρ(i,j). For a fixed window size kхk, ||ρ(i,j)||=k 2 Then the first query feature and the first key feature can satisfy formula (1).

[0081] As shown, this is to facilitate obtaining fine-grained features S (i,j)～ρ(i,j) :

[0082]

[0083] In another possible implementation, positional offsets and mask filling methods can be introduced to obtain another fine-grained feature. For example... Figure 5As shown, in the path for obtaining fine-grained features (hereinafter also referred to as the first path), the positional bias is the relative positional relationship between fine-grained tokens within the first field of view, which can also be understood as the relative positional relationship between each pixel within the first field of view.

[0084] In the first path, pixels at the feature map edges inevitably have similarity calculated with zero padding outside the boundary. To prevent these zero similarities from affecting the softmax operation, a padding mask is used to set these results to -∞.

[0085] Therefore, as Figure 5 As shown, on the first path, the first query feature and the first key feature are cross-multiplied as shown in formula (1) to obtain the first feature to be fused. The first feature to be fused and the position offset after mask filling on the first path are added pixel by pixel to obtain the fine-grained feature.

[0086] In this embodiment, the fine-grained features extracted by combining the features of the target point with those of the pixels in the nearby first field of view have strong expressive power and can lay the foundation for subsequent determination of fault detection results.

[0087] Step A12: Extract coarse-grained features of the target point within the second field of view centered on the target point.

[0088] In some embodiments, the second field of view centered on the target point

[0089] Extracting coarse-grained features of the target point within the range can be implemented as follows:

[0090] Step C1: Determine the second field of view of the target point with the position coordinates of the target point in the first input information as the center.

[0091] like Figure 5 As shown, the second field of view is the range of pixels centered on the target point, encompassing the first field of view as well. Figure 5 The white rectangle shown in the image is illustrated.

[0092] Step C2: Based on the feature values ​​of feature points outside the target point within the second field of view, construct the second key feature and the second value feature of the target point.

[0093] Step C3: Based on the first query feature, the second key feature, and the second value feature of the target point, determine the coarse-grained feature of the target point; wherein, the feature value of the target point in the first input information is the first query feature of the target point.

[0094] In one possible implementation, the set of pixels within a second field of view centered at (i,j) on the feature map constructed from the first input information is defined as σ′(i,j). The set of pixels obtained by pooling within the second field of view is defined as σ(X). For the pooling size H... p ×W p , ||σ(X)||=H p W p In order to obtain coarse-grained features:

[0095]

[0096] In another possible implementation, position offset and mask filling methods can be introduced to obtain another coarse-grained feature. In the coarse-grained feature acquisition path (hereinafter referred to as the second path), the position offset is the relative positional relationship between coarse-grained tokens.

[0097] To further enhance the multi-scale image input expansion capability of pixel focusing attention, different methods are used to calculate the positional bias, which can be B within the second field of view. (i，j) ~σ(X).

[0098] like Figure 5 As shown, the second path aims to perceive global features near the target point. Pooling operations can be performed within the second field of view first. The pooling window and stride can be determined based on the actual situation. The pooling operation generates a new feature map by moving the pooling window across the target image and selecting the pixels with the maximum or average value.

[0099] On the second path, a log-interval continuous position bias (log-CPB) is used, where ReLU (activation function) is applied from the position coordinates (Q) of the first query feature. (i,j) ) and the set of pixels for second key features Spatial relative coordinates Δ (i,j) ~σ(X), and then calculate B (i,j) ~σ(X).

[0100] Therefore, as Figure 5 As shown, the first query feature and the second key feature are cross-multiplied as shown in formula (2) to obtain the second feature to be fused. The second feature to be fused and the position offset are added pixel by pixel to obtain the coarse-grained feature.

[0101] In this embodiment, the coarse-grained features extracted from the target point in combination with the features of the surrounding pixels in the second field of view are more expressive, which can lay the foundation for determining the fault detection results.

[0102] Step A13: The fine-grained features and the coarse-grained features are concatenated to obtain the initial features of the target point.

[0103] The fine-grained features and coarse-grained features are concatenated in an additive manner to obtain the initial features of the target point.

[0104] Step A14: The initial features of the target point are mapped to feature values ​​of the target point using a nonlinear mapping method.

[0105] Step A2: The feature values ​​of the multiple target points are spliced ​​together according to the position information of each target point in the target image to obtain the first sub-feature.

[0106] like Figure 5 As shown, coarse-grained and fine-grained features are input into the connection layer and activation layer to obtain connection features. A separation layer is used to separate these connection features, thus splitting the coarse-grained and fine-grained features, ultimately yielding intermediate coarse-grained and intermediate fine-grained features. A cross product operation is performed between the first-value feature and the intermediate fine-grained feature to obtain the target fine-grained feature. A cross product operation is then performed between the second-value feature and the intermediate coarse-grained feature to obtain the target coarse-grained feature. The target fine-grained feature and the target coarse-grained feature are summed to obtain the initial feature, which is then passed through a nonlinear mapping layer to obtain the feature value of the target point. This operation is performed sequentially for each target point, thereby obtaining the first sub-feature.

[0107] In this embodiment of the disclosure, coarse-grained features are used to capture the overall structure of the image, while fine-grained features are more specific and accurate. By capturing both coarse-grained and fine-grained features of the target point, the fault condition of the false twist component of the texturing machine can be better identified.

[0108] In some embodiments, the residual constructed based on the second input information to obtain the mask image can be implemented as follows:

[0109] Step D1: Perform a Fourier transform on the second input information using a Fourier filter to obtain the time-varying component.

[0110] Specifically, for the second sub-module of the first attention module in the attention network, its second input information is the previously acquired historical images. For the second sub-modules of the second to Nth attention modules in the attention network, their second input information is the residual output of the previous attention module. The framework diagram of the second sub-module of each attention network is as follows: Figure 6 As shown.

[0111] Step D2: Input the time-varying component into the estimation module built on the neural network to obtain the estimated value of the time-varying component.

[0112] Step D3: Determine the residual between the time-varying component and the estimated value of the time-varying component to obtain the mask image.

[0113] Among them, the estimation module based on the neural network can be the time-varying Koopman predictor (Koopa). The Koopa model consists of multiple stackable Koopa basic modules.

[0114] Each Koopa base module focuses on learning specific levels of dynamic characteristics. By stacking Koopa base modules, the model can capture multi-level and complex dynamic changes in time series. Each Koopa base module learns operators hierarchically by using the residuals of the dynamic fit of the previous block as input. The approach proposed in this disclosure improves the prediction accuracy of the time-varying Koopman predictor and enhances the model's adaptability to complex, nonlinear, and non-stationary time series, thus facilitating the extraction of dynamic change characteristics of the false twist component.

[0115] 2) Decoder Network

[0116] In some embodiments, the encoders in the encoder network and the decoders in the decoder network correspond one-to-one; such as Figure 7 As shown, the encoder network illustrates the case with four encoders, and the decoder network illustrates the case with four decoders. Regardless of the number of encoders and decoders, the decoding operation based on the decoder network to obtain the decoded features can be implemented as follows:

[0117] Step E1, for each target decoder in the decoder network, perform the following operations:

[0118] Step E11: Obtain the encoded sub-features output by the encoder corresponding to the target decoder, and use them as the third query feature of the target decoder.

[0119] by Figure 7 Taking the four encoders shown in the encoder network as an example, they process a target image of size H×W×3, where H is the height of the target image and W is the width of the target image. The four encoders i∈{1,2,…,4} respectively generate hierarchical and multi-resolution sub-features E. i ,in C i This represents the weight corresponding to the i-th encoder. In other words, it can be understood as the weight processed by the first encoder. The image is processed by the second encoder; The image; the third encoder processes is The image; the fourth encoder processes the image. The image.

[0120] exist Figure 7In the encoder network, the encoders are executed in the following order: first encoder, second encoder, third encoder, fourth encoder. After the encoder network completes execution, the decoder network is executed in the following order: fourth decoder, third decoder, second decoder, first decoder.

[0121] Among them, such as Figure 7 As shown, the third query feature of the first decoder is the first encoded sub-feature output by the first encoder; the third query feature of the second decoder is the second encoded sub-feature output by the second encoder; the third query feature of the third decoder is the third encoded sub-feature output by the third encoder; and the third query feature of the fourth decoder is the fourth encoded sub-feature output by the fourth encoder.

[0122] Step E12: If the target decoder is the first decoder, obtain the coding sub-features of all encoders in the encoder network, and construct the third value feature and third key feature of the target decoder.

[0123] Step E13: If the target decoder is any decoder other than the first decoder, obtain the decoding sub-features output by each preceding decoder before the target decoder as preferred sub-features; and replace the encoding sub-features of the encoder corresponding to the preceding decoder with the preferred sub-features to obtain the third value feature and the third key feature of the target decoder.

[0124] like Figure 7 As shown, based on the encoder network processing, the first coded sub-feature, second coded sub-feature, third coded sub-feature, and fourth coded sub-feature are obtained. Based on these features, the third value feature and third key feature of the fourth decoder input are determined. After obtaining the fourth decoding sub-feature, the fourth coded sub-feature in the original third value feature and third key feature is replaced and used as the input to the third decoder to obtain the third decoding sub-feature. This process is then repeated sequentially to obtain the first decoding sub-feature, second decoding sub-feature, third decoding sub-feature, and fourth decoding sub-feature output by the decoder network.

[0125] Step E14: Input the third query feature, the third value feature, and the third key feature into the target decoder to obtain the decoded sub-features output by the target decoder.

[0126] In some embodiments, the architecture of each decoder is as follows: Figure 8As shown, the system includes a Mix-Attention mechanism module, a Layer Normalization (LN) module, and a Feedforward Network (FFN). Specifically, the method for obtaining the decoded sub-features output by the target decoder can be implemented as follows:

[0127] Step F1: Input the third query feature, the third key feature, and the third value feature into the hybrid attention mechanism module to obtain the first intermediate feature.

[0128] Step F2: Fuse the first intermediate feature and the third query feature.

[0129] Then, the input layer normalization module obtains the second intermediate feature.

[0130] Step F3: Input the second intermediate feature into the feedforward network to obtain the third intermediate feature.

[0131] Step F4: Fuse the second intermediate feature and the third intermediate feature.

[0132] The decoded sub-features output by the target decoder are obtained.

[0133] Step E2: Construct the decoding features based on the decoding sub-features of each decoder.

[0134] Based on the four decoded sub-features, they are fused to obtain the decoded features. The fusion method can be weighted summation, concatenation, or other similar methods.

[0135] In this embodiment, compared to self-attention, the query features, key features, and value features used to generate are the same and come from the same source, i.e., the same encoder / decoder stage. However, this embodiment employs a hybrid attention mechanism module that uses hybrid features across multiple scale stages, with each feature originating from an independent encoder. This allows query features to find matches at all different stages, enabling matching at the contextual granularity level, thereby enhancing the accuracy of fault detection in the target image.

[0136] Based on the same technical concept, this disclosure proposes a fault detection device 900, such as... Figure 9 As shown, it includes:

[0137] The acquisition unit 901 is used to acquire images of the false twist component of the texturing machine to obtain the target image;

[0138] Construction unit 902 is used to construct a first fusion feature based on at least one attention module in the attention network; wherein, for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature based on fine-grained features within a first field of view and coarse-grained features within a second field of view in the first input information, and the second sub-module uses a residual constructed based on the second input information to obtain a mask image; the mask image is multiplied by the first sub-feature to obtain the output feature of the attention module; the first field of view is smaller than the second field of view;

[0139] The encoding unit 903 is used to input the first fused feature into the encoder network to obtain the encoded feature; wherein the encoder network includes multiple encoders, each encoder outputs a corresponding encoded sub-feature, and the encoded feature includes the encoded sub-feature of at least one encoder;

[0140] Decoding unit 904 is used to perform decoding operation on the encoded features based on the decoder network to obtain decoded features; wherein, the decoder network includes multiple decoders, each decoder outputs a corresponding decoded sub-feature, and the decoded features include the decoded sub-features of multiple decoders;

[0141] The fusion unit 905 is used to fuse the decoded sub-features in the decoded features to obtain a second fused feature;

[0142] The prediction unit 906 is used to input the second fused feature into the multilayer perceptron to obtain the fault detection result of the false twist component of the texturing machine.

[0143] In some embodiments, for the first attention module in the attention network:

[0144] The first input information of the first submodule of the first attention module is the target image;

[0145] The second input information of the second submodule of the first attention module includes the target image and at least one historical image of the false twist component, and the acquisition time interval between the target image and any historical image is less than a preset duration.

[0146] For any attention module other than the first attention module in the attention network:

[0147] The first input information of the first sub-module of any attention module is the output feature of the previous attention module of any attention module;

[0148] The second input information of the second submodule of any attention module is the residual output of the second submodule of the previous attention module of any attention module.

[0149] In some embodiments, the building unit includes:

[0150] A determination subunit is configured to determine multiple target points from the first input information and perform the following operations for each target point:

[0151] Extract fine-grained features of the target point within a first field of view centered on the target point; and

[0152] Extract coarse-grained features of the target point within a second field of view centered on the target point;

[0153] The fine-grained features and the coarse-grained features are concatenated to obtain the initial features of the target point;

[0154] The initial features of the target point are mapped to feature values ​​of the target point using a nonlinear mapping method;

[0155] The splicing subunit is used to splice the feature values ​​of the multiple target points according to the position information of each target point in the target image to obtain the first sub-feature.

[0156] In some embodiments, the determining subunit is specifically used for:

[0157] Determine the feature value of the target point in the first input information, and use it as the first query feature of the target point;

[0158] The first field of view of the target point is determined with the position coordinates of the target point in the first input information as the center.

[0159] Based on the feature values ​​of feature points outside the target point within the first field of view, construct the first key feature and the first value feature of the target point;

[0160] Based on the first query feature, the first key feature, and the first value feature, the fine-grained features of the target point are determined.

[0161] In some embodiments, the determining subunit is specifically used for:

[0162] The second field of view of the target point is determined with the position coordinates of the target point in the first input information as the center.

[0163] Based on the feature values ​​of feature points outside the target point within the second field of view, a second key feature and a second value feature of the target point are constructed.

[0164] Based on the first query feature, the second key feature, and the second value feature of the target point, the coarse-grained feature of the target point is determined; wherein, the feature value of the target point in the first input information is the first query feature of the target point.

[0165] In some embodiments, the building unit is specifically used for:

[0166] The transformation subunit is used to perform a Fourier transform on the second input information using a Fourier filter to obtain the time-varying component;

[0167] An estimation subunit is used to input the time-varying component into an estimation module based on a neural network to obtain an estimated value of the time-varying component;

[0168] The residual determination subunit is used to determine the residual between the time-varying component and the estimated value of the time-varying component to obtain the mask image.

[0169] In some embodiments, the encoders in the encoder network and the decoders in the decoder network correspond one-to-one; the decoding unit includes:

[0170] A decoding sub-feature determination subunit is used for each target decoder in the decoder network, and each target decoder further includes a processing subunit, comprising:

[0171] Obtain the encoded sub-features output by the encoder corresponding to the target decoder, and use them as the third query feature of the target decoder; and,

[0172] When the target decoder is the first decoder, the coding sub-features of all encoders in the encoder network are obtained, and the third value feature and third key feature of the target decoder are constructed.

[0173] When the target decoder is any decoder other than the first decoder, the decoding sub-features output by each preceding decoder before the target decoder are obtained as preferred sub-features; and the preferred sub-features are used to replace the encoding sub-features of the encoder corresponding to the preceding decoder in the feature set constructed for each encoder, to obtain the third value feature and the third key feature of the target decoder.

[0174] The third query feature, the third value feature, and the third key feature are input into the target decoder to obtain the decoded sub-features output by the target decoder;

[0175] A decoding feature construction subunit is used to construct the decoding features based on the decoding sub-features of each decoder.

[0176] In some embodiments, the decoding sub-feature determination subunit is specifically used for:

[0177] The third query feature, the third key feature, and the third value feature are input into the hybrid attention mechanism module to obtain the first intermediate feature;

[0178] After fusing the first intermediate feature and the third query feature, the input layer normalization module is used to obtain the second intermediate feature;

[0179] The second intermediate feature is input into the feedforward network to obtain the third intermediate feature;

[0180] The second intermediate feature and the third intermediate feature are fused to obtain the decoding sub-features output by the target decoder.

[0181] The specific functions and examples of each module, submodule / unit of the apparatus in this disclosure embodiment can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0182] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0183] Figure 10 This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Figure 10 As shown, the electronic device includes a memory 1010 and a processor 1020. The memory 1010 stores a computer program that can run on the processor 1020. The number of memories 1010 and processors 1020 can be one or more. The memory 1010 can store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the methods provided in the above-described method embodiments. The electronic device may also include a communication interface 1030 for communicating with external devices and performing data exchange and transmission.

[0184] If the memory 1010, processor 1020, and communication interface 1030 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0185] Optionally, in a specific implementation, if the memory 1010, processor 1020 and communication interface 1030 are integrated on a single chip, then the memory 1010, processor 1020 and communication interface 1030 can communicate with each other through an internal interface.

[0186] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0187] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAMBUS RAM (DR RAM).

[0188] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, Bluetooth, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)). It is worth noting that the computer-readable storage media mentioned in this disclosure can be non-volatile storage media; in other words, it can be non-transient storage media.

[0189] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0190] In the description of the embodiments of this disclosure, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0191] In the description of the embodiments disclosed herein, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0192] In the description of embodiments of this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0193] The above description is merely an exemplary embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A fault detection method, characterized in that, include: Image acquisition is performed on the false twist component of the texturing machine to obtain the target image; A first fusion feature is constructed based on at least one attention module in an attention network; wherein, for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature based on fine-grained features within a first field of view and coarse-grained features within a second field of view in the first input information, and the second sub-module uses a residual constructed based on the second input information to obtain a mask image; the mask image is multiplied by the first sub-feature to obtain the output feature of the attention module; the first field of view is smaller than the second field of view; For the first attention module in the attention network: The first input information of the first submodule of the first attention module is the target image; The second input information of the second submodule of the first attention module includes the target image and at least one historical image of the false twist component, and the acquisition time interval between the target image and any historical image is less than a preset duration. For any attention module other than the first attention module in the attention network: The first input information of the first sub-module of any attention module is the output feature of the previous attention module of any attention module; The second input information of the second sub-module of any attention module is the residual output of the second sub-module of the previous attention module of any attention module; The output feature of the last attention module in the attention network is the first fusion feature; The first fused feature is input into the encoder network to obtain the encoded feature; wherein, the encoder network includes multiple encoders, each encoder outputs a corresponding encoded sub-feature, and the encoded feature includes the encoded sub-feature of at least one encoder; The encoded features are decoded using a decoder network to obtain decoded features; wherein the decoder network includes multiple decoders, each decoder outputs a corresponding decoded sub-feature, and the decoded features include the decoded sub-features of multiple decoders; The decoded sub-features in the decoded features are fused to obtain the second fused feature; The second fusion feature is input into a multilayer perceptron to obtain the fault detection result of the false twist component of the texturing machine.

2. The method according to claim 1, characterized in that, The construction of the first sub-feature, based on the fine-grained features within the first field of view and the coarse-grained features within the second field of view in the first input information, includes: Multiple target points are identified from the first input information, and the following operations are performed for each target point: Extract fine-grained features of the target point within a first field of view centered on the target point; and Extract coarse-grained features of the target point within a second field of view centered on the target point; The fine-grained features and the coarse-grained features are concatenated to obtain the initial features of the target point; The initial features of the target point are mapped to feature values ​​of the target point using a nonlinear mapping method; The feature values ​​of the multiple target points are concatenated according to the position information of each target point in the target image to obtain the first sub-feature.

3. The method according to claim 2, characterized in that, Extracting fine-grained features of the target point within the first field of view centered on the target point includes: Determine the feature value of the target point in the first input information, and use it as the first query feature of the target point; The first field of view of the target point is determined with the position coordinates of the target point in the first input information as the center. Based on the feature values ​​of feature points outside the target point within the first field of view, construct the first key feature and the first value feature of the target point; Based on the first query feature, the first key feature, and the first value feature, the fine-grained features of the target point are determined.

4. The method according to claim 2, characterized in that, Extracting coarse-grained features of the target point within the second field of view centered on the target point includes: The second field of view of the target point is determined with the position coordinates of the target point in the first input information as the center. Based on the feature values ​​of feature points outside the target point within the second field of view, a second key feature and a second value feature of the target point are constructed. Based on the first query feature, the second key feature, and the second value feature of the target point, the coarse-grained feature of the target point is determined; wherein, the feature value of the target point in the first input information is the first query feature of the target point.

5. The method according to claim 1, characterized in that, The residual constructed based on the second input information yields a mask image, including: The second input information is subjected to Fourier transform using a Fourier filter to obtain the time-varying component; The time-varying component is input into an estimation module based on a neural network to obtain an estimated value of the time-varying component; The mask image is obtained by determining the residual between the time-varying component and the estimated value of the time-varying component.

6. The method according to any one of claims 1-5, characterized in that, The encoders in the encoder network and the decoders in the decoder network correspond one-to-one; the decoding operation based on the decoder network to obtain the decoded features includes: For each target decoder in the decoder network, perform the following operations: Obtain the encoded sub-features output by the encoder corresponding to the target decoder, and use them as the third query feature of the target decoder; and, When the target decoder is the first decoder, the coding sub-features of all encoders in the encoder network are obtained, and the third value feature and third key feature of the target decoder are constructed. When the target decoder is any decoder other than the first decoder, the decoding sub-features output by each preceding decoder before the target decoder are obtained as preferred sub-features; and the preferred sub-features are used to replace the encoding sub-features of the encoder corresponding to the preceding decoder in the feature set constructed for each encoder, to obtain the third value feature and the third key feature of the target decoder. The third query feature, the third value feature, and the third key feature are input into the target decoder to obtain the decoded sub-features output by the target decoder; The decoding features are constructed based on the decoding sub-features of each decoder.

7. The method according to claim 6, characterized in that, The step of inputting the third query feature, the third value feature, and the third key feature into the target decoder to obtain the decoded sub-features output by the target decoder includes: The third query feature, the third key feature, and the third value feature are input into the hybrid attention mechanism module to obtain the first intermediate feature; After fusing the first intermediate feature and the third query feature, the input layer normalization module is used to obtain the second intermediate feature; The second intermediate feature is input into the feedforward network to obtain the third intermediate feature; The second intermediate feature and the third intermediate feature are fused to obtain the decoding sub-features output by the target decoder.

8. A fault detection device, characterized in that, include: The acquisition unit is used to acquire images of the false twist component of the texturing machine to obtain the target image; A construction unit is used to construct a first fusion feature based on at least one attention module in an attention network; wherein, for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature based on fine-grained features within a first field of view and coarse-grained features within a second field of view in the first input information, and the second sub-module uses a residual constructed based on the second input information to obtain a mask image; the mask image is multiplied by the first sub-feature to obtain the output feature of the attention module; the first field of view is smaller than the second field of view; For the first attention module in the attention network: The first input information of the first submodule of the first attention module is the target image; The second input information of the second submodule of the first attention module includes the target image and at least one historical image of the false twist component, and the acquisition time interval between the target image and any historical image is less than a preset duration. For any attention module other than the first attention module in the attention network: The first input information of the first sub-module of any attention module is the output feature of the previous attention module of any attention module; The second input information of the second sub-module of any attention module is the residual output of the second sub-module of the previous attention module of any attention module; The output feature of the last attention module in the attention network is the first fusion feature; An encoding unit is used to input the first fused feature into an encoder network to obtain an encoded feature; wherein the encoder network includes multiple encoders, each encoder outputs a corresponding encoded sub-feature, and the encoded feature includes the encoded sub-feature of at least one encoder; A decoding unit is used to perform a decoding operation on the encoded features based on a decoder network to obtain decoded features; wherein, the decoder network includes multiple decoders, each decoder outputs a corresponding decoded sub-feature, and the decoded features include the decoded sub-features of multiple decoders; A fusion unit is used to fuse the decoded sub-features in the decoded features to obtain a second fused feature; The prediction unit is used to input the second fused feature into the multilayer perceptron to obtain the fault detection result of the false twist component of the texturing machine.

9. The apparatus according to claim 8, characterized in that, The building unit includes: A determination subunit is configured to determine multiple target points from the first input information and perform the following operations for each target point: Extract fine-grained features of the target point within a first field of view centered on the target point; and Extract coarse-grained features of the target point within a second field of view centered on the target point; The fine-grained features and the coarse-grained features are concatenated to obtain the initial features of the target point; The initial features of the target point are mapped to feature values ​​of the target point using a nonlinear mapping method; The splicing subunit is used to splice the feature values ​​of the multiple target points according to the position information of each target point in the target image to obtain the first sub-feature.

10. The apparatus according to claim 9, characterized in that, The determining subunit is specifically used for: Determine the feature value of the target point in the first input information, and use it as the first query feature of the target point; The first field of view of the target point is determined with the position coordinates of the target point in the first input information as the center. Based on the feature values ​​of feature points outside the target point within the first field of view, construct the first key feature and the first value feature of the target point; Based on the first query feature, the first key feature, and the first value feature, the fine-grained features of the target point are determined.

11. The apparatus according to claim 9, characterized in that, The determining subunit is specifically used for: The second field of view of the target point is determined with the position coordinates of the target point in the first input information as the center. Based on the feature values ​​of feature points outside the target point within the second field of view, a second key feature and a second value feature of the target point are constructed. Based on the first query feature, the second key feature, and the second value feature of the target point, the coarse-grained feature of the target point is determined; wherein, the feature value of the target point in the first input information is the first query feature of the target point.

12. The apparatus according to claim 8, characterized in that, The building unit includes: The transformation subunit is used to perform a Fourier transform on the second input information using a Fourier filter to obtain the time-varying component; An estimation subunit is used to input the time-varying component into an estimation module based on a neural network to obtain an estimated value of the time-varying component; The residual determination subunit is used to determine the residual between the time-varying component and the estimated value of the time-varying component to obtain the mask image.

13. The apparatus according to any one of claims 8-12, characterized in that, The encoders in the encoder network and the decoders in the decoder network correspond one-to-one; the decoding unit includes: The decoding sub-feature determination subunit performs the following operations for each target decoder in the decoder network: Obtain the encoded sub-features output by the encoder corresponding to the target decoder, and use them as the third query feature of the target decoder; and, When the target decoder is the first decoder, the coding sub-features of all encoders in the encoder network are obtained, and the third value feature and third key feature of the target decoder are constructed. When the target decoder is any decoder other than the first decoder, the decoding sub-features output by each preceding decoder before the target decoder are obtained as preferred sub-features; and the preferred sub-features are used to replace the encoding sub-features of the encoder corresponding to the preceding decoder in the feature set constructed for each encoder, to obtain the third value feature and the third key feature of the target decoder. The third query feature, the third value feature, and the third key feature are input into the target decoder to obtain the decoded sub-features output by the target decoder; A decoding feature construction subunit is used to construct the decoding features based on the decoding sub-features of each decoder.

14. The apparatus according to claim 13, characterized in that, The decoding sub-feature determination sub-unit is specifically used for: The third query feature, the third key feature, and the third value feature are input into the hybrid attention mechanism module to obtain the first intermediate feature; After fusing the first intermediate feature and the third query feature, the input layer normalization module is used to obtain the second intermediate feature; The second intermediate feature is input into the feedforward network to obtain the third intermediate feature; The second intermediate feature and the third intermediate feature are fused to obtain the decoding sub-features output by the target decoder.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images