Fault detection method, apparatus, device and storage medium

Abstract

To provide a fault detection method, an apparatus, a device and a storage medium for effectively monitoring fault condition of a member.SOLUTION: A method includes: capturing an image of a false twist member of a false twisting machine to obtain a target image; and constructing a first fusion feature based on at least one attention module in an attention network. Each attention module includes a first sub-module and a second sub-module. The first sub-module constructs a first sub feature with respect to fine-granularity features within first visual field ranges and coarse-granularity features within second visual field ranges. The second sub-module obtains a mask map by using a residual constructed based on second input information. The method also includes inputting the first fusion feature into an encoder network to obtain an encoded feature; and decoding the encoded feature based on a decoder network to obtain a decoded feature.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to the field of computers, and more particularly to the fields of artificial intelligence, computer vision, image processing, and the like. [Background technology]

[0002] In the industrial scene of false twisting process, false twisting machine plays an important role in the spinning process flow. In the related art, the yarns related to the false twisting workshop include POY (Pre Oriented Yarn), abbreviated as POY raw yarn, and DTY (Draw Textured Yarn).

[0003] DTY yarn is obtained by processing POY raw yarn in a false-twisting machine. The false-twisting machine includes a false-twisting member. The false-twisting member performs a false-twisting operation on the POY raw yarn. When the false-twisting machine is used frequently, the false-twisting member of the false-twisting machine may wear out, which may affect the false-twisting process flow. Therefore, realizing automatic monitoring of the false-twisting member of the false-twisting machine has been a conventional challenge. Summary of the Invention

[0004] The present disclosure provides a fault detection method, apparatus, device and storage medium for implementing monitoring of a false twist member in a false twist process flow.

[0005] In a first aspect, the present disclosure provides a method for manufacturing a semiconductor device comprising: acquiring a target image by performing image acquisition on a false twist member of a false twisting machine; Constructing a first fusion feature based on at least one attention module in an attention network, where for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature for fine-grained features within a first visual field range and coarse-grained features within a second visual field range in the first input information, the second sub-module obtains a mask map using a residual constructed based on the second input information, and an output feature of the attention module is obtained by multiplying the mask map and the first sub-feature, where the first visual field range is smaller than the second visual field range; inputting the first fusion feature into an encoder network to obtain an encoded feature, where the encoder network includes a plurality of encoders, each encoder outputs a corresponding encoded sub-feature, and the encoded feature includes the encoded sub-feature of at least one encoder; performing a decoding operation on the encoding feature based on a decoder network to obtain a decoding feature, where the decoder network includes a plurality of decoders, each decoder outputs a corresponding decoding sub-feature, and the decoding feature includes the decoding sub-features of the plurality of decoders; Fusing each decoding sub-feature in the decoding feature to obtain a second fused feature; inputting the second fused feature into a multi-layer perceptron to obtain a fault detection result for the false twisting member of the false twisting machine.

[0006] In a second aspect, the present disclosure provides a method for manufacturing a semiconductor device comprising: an acquisition unit for acquiring a target image by performing image acquisition on a false twist member of the false twisting machine; a construction unit for constructing a first fusion feature based on at least one attention module in an attention network, wherein for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature for fine-grained features within a first visual field range and coarse-grained features within a second visual field range in the first input information, the second sub-module obtains a mask map using a residual constructed based on the second input information, and an output feature of the attention module is obtained by multiplying the mask map and the first sub-feature, wherein the first visual field range is smaller than the second visual field range; an encoding unit for inputting the first fusion feature into an encoder network to obtain an encoding feature, the encoder network including a plurality of encoders, each encoder outputting a corresponding encoding sub-feature, the encoding feature including the encoding sub-feature of at least one encoder; a decoding unit for performing a decoding operation on the encoding feature based on a decoder network to obtain a decoding feature, wherein the decoder network includes a plurality of decoders, each decoder outputs a corresponding decoding sub-feature, and the decoding feature includes the decoding sub-features of a plurality of decoders; a fusion unit for fusing each decoding sub-feature in the decoding feature to obtain a second fused feature; a prediction unit for inputting the second fused feature into a multi-layer perceptron to obtain a fault detection result for the false twist member of the false twisting machine.

[0007] In the third phase, at least one processor; a memory communicatively connected to the at least one processor, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform any of the methods of the embodiments of the present disclosure, thereby providing an electronic device.

[0008] In a fourth aspect, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions for causing the computer to perform any of the methods of the embodiments of the present disclosure.

[0009] In a fifth aspect, there is provided a computer program product including a computer program that, when executed by a processor, implements any of the methods of the embodiments of the present disclosure.

[0010] Based on the method according to the embodiment of the present disclosure, an automatic detection flow for the false twist member of the false twisting machine has been realized.

[0011] It should be understood that the contents described in the Summary of the Invention section do not limit the key points or important features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will be readily understood from the following description.

[0012] In the drawings, unless otherwise specified, the same reference numerals in multiple views indicate the same or similar parts or elements. The drawings are not necessarily drawn to scale. It should be understood that the drawings merely illustrate some embodiments provided by the present disclosure and are not to be considered as limiting the scope of the present disclosure. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a schematic diagram of a false twisting machine in one embodiment according to the present disclosure. [Figure 2] FIG. 2 is a schematic diagram of a false twist member in one embodiment according to the present disclosure. [Figure 3] FIG. 3 is a flow diagram of a fault detection method according to one embodiment of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram of the overall architecture of a neural network model in one embodiment according to the present disclosure. [Figure 5] FIG. 5 is a schematic diagram of a first sub-module of the attention module in one embodiment according to the present disclosure. [Figure 6] FIG. 6 is a schematic diagram of a second sub-module of the attention network in one embodiment according to the present disclosure. [Figure 7] FIG. 7 is a schematic diagram of an encoder network and a decoder network in one embodiment according to the present disclosure. [Figure 8] FIG. 8 is a schematic diagram of an encoder in one embodiment according to the present disclosure. [Figure 9] FIG. 9 is a schematic diagram illustrating the configuration of a fault detection device according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a block diagram of an electronic device for implementing the fault detection method according to the embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014] The present disclosure will now be described in more detail with reference to the drawings, in which like reference numerals indicate functionally identical or similar elements, and in which various aspects of the embodiments are shown, but which are not necessarily drawn to scale unless otherwise noted.

[0015] Furthermore, in order to better explain the present disclosure, numerous specific details are described in the following specific embodiments. Those skilled in the art should understand that the present disclosure can be similarly implemented without specific details. In some instances, methods, means, elements, circuits, etc. that are well known to those skilled in the art are not described in detail so as to emphasize the gist of the present disclosure.

[0016] It should be noted that the terms "first" and "second" are for purposes of distinction and should not be understood to indicate or imply relative importance or the number of the indicated components. Therefore, a feature described as "first" or "second" is meant to include, explicitly or implicitly, one or more of the feature. In the description of this disclosure, unless otherwise specified, "plurality" means two or more.

[0017] In an industrial false twisting process, the false twisting member of the false twisting machine may be subject to wear, and therefore the false twisting member of the false twisting machine needs to be monitored.

[0018] As a type of false twisting machine, the core structure thereof is shown in FIG. 1, which includes a yarn rack 101, a yarn cutter 102, a first roller 103, a first heating box 104, a cooling plate 105, a false twisting member 106, a nozzle 107, a second roller 108, a second heating box 109, a third roller 110, a yarn breakage detector 111, and a winding member 112.

[0019] Among them, the first roller 103, the second roller 108, and the third roller 110 are for ensuring that the yarn is processed along a predetermined path. In the false twist process flow, the first roller 103, the second roller 108, and the third roller 110 are synchronized in speed to prevent the yarn product from being torn or piled up.

[0020] Depending on the needs of the product form, the nozzle 107 may be used to process the processed yarn through a false twisting member into a network-type yarn product so that different yarn products have the desired feel and form.

[0021] When producing a high elasticity yarn, the second heating box 109 may not be used. When producing a medium elasticity yarn, the temperature of the second heating box 109 may be adjusted to a first preset temperature, for example, the first preset temperature may be about 140°C. When producing a low elasticity yarn, the temperature of the second heating box 109 may be adjusted to a second preset temperature, for example, the second preset temperature may be between 165 and 195°C.

[0022] When the yarn breakage detection device 111 detects a yarn breakage, it triggers the yarn cutter 102 to cut the yarn so that the POY yarn does not accumulate in the subsequent flow.

[0023] When processing a single yarn, the corresponding false twisting element mainly employs two rubber rolls to twist the yarn product, as shown in 106 in Figure 1. When processing multiple yarns, the false twisting element 106 processes the multiple yarns into a doubled / twisted yarn. In one example, as shown in Figure 2, a first yarn passes through from the upper left side and a second yarn passes through from the upper right side, and the two yarns are then combined to form a single yarn.

[0024] In order to automatically and accurately detect the false twisting member of the false twisting machine, a fault detection method is put forward in an embodiment of the present disclosure, which, as shown in FIG. 3, includes the following steps: S301: Image acquisition is performed on the false twist member of the false twisting machine to obtain a target image.

[0025] An unmanned aerial vehicle may be used to conduct periodic inspections of the twisting member of the twisting machine, or images of the twisting member may be acquired using a monitor, but any method capable of acquiring images of the twisting member can be applied to the embodiments of the present disclosure.

[0026] S302: Construct a first fusion feature based on at least one attention module in the attention network, where for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature for fine-grained features within a first field of view range and coarse-grained features within a second field of view range in the first input information, the second sub-module obtains a mask map using a residual constructed based on the second input information, and the output feature of the attention module is obtained by multiplying the mask map and the first sub-feature, and the first field of view range is smaller than the second field of view range.

[0027] S303: Input the first fusion feature into an encoder network to obtain an encoding feature, where the encoder network includes multiple encoders, each encoder outputs a corresponding encoding sub-feature, and the encoding feature includes the encoding sub-feature of at least one encoder.

[0028] S304: Perform a decoding operation on the encoding feature based on a decoder network to obtain a decoded feature, where the decoder network includes multiple decoders, each decoder outputs a corresponding decoded sub-feature, and the decoded feature includes the decoded sub-features of the multiple decoders.

[0029] S305: Fuse each decoding sub-feature in the decoding feature to obtain a second fused feature.

[0030] S306: The second fused feature is input into a multi-layer perceptron to obtain a fault detection result of the false twisting member of the false twisting machine.

[0031] For the attention network, encoder network, decoder network, and multilayer perceptron, actual images of the false twisted member from the past can be used as sample images to train the parameters of these models (including the attention network, encoder network, decoder network, and multilayer perceptron). Specifically, the sample images can be labeled with a fault location and a fault type, and the fault type can be wear level. The sample images are input sequentially to the attention network awaiting training, the encoder network awaiting training, the decoder network awaiting training, and the multilayer perceptron awaiting training to obtain a predicted fault location and a predicted wear level. A first loss value is obtained based on the loss between the predicted fault location and the actual fault location, and a second loss value is obtained based on the loss between the predicted wear level and the actual wear level. The first loss value and the second loss value are weighted together to obtain a target loss. The parameters of the attention network awaiting training, the encoder network awaiting training, the decoder network awaiting training, and the multilayer perceptron awaiting training are adjusted based on the target loss. When the convergence condition is satisfied, an attention network, an encoder network, a decoder network, and a multilayer perceptron are obtained to perform fault detection on the false twisting member of the false twisting machine in the target image.

[0032] In an embodiment of the present disclosure, the attention network includes at least one attention module, where fine-grained features indicate local details of interest and coarse-grained features indicate features surrounding the local details. The fine-grained and coarse-grained features can obtain information consistent with visual features, and features within different granularity ranges can be fused to enhance the expressive power of the extracted features, thereby contributing to accurate detection of false twisted members. Furthermore, the same attention module also fuses residuals constructed based on second input information of the false twisted members, such that soft attention features are further fused into the features output by the attention module. This allows the same attention model to extract false twisted member features from multiple perspectives, enhancing the expressive power of the features and ultimately improving the accuracy of the detection results. Furthermore, the more layers of attention modules in the attention network, the stronger the expressive power of the extracted output features of the target image. Furthermore, the output features are input to the subsequent encoder and decoder, and for any encoder in the encoder network, there is one decoder corresponding to that encoder in the decoder network. In other words, the decoded sub-features output by the multiple decoders are fused to obtain a second fused feature, and the second fused feature is input to a multi-layer perceptron to obtain the fault detection result for the twisting member of the false twisting machine, thereby realizing an automatic detection flow for the twisting member of the false twisting machine.

[0033] The overall architecture of the neural network model proposed in the embodiments of the present disclosure is as shown in Figure 4. The model includes an attention network, an encoder network, a decoder network, and a multi-layer perceptron. The specific structure of each network is described in detail below. (1) Attention Network In some embodiments, the attention network may include N attention modules (including the first attention module, the second attention module, ..., the Nth attention module in FIG. 4, where N is a positive integer), as shown in FIG. 4. Specific input information and output information of each attention module may be described as follows: [1] Regarding the first attention module in the attention network, The first input information of the first sub-module of the first attention module is the target image. The second input information of the second sub-module of the first attention module includes the target image and at least one previously acquired image of the twisted member, and the acquisition time interval between the target image and any one of the previously acquired images is shorter than a predetermined time length.

[0034] The previously acquired images may be m images acquired by sampling at equal intervals within a predetermined time period before acquiring the target image. For example, if the time point at which the target image is sampled is time t, the predetermined time period is a, and the time interval for acquiring each image is b, then the first previously acquired image is acquired at time ta, the second previously acquired image is acquired at time t-a+b, and so on. Of course, previously acquired images may also be acquired by sampling at non-equidistant intervals within the predetermined time period. Any method capable of acquiring previously acquired images may be applied to the embodiments of the present disclosure.

[0035] [2] For any attention module other than the first attention module in the attention network, The first input information of the first sub-module of any of the attention modules is an output feature output by an attention module immediately before the any of the attention modules; The second input information of the second sub-module of any one of the attention modules is the residual output by the second sub-module of the attention module immediately preceding any one of the attention modules.

[0036] As shown in Figure 4, the first submodule of the first attention module receives the target image as input and outputs the first first subfeature. The second submodule of the first attention module receives the previously acquired image as input and outputs the first residual. The first first subfeature is multiplied by the first residual to obtain the output of the first attention module, i.e., the first output feature. The first output feature is then input to the first submodule of the second attention module, and the output is the second first subfeature. The first residual is then input to the second submodule of the second attention module, and the output is the second residual. Similarly, the N attention modules are executed in sequence as described above until execution is complete, and the output of the attention network is obtained as the first fused feature.

[0037] In the embodiment of the present disclosure, the target image can be used to detect the real-time state of the false twist member, thereby realizing fault detection, and the previously acquired images of the target image can be used to determine the change in the false twist member status within a predetermined time period, and the real-time state of the target image and the past state of the target image are taken together to comprehensively extract the characteristic representation of the false twist member, so as to detect the wear level of the false twist member.

[0038] In some embodiments, constructing first sub-features for fine-grained features within a first field of view range and coarse-grained features within a second field of view range in the first input information can be performed as follows, i.e., can be performed to include the following steps: Step A1: Determine a plurality of target points in the first input information, and perform the following operations for each target point. Step A11: Extract fine-grained features of the target point within the first field of view range centered on the target point.

[0039] In some embodiments, extracting fine-grained features of the target point within the first field of view range centered on the target point can be performed as follows, i.e., can be performed to include the following steps: Step B1: Determine the feature value in the first input information of the target point as a first query feature of the target point.

[0040] The false twisting member of the false twisting machine is cut out from the target image to obtain pixel points that include the false twisting member, that is, target points. The black rectangular frames in FIG. 5 are target points.

[0041] Step B2: The first field of view range of the target point is determined with the position coordinates of the target point in the first input information as the center.

[0042] The first field of view may be 3x3 or 5x5, and can be determined according to the actual situation, and the embodiment of the present disclosure does not limit it. The first field of view in the embodiment of the present disclosure is as shown in the gray rectangular frame around the target point in Figure 5.

[0043] Step B3: Construct a first key feature and a first value feature of the target point based on the feature values of feature points other than the target point within the first field of view.

[0044] Step B4: Determine fine-grained features of the interest points based on the first query feature, the first key feature and the first value feature.

[0045] In one exemplary embodiment, the set of pixels in a sliding window centered at (i,j) in the feature map constructed by the first input information is defined as ρ(i,j). For a fixed window size k×k, JPEG2025115384000002.jpg1345. For the first query feature and the first key feature, the fine-grained feature is calculated by the following formula (1): You can get JPEG2025115384000003.jpg1130. JPEG2025115384000004.jpg12102 In another exemplary embodiment, a position offset and mask filling scheme may be introduced to obtain another fine-grained feature. As shown in Figure 5, in the path of obtaining the fine-grained feature (hereinafter also referred to as the first path), the position bias is the relative position relationship between the fine-grained tokens in the first field of view, and can also be understood as the relative position relationship between each pixel point in the first field of view.

[0046] In the first pass, we necessarily calculate the out-of-bounds zero-filled similarities for the feature map edge pixels. To prevent the calculated similarity values from affecting the softmax operation, we use a padding mask to set these results as -∞.

[0047] As a result, as shown in Figure 5, in the first path, a cross product operation is performed on the first query feature and the first key feature as shown in Equation (1), and then a first fusion-waiting feature is obtained. Then, an addition operation is performed on each pixel between the first fusion-waiting feature and the position offset after the mask filling process in the first path, and fine-grained features are obtained.

[0048] In the embodiment of the present disclosure, the fine-grained features extracted by combining the features of the target point and the pixel points in the first field of view range around it have strong expressive power, and can lay the foundation for determining the fault detection results subsequently.

[0049] Step A12: Extract coarse-grained features of the target point within the second field of view centered on the target point.

[0050] In some embodiments, extracting coarse-grained features of the target point within the second field of view range centered on the target point can be performed as follows, i.e., can be performed to include the following steps: Step C1: The second field of view range of the target point is determined with the position coordinates of the target point in the first input information as the center.

[0051] As shown in FIG. 5, the second field of view is a larger range of pixel points centered on the target point and encompassing the first field of view, i.e., the one shown in the white rectangular frame in FIG.

[0052] Step C2: Construct a second key feature and a second value feature of the target point based on the feature values of feature points other than the target point within the second field of view.

[0053] Step C3: Determine a coarse-grained feature of the target point based on the first query feature, the second key feature and the second value feature of the target point, among which the feature value in the first input information of the target point is the first query feature of the target point.

[0054] In one exemplary embodiment, a pixel set in a second field of view centered at (i, j) in the feature map constructed by the first input information is defined as ρ′(i, j). A pixel set obtained by pooling in the second field of view is defined as σ(X). The size of the pooling is Regarding JPEG2025115384000005.jpg1226, By using JPEG2025115384000006.jpg1351, the following coarse-grained features are obtained. JPEG2025115384000007.jpg13102 In another exemplary embodiment, a position offset and mask filling scheme may be introduced to obtain another coarse-grained feature. In the coarse-grained feature determination pass (hereinafter referred to as the second pass), the position bias is the relative position relationship between the coarse-grained tokens.

[0055] To further enhance the ability of pixel-focused attention to extend multi-scale image input, we adopt a different method to calculate the position bias, which is calculated based on the second field of view. It could also be JPEG2025115384000008.jpg621.

[0056] As shown in Figure 5, in the second pass, a pooling operation may be performed within the second field of view first to detect the general features around the target point, and the pooling window and step size can be determined according to the actual situation. The pooling operation moves the pooling window in the target image to select the maximum or average pixel to generate a new feature map.

[0057] In the second pass, we use log-spaced continuous position bias (log-CPB), in which we use ReLU (the activation function) to find the position coordinates of the first query feature (Q (i,j) ) and the second key feature pixel set ( spatially relative coordinates between From JPEG2025115384000010.jpg1334, Calculate JPEG2025115384000011.jpg934.

[0058] As a result, as shown in Figure 5, the second fusion waiting feature is obtained by performing a cross product operation on the first query feature and the second key feature as shown in Equation (2), and the second fusion waiting feature and the position offset are added together for each pixel to obtain a coarse-grained feature.

[0059] In the embodiment of the present disclosure, the coarse-grained features extracted by combining the features of the target point and the pixel points in the surrounding second field of view range have stronger expressive power, which can lay the foundation for subsequently determining the fault detection results.

[0060] Step A13: Combine the fine-grained features and the coarse-grained features to obtain initial features of the target points.

[0061] The fine-grained features and the coarse-grained features are additively combined to obtain an initial feature for the point of interest.

[0062] Step A14: The initial features of the target points are mapped to feature values of the target points by a nonlinear mapping method.

[0063] Step A2: The feature values of the plurality of target points are combined according to the position information of each target point in the target image to obtain the first sub-feature.

[0064] As shown in Figure 5, the coarse-grained feature and the fine-grained feature are input to the combination layer and the activation layer to obtain the combined feature. The separation layer performs a separation operation on the combined feature to separate the coarse-grained feature and the fine-grained feature, finally obtaining the intermediate coarse-grained feature and the intermediate fine-grained feature. The first value feature and the intermediate fine-grained feature are cross-product-operated to obtain the target fine-grained feature, and the second value feature and the intermediate coarse-grained feature are cross-product-operated to obtain the target coarse-grained feature. An addition operation is performed on the target fine-grained feature and the target coarse-grained feature to obtain the initial feature, and then the initial feature is passed through a nonlinear mapping layer to obtain the feature value of the target point. The first sub-feature is obtained by performing this operation sequentially for each target point.

[0065] In the embodiments of the present disclosure, the coarse-grained features are used to capture the overall structure of the image, and the fine-grained features are relatively specific and accurate. By simultaneously capturing the coarse-grained features and fine-grained features of the target point, the fault status of the false twist member of the false twist machine can be more effectively monitored.

[0066] In some embodiments, obtaining a mask map using residuals constructed based on the second input information can be performed as follows, i.e., can be performed to include the following steps: Step D1: A Fourier transform is performed on the second input information using a Fourier filter to obtain a time-varying component.

[0067] Among them, the second input information of the second sub-module of the first attention module in the attention network is the previously acquired image. The second input information of the second sub-modules of the second attention module to the Nth attention module in the attention network is the residual output by the previous attention module. Figure 6 shows a framework diagram of the second sub-module of each attention network.

[0068] Step D2: The time-varying component is input to an estimation module constructed based on a neural network to obtain an estimate of the time-varying component.

[0069] Step D3: Determine the residual between the time-varying component and the estimate of the time-varying component to obtain the mask map.

[0070] Among them, the estimation module built based on the neural network may be a time-varying Koopman predictor (Koopa), and the Koopa model consists of multi-layered stackable Koopa basic modules.

[0071] Each Koopa basic module focuses on learning the dynamic characteristics of a specific layer, and by stacking Koopa basic modules, the model can capture complex dynamic changes at multiple layers of time series. Each Koopa basic module learns using the residual obtained by fitting the previous Koopa basic module as input to finally obtain an estimate of the time-varying component. The method proposed in the embodiments of the present disclosure can improve the prediction accuracy of the time-varying Koopman predictor and enhance the adaptability of the model to complex nonlinear and non-stationary time series, so as to extract the dynamic change characteristics of false twist members.

[0072] 2) Decoder Network In some embodiments, the encoders in the encoder network correspond one-to-one to the decoders in the decoder network, and Fig. 7 shows that there are four encoders in the encoder network and four decoders in the decoder network. Regardless of the number of encoders and decoders, performing a decoding operation on the encoded features based on the decoder network to obtain decoded features can be implemented as follows, that is, can be implemented to include the following steps: Step E1: For each target decoder in the decoder network, perform the following operations: Step E11: Obtain the encoding sub-features output by the encoder corresponding to the target decoder as the third query features of the target decoder.

[0073] Taking the four encoders in the encoder network shown in Figure 7 as an example, Processing the target image JPEG2025115384000012.jpg936, where H is the height of the target image and W is the width of the target image. JPEG2025115384000013.jpg1455 is a layered, multi-resolution sub-feature E i These are generated respectively, among which: Regarding JPEG2025115384000014.jpg1954, C i is the weight corresponding to the i-th encoder. That is, the first encoder is The second encoder processes the image JPEG2025115384000015.jpg1737. The third encoder processes the image JPEG2025115384000016.jpg1637. The fourth encoder processes the image JPEG2025115384000017.jpg1546. It can be understood that this is processing the image JPEG2025115384000018.jpg1546.

[0074] In Figure 7, the order of execution of the encoders in the encoder network is first encoder, second encoder, third encoder, fourth encoder, etc. After the encoder network execution is complete, the order of execution of the decoder network is fourth decoder, third decoder, second decoder, first decoder, etc.

[0075] As shown in FIG. 7, the third query feature of the first decoder is the first encoded sub-feature output by the first encoder, the third query feature of the second decoder is the second encoded sub-feature output by the second encoder, the third query feature of the third decoder is the third encoded sub-feature output by the third encoder, and the third query feature of the fourth decoder is the fourth encoded sub-feature output by the fourth encoder.

[0076] Step E12: if the target decoder is the first decoder, obtain the encoding sub-features of all encoders in the encoder network, and construct a third value feature and a third key feature of the target decoder.

[0077] Step E13: If the target decoder is any decoder other than the first decoder, obtain the decoding sub-features output by each preceding decoder before the target decoder as preferred sub-features, and for the feature set constructed by the encoding sub-features of each encoder, use the preferred sub-features to replace the encoding sub-features of the encoder corresponding to the preceding decoder, to obtain the third value feature and the third key feature of the target decoder.

[0078] As shown in Figure 7, after the encoder network completes its processing, it obtains the first, second, third, and fourth encoded subfeatures, and determines the third value and third key features to be input to the fourth decoder based on the first, second, third, and fourth encoded subfeatures. After obtaining the fourth decoded subfeature, it uses the fourth decoded subfeature to replace the fourth encoded subfeature in the original third value and third key features as input to the third decoder, thereby obtaining the third decoded subfeature for the third decoder. Further processing is performed to obtain the first, second, third, and fourth decoded subfeatures to be output by the decoder network.

[0079] Step E14: inputting the third query feature, the third value feature and the third key feature into the target decoder, and obtaining the decoding sub-feature output by the target decoder.

[0080] In some embodiments, the architecture of each decoder includes a mix-attention mechanism module (Mix-Attention), a layer normalization module (LN) and a feed-forward network (FFN), as shown in FIG. 8, and the specific manner of obtaining the decoding sub-features output by the target decoder can be implemented as follows, that is, can be implemented to include the following steps: Step F1: Input the third query feature, the third key feature and the third value feature into a mixed attention mechanism module to obtain a first intermediate feature.

[0081] Step F2: The first intermediate feature and the third query feature are fused and input into a layer normalization module to obtain a second intermediate feature.

[0082] Step F3: The second intermediate feature is input into a feedforward network to obtain a third intermediate feature.

[0083] Step F4: Fuse the second intermediate feature and the third intermediate feature to obtain a decoding sub-feature output by the target decoder.

[0084] Step E2: Construct the decoding feature based on the decoding sub-features of each decoder.

[0085] After obtaining the four decoding sub-features, the decoding feature is obtained by fusing them, which may be fused in the form of weighted sum, combination, etc.

[0086] In the embodiment of the present disclosure, in the self-attention mechanism, the sources for generating the query feature, key feature, and value feature are the same, i.e., from the same encoder / decoder. However, in the embodiment of the present disclosure, a mixed attention mechanism module is adopted, and a multi-scale step mixed feature is adopted, where each feature comes from a separate encoder. That is, different query features are allowed to come from different decoding steps, which can achieve different matching degrees for the context granularity, thereby enhancing the accuracy of the function of detecting faults in the target image.

[0087] Based on the same technical concept, an embodiment of the present disclosure provides a fault detection device 900, as shown in FIG. 9 : an acquisition unit 901 for acquiring an image of a false twisting member of a false twisting machine; a construction unit 902 for constructing a first fusion feature based on at least one attention module in an attention network; where, for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature for fine-grained features within a first visual field range and coarse-grained features within a second visual field range in the first input information, the second sub-module obtains a mask map using a residual constructed based on the second input information, and the output feature of the attention module is obtained by multiplying the mask map and the first sub-feature, where the first visual field range is smaller than the second visual field range; an encoding unit 903 for inputting the first fusion feature into an encoder network to obtain an encoding feature, where the encoder network includes a plurality of encoders, each encoder outputs a corresponding encoding sub-feature, and the encoding feature includes the encoding sub-feature of at least one encoder; a decoding unit 904 for performing a decoding operation on the encoding feature based on a decoder network to obtain a decoded feature, where the decoder network includes a plurality of decoders, each decoder outputs a corresponding decoded sub-feature, and the decoded feature includes the decoded sub-features of a plurality of decoders; a fusion unit 905 for fusing each decoding sub-feature in the decoding feature to obtain a second fused feature; a prediction unit 906 for inputting the second fused feature into a multi-layer perceptron to obtain a fault detection result of the false twist member of the false twisting machine.

[0088] In some embodiments, for a first attention module in the attention network: a first input information of a first sub-module of the first attention module is the target image; the second input information of the second sub-module of the first attention module includes the target image and at least one previously acquired image of the false twist member, and the acquisition time interval between the target image and any one of the previously acquired images is shorter than a predetermined time length; For any attention module other than the first attention module in the attention network, The first input information of the first sub-module of any of the attention modules is an output feature output by an attention module immediately before the any of the attention modules; The second input information of the second sub-module of any one of the attention modules is the residual output by the second sub-module of the attention module immediately preceding any one of the attention modules.

[0089] In some embodiments, the building blocks are: a determination subunit for determining a plurality of target points in the first input information, and for each target point, performing the following operations: extracting fine-grained features of the target point within the first field of view range centered on the target point; extracting coarse-grained features of the target point within the second field of view range centered on the target point; combining the fine-grained features and the coarse-grained features to obtain initial features of the target point; and mapping the initial features of the target point by a nonlinear mapping method to obtain feature values of the target point; a synthesis subunit for combining the feature values of the plurality of target points according to position information of each target point in the target image to obtain the first sub-feature.

[0090] In some embodiments, the determining subunit specifically comprises: determining a feature value in the first input information of the target point as a first query feature of the target point; determining the first field of view range of the target point with the position coordinates of the target point in the first input information as a center; constructing a first key feature and a first value feature of the target point based on feature values of feature points other than the target point within the first field of view; and determining fine-grained features of the interest points based on the first query feature, the first key feature, and the first value feature.

[0091] In some embodiments, the determining subunit specifically comprises: determining the second field of view range of the target point with the position coordinates of the target point in the first input information as a center; constructing a second key feature and a second value feature of the target point based on feature values of feature points other than the target point within the second field of view; determining coarse-grained features of the interest points based on a first query feature, the second key feature, and the second value feature of the interest points; The feature value in the first input information of the interest point is a first query feature of the interest point.

[0092] In some embodiments, the building blocks are specifically: a transform subunit for employing a Fourier filter to perform a Fourier transform on the second input information to obtain a time-varying component; an estimation subunit for inputting the time-varying component to an estimation module constructed based on a neural network to obtain an estimate of the time-varying component; a residual determination subunit for determining a residual between the time-varying component and an estimate of the time-varying component to obtain the mask map.

[0093] In some embodiments, the encoders in the encoder network have a one-to-one correspondence with the decoders in the decoder network, and the decoding unit comprises: a decoding sub-feature determination subunit for performing the following operations for each target decoder in the decoder network: obtaining an encoding sub-feature output by an encoder corresponding to the target decoder as a third query feature of the target decoder; if the target decoder is a first decoder, obtaining encoding sub-features of all encoders in the encoder network to construct a third value feature and a third key feature of the target decoder; if the target decoder is any decoder other than the first decoder, obtaining a decoding sub-feature output by each preceding decoder before the target decoder as a preferred sub-feature, and using the preferred sub-feature to replace the encoding sub-feature of the encoder corresponding to the preceding decoder in the feature collection constructed by the encoding sub-features of each encoder, to obtain a third value feature and a third key feature of the target decoder; and inputting the third query feature, the third value feature and the third key feature into the target decoder to obtain a decoding sub-feature output by the target decoder; and a decoding feature construction subunit for constructing the decoding feature based on the decoding sub-features of each decoder.

[0094] In some embodiments, the decoding sub-characterization sub-unit specifically: inputting the third query feature, the third key feature, and the third value feature into a mixed attention mechanism module to obtain a first intermediate feature; The first intermediate feature and the third query feature are fused and input into a layer normalization module to obtain a second intermediate feature; inputting the second intermediate feature into a feedforward network to obtain a third intermediate feature; and fusing the second intermediate feature and the third intermediate feature to obtain a decoding sub-feature output by the target decoder.

[0095] For specific functions and exemplary descriptions of each module, sub-module / unit of the apparatus according to the embodiments of the present disclosure, please refer to the relevant descriptions of the corresponding steps in the above method embodiments, and the description will be omitted here.

[0096] In the technical solution disclosed herein, the acquisition, storage, and application of personal information of relevant users shall comply with the provisions of relevant laws and regulations and shall not violate public order and morals.

[0097] FIG. 10 is a block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 10, the electronic device includes a memory 1010 and a processor 1020. The memory 1010 stores a computer program executable by the processor 1020. The number of memories 1010 and processors 1020 may be one or more. The memory 1010 may store one or more computer programs. When the one or more computer programs are executed by the electronic device, the electronic device can perform the method provided in the above-described method-related embodiment. The electronic device may further include a communication interface 1030 for communicating with an external device and exchanging and transmitting data.

[0098] If the memory 1010, the processor 1020, and the communication interface 1030 are separate, they can be connected to each other and communicate with each other via a bus. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience, only one thick line is shown in FIG. 10, but this does not mean that only one bus or one type of bus exists.

[0099] Optionally, as a specific implementation, when the memory 1010, the processor 1020, and the communication interface 1030 are integrated into one chip, the memory 1010, the processor 1020, and the communication interface 1030 can communicate with each other via an internal interface.

[0100] It should be understood that the processor may be a Central Processing Unit (CPU), other general-purpose processors, Digital Signal Processing (DSP), Application Specific Integrated Circuits (ASIC), Field Programmable Gate Arrays (FPGA) or other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may also be a processor capable of supporting an Advanced Reduced Instruction Set Machine (ARM) architecture.

[0101] Additionally, optionally, the memory may include read-only memory, random access memory, or non-volatile random access memory. The memory may be volatile or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) used as an external cache. The above description is illustrative only and not restrictive. Many forms of RAM are available. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchlink dynamic random access memory (Synchlink DRAM, SLDRAM), and direct memory bus random access memory (Direct RAM BUS RAM, DR RAM) may be used.

[0102] In the above embodiments, all or part of the above may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the above may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed by a computer, the computer program product generates all or part of the flows or functions described in the embodiments of the present disclosure. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, Bluetooth, microwave, etc.). The computer-readable storage medium may be any available medium accessible by a computer, or may be a data storage device such as a server, data center, or the like that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a Digital Versatile Disk (DVD)), a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like. Note that the computer-readable storage medium according to the present disclosure may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0103] Those skilled in the art will understand that all or part of the steps for realizing the above embodiments may be completed by hardware, or may be completed by instructing relevant hardware by a program, and the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0104] In describing embodiments of the present disclosure, the terms "one embodiment," "some embodiments," "examples," "particular examples," or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in any one or more embodiments or examples. Furthermore, unless mutually inconsistent, a person skilled in the art can combine features from different embodiments or examples described herein.

[0105] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means "or", for example, "A / B" can represent "A" or "B". The description of "and / or" in this specification is merely a relation describing related objects, and means that there may be three relations, for example, "A and / or B" can indicate three situations: "A" exists alone, "A" and "B" exist simultaneously, and "B" exists alone.

[0106] In describing the embodiments of the present disclosure, the terms "first" and "second" are for distinguishing purposes and should not be understood to indicate or imply relative importance or the number of the indicated components. Thus, a feature qualified with "first" or "second" can explicitly or implicitly include one or more of the feature. In describing the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

[0107] The above are merely illustrative examples of the present disclosure and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included within the scope of the claims of the present disclosure.

Claims

1. 1. A fault detection method, comprising: acquiring a target image by performing image acquisition on a false twist member of a false twisting machine; Constructing a first fusion feature based on at least one attention module in an attention network, where for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs a first sub-feature for fine-grained features within a first visual field range and coarse-grained features within a second visual field range in the first input information, the second sub-module obtains a mask map using a residual constructed based on the second input information, and output features of the attention module are obtained by multiplying the mask map and the first sub-feature, and the first visual field range is smaller than the second visual field range; inputting the first fusion feature into an encoder network to obtain an encoding feature, where the encoder network includes a plurality of encoders, each encoder outputs a corresponding encoding sub-feature, and the encoding feature includes the encoding sub-feature of at least one encoder; performing a decoding operation on the encoding feature based on a decoder network to obtain a decoding feature, where the decoder network includes a plurality of decoders, each decoder outputs a corresponding decoding sub-feature, and the decoding feature includes the decoding sub-features of the plurality of decoders; Fusing each decoding sub-feature in the decoding feature to obtain a second fused feature; and inputting the second fused feature into a multi-layer perceptron to obtain a fault detection result for the false twist member of the false twisting machine.

2. For the first attention module in the attention network, a first input information of a first sub-module of the first attention module is the target image; the second input information of the second sub-module of the first attention module includes the target image and at least one previously acquired image of the false twist member, and the acquisition time interval between the target image and any one of the previously acquired images is shorter than a predetermined time length; For any attention module other than the first attention module in the attention network, The first input information of the first sub-module of any of the attention modules is an output feature output by an attention module immediately before the any of the attention modules; The second input information of the second sub-module of any of the attention modules is a residual output by the second sub-module of the attention module immediately preceding the any of the attention modules.

2. The method of claim 1 .

3. Constructing first sub-features for fine-grained features within a first field of view range and coarse-grained features within a second field of view range in the first input information includes: determining a plurality of target points in the first input information; for each target point, respectively extracting fine-grained features of the target point within the first field of view range centered on the target point; extracting coarse-grained features of the target point within the second field of view range centered on the target point; combining the fine-grained features and the coarse-grained features to obtain initial features of the target point; and mapping the initial features of the target point to feature values of the target point by a nonlinear mapping method; and combining the feature values of the plurality of target points according to position information of each target point in the target image to obtain the first sub-feature.

3. The method of claim 2.

4. Extracting fine-grained features of the target point within the first field of view centered on the target point includes: determining a feature value in the first input information of the interest point as a first query feature of the interest point; determining the first field of view range of the target point centered on the position coordinates of the target point in the first input information; constructing a first key feature and a first value feature of the target point based on feature values of feature points other than the target point within the first field of view; determining fine-grained features of the interest points based on the first query feature, the first key feature, and the first value feature; 4. The method of claim 3.

5. Extracting coarse-grained features of the target point within the second field of view centered on the target point includes: determining the second field of view range of the target point centered on the position coordinates of the target point in the first input information; constructing a second key feature and a second value feature of the target point based on feature values of feature points other than the target point within the second field of view; determining coarse-grained features of the interest points based on the first query feature, the second key feature, and the second value feature of the interest points; a feature value in the first input information of the interest point is a first query feature of the interest point; 4. The method of claim 3.

6. Obtaining a mask map using residuals constructed based on the second input information includes: employing a Fourier filter to perform a Fourier transform on the second input information to obtain a time-varying component; inputting the time-varying component to an estimation module based on a neural network to obtain an estimate of the time-varying component; determining a residual between the time-varying component and an estimate of the time-varying component to obtain the mask map.

3. The method of claim 2.

7. an encoder in the encoder network has a one-to-one correspondence with a decoder in the decoder network; performing a decoding operation on the encoded features based on the decoder network to obtain decoded features, for each target decoder in the decoder network, perform the following operations: obtain an encoded sub-feature output by an encoder corresponding to the target decoder as a third query feature of the target decoder; if the target decoder is a first decoder, obtain the encoded sub-features of all encoders in the encoder network and construct a third value feature and a third key feature of the target decoder; if the target decoder is any decoder other than the first decoder, obtain a decoded sub-feature output by each preceding decoder before the target decoder as a preferred sub-feature, and, for the feature collection constructed by the encoded sub-features of each encoder, use the preferred sub-feature to replace the encoded sub-feature of the encoder corresponding to the preceding decoder, thereby obtaining a third value feature and a third key feature of the target decoder; and input the third query feature, the third value feature and the third key feature into the target decoder and obtain a decoded sub-feature output by the target decoder; constructing the decoding feature based on the decoding sub-features of each decoder; 7. The method according to claim 1, wherein the first and second electrodes are connected to a first electrode.

8. inputting the third query feature, the third value feature, and the third key feature into the target decoder and obtaining decoding sub-features output by the target decoder, inputting the third query feature, the third key feature, and the third value feature into a mixed attention mechanism module to obtain a first intermediate feature; Fusing the first intermediate features and the third query features into a layer normalization module to obtain second intermediate features; inputting the second intermediate feature into a feedforward network to obtain a third intermediate feature; fusing the second intermediate feature and the third intermediate feature to obtain a decoding sub-feature output by the target decoder.

8. The method of claim 7.

9. A fault detection device, comprising: an acquisition unit for acquiring a target image by performing image acquisition on a false twist member of the false twisting machine; a construction unit for constructing a first fusion feature based on at least one attention module in an attention network, where, for each attention module, the attention module includes a first sub-module and a second sub-module, the first sub-module constructs first sub-features for fine-grained features within a first visual field range and coarse-grained features within a second visual field range in the first input information, the second sub-module obtains a mask map using a residual constructed based on the second input information, and output features of the attention module are obtained by multiplying the mask map and the first sub-features, and the first visual field range is smaller than the second visual field range; an encoding unit for inputting the first fusion feature into an encoder network to obtain an encoded feature, where the encoder network includes a plurality of encoders, each encoder outputs a corresponding encoded sub-feature, and the encoded feature includes the encoded sub-feature of at least one encoder; a decoding unit for performing a decoding operation on the encoding feature based on a decoder network to obtain a decoding feature, where the decoder network includes a plurality of decoders, each decoder outputs a corresponding decoding sub-feature, and the decoding feature includes the decoding sub-features of the plurality of decoders; a fusion unit for fusing each decoding sub-feature in the decoding feature to obtain a second fused feature; a prediction unit for inputting the second fused feature into a multi-layer perceptron to obtain a fault detection result of the false twist member of the false twisting machine. A fault detection device characterized by:

10. For the first attention module in the attention network, a first input information of a first sub-module of the first attention module is the target image; the second input information of the second sub-module of the first attention module includes the target image and at least one previously acquired image of the false twist member, and the acquisition time interval between the target image and any one of the previously acquired images is shorter than a predetermined time length; For any attention module other than the first attention module in the attention network, The first input information of the first sub-module of any of the attention modules is an output feature output by an attention module immediately before the any of the attention modules; The second input information of the second sub-module of any of the attention modules is a residual output by the second sub-module of the attention module immediately preceding the any of the attention modules.

10. The device of claim 9.

11. The building units are: a determination subunit for determining a plurality of target points in the first input information, and for each target point, performing the following operations: extracting fine-grained features of the target point within the first field of view range centered on the target point; extracting coarse-grained features of the target point within the second field of view range centered on the target point; combining the fine-grained features and the coarse-grained features to obtain initial features of the target point; and mapping the initial features of the target point to feature values of the target point by a nonlinear mapping method; a combining subunit for combining the feature values of the plurality of target points according to position information of each target point in the target image to obtain the first sub-feature; 11. The device of claim 10.

12. The defined subunit is determining a feature value in the first input information of the target point as a first query feature of the target point; determining the first field of view range of the target point with the position coordinates of the target point in the first input information as a center; constructing a first key feature and a first value feature of the target point based on feature values of feature points other than the target point within the first field of view; determining fine-grained features of the interest points based on the first query feature, the first key feature, and the first value feature; 12. The device of claim 11 .

13. The defined subunit is determining the second field of view range of the target point with the position coordinates of the target point in the first input information as a center; constructing a second key feature and a second value feature of the target point based on feature values of feature points other than the target point within the second field of view; determining coarse-grained features of the interest points based on a first query feature, the second key feature, and the second value feature of the interest points; a feature value in the first input information of the interest point is a first query feature of the interest point; 12. The device of claim 11 .

14. The building units are: a transform subunit for employing a Fourier filter to perform a Fourier transform on the second input information to obtain a time-varying component; an estimation subunit for inputting the time-varying component to an estimation module constructed based on a neural network to obtain an estimate of the time-varying component; a residual determination subunit for determining a residual between the time-varying component and an estimate of the time-varying component to obtain the mask map.

11. The device of claim 10.

15. an encoder in the encoder network has a one-to-one correspondence with a decoder in the decoder network; The decoding unit a decoding sub-feature determination subunit for performing, for each target decoder in the decoder network, an operation of obtaining an encoding sub-feature output by an encoder corresponding to the target decoder as a third query feature of the target decoder; if the target decoder is a first decoder, an operation of obtaining encoding sub-features of all encoders in the encoder network and constructing a third value feature and a third key feature of the target decoder; if the target decoder is any decoder other than the first decoder, an operation of obtaining a decoding sub-feature output by each preceding decoder before the target decoder as a preferred sub-feature, and using the preferred sub-feature to replace the encoding sub-feature of the encoder corresponding to the preceding decoder in the feature collection constructed by the encoding sub-features of each encoder, thereby obtaining a third value feature and a third key feature of the target decoder; and an operation of inputting the third query feature, the third value feature and the third key feature into the target decoder and obtaining a decoding sub-feature output by the target decoder; a decoding feature construction subunit for constructing the decoding feature based on the decoding sub-features of each decoder; 15. Apparatus according to any one of claims 9 to 14.

16. The decoding sub-characteristic determination sub-unit: inputting the third query feature, the third key feature, and the third value feature into a mixed attention mechanism module to obtain a first intermediate feature; Fusing the first intermediate feature and the third query feature into a layer normalization module to obtain a second intermediate feature; inputting the second intermediate feature into a feedforward network to obtain a third intermediate feature; and fusing the second intermediate feature and the third intermediate feature to obtain a decoding sub-feature output by the target decoder.

16. The device of claim 15.

17. An electronic device, at least one processor; a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the method of any one of claims 1 to 6. electronic equipment.

18. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to carry out the method of any one of claims 1 to 6.

Images