A defect detection method and device based on neural network

Abstract

The present application discloses a defect detection method and device based on a neural network. A defect detection model is constructed, and a textile surface image is input into the defect detection model for model training until the loss function converges, and the defect detection model after training is used to perform defect detection to obtain a defect detection result of the textile; the textile surface image is input into a decomposition transformation module for decomposition to obtain component information, and the component information is fused to generate an initial feature map; the initial feature map is extracted and fused using a convolution decoupling module to obtain an enhanced fused feature map; the enhanced fused feature map is input into a feature pyramid network for multi-scale fusion to obtain a multi-scale feature map, and the multi-scale feature map is enhanced using a bidirectional feature pyramid network to obtain an enhanced multi-scale feature map; the enhanced multi-scale feature map is input into a detection layer to obtain a detection result. The method can automatically detect defects of textiles on a production line with high accuracy.

Description

Technical Field

[0001] The present application relates to the technical field of image defect detection, and in particular to a defect detection method and device based on a neural network. Background Art

[0002] With the continuous development of deep learning technology, especially in the field of image recognition, it has made significant progress and has become the core driving force for the advancement of machine vision technology. Combined with actual application scenarios, in the field of medical diagnosis, deep learning technology can detect and classify medical images, realize auxiliary diagnosis and pathological analysis; in the field of autonomous driving, deep reinforcement learning technology can perceive the surrounding environment through sensor data and realize path planning.

[0003] In the production process of textiles, various defects often occur due to machine failure, operating errors or defects in the raw materials themselves. Fabric defect detection is undoubtedly a very important link. By identifying defects, the defective rate can be reduced and product quality can be improved.

[0004] Traditional textile defect detection methods mainly rely on manual detection and machine vision technology, using methods such as edge detection, template matching, convolutional neural networks and morphological processing to analyze images. Although existing convolutional neural networks perform well in target detection and classification tasks, they have certain limitations when processing high-frequency information and tend to ignore detailed information in the image.

[0005] In addition, before the image is input into the convolutional neural network, the image is usually converted from the spatial domain to the frequency domain through spectrum analysis methods. Commonly used spectrum analysis methods include Fourier transform, Gabor transform, etc. Although the convolutional neural network can identify smaller defects by analyzing the characteristics of the spectrum, it has high requirements for computing resources and is prone to ignoring local deviation information, making it difficult to ensure the accuracy of defect detection. Summary of the invention

[0006] The embodiment of the present application solves the problems of low textile detection accuracy and easy neglect of high-frequency information in the prior art by providing a defect detection method and device based on a neural network. A decomposition transformation module is introduced to extract different frequency features of the obtained textile surface image, a convolution decoupling module is introduced to maximize the information in the downsampling process, and a bidirectional feature pyramid network is used to capture features at different scales while reducing redundant parameters. The detection efficiency and accuracy of textiles on the production line are achieved in real-time automated detection, labor costs and subjective errors are reduced, and product quality and market competitiveness are improved.

[0007] In a first aspect, an embodiment of the present application provides a defect detection method based on a neural network, comprising:

[0008] A defect detection model is constructed, and a textile surface image is input into the defect detection model for model training until the loss function converges, and the trained defect detection model is used for defect detection to obtain a defect detection result of the textile; wherein the defect detection model includes a decomposition transformation module, a convolution decoupling module, a feature pyramid network, a bidirectional feature pyramid network and a detection layer; the textile surface image is input into the decomposition transformation module for decomposition to obtain component information, and the component information is fused to generate an initial feature map; the initial feature map is extracted and fused using the convolution decoupling module to obtain an enhanced fused feature map; wherein the enhanced fused feature map includes a primary enhanced feature map, a secondary enhanced feature map, a first fused feature map and a second fused feature map; the enhanced fused feature map is input into the feature pyramid network for multi-scale fusion to obtain a multi-scale feature map, and the multi-scale feature map is enhanced using the bidirectional feature pyramid network to obtain an enhanced multi-scale feature map; the enhanced multi-scale feature map is input into the detection layer to obtain a detection result.

[0009] In a possible implementation, the inputting of the textile surface image into the decomposition transformation module to decompose the component information and fuse the component information to generate an initial feature map includes: performing wavelet decomposition on the textile surface image to obtain the component information; wherein the component information includes a low-frequency component, a horizontal high-frequency component, a vertical high-frequency component and a diagonal high-frequency component; and fusing the component information to generate an initial feature map containing both the low-frequency information and the high-frequency information.

[0010] In a possible implementation, the fusion is performed to generate an initial feature map as follows:

[0011] ;

[0012] In the formula, represents the initial feature map, represents the weight coefficient of the low-frequency component, represents the low-frequency component, represents the weight coefficient of the horizontal high frequency component, represents the horizontal high frequency component, represents the weight coefficient of the vertical high-frequency component, represents the vertical high-frequency component, represents the weight coefficient of the diagonal high frequency component, represents the diagonal high frequency component.

[0013] In a possible implementation, the convolution decoupling module includes a first convolution layer, a second convolution layer, a first downsampling layer, a second downsampling layer and a plurality of C3 modules.

[0014] In a possible implementation, the use of the convolution decoupling module to extract and fuse features on the initial feature map to obtain an enhanced fused feature map includes: inputting the initial feature map into the first convolution layer for preliminary feature extraction to obtain a first feature map; inputting the first feature map into the C3 module to obtain the first enhanced feature map; inputting the first feature map into the second convolution layer for further feature extraction to obtain a second feature map; inputting the second feature map into the C3 module to obtain the second enhanced feature map; inputting the second feature map into the first downsampling layer to obtain a first downsampling feature map; inputting the first downsampling feature map into the C3 module to obtain the first fused feature map; inputting the first downsampling feature map into the second downsampling layer to obtain a second downsampling feature map; inputting the second downsampling feature map into the C3 module to obtain the second fused feature map; and inputting the first enhanced feature map, the second enhanced feature map, and the first fused feature map into the feature pyramid network.

[0015] In a possible implementation, the loss function is as follows:

[0016] ;

[0017] in, ;

[0018] ;

[0019] In the formula, represents the loss function, Represents the intersection-over-union ratio of the predicted box and the true box, The penalty term representing the center distance between the predicted box and the true box, Represents the aspect ratio penalty term between the predicted box and the true box, Represents the distance between the center point of the predicted box and the true box, Represents the diagonal distance of the minimum closed box covering the predicted box and the real box, Indicates the width of the real box, Indicates the height of the real frame, Indicates the width of the prediction box, Indicates the height of the prediction box.

[0020] In one possible implementation, the step of inputting the textile surface image into the defect detection model for model training until the loss function converges also includes: iteratively executing a model parameter optimization step until the value of the loss function meets a convergence condition; the model parameter optimization step includes: calculating the value of the loss function through forward propagation, and determining the gradient of the loss function relative to the model parameters through back propagation; and updating the model parameters using an optimization algorithm based on the gradient.

[0021] In a second aspect, an embodiment of the present application provides a device for defect detection based on a neural network, comprising: a model building module, for building a defect detection model, and inputting a textile surface image into the defect detection model for model training until the loss function converges, and performing defect detection using the trained defect detection model to obtain a defect detection result of the textile; wherein the defect detection model comprises a decomposition transformation module, a convolution decoupling module, a feature pyramid network, a bidirectional feature pyramid network and a detection layer; a decomposition transformation module, for inputting the textile surface image into the decomposition transformation module for decomposition to obtain component information, and The convolution decoupling module is used to extract and fuse the initial feature map to obtain an enhanced fused feature map; wherein the enhanced fused feature map includes a first enhanced feature map, a second enhanced feature map, a first fused feature map and a second fused feature map; the enhanced fused feature map is input into the feature pyramid network for multi-scale fusion to obtain a multi-scale feature map, and the multi-scale feature map is enhanced by the bidirectional feature pyramid network to obtain an enhanced multi-scale feature map; the detection module is used to input the enhanced multi-scale feature map into the detection layer to obtain a detection result.

[0022] In a third aspect, an embodiment of the present application provides a device for executing a neural network-based defect detection method, the device comprising: a processor; a memory for storing processor executable instructions; when the processor executes the executable instructions, it implements the method described in the first aspect or any possible implementation method of the first aspect.

[0023] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium includes a medium for storing a computer program or instructions, which, when executed, enables the method described in the first aspect or any possible implementation method of the first aspect to be implemented.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0025] The embodiment of the present application adopts a defect detection method and device based on a neural network, and constructs a defect detection model for textiles by decomposing a transformation module and a bidirectional feature pyramid network, combining a convolution decoupling module and a loss function. The high-frequency and low-frequency component information is obtained by the decomposition transformation module, and is input into the convolution decoupling module for feature extraction and fusion, thereby enhancing the ability to capture the details of the textile surface image. The bidirectional feature pyramid network can effectively fuse features at different scales to improve the defect detection model's ability to detect defects of different sizes. The convolution decoupling module retains more textile surface image information during the downsampling process, while reducing redundant parameters and improving the performance of the convolution decoupling module. The loss function further improves the accuracy and stability of the detection results, so that the accuracy and average precision of the convolution decoupling module in textile defect detection are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A flowchart of a defect detection method based on a neural network provided in an embodiment of the present application;

[0028] Figure 2 A flowchart of constructing a defect detection model provided in an embodiment of the present application;

[0029] Figure 3 This is an experimental effect diagram of decomposing component information obtained by the decomposition transformation module provided in the embodiment of the present application;

[0030] Figure 4 An example diagram of the result of defect detection on a textile surface image provided in an embodiment of the present application;

[0031] Figure 5 A schematic diagram of the structure of a neural network-based defect detection device provided in an embodiment of the present application;

[0032] Figure 6 This is a structural diagram of the C3 module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] The following describes some of the techniques involved in the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, some descriptions of well-known functions and structures are omitted in the following description.

[0035] Figure 1 This is a flowchart of a neural network-based defect detection method provided in an embodiment of the present application, including steps 101 to 107. Figure 1 This is only an execution order shown in the embodiment of the present application, and does not represent the only execution order of the defect detection method based on the neural network. In the case where the final result can be achieved, Figure 1 The steps shown may be performed in parallel or in reverse, as follows.

[0036] Step 101: construct a defect detection model, and input the textile surface image into the defect detection model for model training until the loss function converges. Figure 2 As shown in the figure, the defect detection model includes a decomposition transformation module, a convolutional decoupling module, a feature pyramid network, a bidirectional feature pyramid network and a detection layer.

[0037] In the embodiment of the present application, a camera device is used to collect textile surface images in a factory to obtain a data set of textile surface images, wherein the data set is divided into a training set and a validation set in a ratio of 8:2.

[0038] By decomposing the transform module, wavelet transform is introduced at the input of the defect detection model, replacing the first convolutional layer of the backbone network of the traditional deep learning model. The textile surface image is gradually refined at multiple scales, thereby achieving spatial subdivision at high frequencies and frequency subdivision at low frequencies, which can reduce the amount of calculation and improve the efficiency of subsequent processing of textile surface images.

[0039] Specifically, the loss function in the defect detection model is calculated by (Intersection-over-Union ratio, i.e., the ratio of the overlapped intersection area of ​​the predicted box and the real box to the union area), adds a penalty term for the distance between the center points of the predicted box and the real box, as well as a penalty term for the difference in the aspect ratio between the predicted box and the real box. Iterate the model parameter optimization steps until the value of the loss function meets the convergence condition. The loss function is as follows:

[0040] ;

[0041] in, ;

[0042] ;

[0043] ;

[0044] ;

[0045] ;

[0046] ;

[0047] ;

[0048] In the formula, represents the initial loss function, represents the loss function, Represents the intersection-over-union ratio of the predicted box and the true box, and Represent the true box and the predicted box respectively, Represents the distance between the center point of the predicted box and the true box, Represents the diagonal distance of the minimum closed box covering the predicted box and the real box, Indicates the width of the real box, Indicates the height of the real frame, Indicates the width of the prediction box, Indicates the height of the prediction box, Represents the Euclidean distance between the real box and the predicted box, and Represents the center point of the predicted box and the true box, represents a positive trade-off parameter, Indicates the consistency of the aspect ratio between the predicted box and the real box, Indicates the width of the real box, Indicates the height of the real frame, Indicates the width of the prediction box, Indicates the height of the prediction box, ( , ) represents the center point coordinates of the prediction box, ( , ) represents the center point coordinates of the real frame, The penalty term representing the center distance between the predicted box and the true box, Represents the aspect ratio penalty term between the predicted box and the true box.

[0049] During the training process, the initial loss function The model parameters are continuously updated through back propagation, so that the predicted box gradually approaches the real box, and the loss function is obtained. The convergence condition value of the loss function is exemplarily set to 0.0001, and the final loss is minimized, so that the defect detection model has the optimal performance.

[0050] Under the constraint of the loss function, the weights and biases are updated by the back propagation algorithm. Specifically, the value of the loss function is calculated by forward propagation, and then the gradient of the loss function relative to each parameter is calculated by back propagation, where the gradient represents the sensitivity of the loss function to parameter changes. Then, the optimization algorithm is used to update the parameters of the defect detection model according to the calculated gradient. Exemplarily, the optimization algorithm here can use the gradient descent algorithm. By continuously adjusting the parameters of the defect detection model to minimize the value of the loss function, the defect detection model can gradually improve its prediction accuracy and generalization ability, so that it performs better in practical applications.

[0051] Through the above steps, the defect detection model automatically learns each batch of textile surface images, extracts and fuses features, outputs the detection results, and performs backpropagation through the loss function to minimize the loss, ultimately achieving more accurate defect detection.

[0052] In the embodiment of the present application, the training of the defect detection model includes steps 102 to 105, which are specifically as follows.

[0053] Step 102: Input the textile surface image into the decomposition transformation module to decompose and obtain component information, and fuse them to generate an initial feature map. In the embodiment of the present application, a wavelet filter is selected, such as Haar wavelet (first-order discontinuous wavelet), Daubechies wavelet, etc. These wavelet filters are used to perform a two-dimensional discrete wavelet transform (2D-DWT) on the input textile surface image, decompose the textile surface image into two directions of rows and columns, and apply a wavelet filter in each direction to obtain high-frequency and low-frequency components respectively. The low-frequency component ( ) contains the main structural information of the textile surface image, while the high-frequency component ( , , ) contains detailed information of the textile surface image.

[0054] like Figure 3As shown in the figure, it is an experimental effect diagram of the decomposition transformation module to decompose and obtain component information. A low-pass filter is applied to the row and column directions of the textile surface image to obtain a low-frequency component, which contains most of the energy and structural information of the textile surface image. A low-pass filter is applied to the row direction and a high-pass filter is applied to the column direction to obtain a horizontal high-frequency component, which contains horizontal edge information. A high-pass filter is applied to the row direction and a low-pass filter is applied to the column direction to obtain a vertical high-frequency component, which contains vertical edge information. A high-pass filter is applied to both the row and column directions to obtain a diagonal high-frequency component, which contains the details and noise information of the textile surface image. The component information is weighted and fused to generate an initial feature map containing both low-frequency information and high-frequency information, which contains rich detail information, as follows:

[0055] ;

[0056] In the formula, Represents the initial feature map, which is the result of fusion of each component information. represents the weight coefficient of the low-frequency component, represents the low-frequency component, represents the weight coefficient of the horizontal high-frequency component, represents the horizontal high-frequency component, represents the weight coefficient of the vertical high-frequency component, represents the vertical high-frequency component, represents the weight coefficient of the diagonal high-frequency component, Represents the diagonal high-frequency component. Fusion of component information can enhance the effect of feature extraction and improve detection performance by enriching the detail information of textile surface images.

[0057] Step 103: Use the convolution decoupling module to extract and fuse the initial feature map to obtain an enhanced fused feature map, wherein the enhanced fused feature map includes a primary enhanced feature map, a secondary enhanced feature map, a first fused feature map, and a second fused feature map.

[0058] Specifically, the convolutional neural network can more comprehensively capture the details and structural information in the textile surface image, thereby improving the detection accuracy. The initial feature map is input into the first convolution layer for preliminary feature extraction to obtain the first feature map, and then the first feature map is input into the C3 module to obtain a primary enhanced feature map. The first feature map is input into the second convolution layer for further feature extraction to obtain the second feature map, and then the second feature map is input into the C3 module to obtain a secondary enhanced feature map.

[0059] The second feature map is input into the first downsampling layer to obtain a first downsampling feature map, and the first downsampling feature map is input into the C3 module to obtain a first fused feature map.

[0060] Among them, Figure 6 As shown in the figure, the C3 module divides the feature map input at each stage into two parts, one part is processed by multiple Bottleneck (residual blocks), and the other part directly skips these residual blocks for output. Specifically, after the first feature map, the second feature map and the first down-sampling feature map are input into the C3 module, they will be divided into two parts. One part skips the residual block and is directly output, that is, the original first feature map, the second feature map and the first down-sampling feature map are respectively input into the second convolution layer, the first down-sampling layer and the second down-sampling layer, and the other part is processed by multiple residual blocks in the C3 module to obtain the first enhanced feature map, the second enhanced feature map and the first fused feature map, and then the three and the second fused feature map output by the last C3 module are input into the feature pyramid module, that is, the enhanced fused feature map is multi-scale fused. This design of feature separation and fusion can reduce redundant calculations, improve the expression ability of the network, and retain more original information of the textile. The residual block inside the C3 module uses deep convolution to extract features, which enhances the ability to capture features of different scales. By stacking multiple residual blocks, the C3 module can capture deeper feature representations. Residual connections can alleviate the gradient vanishing problem in deep networks and promote the effective transmission of information and the accumulation of features layer by layer. This connection method helps the model better learn complex feature representations and improve detection accuracy.

[0061] The first downsampled feature map is input into the second downsampled layer to obtain the second downsampled feature map, and the second downsampled feature map is input into the C3 module to obtain the second fused feature map. Specifically, the convolutional decoupling module replaces the last two standard convolutional layers of the traditional backbone network by introducing a spatial channel decoupling downsampling layer. First, point-by-point convolution is used to adjust the channel dimension, and then standard convolution is used to perform spatial downsampling. After being processed by the convolutional decoupling module, the spatial resolution of the feature map is increased from 1.11 to 1.11. becomes , the number of channels from becomes The spatial channel decoupling downsampling layer includes a first downsampling layer and a second downsampling layer. In the downsampling process, by decoupling the spatial and channel information, the information of the textile surface image is retained to the maximum extent, while reducing redundant parameters and improving the performance of the defect detection model.

[0062] Step 104: Input the enhanced fused feature map into the feature pyramid network for multi-scale fusion to obtain a multi-scale feature map, and use the bidirectional feature pyramid network to enhance the multi-scale feature map to obtain an enhanced multi-scale feature map. In an embodiment of the present application, a spatial pyramid pooling (SPPF) module is used in the feature pyramid network to process the first enhanced feature map, the second enhanced feature map, the first fused feature map and the second fused feature map output by the last C3 module after being processed by the first three C3 modules, and retain all information in the feature map by element-by-element addition, and automatically adjust the importance of different features through the learning of the feature pyramid network. The spatial pyramid pooling module uses pooling kernels of different sizes to perform multi-scale fusion of the information in the enhanced fused feature map to obtain a multi-scale feature map, which can enhance the defect detection model's ability to detect defects of different sizes.

[0063] In order to better capture features at different scales, a bidirectional feature pyramid network (BiFPN) is introduced. The multi-scale feature map processed by the feature pyramid network is input into the bidirectional feature pyramid network, which can effectively fuse the feature maps of different scales output at different stages to obtain enhanced multi-scale feature maps, thereby further improving the detection network's ability to detect defects of different sizes. The weighted fusion strategy is used to enhance the expression ability of multi-scale feature maps at different scales. The feature capture of the bidirectional feature pyramid network is carried out in both the upper and lower directions. In the top-down path, the high-level feature map is transferred to the low-level feature map layer by layer, that is, the low-resolution but semantically rich textile surface image is transferred to the high-resolution and detailed textile surface image. Through the upsampling operation, the high-level feature map is upsampled to the same resolution as the low-level feature map, and then the upsampled high-level feature map is fused with the corresponding low-level feature map. In the bottom-up path, the low-level feature map is transferred to the high-level feature map layer by layer, as follows:

[0064] ;

[0065] In the formula, Represents the feature fusion result of the bidirectional feature pyramid network, that is, the enhanced multi-scale feature map, and represents the learnable weights, Indicates The high-level feature maps of the input bidirectional feature pyramid network, Indicates The low-level feature maps of the input bidirectional feature pyramid network, represents the number of all feature maps, Represents the multi-scale feature map of the input, Represents the different stages in the convolutional decoupling module in the defect detection model, Represents a smaller value to avoid numerical instability.

[0066] Step 105: Input the enhanced multi-scale feature map into the detection layer to obtain the detection result. Specifically, the enhanced multi-scale feature map output by the bidirectional feature pyramid network is input into the detection layer to perform defect detection. The detection layer generates the detection result based on the fused enhanced multi-scale feature map, such as Figure 4 The figure shows an example of the result of defect detection on a textile surface image, including the location and category information of the defect.

[0067] Although the present application provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in this embodiment is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in sequence or in parallel according to the method shown in this embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment).

[0068] Step 106: Whether the loss function converges. If the convergence condition of the loss function is met, the defect detection model training is terminated and step 107 is executed. Otherwise, training steps 102 to 105 are continued until the convergence condition of the loss function is met.

[0069] Step 107: Perform defect detection using the trained defect detection model to obtain a defect detection result for the textile.

[0070] like Figure 5 As shown, the embodiment of the present application also provides a device 500 for defect detection based on a neural network. The device comprises: a model building module 501, a decomposition and transformation module 502, a feature extraction and fusion module 503 and a detection module 504, which are specifically as follows.

[0071] The model building module 501 is used to build a defect detection model, and input the textile surface image into the defect detection model for model training until the loss function converges, and use the trained defect detection model to perform defect detection to obtain the defect detection result of the textile; wherein the defect detection model includes a decomposition transformation module, a convolution decoupling module, a feature pyramid network, a bidirectional feature pyramid network and a detection layer.

[0072] The decomposition and transformation module 502 is used to input the textile surface image into the decomposition and transformation module for decomposition to obtain component information, and fuse the information to generate an initial feature map.

[0073] The feature extraction and fusion module 503 is used to use the convolution decoupling module to extract and fuse the initial feature map to obtain an enhanced fused feature map; wherein the enhanced fused feature map includes a first enhanced feature map, a second enhanced feature map, a first fused feature map and a second fused feature map; the enhanced fused feature map is input into the feature pyramid network for multi-scale fusion to obtain a multi-scale feature map, and the multi-scale feature map is enhanced by the bidirectional feature pyramid network to obtain an enhanced multi-scale feature map.

[0074] The detection module 504 is used to input the enhanced multi-scale feature map into the detection layer to obtain a detection result.

[0075] The devices or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described in various modules according to their functions. When implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, the module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0076] The method, device or module in the present application can be implemented in the form of a computer-readable program code. The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (English: Application Specific Integrated Circuit; Abbreviation: ASIC), a programmable logic controller and an embedded microcontroller. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, AtmelAT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included in it for implementing various functions can also be regarded as structures within the hardware component. Or even, the means for realizing various functions may be regarded as both a software module for realizing the method and a structure within a hardware component.

[0077] An embodiment of the present application also provides a device for executing a neural network-based defect detection method, the device comprising: a processor; a memory for storing processor executable instructions; when the processor executes the executable instructions, the method of the embodiment of the present application is implemented.

[0078] The embodiments of the present application also provide a non-volatile computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed, the method in the embodiments of the present application is implemented.

[0079] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist independently, or two or more modules may be integrated into one module.

[0080] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary hardware. Based on such an understanding, the technical solution of the present application can be essentially or partly reflected in the prior art in the form of a software product, or it can be reflected in the implementation process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods of each embodiment of the present application or some parts of the embodiments.

[0081] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of this application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0082] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.

Claims

1. A defect detection method based on neural network, characterized in that: include: Constructing a defect detection model, inputting the textile surface image into the defect detection model for model training until the loss function converges, and performing defect detection using the trained defect detection model to obtain a defect detection result of the textile; wherein the defect detection model includes a decomposition transformation module, a convolution decoupling module, a feature pyramid network, a bidirectional feature pyramid network and a detection layer; The textile surface image is input into the decomposition transformation module to be decomposed to obtain component information, and the component information is fused to generate an initial feature map; wherein the initial feature map includes: Performing wavelet decomposition on the textile surface image to obtain the component information; wherein the component information includes a low-frequency component, a horizontal high-frequency component, a vertical high-frequency component and a diagonal high-frequency component; Fusing the component information to generate the initial feature map containing both low-frequency information and high-frequency information; The convolution decoupling module is used to extract and fuse the initial feature map to obtain an enhanced fused feature map; wherein the convolution decoupling module includes a first convolution layer, a second convolution layer, a first downsampling layer, a second downsampling layer and a plurality of C3 modules; the enhanced fused feature map includes a first enhanced feature map, a second enhanced feature map, a first fused feature map and a second fused feature map; the convolution decoupling module is used to extract and fuse the initial feature map to obtain an enhanced fused feature map, including: Inputting the initial feature map into the first convolutional layer to perform preliminary feature extraction to obtain a first feature map; Inputting the first feature map into the C3 module to obtain the primary enhanced feature map; Inputting the first feature map into the second convolutional layer for further feature extraction to obtain a second feature map; Inputting the second feature map into the C3 module to obtain the secondary enhanced feature map; Inputting the second feature map into the first downsampling layer to obtain a first downsampling feature map; Inputting the first down-sampled feature map into the C3 module to obtain the first fused feature map; Inputting the first down-sampling feature map into the second down-sampling layer to obtain a second down-sampling feature map; Inputting the secondary down-sampled feature map into the C3 module to obtain the second fused feature map; Inputting the first enhanced feature map, the second enhanced feature map and the first fused feature map into the feature pyramid network; Inputting the enhanced fused feature map into the feature pyramid network for multi-scale fusion to obtain a multi-scale feature map, and using the bidirectional feature pyramid network to enhance the multi-scale feature map to obtain an enhanced multi-scale feature map; The enhanced multi-scale feature map is input into the detection layer to obtain a detection result.

2. The method according to claim 1, characterized in that The initial feature map is generated by fusing them as follows: ； In the formula, represents the initial feature map, represents the weight coefficient of the low-frequency component, represents the low-frequency component, represents the weight coefficient of the horizontal high frequency component, represents the horizontal high frequency component, represents the weight coefficient of the vertical high-frequency component, represents the vertical high-frequency component, represents the weight coefficient of the diagonal high frequency component, represents the diagonal high frequency component.

3. The method according to claim 1, characterized in that The loss function is as follows: ； in, ; ； In the formula, represents the loss function, Represents the intersection-over-union ratio of the predicted box and the true box, The penalty term representing the center distance between the predicted box and the true box, Represents the aspect ratio penalty term between the predicted box and the true box, Represents the distance between the center point of the predicted box and the real box, Represents the diagonal distance of the minimum closed box covering the predicted box and the true box, Indicates the width of the real box, Indicates the height of the real frame, Indicates the width of the prediction box, Indicates the height of the prediction box.

4. The method according to claim 1, characterized in that: The step of inputting the textile surface image into the defect detection model for model training until the loss function converges further comprises: Iterate the model parameter optimization steps until the value of the loss function meets the convergence condition; The model parameter optimization step comprises: The value of the loss function is calculated through forward propagation, and the gradient of the loss function with respect to the model parameters is determined through backpropagation; The model parameters are updated using an optimization algorithm based on the gradient.

5. A device for defect detection based on neural network, characterized in that: include: A model building module, used to build a defect detection model, input the textile surface image into the defect detection model for model training until the loss function converges, and use the trained defect detection model to perform defect detection to obtain the defect detection result of the textile; wherein the defect detection model includes a decomposition transformation module, a convolution decoupling module, a feature pyramid network, a bidirectional feature pyramid network and a detection layer; A decomposition transformation module is used to input the textile surface image into the decomposition transformation module for decomposition to obtain component information, and fuse the component information to generate an initial feature map; wherein the initial feature map includes: Performing wavelet decomposition on the textile surface image to obtain the component information; wherein the component information includes a low-frequency component, a horizontal high-frequency component, a vertical high-frequency component and a diagonal high-frequency component; Fusing the component information to generate the initial feature map containing both low-frequency information and high-frequency information; A feature extraction and fusion module, used to use the convolution decoupling module to extract and fuse the initial feature map to obtain an enhanced fused feature map; wherein the convolution decoupling module includes a first convolution layer, a second convolution layer, a first downsampling layer, a second downsampling layer and a plurality of C3 modules; the enhanced fused feature map includes a first enhanced feature map, a second enhanced feature map, a first fused feature map and a second fused feature map; the use of the convolution decoupling module to extract and fuse the initial feature map to obtain an enhanced fused feature map includes: Inputting the initial feature map into the first convolutional layer to perform preliminary feature extraction to obtain a first feature map; Inputting the first feature map into the C3 module to obtain the primary enhanced feature map; Inputting the first feature map into the second convolutional layer for further feature extraction to obtain a second feature map; Inputting the second feature map into the C3 module to obtain the secondary enhanced feature map; Inputting the second feature map into the first downsampling layer to obtain a first downsampling feature map; Inputting the first down-sampled feature map into the C3 module to obtain the first fused feature map; Inputting the first down-sampling feature map into the second down-sampling layer to obtain a second down-sampling feature map; Inputting the secondary down-sampled feature map into the C3 module to obtain the second fused feature map; Inputting the first enhanced feature map, the second enhanced feature map and the first fused feature map into the feature pyramid network; Inputting the enhanced fused feature map into the feature pyramid network for multi-scale fusion to obtain a multi-scale feature map, and using the bidirectional feature pyramid network to enhance the multi-scale feature map to obtain an enhanced multi-scale feature map; The detection module is used to input the enhanced multi-scale feature map into the detection layer to obtain a detection result.

6. A device for performing a defect detection method based on a neural network, characterized in that: include: processor; a memory for storing processor-executable instructions; When the processor executes the executable instructions, the method according to any one of claims 1 to 4 is implemented.

7. A non-volatile computer-readable storage medium, characterized in that: The device comprises a computer program or an instruction for storing the computer program or the instruction, which, when executed, enables the method according to any one of claims 1 to 4 to be implemented.

Images