Feature extraction method and detection system based on directional space and multi-scale feature extraction network

Abstract

The invention discloses a feature extraction method based on a directional space and a multi-scale feature extraction network and a detection system, relates to the technical field of cloth flaw detection, and solves the problem of low feature extraction precision when feature extraction is performed on fine flaws with different shapes in the prior art. The method comprises the steps that a textile cloth image is acquired, feature extraction is conducted on the textile cloth image through a directional space and multi-scale feature extraction module, and the directional space and multi-scale feature extraction module carries out feature extraction on the textile cloth image; the directional space and multi-scale feature extraction module adopts a multi-branch parallel gradient stacking mode to arrange an information flow of network internal feature extraction, and a re-parameterization convolution module is adopted as a branch feature extraction component in the directional space and multi-scale feature extraction module, so that spatial features of warp and weft flaws can be extracted more effectively; therefore, the feature extraction is more accurate, and the defect features from the global to the local and from the small scale to the large scale can be captured.

Description

Technical Field

[0001] The present application relates to the technical field of cloth defect detection, and in particular to a feature extraction method and a detection system based on a directional space and multi-scale feature extraction network. Background Art

[0002] As an indispensable raw material in daily life and industrial production, the quality of textile fabrics directly affects the durability and reliability of the final product. Traditional fabric defect detection methods mainly rely on offline detection by artificial vision, which is not only inefficient, but also easily affected by subjective factors. With the rapid development of the textile industry, automated and intelligent fabric defect detection technology has emerged and has become the key to fabric production quality control in the textile industry. In recent years, breakthroughs in deep learning technology have provided strong support for efficient and accurate textile fabric defect detection. By constructing a detection model based on deep learning, the detection speed and accuracy can be greatly improved, while reducing labor costs and subjective errors. Although recent research has made some progress in improving the performance of fabric defect detection, it still has shortcomings when dealing with complex backgrounds, especially when dealing with defects of various shapes and subtle defects. These studies have failed to fully combine the natural morphological characteristics of textile fabric defects for in-depth analysis, resulting in insufficient coupling between target detection and defect characteristics in various detection methods.

[0003] In order to improve the accuracy and precision of detection, it is necessary to ensure that various defects can be detected. Currently, there are two main types of textile fabric defect detection technologies: image segmentation-based methods and object regression methods. However, segmentation-based methods have limitations when processing images with complex textures and changing backgrounds. Because they rely on precise pixel-level annotation and template matching, each defect requires an accurate pixel template, which is costly and time-consuming in actual industrial applications and is not suitable for use in a flexible weaving production environment with multiple varieties and changes. The multi-stage processing flow of the object regression method results in a slow detection speed, which makes it difficult to meet the real-time application scenarios with high delay sensitivity such as fabric quality inspection. In addition, there are significant differences in the morphology and size of warp and weft defects, which usually show extreme aspect ratios and long-distance dependence characteristics. The spatial characteristics of the defects play a key role in the feature extraction performance of the detection model. These factors have an important impact on the overall detection performance of the model. Summary of the invention

[0004] The purpose of the present application is to overcome the problem of low accuracy of feature extraction in the prior art when extracting features of defects of various shapes and subtle defects, and to provide a feature extraction method and detection system based on directional space and multi-scale feature extraction network.

[0005] In a first aspect, a feature extraction method based on a directional space and a multi-scale feature extraction network is provided, comprising:

[0006] Acquire textile cloth images;

[0007] The textile cloth image is subjected to feature extraction using a directional space and multi-scale feature extraction module, wherein the directional space and multi-scale feature extraction module arranges the information flow of feature extraction within the network in a multi-branch parallel gradient stacking manner, and a re-parameterized convolution module is used as a branch feature extraction component in the directional space and multi-scale feature extraction module, and the branch feature extraction component includes a structural re-parameterized convolution RepConv and a multi-scale coordinate weight guidance module, and the structural re-parameterized convolution RepConv uses a multi-branch structure to enrich the expression level of the defect feature map in the training stage, captures defect features from global to local, from small scale to large scale, and converts the multi-branch structure into a single branch form through a re-parameterized design in the inference stage, and the multi-scale coordinate weight guidance module adopts a parallel strategy and a multi-scale attention mechanism to perform spatial direction aggregation features along the radial, latitudinal, small target convolution kernel and large target convolution kernel dimensions of the input feature map.

[0008] In some possible implementations, the processing of the structured reparameterized convolution RepConv includes:

[0009] Perform weighted summation of the input tensor through the weight matrix;

[0010] Normalization is performed through a batch normalization layer;

[0011] Scaling and offset are performed through learnable adjustable parameters to obtain the extracted convolution kernel weight parameters and bias.

[0012] In some possible implementations, the processing of the multi-scale coordinate weight guidance module includes:

[0013] The input features are spatially averaged and pooled in four dimensions: longitude, latitude, small scale, and large scale to obtain compressed spatial perception features.

[0014] The compressed spatial perception features are re-encoded and refined through the information encoding layer;

[0015] Channel-aligned convolution and Sigmod function are used to generate the coordinate attention weights of the four dimensions of feature information;

[0016] The generated coordinate attention weights are applied to the input features to achieve enhanced extraction of defect features at all scales.

[0017] In some possible implementations, before using the directional space and multi-scale feature extraction module to extract features from the textile cloth image, it also includes: preprocessing the input textile cloth image, and the preprocessing includes image enhancement and size adjustment.

[0018] In a second aspect, a detection system based on a directional space and multi-scale feature extraction network is provided, comprising:

[0019] A feature extraction module, used to extract features of a textile cloth image using the method described in any one of the implementations of the first aspect above;

[0020] The defect classification and positioning module is used to identify, classify and locate defects based on the feature data extracted by the feature extraction module.

[0021] In some possible implementations, the method further includes: a preprocessing module, which is used to preprocess the input textile cloth image before the feature extraction module performs feature extraction, and the preprocessing includes image enhancement and size adjustment.

[0022] In some possible implementations, the system further includes: a display module for visually displaying the results of defect identification, classification, and positioning.

[0023] In a third aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

[0024] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable medium stores a program code for execution by a device, wherein the program code includes steps for executing the method in any one of the implementations of the first aspect described above.

[0025] In a fifth aspect, an electronic device is provided, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements a method as in any one of the implementations in the first aspect above.

[0026] This application has the following beneficial effects:

[0027] 1. The directional space and multi-scale feature extraction module of the present application can more effectively extract the spatial features of the defects in the longitude and latitude directions, especially for the defects with extreme aspect ratio and long-distance dependence characteristics, which significantly improves the detection accuracy of the model for such defects. And after the introduction of the directional space and multi-scale feature extraction module, the model can focus more on the target defects, feature extraction is more accurate, the interference of background noise is reduced, and the false detection rate is reduced;

[0028] 2. This application can capture defect features at different scales through convolution attention weight operations in multi-scale spatial dimensions, and enhance the adaptability to diversified defects. Secondly, the introduction of structural re-parameterized convolution RepConv and multi-scale coordinate weight guidance module can enable the model to make full use of the multi-branch structure in the training stage to enrich the expression level of the cloth defect feature map, and can capture defect features from global to local, from small scale to large scale, effectively improving the ability of feature extraction;

[0029] 3. The detection system of the present application pays more attention to target defects, extracts features more accurately, reduces interference from background noise, and reduces false detection rates. Secondly, in the reasoning stage, the re-parameterized design transforms the multi-branch structure into a more compact single-branch form. This process can effectively improve reasoning efficiency without reducing detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application.

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 is a flow chart of a feature extraction method based on directional space and multi-scale feature extraction network according to Embodiment 1 of the present application;

[0033] Figure 2 It is a structural diagram of a directional space and multi-scale feature extraction module in the feature extraction method based on a directional space and multi-scale feature extraction network in Example 1 of the present application;

[0034] Figure 3 It is a structural diagram of a multi-scale coordinate weight guidance module in a feature extraction method based on directional space and multi-scale feature extraction network in Example 1 of the present application;

[0035] Figure 4 It is a schematic diagram of the convolution attention weight calculation principle of the latitude and longitude space and multi-scale space dimensions in the feature extraction method based on the directional space and multi-scale feature extraction network of Example 1 of the present application;

[0036] Figure 5 It is a structural block diagram of a detection system based on directional space and multi-scale feature extraction network of Example 2 of the present application;

[0037] Figure 6 It is a backbone network feature extraction attention heat map of the detection system based on directional space and multi-scale feature extraction network in Example 2 of the present application;

[0038] Figure 7 It is a schematic diagram of the internal structure of the electronic device of Example 5 of the present application.

[0039] Reference numerals:

[0040] 100, feature extraction module; 200, defect classification and positioning module; 300, preprocessing module; 400, display module. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Example 1

[0043] like Figure 1 As shown, a feature extraction method based on directional space and multi-scale feature extraction network involved in Example 1 of the present application includes:

[0044] S100, obtaining textile fabric images, i.e., using a camera, a still camera, or a camcorder to collect image data containing textile fabric defects. Of course, during the model training stage, crawlers can also be used to crawl pictures of textile fabric defects from the Internet, especially pictures of textile fabrics containing slender defects with extreme aspect ratios.

[0045] In a further embodiment, the acquired textile cloth image is preprocessed, including image enhancement, size adjustment, etc.

[0046] S200: extracting features from the textile cloth image using a directional space and multi-scale feature extraction module.

[0047] like Figure 2 As shown in the figure, the oriented spatial and multi-scale feature extraction module (i.e., OSMSFEM) adopts a multi-branch parallel gradient stacking approach to arrange the information flow of feature extraction within the network to efficiently capture the semantic and spatial information of different defect features.

[0048] In OSMSFEM, a reparameterized convolution module (i.e., Rep MSCWG) is used as the branch feature extraction component;

[0049] like Figure 2 As shown in the figure, the Rep MSCWG structure and internal Bottleneck can be seen. This component includes the structural re-parameterized convolution RepConv and the multi-scale coordinate weight guidance module.

[0050] The introduction of the structural reparameterized convolution RepConv enables the model to make full use of the multi-branch structure in the training stage to enrich the expression level of the fabric defect feature map and capture defect features from global to local and from small scale to large scale. In the inference stage, the reparameterized design transforms the multi-branch structure into a more compact single-branch form. This process can effectively improve the inference efficiency without reducing the model detection performance. The reparameterization process is carried out by extracting the parameters of the convolution layer and the batch normalization layer. Specifically:

[0051] First, the convolutional layer is calculated, and the calculation process is shown in the following formula:

[0052]

[0053] In the above formula: represents the input tensor, ; is a weight matrix used to perform weighted summation on the input signal; represents the sample mean; represents the variance of the sample; and are learnable adjustment parameters, corresponding to the scaling factor and offset factor respectively.

[0054] Based on the above formula, let:

[0055]

[0056] In the above formula: and They are the extracted convolution kernel weight parameters and bias respectively.

[0057] The final result of convolution and BN layer extraction is:

[0058]

[0059] Based on the above calculations, the RepConv multi-branch structure in the training phase is finally converted into a single-branch structure through structural reparameterization.

[0060] like Figure 3As shown in the figure, the multi-scale coordinate weighted guidance module (i.e., Multi-scale coordinate weighted guidance module) adopts a parallel strategy and a multi-scale attention mechanism to aggregate features in the spatial direction along the radial and latitudinal directions of the input feature map, as well as the small target convolution kernel and the large target convolution kernel. This can enhance the model's attention to defects of various scales, thereby improving the model's defect feature extraction capability and further enhancing the model's defect detection performance. Specifically:

[0061] like Figure 3 As shown, first, the input feature The coordinates of spatial information are embedded by compressing the spatial perception features into length C / 4 through the 4-dimensional spatial average pooling layers of longitude, latitude, small scale and large scale.

[0062] Then, after the information is re-encoded and refined by the information coding layer CBH (channel compression convolution, batch normalization layer and hard-swish non-linear activation function), channel alignment convolution and Sigmod are used to generate the coordinate attention weights of the four dimensions of the feature information.

[0063] like Figure 4 As shown in the figure, the generation process of attention weights of four dimensional directions and spatial perception information is shown. The convolution attention weight operation of the longitude and latitude spatial dimensions uses a size of and The convolution kernel of the input features The spatial latitude and radial directions are globally averaged and pooled, and the feature tensor with latitude and longitude spatial perception is output. The spatial feature formulas in the latitude and radial directions can be expressed as:

[0064]

[0065] In the above formula: is the characteristic element of the latitudinal characteristic tensor at height h in the cth channel; is the characteristic element of the radial feature tensor at width w in the cth channel; H and W represent the height and width of the original input feature size respectively; Represents the feature distribution of the cth channel of the input feature.

[0066] The generated latitudinal and radial attention weights are formulated as follows:

[0067]

[0068] In the above formula: and are the attention weights of spatial perception information in the latitudinal and radial directions respectively; It is a point convolution block with a kernel of 1X1 and a stride of 1, which is used for channel unification; Information encoding layer (channel compression convolution, batch normalization layer and hard-swish nonlinear activation function); is the Sigmod activation function, which is used to finally generate the spatial attention weights.

[0069] Convolutional attention weight operations in multi-scale spatial dimensions use 3X3 convolution and 5X5 convolution to check input features. The global average pooling operation is performed on the small and large scales, and the feature tensor with large and small scale spatial perception is output. According to the specific calculation process of the longitude and latitude spatial dimensions mentioned above, the convolution attention weights of multi-scale spatial dimensions can be obtained in the same way.

[0070] Example 2

[0071] like Figure 5 As shown, a detection system based on a directional space and multi-scale feature extraction network involved in Example 2 of the present application is improved and trained with YOLOv5s as the base model in this embodiment, and the detection system includes:

[0072] The preprocessing module 300 is used to preprocess the input textile cloth image before the feature extraction module 100 performs feature extraction, and the preprocessing includes image enhancement and size adjustment.

[0073] A feature extraction module 100, configured to extract features of a textile cloth image using a method as described in any one of the embodiments of the first embodiment;

[0074] The defect classification and positioning module 200 is used to identify, classify and locate defects according to the feature data extracted by the feature extraction module.

[0075] The display module 400 is used to visually display the results of defect identification, classification and positioning.

[0076] like Figure 6 As shown, the first column is the input textile cloth image, the second column is the attention heat map extracted by the detection system in this embodiment (i.e., the feature extraction effect), and the third column is the backbone feature extraction effect of the benchmark model yolov5. It can be seen that the detection system in this embodiment can effectively extract features of slender defects and tiny defects, as shown in the following Table 1, which is an effectiveness analysis table of the detection system in this embodiment:

[0077] Table 1:

[0078] Evaluation Metrics YOLOv5s + OSMSFEM Baseline(YOLOv5s) P(%) 70.3 63.3 R(%) 72.0 67.1 mAP@.5(%) 71.8 68.4 F1(%) 71.1 65.1 WeightSize(MB) 15.9 14.5 Parameters(MB) 7.6 7.0 FPS 31 37

[0079] As can be seen from Table 1, in this embodiment, by introducing re-parameterized convolution and designing a multi-scale coordinate weight module, the cloth defect detection performance is significantly improved. The system accurately captures warp, weft and multi-scale features, and adapts to complex forms and small target defects. Key indicators such as accuracy increased from 63.3% to 70.3%, and mAP@.5 increased from 68.4% to 71.8%. At the same time, re-parameterization optimizes reasoning efficiency, reduces false detection rate, enhances system robustness and real-time performance, and is suitable for diverse industrial scenarios.

[0080] It should be noted that the detection system of this embodiment pays more attention to the target defects, the feature extraction is more accurate, the interference of background noise is reduced, and the false detection rate is reduced. Secondly, in the reasoning stage, the re-parameterization design transforms the multi-branch structure into a more compact single-branch form, which not only improves the detection performance and feature extraction capabilities, but also enhances the robustness of the system while maintaining computational efficiency, thereby enhancing the defect detection performance of the system. For other specific implementations of the detection system based on directional space and multi-scale feature extraction network in this embodiment, please refer to the specific implementation of the feature extraction method based on directional space and multi-scale feature extraction network mentioned above. To avoid redundancy, they will not be repeated here.

[0081] Example 3

[0082] A computer program product involved in Example 3 of the present application is used to store a computer program. When the computer program runs on a computer, the method in any one of the implementation modes in Example 1 of the present application is implemented.

[0083] Example 4

[0084] A computer-readable storage medium according to Embodiment 4 of the present application, wherein the computer-readable storage medium stores a program code for execution by a device, wherein the program code includes steps for executing a method in any one of the implementations in Embodiment 1 of the present application;

[0085] Among them, the computer-readable storage medium can be a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM); the computer-readable storage medium can store program code, and when the program stored in the computer-readable storage medium is executed by the processor, the processor is used to execute the steps of the method in any one of the implementation methods in Example 1 of the present application.

[0086] Example 5

[0087] like Figure 7As shown, an electronic device involved in Embodiment 5 of the present application includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and when the program or instruction is executed by the processor, a method in any one of the implementations in Embodiment 1 of the present application is implemented;

[0088] Among them, the processor can adopt a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the method in any one of the implementation methods in Example 1 of the present application.

[0089] The processor may also be an integrated circuit electronic device with signal processing capability. In the implementation process, each step of the method in any implementation of Embodiment 1 of the present application may be completed by an integrated logic circuit of hardware in the processor or by instructions in software form.

[0090] The above-mentioned processor may also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the functions required to be performed by the unit included in the data processing device of the embodiment of the present application in combination with its hardware, or executes the method in any one of the implementation modes in the embodiment 1 of the present application.

[0091] The above are only preferred specific implementations of the present application; however, the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and its improved ideas of the present application within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A feature extraction method based on directional space and multi-scale feature extraction network, characterized in that: include: Acquire textile cloth images; The textile cloth image is subjected to feature extraction using a directional space and multi-scale feature extraction module, wherein the directional space and multi-scale feature extraction module arranges the information flow of feature extraction within the network in a multi-branch parallel gradient stacking manner, and a re-parameterized convolution module is used as a branch feature extraction component in the directional space and multi-scale feature extraction module, and the branch feature extraction component includes a structural re-parameterized convolution RepConv and a multi-scale coordinate weight guidance module, and the structural re-parameterized convolution RepConv uses a multi-branch structure to enrich the expression level of the defect feature map in the training stage, captures defect features from global to local, from small scale to large scale, and converts the multi-branch structure into a single branch form through a re-parameterized design in the inference stage, and the multi-scale coordinate weight guidance module adopts a parallel strategy and a multi-scale attention mechanism to perform spatial direction aggregation features along the radial, latitudinal, small target convolution kernel and large target convolution kernel dimensions of the input feature map.

2. The feature extraction method based on directional space and multi-scale feature extraction network according to claim 1, characterized in that: The processing process of the structured reparameterized convolution RepConv includes: Perform weighted summation of the input tensor through the weight matrix; Normalization is performed through a batch normalization layer; Scaling and offset are performed through learnable adjustable parameters to obtain the extracted convolution kernel weight parameters and bias.

3. The feature extraction method based on directional space and multi-scale feature extraction network according to claim 1, characterized in that: The processing process of the multi-scale coordinate weight guidance module includes: The input features are spatially averaged and pooled in four dimensions: longitude, latitude, small scale, and large scale to obtain compressed spatial perception features. The compressed spatial perception features are re-encoded and refined through the information encoding layer; Channel-aligned convolution and Sigmod function are used to generate the coordinate attention weights of the four dimensions of feature information; The generated coordinate attention weights are applied to the input features to achieve enhanced extraction of defect features at all scales.

4. The feature extraction method based on directional space and multi-scale feature extraction network according to claim 1, characterized in that: Before extracting features from the textile cloth image using the directional space and multi-scale feature extraction module, the method further includes: preprocessing the input textile cloth image, wherein the preprocessing includes image enhancement and size adjustment.

5. A detection system based on directional space and multi-scale feature extraction network, characterized in that: include: A feature extraction module, used for extracting features of a textile cloth image using the method according to any one of claims 1 to 3; The defect classification and positioning module is used to identify, classify and locate defects based on the feature data extracted by the feature extraction module.

6. The detection system based on directional space and multi-scale feature extraction network according to claim 5, characterized in that: Also includes: The preprocessing module is used to preprocess the input textile cloth image before the feature extraction module performs feature extraction, and the preprocessing includes image enhancement and size adjustment.

7. The detection system based on directional space and multi-scale feature extraction network according to claim 5, characterized in that: Also includes: The display module is used to visually display the results of defect identification, classification and positioning.

8. A computer program product, characterized in that The computer program product stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code for execution by a device, wherein the program code includes steps for executing the method according to any one of claims 1 to 4.

10. An electronic device, characterized in that: The electronic device comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction implements the method as claimed in any one of claims 1 to 4 when executed by the processor.

Images