A wood surface defect detection method, system, medium and equipment

Abstract

The present invention proposes a method, system, medium and equipment for wood surface defect detection, which belongs to the technical field of wood surface defect detection. The method includes: obtaining an image of eucalyptus veneer and performing preprocessing; inputting the preprocessed image into a wood defect recognition network, extracting the feature map of the image using an anti-depth separable stem module and performing feature aggregation to obtain a first feature map; extracting global features through a rectangular self-calibration module to form a second feature map, and then using an adaptive frequency attention network module to extract the key frequency components of the second feature map and output a third feature map; inputting the third feature map into a frequency enhancement channel attention module to obtain a fourth feature map; fusing the fourth feature map with the first feature map to obtain a fifth feature map and predicting it through a segmentation head, and outputting the final identified bark defect area. The problem of poor network feature extraction effect and low semantic segmentation accuracy caused by bark defects in the process of wood defect detection is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of wood surface defect detection, and in particular relates to a wood surface defect detection method, system, medium and equipment. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Eucalyptus veneer is a natural and environmentally friendly material, mainly used in the production of plywood. However, during the production process, defects on the surface of the veneer will greatly affect its commercial value and practical value. In particular, bark defects not only affect the bonding performance of plywood, but may also cause inconsistent thickness of the board, thereby affecting the structural strength and stability of the product. At present, the detection of wood surface defects mainly relies on manual labor, which not only leads to low detection efficiency, but also seriously wastes wood resources.

[0004] In order to solve this problem, many non-destructive wood defect detection technologies have been developed, such as ultrasound, stress wave, computer tomography, laser scanning, etc. Although these methods have high detection accuracy in theory, there are some challenges in practical application: first, they usually require expensive professional equipment, resulting in high detection costs; second, these technologies are slow and inefficient when conducting large-scale inspections; in addition, these methods usually require complex operations and professional personnel for analysis, and are not suitable for online real-time detection in industrial production.

[0005] With the rise of convolutional neural networks (CNN), computer vision solutions can better solve the above problems. In recent years, researchers have proposed many detection methods based on deep learning, such as the patent with announcement number CN114359235A, which discloses a wood surface defect detection method based on an improved YOLOv5l network. Although the above method can detect wood defects more accurately and quickly, the bark on the wood veneer often overlaps with multiple defects, and the data set used by the above method mainly annotates various defects through anchor frames, which makes it easy for these methods to extract wrong features when dealing with multiple overlapping targets, thereby reducing the recognition accuracy.

[0006] Although traditional semantic segmentation methods can process each pixel independently to extract defect features more accurately, they can only achieve relatively good accuracy on defects such as dead knots, live knots and cracks with relatively uniform features. Complex defects such as bark with diverse colors, shapes and textures will cause the network to extract its features poorly, reducing the accuracy of semantic segmentation. Bark defects refer to the bark remaining on the veneer when the wood is processed into wood veneer by rotary cutting. Most of them are irregular long oval strips and are mostly dark brown, light brown and gray in color. On the other hand, the current models for semantic segmentation of wood defects usually have a large number of parameters and high computational costs, which limits their feasibility in practical applications. Summary of the invention

[0007] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a method, system, medium and equipment for detecting wood surface defects.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] The first aspect of the present invention provides a wood surface defect detection method, which adopts a lightweight frequency enhanced semantic segmentation network (LFENet) based on AFFormer improvement, takes texture features as the entry point, uses an inverse depth separable stem (IDS-Stem) to extract texture information in the feature map, and uses a rectangular self-calibrated adaptive frequency attention network (RCAF) to extract important frequency information in the feature map. Finally, a frequency enhanced channel attention module (FECA) is used to restore the local detail information lost during downsampling and make full use of the frequency information in the feature map of each stage, providing more effective feature support for the segmentation head.

[0010] A method for detecting wood surface defects, comprising:

[0011] Acquire eucalyptus veneer images and perform preprocessing;

[0012] The preprocessed eucalyptus veneer image is input into a wood defect recognition network, a feature map of the eucalyptus veneer image is extracted using an inverse depth separable stem module, and feature aggregation is performed on the feature map to obtain a first feature map;

[0013] The first feature map is input into the rectangular self-calibration adaptive frequency attention network module of the wood defect recognition network. The rectangular self-calibration module extracts global features from both horizontal and vertical directions to form a second feature map. Then, a four-layer adaptive frequency attention network module is used to extract the key frequency components of the second feature map and enhance it. Each layer of the adaptive frequency attention network module outputs a third feature map containing frequency information that is beneficial to semantic segmentation.

[0014] The third feature map output by each layer of the adaptive frequency attention network module, which contains frequency information that is beneficial to semantic segmentation, is input into the frequency enhancement channel attention module, the frequency information of each layer is fused and the local information is restored to obtain the fourth feature map, the fourth feature map is fused with the first feature map to obtain the fifth feature map, the fifth feature map is input to the segmentation head for prediction, and the final identified bark defect area is output.

[0015] As a further technical solution, the pretreatment process is:

[0016] Select images with bark defects from the acquired eucalyptus veneer images and annotate them using Roboflow;

[0017] The annotated images are randomly rotated and spliced ​​for data augmentation and the image size is adjusted.

[0018] As a further technical solution, the process of extracting the feature map of the eucalyptus veneer image using the inverse depth separable stem module and performing feature aggregation on the feature map to obtain the first feature map is as follows:

[0019] The eucalyptus veneer image is input into the down-sampled anti-depth separable convolutional block of the first layer of the anti-depth separable stem module to preliminarily extract the underlying semantic information in the image;

[0020] Inputting the underlying semantic information into the dual-branch structure of the second layer of the anti-depth separable stem module to increase the ability to capture multi-scale and underlying texture features, and outputting two feature maps;

[0021] The anti-depth-separable convolutional block of the third layer of the anti-depth-separable stem module is used to aggregate the features of the two feature maps to obtain the first feature map.

[0022] As a further technical solution, the process of extracting global features from the horizontal and vertical directions through the rectangular self-calibration module and forming the second feature map also includes:

[0023] The shape self-calibration method is used to calibrate the regional shape from the horizontal and vertical directions respectively, which can be expressed as:

[0024] ;

[0025] In the formula, and Respectively represent large kernel strip convolution in the horizontal and vertical directions, represents the normalization layer, represents the ReLU activation function, Represents the Sigmoid activation function.

[0026] As a further technical solution, the process of using a four-layer adaptive frequency attention network module to extract the key frequency components of the second feature map and enhance it, and each layer of the adaptive frequency attention network module outputs a third feature map containing frequency information that is beneficial to semantic segmentation is:

[0027] Using the frequency similarity kernel to obtain the frequency information in the second feature map;

[0028] A dynamic low-pass filter and a dynamic high-pass filter are used to extract low-frequency information and high-frequency information in the second feature map;

[0029] The extracted low-frequency information and high-frequency information are superimposed with the enhanced frequency information to output a third feature map containing frequency information that is beneficial to semantic segmentation.

[0030] As a further technical solution, the process of using the frequency similarity kernel to obtain the frequency information in the enhanced second feature map is:

[0031] Use convolutional layer to encode the position to get features , and then use a fixed-size similarity kernel A to learn the corresponding relationship between different frequencies. It is transformed into key K, value V and query Q through the linear layer, and then K is normalized using the Softmax operation, and each frequency component is integrated into a similarity kernel. , the formula is:

[0032] ;

[0033] in Indicates the first frequency components, Indicates the first frequency components, and then use the similarity kernel and Q interaction to enhance the important frequency components.

[0034] As a further technical solution, the process of inputting the third feature map containing frequency information beneficial to semantic segmentation in each layer into the frequency enhancement channel attention module, fusing the frequency information of each layer and restoring the local information, and obtaining the fourth feature map also includes:

[0035] The residual channel attention mechanism is used to sort out the features in many channels, enhance the channels that are critical to segmentation, and suppress redundant channels; the process is as follows:

[0036] Use two 3×3 DW convolutions to independently convolve the features of each channel to enhance the feature response within the channel;

[0037] Use average pooling to compress the spatial dimension of each channel to 1, and then use two fully connected layers to perform nonlinear transformation on the compressed feature vector to learn cross-channel dependencies and determine the importance of each channel;

[0038] The importance weight of each channel is generated based on the Hard-Sigmoid activation function, and the Hadamard product operation is performed on the feature map before compression to increase the weight of important channels.

[0039] A second aspect of the present invention provides a wood surface defect detection system.

[0040] A wood surface defect detection system, comprising:

[0041] The preprocessing module is configured to: obtain the eucalyptus veneer image and perform preprocessing;

[0042] A first feature map acquisition module is configured to: input the preprocessed eucalyptus veneer image into the wood defect recognition network, extract a feature map of the eucalyptus veneer image using an inverse depth separable stem module, and perform feature aggregation on the feature map to obtain a first feature map;

[0043] The frequency feature enhancement module is configured to: input the first feature map into the rectangular self-calibration adaptive frequency attention network module of the wood defect recognition network, extract global features from the horizontal and vertical directions through the rectangular self-calibration module, and form a second feature map, and then use the four-layer adaptive frequency attention network module to extract the key frequency components of the second feature map and enhance it, and each layer of the adaptive frequency attention network module outputs a third feature map containing frequency information that is beneficial to semantic segmentation;

[0044] The bark defect area display module is configured as follows: input the third feature map containing frequency information beneficial to semantic segmentation output by each layer of the adaptive frequency attention network module into the frequency enhancement channel attention module, fuse the frequency information of each layer and restore the local information to obtain the fourth feature map, fuse the fourth feature map with the first feature map to obtain the fifth feature map, input the fifth feature map to the segmentation head for prediction, and output the final identified bark defect area.

[0045] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a wood surface defect detection method as described in the first aspect of the present invention.

[0046] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for detecting wood surface defects as described in the first aspect of the present invention are implemented.

[0047] One or more of the above technical solutions have the following beneficial effects:

[0048] This paper proposes a wood defect recognition network (LFENet) based on lightweight frequency enhanced semantic segmentation improved by AFFormer. It takes texture features as the entry point, uses the inverse depth separable stem (IDS-Stem) to extract texture information in the feature map, and uses the rectangular self-calibrated adaptive frequency attention network (RCAF) to extract important frequency information in the feature map. Finally, the frequency enhanced channel attention module (FECA) is used to restore the local detail information lost during downsampling and make full use of the frequency information in the feature map of each stage, providing more effective feature support for the segmentation head. This solves the problem that in the process of wood defect detection, the network extracts features of complex defects such as bark with diverse colors, shapes and textures, which leads to poor results and reduced semantic segmentation accuracy.

[0049] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0051] Figure 1 This is a flow chart of the method of the first embodiment.

[0052] Figure 2 It is a schematic diagram of the structure of the eucalyptus image acquisition device in the first embodiment.

[0053] FIG. 3( a ), FIG. 3( b ), FIG. 3( c ), and FIG. 3( d ) are schematic diagrams of bark defects of different colors and shapes in the eucalyptus veneer image in the first embodiment.

[0054] FIG. 4( a ) shows the eucalyptus veneer image before being labeled by Roboflow in the first embodiment; FIG. 4( b ) shows the eucalyptus veneer image after being labeled by Roboflow.

[0055] Figure 5 This is the overall architecture diagram of the wood defect recognition network (LFENet) in Example 1.

[0056] Figure 6 Schematic diagram of the structure of the Inverse Depth Separable Stem (IDS-Stem) module in the first embodiment.

[0057] Figure 7Schematic diagram of the structure of the inverse depthwise separable convolution block (IDS block) in the first embodiment.

[0058] Figure 8 Schematic diagram of the hierarchical structure of the rectangular self-calibrated adaptive frequency attention network (RCAF) in the first embodiment.

[0059] Fig. 9 Schematic diagram of the structure of the rectangular self-calibration module (RCM) in the first embodiment.

[0060] Fig.10 Schematic diagram of the structure of rectangular self-calibration attention (RCA) in the first embodiment.

[0061] Fig.11 Schematic diagram of the structure of the frequency enhanced channel attention module (FECA) in the first embodiment.

[0062] Fig.12 This is a schematic diagram of the structure of the residual channel attention (Res-Channel Attention) in the first embodiment.

[0063] FIG13 (a) is a comparison diagram of Bark.IoU between different module configurations in the first embodiment and AFFormer; FIG13 (b) is a comparison diagram of FLOPs between different module configurations in the first embodiment and AFFormer; FIG13 (c) is a comparison diagram of Params between different module configurations in the first embodiment and AFFormer.

[0064] Fig.14 This is a comparison chart of the coordination effect data of each module in the first embodiment.

[0065] Fig.15 This is a comparison chart of the effects of various segmentation methods in the first embodiment.

[0066] Fig.16 This is a comparison chart of FPS of various segmentation methods in the first embodiment.

[0067] Fig.17 This is a comparison chart of the visualized segmentation effects of various segmentation methods in the first embodiment.

[0068] Fig.18 It is a system structure diagram of the second embodiment. DETAILED DESCRIPTION

[0069] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0070] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0071] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0072] Embodiment 1

[0073] This embodiment discloses a method for detecting wood surface defects.

[0074] like Figure 1 As shown, a method for detecting wood surface defects comprises:

[0075] Step S1, obtaining an image of a eucalyptus veneer and performing preprocessing. Step S1 also includes the following steps:

[0076] Step S11, in the process of acquiring the eucalyptus veneer image, the eucalyptus image acquisition device is used to acquire the image, specifically, Figure 2 As shown in the figure, the eucalyptus image acquisition equipment consists of a board loader, a photoelectric sensor, a warm white LED light, a 12-megapixel CMOS camera, a light shielding box, and a conveyor belt. The acquisition process is carried out in the light shielding box and LED lights are used to ensure stable lighting conditions. The size of the eucalyptus veneer is 61cm×127cm. After the camera captures the surface image of the veneer, it is stored as a 3000×4096×3 image.

[0077] Step S12, select eucalyptus veneer images with bark defects from the acquired eucalyptus veneer images. Bark defects refer to the bark remaining on the veneer when the wood is processed into wood veneer by rotary cutting. They are mostly irregular long oval shapes, and the colors are mostly dark brown, light brown, and gray, as shown in Figure 3 (a)-Figure 3 (d). The bark defects are annotated using Roboflow. The annotation process and structure are shown in Figure 4 (a) and Figure 4 (b), where Figure 4 (a) is the image before annotation, and Figure 4 (b) refers to the veneer image with bark defects annotated. Specifically, the green part in Figure 4 (b) represents the bark defect area. The annotated eucalyptus veneer images are divided into training sets and test sets to facilitate subsequent model training.

[0078] Step S12, data enhancement is performed on the labeled image by random rotation, splicing and other methods, and the image after data enhancement is resized to 1024×1024×3.

[0079] Step S2, input the pre-processed eucalyptus veneer image into the wood defect recognition network. The overall architecture of the wood defect recognition network is as follows: Figure 5The anti-depth separable stem module is used to extract the feature map of the eucalyptus veneer image and perform feature aggregation on the feature map to obtain a first feature map.

[0080] In step S2, the inverse depth-wise separable stem module consists of an inverse depth-wise separable convolution block (IDS block) and a 2×2 maximum pooling. Compared to the depth-wise separable convolution block, which first extracts features through a 3×3 depth-wise (DW) convolution and then increases the number of channels through a 1×1 point-wise (PW) convolution. Figure 7 , the inverse depth-separable convolution block in this embodiment first uses 1×1 PW convolution to increase the number of channels to enrich the semantic information, and then uses 3×3 DW convolution to extract local features and improve spatial information to obtain richer semantic information. When downsampling is not performed, the step size of the 3×3 DW convolution is 1 and the padding is 1; when downsampling is performed, the step size S of the 3×3 DW convolution is 2 and the padding is 1. The convolution block can be written as:

[0081]

[0082]

[0083] Where X represents the input feature map, represents 1×1 PW convolution, represents a 3×3 DW convolution, s and p represent the stride and padding size, respectively.

[0084] The IDS-stem module has a three-layer structure, such as Figure 6 The process of extracting the feature map of the eucalyptus veneer image using the inverse depth separable stem module and performing feature aggregation on the feature map to obtain the first feature map is as follows:

[0085] The image is first input to the down-sampling inverse depth separable convolution block of the first layer to preliminarily extract the underlying semantic information in the image. The second layer adopts a dual-branch structure, using the inverse depth separable convolution block for down-sampling on one side and 2×2 max pooling with a step size of 2 on the other side, so that the wood defect recognition network automatically finds a suitable down-sampling combination, which saves calculations while extracting richer features, and increases the network's ability to capture multi-scale and underlying texture features. The feature maps output by the two branches are spliced ​​and input to the inverse depth separable convolution block of the third layer for feature aggregation, which ignores redundant feature information while enhancing the network's response to significant features, and reduces the computational complexity of subsequent layers. After these three layers of operations, the feature map of 1024×1024×3 is converted to the first feature map of 256×256×32. This process can be written as:

[0086]

[0087] in represents the inverse depthwise separable convolutional block, Represents an inverse depth-separable convolutional block for downsampling. Each convolutional layer in the convolutional block is followed by a normalization layer and a Hardswish activation function. represents 2×2 max pooling with a step size of 2, Indicates a splicing operation.

[0088] Step S3, input the texture feature map into the rectangular self-calibration adaptive frequency attention network module (RCAF) of the wood defect recognition network, extract global features from the horizontal and vertical directions through the rectangular self-calibration module, and form a second feature map, and then use the four-layer adaptive frequency attention network module to extract the key frequency components of the second feature map and enhance it. Each layer of the adaptive frequency attention network module outputs a third feature map containing frequency information that is beneficial to semantic segmentation. In this embodiment, combined with Figure 7 ,RCAF network is a rectangular self-calibration adaptive frequency attention network, which consists of a rectangular self-calibration module (RCM) and an adaptive frequency attention network (AFFormer). Its structure is as follows Figure 8 As shown in Figure 2, RCM consists of rectangular self-calibration attention (RCA) and multi-layer perceptron (MLP), as shown in Figure 2. Fig. 9 As shown in Figure 2, the adaptive frequency attention network (AFFormer) is used to capture the frequency information in the feature map that is beneficial to semantic segmentation. It extracts key frequency components from the feature map through an adaptive filter (AFF), which includes a frequency similarity kernel (FSK), a dynamic low-pass filter (DLF), and a dynamic high-pass filter (DHF).

[0089] Combination Fig.10 , the process of forming the second feature map and extracting the key frequency components of the second feature map for enhancement, and outputting the third feature map containing frequency information beneficial to semantic segmentation includes:

[0090] Step S31, after the first feature map is processed by RCM, the weight of the bark defect area in the first feature map can be increased, and the attention of the wood defect recognition network is further focused on the bark defect;

[0091] Step S32, RCA in RCM uses average pooling to extract global features from the horizontal and vertical directions, generate axis vectors in two directions, add these two axis vectors to form a second feature map, so as to preliminarily identify and confirm the area and range of bark defects on the wood.

[0092] Step S33, using the shape self-calibration method (SSF) to correct the area of ​​bark defects. This method consists of two large-kernel band convolutions, which calibrate the area shape from the horizontal and vertical directions respectively, so that it can adapt to various irregular shapes and make it fit the shape of bark defects better. The function can be expressed as:

[0093]

[0094] Here and Respectively represent large kernel strip convolution in the horizontal and vertical directions, represents the normalization layer, represents the ReLU activation function, Represents the Sigmoid activation function.

[0095] In step S34, RCA also includes a 3×3 DW convolution, which is used to further extract local features in the first feature map. The attention feature output by SSF is weighted by the Hadamard product on the extracted features, and finally the attention can be expressed as

[0096]

[0097] here represents 3×3 DW convolution, represents the Hadamard product operation, and They represent horizontal average pooling and vertical average pooling respectively.

[0098] Step S35, using a 4-layer AFFormer to capture the frequency information in the second feature map that is beneficial to semantic segmentation, and using a frequency similarity kernel to obtain the frequency information in the enhanced second feature map; FSK is used to enhance the important frequency information in the feature map, and the convolutional layer is first used for position encoding to obtain the feature map. , and then use a fixed-size similarity kernel A to learn the corresponding relationship between different frequencies. It is transformed into key K, value V and query Q through the linear layer, and then K is normalized using the Softmax operation, and each frequency component is integrated into a similarity kernel. ,Right now:

[0099]

[0100] in Indicates the first frequency components, Indicates the first frequency components, and then use the similarity kernel and Q interaction to enhance the important frequency components.

[0101] Step S36: extract low-frequency information and high-frequency information from the second feature map using a dynamic low-pass filter (DLF) and a dynamic high-pass filter (DHF).

[0102] DLF uses multiple groups of average pooling to simulate low-pass filters. The kernels and step sizes of these average pooling are different to adapt to different cutoff frequencies of different images to achieve the effect of dynamic filtering. After filtering, the bilinear interpolation method is used to upsample the pooled frequency information to the initial size for fusion. This process can be expressed as

[0103]

[0104] here represents the mth group of low-pass filters, It means the mth value after V is grouped by channel. represents bilinear interpolation, Indicates the output size is DHF uses multiple groups of deep convolutions to simulate high-pass filters. In order to fully retain the useful high-frequency information on each image, the value V is divided into n groups. For each group, convolutions with different kernel sizes are used to simulate high-pass filters with different cutoff frequencies. This process can be expressed as

[0105]

[0106] Indicates that the convolution kernel size is The depth of convolution.

[0107] Step S37, superimpose the frequency information extracted by DLF and DHF with the frequency information enhanced by FSK, and output a third feature map containing frequency information that is beneficial to semantic segmentation. As shown in formula (9):

[0108]

[0109] In the formula, Indicates the concatenation operation of the feature maps divided into H groups.

[0110] Step S4, input the third feature map of each layer containing frequency information beneficial to semantic segmentation into the frequency enhancement channel attention module, fuse the frequency information of each layer and restore the local information to obtain the fourth feature map, fuse the fourth feature map with the first feature map to obtain the fifth feature map, input the fifth feature map to the segmentation head for prediction, and output the final identified bark defect area.

[0111] Combination Fig.11The frequency enhanced channel attention module (FECA) adopts the idea of ​​expansion. First, a 1×1 convolution is used to expand the third feature map output by the second layer AFFormer to the same number of channels as the third feature map output by the fourth layer AFFormer, so as to make the feature representation in the third feature map richer. Then, the third feature map output by the fourth layer AFFormer is added to the third feature map output by the third layer AFFormer. After fusion, a 1×1 convolution is used to integrate the feature information to enhance the strength of important frequency information, so that the wood defect recognition network can select the frequency information useful for semantic segmentation. Repeat the above operation with the feature map after the fusion of the third and fourth layers of AFFormer and the third feature map output by the second layer AFFormer, and then upsample it to the same size as the third feature map output by the first layer AFFormer, and finally obtain the fourth feature map after the frequency information of each layer is fused and the local information is restored.

[0112] The entire processing process of FECA is as follows:

[0113] (10)

[0114] In the formula, , , , They represent the feature maps of the four AFF outputs in RCAF respectively. represents a 1×1 convolution and a normalization layer, represents adaptive dynamic sampling, represents residual channel attention.

[0115] In this embodiment, dynamic upsampling is used instead of traditional bilinear interpolation upsampling, which adopts point sampling to avoid time-consuming dynamic convolution and additional sub-network generation, thereby significantly reducing the consumption of computing resources. Compared with bilinear interpolation, which simply interpolates based on fixed rules, it converts the input feature map into a continuous mapping, and then generates sampling points based on content perception to resample the continuous mapping, which can more intelligently process the details and edges of the image, better retain high-frequency details, and achieve high-quality upsampling effects.

[0116] After upsampling, the feature map and the third feature map output by the first layer AFFormer are expanded and fused at the same time to restore the detail information lost during downsampling. At this time, the output feature map has a large number of independent channels and rich feature information, but there may be redundancy in the information of each channel, which will affect the segmentation effect. Further, combined with Fig.12In this embodiment, Res-Channel Attention is introduced to effectively sort out the features in many channels, enhance those channels that are critical to segmentation, and suppress redundant channels. Res-Channel Attention first uses two 3×3DW convolutions to independently convolve the features of each channel while reducing the amount of calculation and parameters, enhancing the feature response within the channel, and then uses average pooling (APPlooing) to compress the spatial dimension of each channel to 1, and then uses two fully connected layers to perform nonlinear transformation on the compressed feature vector, learn cross-channel dependencies and determine the importance of each channel, and finally use the Hard-Sigmoid activation function to generate the importance weight of each channel, and perform the Hadamard product operation on the feature map before compression to increase the weight of important channels.

[0117] The first feature map output by IDS-Stem is expanded to the same number of channels as the fourth feature map output by FECA through 1×1 convolution, and the two feature maps are added to obtain the fifth feature map that restores spatial information. The fifth feature map is input to the segmentation head, which is used to convert the features extracted from the network into the final pixel-level classification results. Each pixel in the image is classified to determine which category it belongs to. To maintain lightweight, the segmentation head only uses a 1×1 convolution to predict based on the semantic information in the feature map, and overlays the prediction result on the original image to show the area of ​​bark defects.

[0118] Furthermore, in this embodiment, ablation experiments and comparative experiments are performed to verify the segmentation effect of the LFENet proposed in this embodiment on complex defects such as bark.

[0119] In the ablation experiment, LFENet uses the same batch size, training iteration strategy and data augmentation method as AFFormer. All models are trained on 2 A100 GPUs with 80K iterations and 8 batch sizes. The present invention uses AdamW as the optimizer, with an initial learning rate of 0.0007 and a weight decay of 0.01. The poly strategy is used to dynamically adjust the learning rate, with the coefficient set to 1.0. During training, the training image is adjusted to a size of 1024×1024×3.

[0120] The effects of IDS-Stem, RCAF and FECA on LFENet are shown in Figure 13 (a) to Figure 13 (c). In the figure, Bark.IoU is the intersection-over-union ratio of bark defects; FLOPs is the amount of model calculation; FLOPs is the amount of model parameters. It can be seen that IDS-Stem increased the IoU (Bark.IoU) of bark defects by 1.52% while keeping the parameters and calculation amount almost unchanged. After using RCAF to replace AFFormer as the backbone network, Bark.IoU increased from 73.58% to 74.39%. Although the use of FECA increases the amount of calculation and the amount of parameters, the accuracy of segmentation has been significantly improved, and Bark.IoU has reached 77.15%. When the three modules work together, the segmentation performance is best. The comparison results of FLOPs and FLOPs are close to the Bark.IoU results. As shown Fig.14 As shown in the figure, Bark.IoU is improved by 4.23% compared to AFFormer.

[0121] In the comparative experiment, LFENet was compared with several mainstream semantic segmentation methods proposed in recent years, including Topformer, Seaformer, HRNet, LITv2, and AFFormer. To ensure the fairness of the comparison, all methods including LFENet did not use pre-training, but were directly trained on the eucalyptus bark dataset collected by the present invention. In this way, the ability of the network to learn from unfamiliar datasets can be evaluated. Fig.15 The experimental results shown in the figure show that LFENet is significantly better than other methods in several key indicators such as Bark.IoU, mRecall and F1. Fig.15 In , mPrecision is the average precision; mRecall is the average recall; and F1 is the weighted harmonic mean. Fig.16 As shown in Figure 2, LFENet’s FPS exceeds that of the lightweight SeaFormer, where FPS refers to the number of images processed per second. Fig.17As shown, LFENet can more accurately identify bark defects. In contrast, SeaFormer aims at lightweighting and significantly reduces the resolution of the image, resulting in loss of local details, and thus has poor segmentation performance for smaller bark areas. TopFormer recovers some local information by upsampling and fusing with a larger feature map, but fails to detect bark when the color of the bark is similar to the surrounding environment. HRNet showed deficiencies in segmenting lighter bark defects, indicating that changes in bark color can affect HRNet's ability to identify bark. Both LITv2 and AFFormer consider frequency information, but do not effectively restore local details, resulting in the inability to identify small bark at the edge. These results show that LFENet has stronger robustness, learning ability, and accuracy when dealing with highly diverse targets.

[0122] Embodiment 2

[0123] This embodiment discloses a wood surface defect detection system;

[0124] like Fig.18 As shown, a wood surface defect detection system comprises:

[0125] The preprocessing module is configured to: obtain the eucalyptus veneer image and perform preprocessing;

[0126] A first feature map acquisition module is configured to: input the preprocessed eucalyptus veneer image into the wood defect recognition network, extract a feature map of the eucalyptus veneer image using an inverse depth separable stem module, and perform feature aggregation on the feature map to obtain a first feature map;

[0127] The frequency feature enhancement module is configured to: input the first feature map into the rectangular self-calibration adaptive frequency attention network module of the wood defect recognition network, extract global features from the horizontal and vertical directions through the rectangular self-calibration module, and form a second feature map, and then use the four-layer adaptive frequency attention network module to extract the key frequency components of the second feature map and enhance it, and each layer of the adaptive frequency attention network module outputs a third feature map containing frequency information that is beneficial to semantic segmentation;

[0128] The bark defect area display module is configured as follows: input the third feature map containing frequency information beneficial to semantic segmentation output by each layer of the adaptive frequency attention network module into the frequency enhancement channel attention module, fuse the frequency information of each layer and restore the local information to obtain the fourth feature map, fuse the fourth feature map with the first feature map to obtain the fifth feature map, input the fifth feature map to the segmentation head for prediction, and output the final identified bark defect area.

[0129] Embodiment 3

[0130] The purpose of this embodiment is to provide a computer-readable storage medium.

[0131] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in a wood surface defect detection method as described in Example 1.

[0132] Embodiment 4

[0133] The purpose of this embodiment is to provide an electronic device.

[0134] An electronic device comprises a memory, a processor and a program stored in the memory and executable on the processor, wherein the processor implements the steps in a wood surface defect detection method as described in Example 1 when executing the program.

[0135] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0136] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0137] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for detecting wood surface defects, characterized in that: include: Acquire eucalyptus veneer images and perform preprocessing; The preprocessed eucalyptus veneer image is input into the wood defect recognition network, and the feature map of the eucalyptus veneer image is extracted by using the inverse depth separable stem module and the feature map is aggregated to obtain the first feature map; wherein, the inverse depth separable stem module is composed of an inverse depth separable convolution block and a 2×2 maximum pooling, and the inverse depth separable convolution block first uses a 1×1 PW convolution to increase the number of channels to enrich the semantic information, and then uses a 3×3 DW convolution to extract local features and improve the spatial information to obtain richer semantic information; the inverse depth separable convolution block is as follows: Where X represents the input feature map, represents 1×1 PW convolution, represents a 3×3 DW convolution, s and p represent the step size and padding size respectively; The process of extracting the feature map of the eucalyptus veneer image using the inverse depth separable stem module and performing feature aggregation on the feature map to obtain the first feature map is as follows: The eucalyptus veneer image is input into the down-sampled anti-depth separable convolutional block of the first layer of the anti-depth separable stem module to preliminarily extract the underlying semantic information in the image; Inputting the underlying semantic information into the dual-branch structure of the second layer of the anti-depth separable stem module to increase the ability to capture multi-scale and underlying texture features, and outputting two feature maps; The anti-depth-separable convolutional block of the third layer of the anti-depth-separable stem module is used to aggregate the features of the two feature maps to obtain a first feature map; The first feature map is input into the rectangular self-calibration adaptive frequency attention network module of the wood defect recognition network. The rectangular self-calibration module extracts global features from both horizontal and vertical directions to form a second feature map. Then, a four-layer adaptive frequency attention network module is used to extract the key frequency components of the second feature map and enhance it. Each layer of the adaptive frequency attention network module outputs a third feature map containing semantic segmentation frequency information. The third feature map containing the semantic segmentation frequency information output by each layer of the adaptive frequency attention network module is input into the frequency enhancement channel attention module, the frequency information of each layer is fused and the local information is restored to obtain the fourth feature map, the frequency enhancement channel attention module first uses 1×1 convolution to expand the third feature map output by the second layer of the adaptive frequency attention network to the same number of channels as the third feature map output by the fourth layer of the adaptive frequency attention network, and then adds the third feature map output by the fourth layer of the adaptive frequency attention network to the third feature map output by the third layer of the adaptive frequency attention network, and then uses a 1×1 convolution to integrate the feature information after fusion, so that the wood defect recognition network selects the frequency information useful for semantic segmentation by itself; repeat the above operation with the feature map after the fusion of the third and fourth layers of the adaptive frequency attention network and the third feature map output by the second layer of the adaptive frequency attention network, and then use upsampling to restore it to the same size as the third feature map output by the first layer of the adaptive frequency attention network, and finally obtain the fourth feature map after the frequency information of each layer is fused and the local information is restored; The entire processing process of the frequency enhancement channel attention module is as follows: In the formula, , , , They represent the feature maps of the four adaptive filter outputs in the rectangular self-calibrated adaptive frequency attention network module, represents a 1×1 convolution and a normalization layer, represents adaptive dynamic sampling, represents residual channel attention; The fourth feature map is fused with the first feature map to obtain the fifth feature map, which is input to the segmentation head for prediction, and the final identified bark defect area is output.

2. A wood surface defect detection method according to claim 1, characterized in that: The pre-processing process is as follows: Select images with bark defects from the acquired eucalyptus veneer images and annotate them using Roboflow; The annotated images are randomly rotated and spliced ​​for data augmentation and the image size is adjusted.

3. A wood surface defect detection method according to claim 1, characterized in that: The process of extracting global features from the horizontal and vertical directions through the rectangular self-calibration module and forming a second feature map also includes: The shape self-calibration method is used to calibrate the regional shape from the horizontal and vertical directions respectively, which can be expressed as: ； In the formula, and Respectively represent large kernel strip convolution in the horizontal and vertical directions, represents the normalization layer, represents the ReLU activation function, Represents the Sigmoid activation function.

4. A wood surface defect detection method according to claim 1, characterized in that: The process of using a four-layer adaptive frequency attention network module to extract the key frequency components of the second feature map and enhance it, and each layer of the adaptive frequency attention network module outputs the third feature map after the frequency feature is enhanced is: Using the frequency similarity kernel to obtain the frequency information in the second feature map; A dynamic low-pass filter and a dynamic high-pass filter are used to extract low-frequency information and high-frequency information in the second feature map; The extracted low-frequency information and high-frequency information are superimposed on the enhanced frequency information, and a third feature map with enhanced frequency features is output.

5. A wood surface defect detection method as claimed in claim 4, characterized in that: The process of using the frequency similarity kernel to obtain the frequency information in the enhanced second feature map is: Use convolutional layer to encode the position to get features , and then use a fixed-size similarity kernel A to learn the corresponding relationship between different frequencies. It is transformed into key K, value V and query Q through the linear layer, and then K is normalized using the Softmax operation, and each frequency component is integrated into a similarity kernel. , the formula is: ； in Indicates the first frequency components, Indicates the first frequency components, and then use the similarity kernel and Q interaction to enhance the important frequency components.

6. A wood surface defect detection method according to claim 1, characterized in that: The process of inputting the third feature map containing the semantic segmentation frequency information of each layer into the frequency enhancement channel attention module, fusing the frequency information of each layer and restoring the local information to obtain the fourth feature map also includes: The residual channel attention mechanism is used to sort out the features in many channels, enhance the channels that are critical to segmentation, and suppress redundant channels; the process is as follows: Use two 3×3 DW convolutions to independently convolve the features of each channel to enhance the feature response within the channel; Use average pooling to compress the spatial dimension of each channel to 1, and then use two fully connected layers to perform nonlinear transformation on the compressed feature vector to learn cross-channel dependencies and determine the importance of each channel; The importance weight of each channel is generated based on the Hard-Sigmoid activation function, and the Hadamard product operation is performed on the feature map before compression to increase the weight of important channels.

7. A wood surface defect detection system, characterized in that: include: The preprocessing module is configured to: obtain the eucalyptus veneer image and perform preprocessing; The first feature map acquisition module is configured to: input the preprocessed eucalyptus veneer image into the wood defect recognition network, use the inverse depth separable stem module to extract the feature map of the eucalyptus veneer image and perform feature aggregation on the feature map to obtain the first feature map; wherein the inverse depth separable stem module is composed of an inverse depth separable convolution block and a 2×2 maximum pooling, the inverse depth separable convolution block first uses a 1×1 PW convolution to increase the number of channels to enrich the semantic information, and then uses a 3×3 DW convolution to extract local features and improve spatial information to obtain richer semantic information; the inverse depth separable convolution block is as follows: Where X represents the input feature map, represents 1×1 PW convolution, represents a 3×3 DW convolution, s and p represent the step size and padding size respectively; The process of extracting the feature map of the eucalyptus veneer image using the inverse depth separable stem module and performing feature aggregation on the feature map to obtain the first feature map is as follows: The eucalyptus veneer image is input into the down-sampled anti-depth separable convolutional block of the first layer of the anti-depth separable stem module to preliminarily extract the underlying semantic information in the image; Inputting the underlying semantic information into the dual-branch structure of the second layer of the anti-depth separable stem module to increase the ability to capture multi-scale and underlying texture features, and outputting two feature maps; The anti-depth-separable convolutional block of the third layer of the anti-depth-separable stem module is used to aggregate the features of the two feature maps to obtain a first feature map; The frequency feature enhancement module is configured to: input the first feature map into the rectangular self-calibration adaptive frequency attention network module of the wood defect recognition network, extract global features from the horizontal and vertical directions through the rectangular self-calibration module, and form a second feature map, and then use the four-layer adaptive frequency attention network module to extract the key frequency components of the second feature map and enhance it, and each layer of the adaptive frequency attention network module outputs a third feature map containing semantic segmentation frequency information; The bark defect area display module is configured to: input the third feature map containing semantic segmentation frequency information output by each layer of the adaptive frequency attention network module into the frequency enhancement channel attention module, fuse the frequency information of each layer and restore the local information to obtain the fourth feature map, the frequency enhancement channel attention module first uses 1×1 convolution to expand the third feature map output by the second layer of the adaptive frequency attention network to the same number of channels as the third feature map output by the fourth layer of the adaptive frequency attention network, and then adds the third feature map output by the fourth layer of the adaptive frequency attention network to the third feature map output by the third layer of the adaptive frequency attention network, and then uses a 1×1 convolution to integrate the feature information after fusion, so that the wood defect recognition network selects the frequency information useful for semantic segmentation by itself; repeat the above operation with the feature map after the fusion of the third and fourth layers of the adaptive frequency attention network and the third feature map output by the second layer of the adaptive frequency attention network, and then use upsampling to restore it to the same size as the third feature map output by the first layer of the adaptive frequency attention network, and finally obtain the fourth feature map after the frequency information of each layer is fused and the local information is restored; The entire processing process of the frequency enhancement channel attention module is as follows: In the formula, , , , They represent the feature maps of the four adaptive filter outputs in the rectangular self-calibrated adaptive frequency attention network module, represents a 1×1 convolution and a normalization layer, represents adaptive dynamic sampling, represents residual channel attention; The fourth feature map is fused with the first feature map to obtain the fifth feature map, which is input to the segmentation head for prediction, and the final identified bark defect area is output.

8. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of the wood surface defect detection method as described in any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the wood surface defect detection method according to any one of claims 1 to 6 are implemented.

Images