Wood similar texture matching method based on twin network

Abstract

The invention relates to the technical field of image processing, in particular to a wood similar texture matching method based on a twin network, which comprises the following steps: constructing a target image and a comparison image of wood texture, and generating mask images corresponding to the target image and the comparison image; inputting the target image, the comparison image and the corresponding mask image into first to fourth MRF-Resnet modules respectively to obtain four multiple receptive field feature maps respectively; and inputting the four multi-receptive field feature maps into an improved feature fusion module for feature fusion, and inputting the fused features into a similarity calculation module with depth separable convolution improvement. According to the method, the problems of fixed receptive field, multiple attention mechanisms, calculation complexity and efficiency in the existing twin network are solved.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a wood similar texture matching method based on a twin network. Background Art

[0002] Texture is an important visual feature of wood, reflecting the local structure and surface details of wood. Wood texture similarity matching refers to judging the similarity of textures between different wood samples by calculating and analyzing the similarity of wood surface textures. The mainstream texture similarity matching methods are mainly divided into the following three types:

[0003] 1. Texture similarity comparison method based on pixel difference, such as Srivastava D et al. proposed the use of gray-level co-occurrence matrix GLCM. This method is not sensitive enough to texture details and color changes, and it is difficult to accurately capture the complex and delicate texture characteristics of the wood surface.

[0004] 2. The bag-of-visual-words method based on local feature extraction and clustering, such as BoVW, has the limitation of ignoring the spatial position relationship of texture features in wood texture similarity matching; when processing wood texture similarity matching, it performs poorly because it cannot reflect the global layout of the texture.

[0005] 3. Based on deep learning, such as the Siamese Net, but the receptive field of the Siamese Net is fixed and cannot adapt to texture features of different scales, which limits the model's ability to capture the diversity of complex wood textures; the continuous use of multiple attention mechanisms may cause mutual interference between these mechanisms, limiting the ability of each attention mechanism to capture features; the feature fusion method is too simple, destroying the spatial structure of the original data, thereby weakening the global understanding of the texture; directly using a fully connected network for similarity calculation leads to a surge in the number of model parameters, increasing computational complexity and reducing efficiency. Summary of the invention

[0006] In view of the shortcomings of the existing methods, the present invention solves the problems existing in the above-mentioned existing twin networks.

[0007] The technical solution adopted by the present invention is: a wood similar texture matching method based on twin network comprises the following steps:

[0008] Step 1: construct a target image and a contrast image of wood texture, and generate a mask image corresponding to the target image and the contrast image;

[0009] As a preferred embodiment of the present invention, the mask image is obtained by multiplying the wood texture image and the mask matrix in position to obtain an edge texture image.

[0010] Step 2: Input the target image, the comparison image and the corresponding mask image into the first to fourth MRF-Resnet modules respectively to obtain four multiple receptive field feature maps;

[0011] As a preferred embodiment of the present invention, the MRF-Resnet module includes:

[0012] With Resnet as the backbone, the CBAM module is inserted before and after the basic residual block of stage 1.

[0013] As a preferred embodiment of the present invention, the MRF-Resnet module further includes:

[0014] The output feature maps from stage 1 to stage 4 are residually connected and then multiple receptive fields are fused.

[0015] As a preferred embodiment of the present invention, the fusion of multiple receptive fields includes:

[0016] First, retain the feature maps processed by the bottleneck residual blocks from stage1 to stage4;

[0017] Secondly, the shape of the feature map processed by the four-stage bottleneck residual block is transformed into

[0018]

[0019] Finally, all deformed feature maps are connected in the channel dimension to obtain Multiple receptive field feature maps.

[0020] Step 3: Input the four multi-receptive field feature maps into the improved feature fusion module for feature fusion, and input the fused features into the similarity calculation module improved by the depthwise separable convolution;

[0021] As a preferred embodiment of the present invention, the improved feature fusion module performs feature fusion including:

[0022] First, the global features of the target image and the edge features of the target image are spliced ​​in the width direction;

[0023] Secondly, the global features of the contrast image and the edge features of the contrast image are spliced ​​in the width direction;

[0024] Finally, the already stitched target image features and comparison image features are stitched together in height to obtain the fused new feature map f mix .

[0025] As a preferred embodiment of the present invention, inputting the fused features into the depth-separable convolution improved similarity calculation module includes:

[0026] First, each channel of the feature map is convolved separately using depthwise convolution;

[0027] Secondly, a 1×1 convolution kernel is used to perform point convolution on the features of different channels;

[0028] Again, the feature vector after point convolution is straightened, and the structural similarity, Euclidean distance and hash similarity of the target image and the comparison image are added to the end of the straightened vector.

[0029] As a preferred embodiment of the present invention, a wood similar texture matching system based on a twin network includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement a wood similar texture matching method based on a twin network.

[0030] As a preferred embodiment of the present invention, a computer readable medium stores a computer program code, and when the computer program code is executed by a processor, a wood similar texture matching method based on a twin network is implemented.

[0031] Beneficial effects of the present invention:

[0032] 1. In the wood texture similarity matching task, the reliability of the similarity score depends not only on the performance of the model in extracting texture features, but also on the degree of adaptation of the similarity measurement function to the task; WTSM-SiameseNet efficiently extracts wood texture features through a concurrent attention mechanism and a multi-receptive field feature extraction network; and then improves the similarity measurement module through feature fusion based on feature correlation and deep separable convolution, adaptively generates a wood texture similarity score that conforms to human subjective feelings while reducing the calculation of model parameters;

[0033] 2. The evaluation results on the wood texture similarity dataset show that the scoring of wood texture similarity by WTSM-SiameseNet is consistent with people's subjective feelings;

[0034] 3. Feature extraction network with multiple receptive fields fusion. By fusing feature extraction networks with different receptive fields, the model can adaptively select the optimal receptive field, solving the problem that fixed receptive fields cannot adapt to textures of different sizes. It has better flexibility and accuracy when facing wood textures of different scales.

[0035] 3. Through the dual-stream parallel attention mechanism, the interdependence problem between layers in the traditional serial attention mechanism is reduced, and the performance of the overall attention mechanism is improved. This not only enhances the ability to capture features, but also avoids interference between multiple attention mechanisms;

[0036] 4. A feature fusion method based on feature correlation was designed. The newly designed feature fusion method not only retains the original texture feature space structure, but also improves the interpretability of the fused features, optimizes the expression ability of the overall model, reduces the parameter complexity, and improves the stability and accuracy of the model;

[0037] 5. By using deep separable convolution to optimize the twin network structure, the number of model parameters is greatly reduced, thereby improving training efficiency. This optimization significantly reduces the consumption of computing resources while ensuring model performance;

[0038] 6. The accuracy of WTSM-SiameseNet on the test set reached 96.67%, which was 12.91% higher and 14.21% higher than the previous SiameseNet. Compared with CS-SiameseNet, the accuracy was 2.86% higher and 6.58% higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the WTSM-SiameseNet network structure of the present invention;

[0040] Figure 2 This is a schematic diagram of the existing SiameseNet network structure;

[0041] Figure 3 This is the MRF-Resnet architecture diagram;

[0042] Figure 4 It is the schematic diagram of the feature fusion module;

[0043] Figure 5 It is a schematic diagram of the concurrent attention mechanism;

[0044] Figure 6 It is the CBAM attention mechanism;

[0045] Figure 7 This is the architecture diagram of the similarity calculation module;

[0046] Figure 8 is a dataset example;

[0047] Fig. 9 This is a graph comparing the changes in training loss of the experimental model;

[0048] Fig.10is a wood texture similarity matching example. DETAILED DESCRIPTION

[0049] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0050] A wood similar texture matching method based on twin network includes the following steps:

[0051] like Figure 2 This is the structure diagram of the existing SiameseNet network, which is often used to analyze the similarity between data. Its core includes the Texture Feature Extraction Module and the Texture Feature Aggregation and Matching Module. Figure 2 (a) The texture feature extraction module uses two identical feature extraction networks to extract features from the target image x1 and the comparison image x2 respectively; Figure 2 The (b) texture feature aggregation matching module first uses the feature fusion module to fuse all texture features into a new feature, and then uses the similarity calculation module to obtain the similarity of the two images; however, in real scenes, wood textures have obvious differences in size, shape and details, and the existing texture feature extraction module has a fixed receptive field and cannot automatically adjust according to the characteristics of the texture. Therefore, it is not suitable for extracting all types of wood texture features; this limitation will lead to feature loss during feature extraction, and then lead to misjudgment of the similarity of wood textures; secondly, the feature fusion module and similarity calculation module in the texture feature aggregation matching module are too simple in structure, making it difficult to fit the mapping relationship between texture features and similarity scores, resulting in inaccurate similarity scores.

[0052] against Figure 2 Model problems, such as Figure 1 As shown in the figure, a WTSM-SiameseNet (Wood Texture Similarity Matching-SiameseNet) suitable for wood texture similarity matching is designed.

[0053] Step 1: construct a target image x1 and a comparison image x2 of wood texture, and generate a mask image corresponding to the target image and the comparison image;

[0054] Step 2: Input the target image, the comparison image and the corresponding mask image into the first to fourth MRF-Resnet modules respectively to obtain four multi-receptive field feature maps F'1, F'2, F'3, F'4;

[0055] In the texture feature extraction module, a concurrent attention mechanism is introduced, and a feature extraction module MRF-Resnet (Multi-scale Receptive Field-Resnet) with multiple receptive fields is designed, which improves the receptive field dimension and edge attention of feature extraction and enhances the model's ability to evaluate texture similarity.

[0056] Shallow feature extraction networks are difficult to extract complex texture features, and fixed receptive fields easily lead to insufficient texture feature provision; in the texture feature extraction module before the improvement, that is, Figure 2 The feature extraction network is a convolutional network with a depth of 4. Due to the diversity and complexity of wood texture, the network has a poor effect on extracting small and complex features in wood texture. To address this problem, the present invention combines Resnet with the idea of ​​multiple receptive fields and designs a feature extraction network MRF-Resnet with multiple receptive fields to solve the problem of insufficient feature extraction caused by insufficient network depth and limited receptive fields. In addition, the edge texture of wood has a greater impact on the texture consistency at the edge joint in tasks such as board splicing and repair. Therefore, when performing wood similarity matching, more attention needs to be paid to the edge texture. To address this problem, a concurrent attention mechanism is introduced in the texture feature extraction module to improve the influence of edge texture in the similarity calculation process.

[0057] as follows Figure 3 As shown in the figure, the present invention designs MRF-Resnet with Resnet as the backbone, and its core includes three parts: base residual block (Base Residual Block), bottleneck residual block (Bottlcncck Residual Block) and multi-receptive field fusion (Multi-Receptive Field Fusion).

[0058] Figure 3 The basic residual block of (a) contains three layers of convolution. The first layer of convolution is used to reduce the number of channels of the input feature map, thereby reducing the amount of calculation and the number of parameters; the second layer of convolution is responsible for feature extraction and conversion, capturing the local texture features in the input feature map and converting them into a higher-level feature representation; the third layer of convolution is used to restore the feature dimension so that the output feature map has the same size as the input feature map.

[0059] Figure 3 The bottleneck residual block in (b) has a similar structure to the basic residual block, but the convolution step size of the second layer is set to 2, so that the bottleneck residual block has the function of downsampling and improves the range of the receptive field.

[0060] The MRF-Resnet module uses Resnet as the backbone, inserts the CBAM module before and after the basic residual block of stage 1, and performs residual connection on the output feature maps from stage 1 to stage 4 and then performs multi-receptive field fusion;

[0061] The receptive field is the field of view of the model on the input image during feature extraction. Small textures of wood only require a smaller receptive field to cover, while large textures require a larger receptive field. Therefore, the present invention designs multiple receptive field fusion, which improves the receptive field dimension in the feature by fusing the features of different receptive fields extracted by the network. The principle of the multiple receptive field fusion module is as follows: Figure 4 As shown:

[0062] Since the bottleneck residual block will change the feature dimension while expanding the receptive field of the feature map, the fusion of features with different receptive fields requires the dimension transformation of the features first; the size of the feature map of stage 1, that is, the initial feature map, is (C, K, K), where C represents the number of channels of the feature map and K represents the length and width of the feature map; each time the feature map passes through the bottleneck residual block, the number of channels will become twice the original, and the size of the feature map will become the original After stage 4, the size of the feature map will become

[0063] First, retain the feature map processed by the bottleneck residual block of each stage;

[0064] Second, a convolution operation is used to transform the shape of all feature maps into The size transformation process is shown in the following formula (1):

[0065]

[0066] Among them, F i,j Represents the pixel value at position (i, j) in the output feature map; f (i·S+m),(j·S+n) K represents the pixel value at position (i·S+m) and (j·S+n) in the input feature map; m,n Represents the convolution kernel weight.

[0067] Finally, all deformed feature maps are connected in the channel dimension to obtain a shape of The new feature map is a multi-receptive field feature map F'; the feature map has the same receptive field in space, but different receptive fields in channels. The connection formula of the new feature is shown in the following formula (2):

[0068] F'=concat(F1,F2,F3,F4,dim=1) (2)

[0069] Among them, F' represents the new feature map obtained after MSRFNet feature extraction; F1, F2, F3, and F4 respectively represent the feature maps of the four bottleneck residual block output feature maps after convolution dimension transformation; concat (dim = 1) means connection in the channel dimension; and the feature maps F'1, F'2, F'3, and F'4 are obtained by formula 2.

[0070] In traditional feature extraction networks, the weights of all features are the same; however, for the wood texture similarity matching task, the focus of feature extraction should be on texture features rather than background features; in order to increase the attention to texture features in the MRF-Resnet module, the CBAM attention mechanism (concurrent attention) is introduced to enhance the attention to texture during global feature extraction; in addition, the edge texture of wood plays a key role in subsequent processing tasks such as splicing, because it directly affects the texture consistency at the splicing; therefore, when extracting texture features, it is necessary to increase the attention to edge texture; however, the attention mechanisms of the serial structure may interfere with each other, resulting in a weakened effect; to solve this problem, the CBAM attention mechanism is designed to more effectively extract texture and edge features.

[0071] CBAM attention mechanism Figure 5 As shown in the figure; MRF-ResNet is used to extract features from the complete wood texture image and the wood image containing only edge texture, and while obtaining global features and edge features, these features are ensured to be structurally consistent; in addition, since MRF-ResNet only contains the CBAM attention module, the complete global texture features can be retained during global feature extraction; and during edge feature extraction, since only the edge texture image is used, the extracted features only contain edge texture information and will not be interfered by features in other regions; by fusing global features and edge features, the weight of edge features can be increased while retaining global features as much as possible.

[0072] The CBAM (Convolutional Attention Module) attention mechanism is an attention mechanism that combines channel attention and spatial attention. Figure 6 As shown, through Figure 6 (a) Channel attention can adjust the importance of each channel in the feature map, so that the network pays more attention to the most relevant and important channels in texture feature extraction; Figure 6 (b) Spatial attention can increase the weight of areas with rich texture information and reduce the weight of areas with less texture information.

[0073] The calculation formula of the channel attention module is shown in formula (3):

[0074] f c =σ(MLP(AvgPool(F))+MLP(MaxPool(F))) (3)

[0075] Among them, F represents the input feature map; AvgPool and MaxPool represent global average pooling and global maximum pooling respectively; MLP represents multi-layer perceptron; σ represents Sigmoid function.

[0076] The calculation formula of the spatial attention module is shown in formula (4):

[0077] f s =σ(c 7×7 ([AvgPool(F c );MaxPool(F c )])) (4)

[0078] Among them, F c Represents the feature map after using channel attention; c 7×7 Represents a convolution operation with a convolution kernel size of 7×7; [AvgPool(F c );MaxPool(F c )] means connecting the average pooling and maximum pooling structures in the channel direction.

[0079] In edge feature extraction, the edge texture image is obtained by multiplying the wood texture image with the mask matrix. The mask formula is shown in the following formula (5):

[0080] Edge_Pattern i,j =Wood_Texture i,j ⊙M i,j (5)

[0081] Wherein, Edge_Pattern represents the edge texture image; Wood_Texture represents the input wood texture image; M represents the mask matrix with the same shape as Wood_Texture, and the form of M is shown in the following formula (6):

[0082]

[0083] Step 3: Input the four multi-receptive field feature maps obtained after MSRFNet feature extraction into the improved feature fusion module for feature fusion, that is, input the feature maps F'1, F'2, F'3, and F'4 into the improved feature fusion module for feature fusion; and input the fused features into the similarity calculation module improved by depthwise separable convolution;

[0084] In the texture feature aggregation matching module, the feature fusion module (Feature Fusion Module) is improved based on feature correlation, and the similarity calculation module (Similarity Computation Module) is improved using depthwise separable convolution, thereby reducing the number of model parameters and improving the accuracy of similarity calculation;

[0085] The texture feature aggregation matching module consists of a feature fusion module and a similarity calculation module. The original feature fusion method uses data to obtain the absolute value of phase subtraction. This method ignores the internal relationship of the features, destroys the original structure of the features, and reduces the expressiveness of the features. In addition, the original similarity calculation module only uses a two-layer fully connected network to learn the mapping relationship from features to similarities. However, the real similarity measurement function is a complex function, and it is difficult to effectively fit the real measurement function using the original method, which makes it difficult for the model to converge. In order to solve these two problems, the present invention improves the feature fusion module from the perspective of feature correlation. At the same time, the similarity calculation module is optimized by using deep convolution and point convolution. The improved texture feature aggregation module is as follows: Figure 7 shown.

[0086] The function of the feature fusion module is to fuse the global features of the target image x1 obtained by the texture feature extraction module, the edge features of the target image x1, the global features of the contrast image x2, and the edge features of the contrast image x2; because the four features are different in source and type, the meaning represented by each feature at the corresponding position is also different; therefore, it is not appropriate to directly use the feature fusion method of taking the absolute value of phase subtraction; however, these four features all come from the same feature extraction network, so they have something in common in structure, that is, the shape of the features is the same, and they have the same receptive field on the same channel; based on these characteristics, a feature fusion module based on feature correlation is designed, and the specific fusion method is as follows:

[0087] First, the global features of the target image and the edge features of the target image are spliced ​​in the width (W) direction;

[0088] Secondly, the global features of the contrast image and the edge features of the contrast image are spliced ​​in the same way; the splicing formula is shown in the following formula (7):

[0089] f target(contrast) =concat(f global ,f edge , dim=2) (7)

[0090] Among them, f target(contrast) Represents the fused target image or contrast image features, i.e., f target(contrast) Represents the fused target image f target Or compare image features f contrast ;f global represents the global feature; f edge Indicates edge features; concat(dim=2) indicates connection at the width level;

[0091] like Figure 7 (a) f taget_global 、f contrast_global 、f taget_edge 、f contrast_edge Corresponding to F'1, F'2, F'3, and F'4 respectively.

[0092] Finally, the already stitched target image features and comparison image features are stitched at height (H), and the stitching formula is shown in the following formula (8):

[0093] f mix =concat(f target ,f contrast , dim=3) (8)

[0094] Among them, f target and f contrast represents the target image features and the comparison image features; concat(dim=3) represents the connection at the height level; f mix Represents the new features after fusion.

[0095] Through feature fusion, the internal relationship of the original features can be protected to the greatest extent; at the same time, the new features combine all the features of the reference image and the comparison image in space, and still have different receptive fields in the channel.

[0096] The original similarity measurement module directly flattens the fused features into a one-dimensional vector and calculates the similarity through a fully connected network. This method ignores the internal connection of the features and cannot analyze the relationship between the target image and the comparison image from the dimensions of space and channels. Therefore, it is difficult to accurately fit the mapping relationship between the features and the similarity. Therefore, the present invention designs an optimized similarity calculation module, which uses deep convolution and point convolution to improve the similarity calculation. At the same time, in order to improve the accuracy of the similarity calculation, the structural similarity, Euclidean distance and hash similarity of the target image and the comparison image are input into the network as additional features.

[0097] The specific steps for similarity calculation are as follows:

[0098] First, use depthwise convolution to convolve each channel of the feature map separately, so as to integrate global features and edge features, and learn the relationship between the target image and the comparison image under the same receptive field;

[0099] Secondly, a 1×1 convolution kernel is used to perform pointwise convolution on the features of different channels to learn the connection between features in different receptive fields;

[0100] Again, the feature vector after point convolution is straightened, and the structural similarity (SSIM), Euclidean distance and hash similarity (Hashsimilarity) of the target image and the comparison image are added to the end of the straightened vector, thereby improving the feature dimension of the model when performing similarity calculation.

[0101] The multi-dimensional features (such as brightness, contrast and structural information) provided by SSIM can enrich the input features of the neural network and help the model evaluate the similarity of wood texture from multiple aspects. The formula for calculating the structural similarity is shown in the following formula (9):

[0102]

[0103] Where x and y represent the reference texture image and the contrast texture image respectively; μ x and μ y represents the local mean of x and y; σ x and σ y represents the local standard deviation of x and y; σ xy represents the local covariance of x and y; c1 and c2 represent two constants.

[0104] The Euclidean distance can be used in combination with other features to supplement features with higher complexity. The Euclidean distance calculation formula is shown in the following formula (10):

[0105]

[0106] Among them, x (i,j) and (i,j) Respectively represent the pixel values ​​of the reference texture image and the comparison texture image at position (i, j).

[0107] Hash similarity is robust to slight changes in the image (such as rotation, scaling, slight noise, etc.), which can help the model accurately judge the similarity when facing slight changes in wood texture. The hash similarity calculation formula is shown in the following formula (11):

[0108]

[0109] Wherein, H1(i) and H2(i) represent the hash codes of the reference texture image and the comparison texture image; n represents the length of the hash code; Represents a bitwise exclusive-or operation.

[0110] Finally, a fully connected network is used to establish the mapping relationship between texture features and texture similarity, and the texture similarity score is output through the sigmoid function.

[0111] Due to the lack of public datasets in the field of wood texture similarity matching, a self-made wood texture similarity matching dataset is designed. The purpose of this dataset is to simulate human visual perception through machine vision, so that the model can better identify and judge the similarity of wood images; the classification of labels in the dataset is based on the human visual perception of color and texture.

[0112] Class 0 labels: The colors are consistent and the textures are similar, representing the human visual recognition of the same material and surface features;

[0113] Class 1 labels: The color is the same but the texture is different, which is designed to simulate how humans recognize the difference in texture under the same color;

[0114] 2-category labels: The color difference is large, which is intended to examine whether the model can distinguish through texture features under different color conditions; this label classification method can help the model learn features in more dimensions, especially the ability to distinguish similarities and dissimilarities under different conditions, thereby improving the generalization of the model.

[0115] In formulating label 0 and label 1, the impact of similarity based on overall vision and similarity based on content on the experimental results was also considered; if the dataset is annotated based only on overall vision, the image returned by the model may be similar in color to the target image, but not similar in texture details such as the direction and spacing of the stripes; while annotating the dataset based only on content will have the opposite effect; based on these two comparisons, the dataset is designed to clearly distinguish between label 0 (similar in color and texture) and label 1 (same color but different texture), which helps the model accurately identify the wood images that truly have similar texture features required for the experiment; this distinction can help the model better learn the difference between texture structure and overall vision during training, thereby improving the classification accuracy and performance of the model.

[0116] Labeling strategies and their impact on experiments:

[0117] Since labels are manually labeled, they are subjective to a certain extent. This subjectivity may cause some deviations in the data set, but this is exactly in line with the purpose of this experiment: machine vision simulates human visual perception, and human understanding of images is often subjective. Therefore, the subjectivity in the labeling process does not necessarily reduce the effectiveness of the experiment, but can make the model closer to the judgment logic of human vision. However, the subjectivity of manual labels may bring some uncertainties in model training, such as:

[0118] Different annotators may have slightly different definitions of similarity, resulting in insufficient label consistency in the data;

[0119] Some image pairs with blurred boundaries (i.e., cases where color or texture is difficult to judge) may produce different classification results in different experiments;

[0120] In order to reduce the interference caused by subjectivity, the following strategies can be adopted:

[0121] Multi-person scoring and averaging: Invite multiple annotators to score the same pair of images, and take the average value as the final label; this method can effectively reduce the impact of individual differences on the accuracy of the dataset, while improving the consistency and reliability of the labels.

[0122] Label optimization based on model feedback: After preliminary model training, the results are analyzed and labels are adjusted in a targeted manner to make the data more closely match the expected performance of the model.

[0123] Dataset creation:

[0124] like Figure 8 As a data set example, the data set used is a self-made wood texture similarity matching data set; the production of this data set includes three steps: wood texture image acquisition, texture image preprocessing, and label production, as follows:

[0125] During the wood texture image acquisition stage, in order to ensure uniform lighting on the wood surface during the acquisition process and reduce the impact of factors such as exposure on image quality, a fill light was used to fill the shooting environment; then, an Oscar F810C industrial camera was used to acquire 3,000 high-resolution original wood texture images.

[0126] In the texture image preprocessing stage, the wood texture images were expanded to 5000 through rotation, translation, texture generation and other methods; then, the sliding window method was used to cut the original wood texture map into 176*176 pixel wood texture images, and 10000 of them were selected for the production of the wood texture similarity dataset.

[0127] In the labeling stage, two images were randomly selected from the 10,000 processed wood texture images to form an image pair, and a total of 7,588 groups were made; one image in each pair was used as the target image and the other as the comparison image, and then these image pairs were manually classified by similarity; the similarity classification criteria are shown in Table 1 below:

[0128] Table 1 Similarity labels

[0129]

[0130] Experiment and result analysis:

[0131] In the experiment, the data set is randomly divided into a training set and a test set in a ratio of 4:3, where the training set is used for model training and the test set is used for model evaluation. In addition, the WTSM-SiameseNet of the present invention uses the Pytorch deep learning framework, and the Adam optimizer is used in the training process, with a learning rate of 1e-3 and a batch size of 10. The hardware equipment and software platform related to the rest of the experiments are shown in Table 2:

[0132] Table 2 Hardware equipment and software platform

[0133]

[0134] Training process:

[0135] In order to train the model, we first need to set the training parameters, including selecting the optimizer Adam, setting the Epoch to 200, and the Batch Size to 10; secondly, input the texture image pair and its corresponding similarity label; thirdly, use WTSM-SiameseNet to calculate the similarity of the two images; finally, perform a cross entropy loss operation on the similarity category and the similarity label of the image pair to obtain the training loss, and optimize the parameters in WTSM-SiameseNet through back propagation so that the model can more accurately predict the similarity of the input image pair. The calculation of the cross entropy loss function is shown in the following formula (12):

[0136]

[0137] Among them, y represents the similarity label, only the elements in the corresponding class are 1, and the others are 0; Represents the predicted value of the model, which is a vector representing the probability distribution of the model's predicted category; y i is the i-th element in y; yes The i-th element in .

[0138] Evaluation indicators:

[0139] During the model performance evaluation phase, accuracy, precision, recall, and F1-score are used as evaluation indicators of model performance; accuracy represents the probability that the model classifies correctly among all image pairs; precision represents the probability that the model classifies correctly among image pairs classified as having similar textures; recall represents the probability that the model classifies correctly among all image pairs with similar textures; F1-score is the harmonic mean of precision and recall, which is used to comprehensively consider the accuracy and completeness of the model.

[0140] The calculation formulas for accuracy, precision, recall and F1 value are shown in the following equations (13), (14), (15) and (16):

[0141]

[0142] in, TP It represents the probability that the model accurately identifies similar textures; TN represents the probability that the model accurately identifies dissimilar textures; FP It represents the probability that the model misjudges dissimilar textures as similar; FN represents the probability that the model misjudges similar textures as dissimilar.

[0143] Comparative experiment:

[0144] In order to verify the performance of WTSM-SiameseNet in the wood texture similarity matching task, SiameseNet, SE-SiameseNet, Res-SiameseNet and CS-SiameseNet were selected for comparative experiments; among them, SiameseNet is the twin network model based on SiameseNet, the feature extraction module is a 9-layer convolutional network, and the similarity calculation uses a 2-layer fully connected network; SE-SiameseNet is a twin network improved by SEnet50 as the backbone proposed by Yu et al. in 2020; Res-SiameseNet was also proposed by Yu et al. in 2020. This method uses ResNet50 as the backbone and further improves the twin network using the attention mechanism; CS-SiameseNet is a model proposed by Yan et al. in 2023. This method uses VGGnet improved by CBAM attention as the backbone to improve the performance of the twin network; the minimum training loss of each group of models and the number of training rounds for the best model are shown in Table 3 below:

[0145] Table 3. Best loss and optimal model for model training

[0146]

[0147] Comparison of experimental model training loss changes Fig. 9As shown:

[0148] As can be seen from the figure, the original SiameseNet model has the fastest fitting speed, and the loss tends to be flat after the 50th epoch. The losses of the other models tend to be flat after the 110th epoch. This is because the SiameseNet network structure is relatively simple. In terms of the loss of the training model, the WTSM-SiameseNet designed by the present invention performs better than other models.

[0149] In order to further compare the generalization ability of each model, experiments were conducted on the test set. The comparative experimental results are shown in Table 4:

[0150] Table 4 Model generalization comparison experiment

[0151]

[0152] The results in Table 4 show that as a basic model, SiameseNet performs relatively poorly, which may be due to its simple structure, which only uses a 9-layer convolutional network for feature extraction and lacks the ability to learn complex texture features. In addition, using a 2-layer fully connected network for similarity calculation may lead to information loss and affect classification performance. SE-SiameseNet introduces SEnet50 as the backbone, which significantly improves the performance of the model, increasing the accuracy of the model by 4.09% and the precision by 5.36%. This shows that the SE mechanism (Squeeze-and-Excitation) effectively enhances the model's attention to important features, thereby improving accuracy and precision. Compared with SiameseNet, SE-SiameseNet can better capture the subtle differences in texture features. Res-SiameseNet uses ResNet50 as the backbone and uses residual connections to improve gradient propagation, which promotes the training effect of the model. Although it has been greatly improved compared to SiameseNet, it has similar performance compared to SE-SiameseNet, with an improvement of only 1.23% in accuracy. %, and the precision rate was only improved by 0.06%; this may be because the attention mechanism of Res-SiameseNet may not be fully utilized when comparing similarities, resulting in a relatively small improvement; CS-SiameseNet adopts VGGnet with CBAM attention mechanism as the backbone, which greatly improves the model performance; CBAM makes the model more accurate in judging texture similarity by enhancing important features and suppressing irrelevant features, which significantly improves the precision rate and recall rate, which are 4.73% and 2.21% higher than Res-SiameseNet respectively; WTSM-SiameseNet of the present invention makes the model more suitable for the wood texture similarity matching task by improving the texture feature extraction module and the texture feature aggregation matching module; in the test set, the accuracy rate of WTSM-SiameseNet reaches 96.67%, which is 2.86% higher than CSNet; at the same time, the model reaches 97.24% in precision, which is significantly higher than other models; in addition, when the number of parameters of WTSM-SiameseNet is moderate, the model can still achieve high-precision performance, indicating the efficiency of its structural design.

[0153] Optimization experiment

[0154] In order to verify the effectiveness of concurrent attention, the present invention designs an optimization experiment to evaluate the contribution of each attention mechanism to the model performance. It should be noted that in the experiment without edge attention mechanism, due to the change in the number of features, the feature fusion method based on feature correlation cannot be used, so the fusion is performed by subtracting the absolute value of the vector, and the other modules remain consistent. The optimization experiment of concurrent attention mechanism is shown in Table 5 below:

[0155] Table 5 ConcurrentAttention optimization experiment

[0156]

[0157] From the data in the table, we can see that from not using the attention mechanism to only using the CBAM attention mechanism, the accuracy increased by 8.15% and the F1-score increased by 0.080, indicating that the CBAM attention mechanism can help the model focus on key texture information by enhancing important features and suppressing noise, thereby improving classification accuracy; when the concurrent attention mechanism is introduced, the accuracy of the model is further improved to 96.67%, and the F1-score reaches 0.970, indicating that the use of the concurrent attention mechanism further optimizes the model performance in feature extraction and similarity calculation; this also proves that the concurrent attention mechanism combines the advantages of CBAM and edge attention, enabling the model to simultaneously Focusing on global and local features improves the ability to learn complex texture features; this information complementarity allows the model to understand the image content more comprehensively, thereby improving performance; through different types of attention mechanisms, the model may be able to better generalize to unseen samples, reducing overfitting on the training data; this ability improves the performance of the model on the test set; in summary, the attention mechanism significantly improves the model performance, especially the introduction of the concurrent attention mechanism, which further enhances the understanding and classification of texture features; through the combination of different mechanisms, the model can better capture the complex features of wood texture, thereby achieving higher accuracy and F1-score.

[0158] Ablation experiment:

[0159] In order to verify the impact of improvements in the attention mechanism (Attention, Att), feature extraction network (Feature Extraction Network, FEN), feature fusion module (Feature Fusion Module, FFM) and similarity computation module (Similarity Computation Module, SCM) on the model performance, ablation experiments were conducted on the four modules. The ablation test results are shown in Table 6 below:

[0160] Table 6 Ablation experiment of main modules of M-MSRFNet

[0161]

[0162] From the data in Table 6, it can be seen that the use of the concurrent attention mechanism alone will lead to a decrease in model performance. This is because four texture features will be generated when the concurrent attention mechanism is used. If the vector subtraction and absolute value are directly used for fusion, the complexity of the features will increase, thereby increasing the difficulty of similarity calculation. Secondly, the improved feature extraction network effectively alleviates the problem of limited feature receptive field by increasing the dimension of the feature receptive field, thereby improving the quality of the features, and improving the accuracy of the model by 1.13% and the precision by 3.42%; Thirdly, the improved feature fusion module can fuse according to the correlation between features, effectively reducing the damage to the original structure of the features, thereby improving the expressiveness of the features; The experimental results show that the accuracy, precision, recall and F1 of the model are improved by 8.52%, 7.01%, 9.72% and 0.084 respectively; Finally, the improved similarity calculation module uses depthwise separable convolution, although it has limited effect on improving the accuracy of the model, only increasing it by 1.29%, but reducing the number of parameters by 86.8%.

[0163] Wood texture similarity matching example:

[0164] In order to more intuitively observe the matching effect of WTSM-SiameseNet wood texture similarity, three groups of defective wood repair examples are designed. In the early stage of the project, the texture generation of wood defect parts was studied based on texture generation technology. The purpose of this experiment is to use the generated texture as the target image and search for similar boards in the texture library to realize the actual wood repair work. The specific steps of the experiment are as follows:

[0165] First, locate and capture the defective part of the wood in the original image of the solid wood board (Original Image); secondly, use MRS-Transformer to generate textures for the defective part to obtain the generated textures (Generating Textures); thirdly, use WTSM-SiameseNet to compare the generated wood textures with the real textures in the database, and select the matching textures with the highest similarity (Matching Textures); finally, use the matching textures instead of the generated textures to repair the defective parts in the original solid wood board. Fig.10 shown.

[0166] Experiments show that WTSM-SiameseNet shows good results in the task of wood texture similarity matching. With the help of this method, tasks such as wood defect repair can significantly improve the quality and consistency of the repaired texture, and make the texture better meet the subjective perception of the human eye.

[0167] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A wood similar texture matching method based on twin network, characterized in that: The following steps are involved: Step 1: construct a target image and a contrast image of wood texture, and generate a mask image corresponding to the target image and the contrast image; Step 2: Input the target image, the comparison image and the corresponding mask image into the first to fourth MRF-Resnet modules respectively to obtain four multiple receptive field feature maps; Step 3: Input the four multi-receptive field feature maps into the improved feature fusion module for feature fusion, and input the fused features into the depthwise separable convolution and improved similarity calculation module.

2. The wood similar texture matching method based on twin network according to claim 1 is characterized in that: The MRF-Resnet module includes: With Resnet as the backbone, the CBAM module is inserted before and after the basic residual block of stage 1.

3. The wood similar texture matching method based on twin network according to claim 2 is characterized in that: The MRF-Resnet module also includes: performing residual connections on the output feature maps from stage 1 to stage 4 and then performing multiple receptive field fusion.

4. The wood similar texture matching method based on twin network according to claim 3 is characterized in that: Multiple receptive field fusion includes: First, retain the feature maps processed by the bottleneck residual blocks from stage1 to stage4; Secondly, the shape of the feature map processed by the four-stage bottleneck residual block is transformed into Finally, all deformed feature maps are connected in the channel dimension to obtain Multiple receptive field feature maps.

5. The wood similar texture matching method based on twin network according to claim 1, characterized in that: The improved feature fusion module performs feature fusion including: First, the global features of the target image and the edge features of the target image are spliced ​​in the width direction; Secondly, the global features of the contrast image and the edge features of the contrast image are spliced ​​in the width direction; Finally, the spliced ​​target image features and comparison image features are spliced ​​in height to obtain a fused new feature map.

6. The wood similar texture matching method based on twin network according to claim 5 is characterized in that: The fused features are input into the depth-separable convolution and the improved similarity calculation module includes: First, each channel of the feature map is convolved separately using depthwise convolution; Second, use the shape 1×1 The convolution kernel performs point convolution on the features of different channels; Again, the feature vector after point convolution is straightened, and the structural similarity, Euclidean distance and hash similarity of the target image and the comparison image are added to the end of the straightened vector.

7. The wood similar texture matching method based on twin network according to claim 1, characterized in that: The mask image is obtained by multiplying the wood texture image with the mask matrix to obtain an edge texture image.

8. A wood similarity texture matching system based on twin networks, characterized in that: include: a memory for storing instructions executable by a processor; A processor, used to execute instructions to implement the wood similar texture matching method based on a twin network as described in any one of claims 1 to 7.

9. A computer readable medium storing a computer program code, characterized in that: When the computer program code is executed by a processor, the computer program code implements the wood similar texture matching method based on a twin network as described in any one of claims 1 to 7.

Images