Chrysanthemum classification method based on front and back fusion cross attention mechanism

Abstract

The application discloses a chrysanthemum classification method based on front and back fusion cross attention mechanism, comprising the following steps: on the basis of Swin Transformer, a double-path neural network model is built by using a cross attention mechanism combined with a residual module design; front and back images of chrysanthemums in a chrysanthemum image database are taken as double-path neural network inputs for model training and testing, and the trained model is taken as a chrysanthemum type prediction model; front and back images of chrysanthemums to be detected are collected and input into the chrysanthemum type prediction model, and a classification result is output through the chrysanthemum type prediction model. The chrysanthemum classification method based on front and back fusion cross attention mechanism has the advantages of fast chrysanthemum type identification speed, high accuracy and strong stability.

Description

Technical Field

[0001] This invention belongs to the field of medicinal material detection technology, and relates to the identification technology of dried chrysanthemum species, specifically to a chrysanthemum classification method based on a positive and negative cross-attention mechanism. Background Technology

[0002] Chrysanthemums originated in my country and, as one of the country's ten traditional famous flowers, are deeply loved by the people and play an important role in my country's material and cultural life. Moreover, chrysanthemums are renowned for their unique ornamental value, rich economic value, and medicinal value. Chrysanthemums contain various chemical components, including flavonoids, volatile oils, phenylpropanoids, terpenoids, and amino acids, among which flavonoids and phenylpropanoids are their main medicinal components. Their pleasant fragrance, captivating color, and significant medicinal value have led to their widespread use in food, tea, decoration, and medicine. However, the quality and price of chrysanthemums are affected by their place of origin. Therefore, tracing the origin of chrysanthemums is of great significance for ensuring product quality, enhancing consumer trust, promoting sustainable industrial development, and meeting market demand, and has become one of the important directions in modern agricultural development.

[0003] Traditional methods for determining the origin of chrysanthemums mainly fall into two categories: plant phenotypic analysis and physicochemical analysis. Plant phenotypic analysis focuses on observing the external characteristics of chrysanthemums, combining sensory factors such as smell and taste to determine their origin. However, this method often requires experienced appraisers and is subject to subjectivity and uncertainty, hindering efficient and large-scale testing of Chinese medicinal materials. In the physicochemical field, modern testing instruments, such as high-performance liquid chromatography (HPLC), are used by many third-party testing institutions to identify the origin of Chinese medicinal materials. However, these analytical methods also suffer from drawbacks such as complex sample pretreatment, long analysis times, and high costs, making them difficult to use frequently and efficiently in the circulation of Chinese medicinal materials. In this context, we can utilize computer vision technology to extract feature information from images of chrysanthemums and use image classification methods to achieve rapid identification of chrysanthemum origins. This method not only improves the detection speed and accelerates the speed and efficiency of chrysanthemum origin identification but also has significant implications for chrysanthemum classification.

[0004] The patent document with publication number CN117690135A discloses a chrysanthemum classification method based on the fusion of front and back depth visual features. Although it has achieved the identification and classification of chrysanthemum species, its recognition performance is not very good when faced with complex or difficult-to-distinguish images, and it is difficult to achieve the ideal recognition effect. Summary of the Invention

[0005] Purpose of the invention: In order to overcome the shortcomings of the existing technology, a chrysanthemum classification method based on the positive and negative cross-attention mechanism is provided, which can accurately identify the origin of chrysanthemums and realize the intelligent and accurate identification of chrysanthemum varieties.

[0006] Technical Solution: To achieve the above objectives, this invention provides a daisy classification method based on a positive-negative fusion cross-attention mechanism, comprising the following steps:

[0007] S1: Based on the Swin Transformer, a dual-path neural network model is built by combining cross-attention mechanism with residual module design.

[0008] S2: Use the front and back images of chrysanthemums in the chrysanthemum image database as inputs to a dual-path neural network for model training and testing, and use the trained model as a chrysanthemum type prediction model.

[0009] S3: Collect front and back images of the chrysanthemum to be detected and input them into the chrysanthemum type prediction model. The chrysanthemum type prediction model will output the classification result.

[0010] Furthermore, in step S1, the dual-path neural network model uses the Swin Transformer as the backbone of the dual-stream network to extract features from the front and back images of chrysanthemums. The input front and back images are processed by the Swin Transformer to obtain the features of the front and back images, and then the features from the upper and lower paths are concatenated. Finally, the final output feature vector is obtained through a multilayer perceptron (fully connected layer). The dual-stream design considers the possibility that different types of chrysanthemums may have similar front features but dissimilar back features, or vice versa. It fully utilizes and integrates the front and back features of chrysanthemums to make more accurate identification of chrysanthemum types.

[0011] Furthermore, the Swin Transformer includes a Linear Embedding layer and four Stages. The four Stages are composed of 2, 2, 18, and 2 Swin Transformer Blocks, respectively. The first three Stages are equipped with PatchMerging layers for downsampling, while the last Stage is not equipped with a PatchMerging layer. The Swin TransformerBlock alternately uses window attention and moving window attention mechanisms.

[0012] Furthermore, the method of using the cross-attention mechanism combined with the residual module in step S1 is as follows: A cross-attention mechanism is added between adjacent stages in the Swing Transformer. The residuals of the front flow, the output of the front flow, and the residuals of the back flow are respectively used as QKV values. The correlation between the residuals and outputs of the front flow is calculated to obtain an attention score. This score is then multiplied by the residual of the back flow to obtain the front-back overlapping features. At this point, the feature information of the front flow is affected by the feature information of the back image, thus it can be considered that the two types of feature information have been fused. The calculation formula for the cross-attention mechanism is as follows:

[0013] Q n =X re sW Q

[0014] K n =XW K

[0015] V n =Y re sW V

[0016]

[0017] Among them, X res X is the residual of the current stream, X is the output of the previous layer of the current stream, and Y is the residual of the current stream. res It is another kind of residual, W Q W K and W V Both are linear mappings, d k Given the dimension K, the final output of the cross-attention layer is:

[0018] X out =CrossAttention+X in

[0019] By introducing skip connections, the original features of the input layer can be preserved, while the cross-attention layer only needs to learn the feature fusion part between the two paths. This helps to solve the problems of gradient vanishing and gradient explosion, effectively reduces the decay of gradient correlation, stabilizes gradients, prevents network degradation, and can also help the model converge faster.

[0020] Furthermore, in the Swing Transformer, the residual of the forward flow comes from Stage 1, and the residual of the reverse flow comes from the previous Stage. The specific formula is as follows:

[0021]

[0022] in, and These are the residuals of the previous layer of the input cross-attention layer, X. 0 Y is a feature extracted from an image after passing through a LinearEmbedding layer. n-1 This is the output of the previous stage, and F is the direct mapping. This design allows deeper features to benefit from information from lower-level features, helping the network better capture features at different scales and levels of abstraction in the image.

[0023] Furthermore, in the dual-path neural network model of step S1, the features from the upper and lower paths are concatenated to obtain a feature vector, which is then passed through a fully connected layer. A BatchNorm layer, a Dropout layer, and a GELU activation function are added in between to finally obtain the output feature vector. The BatchNorm layer ensures that the input of each layer remains within a relatively stable range, helping to alleviate the gradient vanishing problem. The Dropout layer reduces the interaction between neurons, forcing the network to learn more robust and generalized feature representations, thus helping to reduce overfitting. GELU is a non-linear function that introduces non-linear mappings into the neural network, and its expression is as follows:

[0024]

[0025] Furthermore, in step S2, all chrysanthemum images undergo preprocessing before being input into the network. Specifically, each captured image is first randomly rotated left and right by 15°. Next, all images are normalized and scaled to a fixed image size to ensure a uniform input format, increase data diversity and generalization ability, and facilitate subsequent image processing and feature extraction.

[0026] Furthermore, the specific method for training the chrysanthemum variety prediction model in step S2 is as follows: the pre-trained weights are loaded into the upper and lower network paths respectively to facilitate rapid network convergence. The Adam optimizer is used to optimize the model, and cross-entropy loss is selected as the loss function. After each training session, the model is tested on the validation set. When a better result is obtained, the model trained this time is saved to obtain the optimal solution of the network.

[0027] Furthermore, in step S2, the network model is trained using a loss function that minimizes the cross-entropy, the formula of which is as follows:

[0028]

[0029] Among them, y i For the true value, These are predicted values.

[0030] Furthermore, the prediction method of the chrysanthemum variety prediction model in step S3 is as follows: the chrysanthemum variety prediction model uses Softmax to transform the output feature vector into a probability matrix, and takes the category with the highest probability as the final classification result. The formula for Softmax is as follows:

[0031]

[0032] Among them, z i It is the i-th element of the input vector, σ(z) i ) is the i-th element of the probability distribution output by the Softmax function.

[0033] This invention introduces the Swing Transformer and cross-attention mechanism combined with residual module design. Its powerful representation ability can better extract the deep features of chrysanthemum images, especially the details and global information of complex images, which is more effective in capturing them and solves the problem of poor recognition effect of complex or difficult-to-distinguish images.

[0034] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0035] 1. This invention adopts a neural network model recognition method based on the fusion of front and back depth visual features. Compared with the time-consuming, labor-intensive and unstable manual identification, it has the advantages of being fast, non-destructive, convenient, low-cost, highly accurate and stable.

[0036] 2. This invention utilizes a cross-attention mechanism combined with residual module design based on the Swing Transformer. Window attention and moving window attention mechanisms are used alternately within the Swing Transformer Block. By introducing skip connections, the original features of the input layer can be preserved, while the cross-attention layer only needs to learn the feature fusion part between the two paths. This helps to solve the problems of gradient vanishing and gradient exploding, effectively reduces the decay of gradient correlation, stabilizes gradients, prevents network degradation, and also helps the model converge faster.

[0037] 3. The interactive mechanism provided by this invention takes into account the characteristics that different types of chrysanthemums have similar frontal features but dissimilar backal features, or similar backal features but dissimilar frontal features. It can better integrate the features of both front and back, make more accurate identification of chrysanthemum types, and make accurate identification even when faced with complex or difficult-to-distinguish images.

[0038] 4. This invention provides a new method for identifying the origin and variety of chrysanthemums, effectively meeting the needs for accurate and rapid traceability testing of chrysanthemum origins in different scenarios, and further ensuring the quality and safety of the Chinese medicinal materials supply chain. Furthermore, this invention can also be applied to more industries and fields such as food, pharmaceuticals, and industrial raw materials. Attached Figure Description

[0039] Figure 1 This is a schematic flowchart of the method of the present invention;

[0040] Figure 2 This is a display of images of chrysanthemum medicinal materials from different origins in this embodiment;

[0041] Figure 3 This is a schematic diagram of window attention and moving window attention in the chrysanthemum species prediction model. Detailed Implementation

[0042] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0043] like Figure 1 As shown, this invention provides a daisy classification method based on a positive-negative fusion cross-attention mechanism, comprising the following steps:

[0044] S1: Based on the Swin Transformer, a dual-path neural network model is built by combining cross-attention mechanism with residual module design:

[0045] Reference Figure 1 The chrysanthemum variety prediction model uses the Swin Transformer as the backbone of a two-stream network, leveraging its powerful feature representation capabilities to extract features from both the front and back images of chrysanthemums. The input front and back images are processed through a LinearEmbedding layer and four stages to obtain the features of both sides. These features are then concatenated and finally passed through a multilayer perceptron to obtain the final output feature vector. The two-stream design considers the possibility that different chrysanthemum varieties may have similar frontal features but dissimilar back features, or vice versa. By fully utilizing and fusing the front and back features of chrysanthemums, the model can more accurately identify chrysanthemum varieties.

[0046] The Swin Transformer consists of four stages, each composed of 2, 2, 18, and 2 SwinTransformer Blocks respectively. The first three stages include a Patch Merging layer for downsampling, while the final stage does not. The Swin Transformer Blocks internally alternate between window attention and moving window attention mechanisms, as detailed below. Figure 3 As shown.

[0047] In this invention, a cross-attention mechanism is added between adjacent stages in the Swing Transformer. The residuals of the front flow, the output of the front flow, and the residuals of the back flow are respectively used as QKV values. The correlation between the residuals and outputs of the front flow is calculated to obtain an attention score. This score is then multiplied by the residual of the back flow to obtain the front-back overlapping feature. At this point, the feature information of the front flow is influenced by the feature information of the back image, thus it can be considered that the two types of feature information have been fused. The calculation formula for the cross-attention mechanism is as follows:

[0048] Q n =X res W Q

[0049] K n =XW K

[0050] V n =Y res W V

[0051]

[0052] Among them, X res X is the residual of the current stream, X is the output of the previous layer of the current stream, and Y is the residual of the current stream. res It is another kind of residual, W Q W K and W V Both are linear mappings, d k Given the dimension K, the final output of the cross-attention layer is:

[0053] X out =CrossAttention+X in

[0054] By introducing skip connections, the original features of the input layer can be preserved, while the cross-attention layer only needs to learn the feature fusion part between the two paths. This helps to solve the problems of gradient vanishing and gradient explosion, effectively reduces the decay of gradient correlation, stabilizes gradients, prevents network degradation, and can also help the model converge faster.

[0055] Through this interaction, the characteristics of different chrysanthemum species being similar in positive features but dissimilar in negative features, or similar in negative features but dissimilar in positive features, are taken into account, so as to better integrate the positive and negative features and make more accurate identification of chrysanthemum species.

[0056] In this embodiment, the input is a 224*224*3 image of the front and back of a chrysanthemum. This image is passed through a 4*4 linear embedding layer with a stride of 4 and padding of 0, resulting in an output of 56*56*96. After Stage 1, the image size is 28*28*192; after Stage 2, it's 14*14*384; after Stage 3, it's 7*7*768. Stage 4 does not have a PatchMerging (downsampling) layer, so the image size remains unchanged after the final stage. Three cross-attention modules are inserted between the four stages; these do not alter the image feature size. Finally, a flat pooling layer is applied, resulting in an output of 1*1*768. The features from the upper and lower paths are concatenated to obtain a 1*1536 dimensional feature vector. This feature vector is then passed through a fully connected layer to become 1*512, with a BatchNorm layer, Dropout layer, and GELU activation function added in between. This process is repeated to obtain the final 1*18 dimensional feature vector. The BatchNorm layer keeps the input of each layer within a relatively stable range, helping to alleviate the vanishing gradient problem. The Dropout layer reduces the interaction between neurons, forcing the network to learn more robust and generalized feature representations, which helps reduce overfitting. GELU is a non-linear function that introduces non-linear mappings into neural networks; its expression is as follows:

[0057]

[0058] Compared to ReLU, the GELU function has a non-zero gradient near the origin, reducing the gradient vanishing problem during training. Furthermore, its derivative is smooth and uninterrupted, making backpropagation easier.

[0059] S2: Use front and back images of chrysanthemums from the chrysanthemum image database as inputs to a dual-path neural network for model training and testing. Use the trained model as a chrysanthemum type prediction model.

[0060] Regarding model training:

[0061] The dataset in this embodiment contains 18 varieties of chrysanthemums from 17 counties and cities in 8 provinces, including Anhui, Henan, and Jiangsu. Each variety has 150 images of both the front and back, totaling approximately 5700 images. (Details are as follows...) Figure 2 As shown.

[0062] All chrysanthemum images undergo preprocessing before being input into the network: First, each captured image is randomly rotated left and right by 15°. Next, all images are normalized and scaled to a fixed size to ensure a uniform input format, increase data diversity and generalization ability, and facilitate subsequent image processing and feature extraction.

[0063] In this embodiment, the dataset is divided into training, validation, and test sets in a 6:2:2 ratio. At the start of training, the pre-trained weights of the Swin Transformer are loaded into the upper and lower network layers respectively to facilitate rapid network convergence. The number of iterations is set to 50, the learning rate to 0.0002, and the batch size to 24. The Adam optimizer is used to optimize the model, with cross-entropy loss chosen as the loss function. After each training iteration, the model is tested on the validation set. If a better result is found, the model from this training iteration is saved to obtain the optimal solution for the network. To avoid experimental errors caused by random sampling, the predicted train-test process is repeated 50 times, and the average accuracy is taken.

[0064] S3: Reference Figure 1 The front and back images of the chrysanthemum to be detected are collected and input into the chrysanthemum type prediction model. The front and back images of the chrysanthemum are used as two inputs to the network model. The features of the two inputs are concatenated and then output through a fully connected layer to obtain the classification result of the chrysanthemum type.

[0065] The chrysanthemum variety prediction model first obtains a 1*18 feature matrix, then uses Softmax to transform the output feature vector into a probability matrix, and takes the category with the highest probability as the final classification result. The formula for Softmax is as follows:

[0066]

[0067] Among them, z i It is the i-th element of the input vector, σ(z) i ) is the i-th element of the probability distribution output by the Softmax function.

[0068] To verify the classification performance of the model of this invention, the classification accuracy of the chrysanthemum category prediction model is analyzed in this embodiment. Table 1 shows the error situation of chrysanthemum category prediction. Model 1 is the ConvNextV2 model, Model 2 is a single-stream Swin Transformer that only utilizes the frontal image, and Model 3 is the network model used in this invention. It can be seen that the currently popular ConvNextV2 does not perform very well in chrysanthemum image category recognition, with an accuracy of only 86.72%. The single-stream Swin Transformer shows improvement, but it does not fully utilize the features of the chrysanthemum image, achieving an accuracy of 93.97%. The model of this invention fully utilizes the features of both front and back sides and fully integrates them through an attention mechanism, resulting in a further improvement in accuracy, reaching 96.41%.

[0069] Table 1 Errors in Chrysanthemum Origin Prediction

[0070]

[0071]

[0072] The experimental results above show that the present invention can effectively integrate the features of the front and back images of chrysanthemums and accurately identify the types of chrysanthemums.

Claims

1. A chrysanthemum classification method based on front and back fusion cross attention mechanism, characterized by, Includes the following steps: S1: Based on the Swin Transformer, a dual-path neural network model is built by combining cross-attention mechanism with residual module design. S2: Use the front and back images of chrysanthemums in the chrysanthemum image database as inputs to a dual-path neural network for model training and testing, and use the trained model as a chrysanthemum type prediction model. S3: Collect front and back images of the chrysanthemum to be detected and input them into the chrysanthemum type prediction model. The chrysanthemum type prediction model will output the classification result. In step S1, the dual-path neural network model uses the Swin Transformer as the backbone of the dual-path network to extract features from the front and back images of the chrysanthemum. The input front and back images are processed by the Swin Transformer to obtain the features of the front and back images. Then, the features of the upper and lower paths are concatenated and finally passed through a multilayer perceptron to obtain the final output feature vector. The Swin Transformer consists of a Linear Embedding layer and four Stages. The four Stages are composed of 2, 2, 18, and 2 Swin Transformer Blocks, respectively. The first three Stages have Patch Merging layers for downsampling, while the last Stage does not have a Patch Merging layer. The Swin Transformer Blocks alternately use window attention and moving window attention mechanisms. The method for designing the cross-attention mechanism combined with the residual module in step S1 is as follows: A cross-attention mechanism is added between adjacent stages in the Swing Transformer. The residuals of the forward flow, the output of the forward flow, and the residuals of the reverse flow are respectively used as QKV values. The correlation between the residuals and outputs of the forward flow is calculated to obtain the attention score, which is then multiplied by the residual of the reverse flow to obtain the forward and reverse overlap features. The calculation formula for the cross-attention mechanism is as follows: ； ； ； ； in, It is the residual of the current flow. It is the output of the layer above the current stream. It is another kind of residual. , and They are all linear mappings. yes The dimension, and the final output of the cross-attention layer is: ； In the Swin Transformer, the residual of the forward flow comes from Stage 1, and the residual of the reverse flow comes from the previous Stage. The specific formulas are as follows: ； ； in, and These are the residuals of the previous layer of the input cross-attention layer. These are features extracted from an image after passing through a LinearEmbedding layer. This is the output of the previous stage. It is a direct mapping.

2. The chrysanthemum classification method based on a positive-negative fusion cross-attention mechanism according to claim 1, characterized in that, In the dual-path neural network model of step S1, the features from the upper and lower paths are concatenated to obtain a feature vector, which is then passed through a fully connected layer. A BatchNorm layer, a Dropout layer, and the GELU activation function are added in between to finally obtain the output feature vector. The BatchNorm layer keeps the input of each layer within a relatively stable range, helping to alleviate the gradient vanishing problem. The Dropout layer reduces the interaction between neurons, forcing the network to learn more robust and generalized feature representations, which helps reduce overfitting. GELU is a non-linear function that introduces non-linear mappings into the neural network, and its expression is as follows: 。 3. The chrysanthemum classification method based on a positive-negative fusion cross-attention mechanism according to claim 1, characterized in that, In step S2, all chrysanthemum images need to be preprocessed before being input into the network. Specifically, each captured image is randomly rotated left and right by 15°; all images are normalized and scaled to a fixed image size.

4. The chrysanthemum classification method based on a positive-negative fusion cross-attention mechanism according to claim 1, characterized in that, The specific method for training the chrysanthemum variety prediction model in step S2 is as follows: the pre-trained weights are loaded into the upper and lower network paths respectively to facilitate rapid network convergence. The Adam optimizer is used to optimize the model, and cross-entropy loss is selected as the loss function. After each training session, the model is tested on the validation set. When a better result is obtained, the model trained this time is saved to obtain the optimal solution of the network.

5. The chrysanthemum classification method based on a positive-negative fusion cross-attention mechanism according to claim 4, characterized in that, In step S2, the network model is trained using a loss function that minimizes the cross-entropy, and its formula is as follows: ； in, For the true value, These are predicted values.

6. The chrysanthemum classification method based on a positive-negative fusion cross-attention mechanism according to claim 1, characterized in that, The prediction method of the chrysanthemum type prediction model in step S3 is as follows: the chrysanthemum type prediction model uses Softmax to transform the output feature vector into a probability matrix, and takes the category with the highest probability as the final classification result; the formula of Softmax is as follows: ； where z i is the i-th element of the input vector, σ(z i ) is the i-th element of the probability distribution output by the Softmax function.

Images