A tea grade and disease collaborative detection method based on bilinear feature fusion

Abstract

This invention discloses a method for collaborative detection of tea grade and disease based on bilinear feature fusion. The method involves acquiring tea images and preprocessing them; using a bi-branch approach to extract dual-stream features from the preprocessed tea image data and performing global average pooling; aligning the extracted dual-stream features using spatial projection to map heterogeneous features to the same dimension; applying the Hadamard product to linearly interact with the spatially projected dual-stream features and capturing the nonlinear correlation between the features to obtain the interacted fused features; balancing the dynamic range of the interacted fused features using the signed square root method and normalizing the dynamically balanced features using Euclidean norm; and inputting the normalized features into two independent fully connected heads for tea grade prediction and tea disease identification, respectively.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and smart agriculture technology, and in particular to a method for the collaborative detection of tea grade and disease based on bilinear feature fusion. Background Technology

[0002] In the field of automated tea production and plant protection monitoring, tea grading and disease identification are two core tasks. For example, Jiang Yanhua et al.'s patent (patent number CN202411990035.4) uses machine learning methods for tea quality grading, while Sun Daozong et al.'s patent (patent number CN202111663816.9) uses a lightweight optimized Yolov4 model for tea disease identification. However, existing technical solutions typically treat these two tasks as independent business processes, training and deploying different models separately.

[0003] However, the current task separation approach has significant drawbacks: First, in practical applications, multiple models need to be loaded simultaneously, severely consuming computational resources and making it difficult to meet real-time requirements; second, the grading characteristics and disease characteristics of tea are correlated in the underlying visual information, and task separation prevents models from sharing feature representations, limiting the improvement of generalization ability. Currently, there is no efficient algorithm that can uniformly handle the assessment of healthy tea grading and the diagnosis of disease types. Summary of the Invention

[0004] The purpose of this paper is to address the problems of existing technologies by providing a method for the collaborative detection of tea grade and diseases based on bilinear feature fusion.

[0005] The technical solution adopted to achieve the purpose of this invention is: A method for the synergistic detection of tea grade and disease based on bilinear feature fusion includes the following steps: Step 1: Acquire tea leaf images and preprocess the acquired tea leaf images to obtain a preprocessed tea leaf image dataset. X ; Step 2: Use a two-branch approach on the tea image dataset preprocessed in Step 1. X Perform dual-stream feature extraction and global average pooling; Step 3: Use the spatial projection method to align the dual-stream features extracted in Step 2, mapping the heterogeneous features to the same dimension; Step 4: Use the Hadamard product to perform linear interaction on the two-stream features after spatial projection in Step 3, and capture the nonlinear correlation between the two-stream features to obtain the interactively fused features. Step 5: Use the signed square root method to balance the dynamic range of the features after interactive fusion in Step 4, and use Euclidean norm to normalize the features after dynamic range balancing. Step 6: Input the normalized features from Step 5 into two independent fully connected heads to perform tea grade prediction and tea disease identification, respectively.

[0006] In the above technical solution, step 2 includes the following steps: S201: One branch uses the MobileNet3 network as the backbone module to process the tea image dataset preprocessed in step 1 by inverting the residual structure. X Feature extraction is performed, followed by global average pooling. S202: Another branch uses the ShuffleNetv2 network as the backbone module, and processes the tea image dataset preprocessed in step 1 through channel shuffling and segmentation structures. X Feature extraction is performed, followed by global average pooling.

[0007] In the above technical solution, the MobileNet3 network expression is as follows:

[0008] In the formula, This represents the features extracted using the MobileNet3 network, with dimensions of [missing information]. B ×576; This represents a global average pooling operation; This represents the MobileNetv3 network with the classification header removed; This represents a dataset of tea leaf images.

[0009] In the above technical solution, the ShuffleNetv2 network expression is as follows:

[0010] In the formula, This represents the features extracted using the ShuffleNetv2 network, with dimensions of [dimensionality missing]. B ×1024; This represents a global average pooling operation; This represents the ShuffleNetv2 network with the classification header removed; This represents a dataset of tea leaf images.

[0011] In the above technical solution, step 3 includes: using two linear projection operators to map the two sets of heterogeneous features extracted by MobileNet3 network and ShuffleNetv2 network respectively to a common dimension; The expression for mapping the two sets of heterogeneous features to a common dimension is as follows:

[0012] In the formula, , These represent the features extracted from the two branches after spatial projection, with dimensions of [dimension 1] and [dimension 2]. B ×1024; , Each represents the weight matrix of the projection operator; This represents features extracted using the MobileNet3 network; This represents the features extracted using the ShuffleNetv2 network.

[0013] In the above technical solution, the feature expression after interactive fusion is as follows:

[0014] In the formula, Represents the characteristics after interactive integration; , These represent the features extracted from the two branches after spatial projection; Represents the Hadamard product.

[0015] In the above technical solution, the dynamic range expression of the features after interactive fusion in step 4, which utilizes the signed square root method, is as follows:

[0016] In the formula, Represents the characteristics after dynamic range equilibrium; Represents a symbolic function; Represents the features after interactive fusion; represents the absolute value function; eps This represents a very small positive number to ensure numerical stability. eps =1×10 -8 ; The expression for normalizing the features after dynamic range balancing using Euclidean norm is as follows:

[0017] In the formula, The feature represents the Euclidean norm normalized feature, and its dimension is B ×1024; Represents the features after dynamic range balance; represents the Euclidean norm, which maps vectors to a unit sphere, enhancing training stability.

[0018] In the above technical solution, the two independent fully connected heads include a health level head and a disease type head; in step 5, the normalized features are input into the health level head to predict the level and output the level of healthy tea; in step 5, the normalized features are input into the disease type head to identify diseases and output the types of tea diseases.

[0019] In the above technical solution, the normalized feature input health level head in step 5 is used to perform the level prediction expression as follows:

[0020] In the formula, This represents the predicted grade of the tea. , Each represents a weight matrix representing a health level head; This represents a discard operation; This represents the characteristics after Euclidean norm normalization.

[0021] In the above technical solution, the normalized feature input disease type header in step 5 is used for disease identification expression as follows:

[0022] In the formula, represent; , Each represents a weight matrix for the disease type; This represents a discard operation; This represents the characteristics after Euclidean norm normalization.

[0023] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention solves the problem of discrete deployment and complex maintenance of grade classification and disease classification tasks in traditional schemes by constructing a unified multi-task learning framework.

[0024] 2. This invention extracts the second-order co-occurrence relationship between heterogeneous features through bilinear fusion, and enhances the recognition of micro-texture and pathological distribution by utilizing higher-order interactions, thereby achieving deep synergy between fine-grained grading and disease detection within a single model.

[0025] 3. Compared with the MobileNetv3 method, the bilinear feature fusion-based tea grade and disease co-detection method of this invention improves the classification accuracy of tea grade and disease co-detection by 2.27% and 4.38%, respectively; compared with the ShuffleNetv2 method, it improves the classification accuracy of tea grade and disease co-detection by 5.96% and 110.03%, respectively. Attached Figure Description

[0026] Figure 1The diagram shown is a flowchart of the method for synergistic detection of tea grade and disease based on bilinear feature fusion as described in this invention. Detailed Implementation

[0027] The present invention will be further described in detail below with reference to specific embodiments. Example 1

[0028] A method for the synergistic detection of tea grade and disease based on bilinear feature fusion, see [link to relevant documentation]. Figure 1 This includes the following steps: Step 1: Acquire tea leaf images and preprocess the acquired tea leaf images to obtain a preprocessed tea leaf image dataset. X In this embodiment, the tea images acquired are in RGB format, and the tea image dataset... X The dimension is B ×3×224×224, B This represents the batch size. The number of channels in the tea image is 3, and the height and width of the tea image are both 224.

[0029] The preprocessing of the acquired tea images includes unifying the size of the acquired tea images.

[0030] Step 2: Use a two-branch approach on the tea image dataset preprocessed in Step 1. X Perform dual-stream feature extraction and global average pooling.

[0031] Step 2 includes the following steps: S201: One branch uses the MobileNet3 network as the backbone module to process the tea image dataset preprocessed in step 1 by inverting the residual structure. X Feature extraction is performed, followed by global average pooling.

[0032] The MobileNet3 network expression is as follows:

[0033] In the formula, This represents the features extracted using the MobileNet3 network, with dimensions of [missing information]. B ×576; This represents a global average pooling operation; This represents the MobileNetv3 network with the classification header removed; This represents a dataset of tea leaf images.

[0034] S202: Another branch uses the ShuffleNetv2 network as the backbone module, and processes the tea image dataset preprocessed in step 1 through channel shuffling and segmentation structures. XFeature extraction is performed, followed by global average pooling.

[0035] The ShuffleNetv2 network expression is as follows:

[0036] In the formula, This represents the features extracted using the ShuffleNetv2 network, with dimensions of [dimensionality missing]. B ×1024; This represents a global average pooling operation; This represents the ShuffleNetv2 network with the classification header removed; This represents a dataset of tea leaf images.

[0037] Step 3: Use the spatial projection method to align the dual-stream features extracted in Step 2, mapping the heterogeneous features to the same dimension.

[0038] Step 3 includes: using two linear projection operators to map the two sets of heterogeneous features extracted by the MobileNet3 network and the ShuffleNetv2 network respectively to a common dimension.

[0039] The expression for mapping the two sets of heterogeneous features to a common dimension is as follows:

[0040] In the formula, , These represent the features extracted from the two branches after spatial projection, with dimensions of [dimension 1] and [dimension 2]. B ×1024; , Each represents the weight matrix of the projection operator; This represents features extracted using the MobileNet3 network; This represents the features extracted using the ShuffleNetv2 network.

[0041] Step 4: Use the Hadamard product to perform linear interaction on the two-stream features after spatial projection in Step 3, and capture the nonlinear correlation between the two-stream features to obtain the interactively fused features.

[0042] The feature expression after interactive fusion is as follows:

[0043] In the formula, Represents the characteristics after interactive integration; , These represent the features extracted from the two branches after spatial projection; Represents the Hadamard product.

[0044] Step 5: The dynamic range of the features after interactive fusion in Step 4 is balanced using the Signed Square Root (SSR) method to alleviate the feature value distribution shift caused by element-wise multiplication and suppress the influence of extreme values. The Euclidean norm is used to normalize the features after dynamic range balance, scaling the feature vector to a unit length to enhance the features after interactive fusion.

[0045] The dynamic range expression for balancing the features after interactive fusion in step 4 using the signed square root method is as follows:

[0046] In the formula, Represents the characteristics after dynamic range equilibrium; Represents a symbolic function; Represents the features after interactive fusion; represents the absolute value function; eps This represents a very small positive number to ensure numerical stability. eps =1×10 -8 .

[0047] The expression for normalizing the features after dynamic range balancing using Euclidean norm is as follows:

[0048] In the formula, The feature represents the Euclidean norm normalized feature, and its dimension is B ×1024; Represents the features after dynamic range balance; represents the Euclidean norm, which maps vectors to a unit sphere, enhancing training stability.

[0049] Step 6: Input the normalized features from Step 5 into two independent fully connected heads to perform tea grade prediction and tea disease identification, respectively.

[0050] The two independent fully connected headers include a health grade header and a disease type header. In step 5, the normalized features are input into the health grade header for grade prediction, outputting the grade of healthy tea leaves; in step 5, the normalized features are input into the disease type header for disease identification, outputting the type of tea disease.

[0051] The normalized features from step 5 are input into the health level head for level prediction, as shown in the following expression:

[0052] In the formula, This represents the predicted grade of the tea. , Each represents a weight matrix representing a health level head; This represents a discard operation; This represents the characteristics after Euclidean norm normalization.

[0053] The normalized features from step 5 are input into the disease type header for disease identification, as shown in the following expression:

[0054] In the formula, represent; , Each represents a weight matrix for the disease type; This represents a discard operation; This represents the characteristics after Euclidean norm normalization.

[0055] In this embodiment, tea grade prediction and tea disease identification are denoted as the FMSNet method. As shown in Table 1, the FMSNet method is trained in this embodiment using the Adam optimizer, with a batch size of 32, a learning rate of 0.0001, 20 iterations, and a dropout rate of 0.3. The weights of the loss function for the two classification tasks are set to 0.5, indicating that the two tasks have the same weight.

[0056] Table 1 Parameter settings used in implementation In this embodiment, MobileNet3 and ShuffleNetv2 are used as baselines and compared with FMSNet. Five-fold cross-validation is used, and the simulation results are shown in Table 2.

[0057] Table 2 Results of tea grading and disease identification Table 2 shows the macro-average accuracy results of the three methods, presented as mean ± standard deviation. It can be seen that the macro-average accuracy of the FMSNet method is 85.71% and 89.83%, respectively. Compared with MobileNetv3, the FMSNet method improves accuracy by 1.90% and 3.77% in tea grade and disease co-classification, respectively; compared with ShuffleNetv2, the FMSNet method improves accuracy by 4.82% and 47.06% in tea grade and disease co-classification, respectively. Except for the standard deviation of MobileNetv3 in disease prediction, the standard deviation of the FMSNet method is also lower than that of MobileNetv3 and ShuffleNetv2, indicating that the method in this invention has good reliability.

[0058] For ease of explanation, spatial relative terms such as “up,” “down,” “left,” and “right” are used in the embodiments to describe the relationship of one element or feature shown in the figures relative to another element or feature. It should be understood that, in addition to the orientations shown in the figures, spatial terms are intended to include different orientations of the device in use or operation. For example, if the device in the figures is inverted, an element described as being “down” of other elements or features would be positioned “up” of those other elements or features. Therefore, the exemplary term “down” can encompass both up and down orientations. The device may be positioned in other ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0059] Moreover, relational terms such as “first” and “second” are used merely to distinguish one component from another that has the same name, without necessarily requiring or implying any such actual relationship or order between the components.

[0060] The above description is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for the synergistic detection of tea grade and disease based on bilinear feature fusion, characterized in that, Includes the following steps: Step 1: Acquire tea leaf images and preprocess the acquired tea leaf images to obtain a preprocessed tea leaf image dataset. X ; Step 2: Use a two-branch approach on the tea image dataset preprocessed in Step 1. X Perform dual-stream feature extraction and global average pooling; Step 3: Use the spatial projection method to align the dual-stream features extracted in Step 2, mapping the heterogeneous features to the same dimension; Step 4: Use the Hadamard product to perform linear interaction on the two-stream features after spatial projection in Step 3, and capture the nonlinear correlation between the two-stream features to obtain the interactively fused features. Step 5: Use the signed square root method to balance the dynamic range of the features after interactive fusion in Step 4, and use Euclidean norm to normalize the features after dynamic range balancing. Step 6: Input the normalized features from Step 5 into two independent fully connected heads to perform tea grade prediction and tea disease identification, respectively.

2. The method for synergistic detection of tea grade and disease based on bilinear feature fusion according to claim 1, characterized in that, Step 2 includes the following steps: S201: One branch uses the MobileNet3 network as the backbone module to process the tea image dataset preprocessed in step 1 by inverting the residual structure. X Feature extraction is performed, followed by global average pooling. S202: Another branch uses the ShuffleNetv2 network as the backbone module, and processes the tea image dataset preprocessed in step 1 through channel shuffling and segmentation structures. X Feature extraction is performed, followed by global average pooling.

3. The method for synergistic detection of tea grade and disease based on bilinear feature fusion according to claim 2, characterized in that, The MobileNet3 network expression is as follows: In the formula, This represents the features extracted using the MobileNet3 network, with dimensions of [missing information]. B ×576; This represents a global average pooling operation; This represents the MobileNetv3 network with the classification header removed; This represents a dataset of tea leaf images.

4. The method for synergistic detection of tea grade and disease based on bilinear feature fusion according to claim 2, characterized in that, The ShuffleNetv2 network expression is as follows: In the formula, This represents the features extracted using the ShuffleNetv2 network, with dimensions of [dimensionality missing]. B ×1024; This represents a global average pooling operation; This represents the ShuffleNetv2 network with the classification header removed; This represents a dataset of tea leaf images.

5. The method for synergistic detection of tea grade and disease based on bilinear feature fusion according to claim 1, characterized in that, Step 3 includes: using two linear projection operators to map the two sets of heterogeneous features extracted by MobileNet3 network and ShuffleNetv2 network respectively to a common dimension; The expression for mapping the two sets of heterogeneous features to a common dimension is as follows: In the formula, , These represent the features extracted from the two branches after spatial projection, with dimensions of [dimension 1] and [dimension 2]. B ×1024; , Each represents the weight matrix of the projection operator; This represents features extracted using the MobileNet3 network; This represents the features extracted using the ShuffleNetv2 network.

6. The method for synergistic detection of tea grade and disease based on bilinear feature fusion according to claim 1, characterized in that, The feature expression after interactive fusion is as follows: In the formula, Represents the characteristics after interactive integration; , These represent the features extracted from the two branches after spatial projection; Represents the Hadamard product.

7. The method for synergistic detection of tea grade and disease based on bilinear feature fusion according to claim 1, characterized in that, The dynamic range expression for balancing the features after interactive fusion in step 4 using the signed square root method is as follows: In the formula, Represents the characteristics after dynamic range equilibrium; Represents a symbolic function; Represents the characteristics after interactive integration; represents Absolute value function; eps This represents a very small positive number to ensure numerical stability. eps =1×10 -8 ; The expression for normalizing the features after dynamic range balancing using Euclidean norm is as follows: In the formula, The feature represents the Euclidean norm normalized feature, and its dimension is B ×1024; Represents the characteristics after dynamic range equilibrium; Representing the Euclidean norm, Euclidean norm normalization maps vectors to a unit sphere, enhancing the stability of training.

8. The method for synergistic detection of tea grade and disease based on bilinear feature fusion according to claim 1, characterized in that, Two independent fully connected headers include a health grade header and a disease type header; in step 5, the normalized features are input into the health grade header for grade prediction, and the grade of healthy tea is output; in step 5, the normalized features are input into the disease type header for disease identification, and the type of tea disease is output.

9. The method for synergistic detection of tea grade and disease based on bilinear feature fusion according to claim 8, characterized in that, The normalized features from step 5 are input into the health level head for level prediction, as shown in the following expression: In the formula, This represents the predicted grade of the tea. , Each represents a weight matrix representing a health level head; This represents a discard operation; This represents the characteristics after Euclidean norm normalization.

10. The method for synergistic detection of tea grade and disease based on bilinear feature fusion according to claim 8, characterized in that, The normalized features from step 5 are input into the disease type header for disease identification, as shown in the following expression: In the formula, represent; , Each represents a weight matrix for the disease type; This represents a discard operation; This represents the characteristics after Euclidean norm normalization.

Images