Plant image classification method

Abstract

The invention relates to the field of computer vision and image processing, discloses a plant image classification method, and solves the problems that the existing plant image classification technology is poor in generalization ability and low in classification precision, and lacks self-diagnosis classification uncertainty and a continuous optimization mechanism, so that difficult sample classification errors and repetition occur. According to the scheme, the method comprises the steps of obtaining a to-be-classified plant image and performing preprocessing; performing feature extraction on the plant image through a pre-trained feature extraction network to obtain a plurality of image features including different abstract levels; fusing the plurality of image features to obtain a fused feature, and according to the fused feature, generating a classification probability that the plant image belongs to each of the plurality of plant categories; determining an uncertainty metric value of the classification result according to the classification probability; and if the uncertainty metric value satisfies a preset condition, sending the plant image to a labeling terminal for requesting manual labeling of a label, and updating the feature extraction network according to the received manual labeling label and the plant image.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing, specifically to a method for classifying plant images. Background Technology

[0002] Plant image classification is an important application of computer vision technology in agriculture, forestry, ecology, and education. This technology enables rapid species identification and classification through automated analysis of digital plant images, providing an efficient tool for biodiversity surveys, crop growth monitoring, rare plant protection, and popularization of botanical knowledge. Especially with the widespread use of mobile internet and smart terminals, it meets the urgent need for users to instantly obtain plant species information by taking photos, driving the technology towards higher precision and stronger robustness.

[0003] Existing plant image classification technologies mostly employ rule-based implementation schemes. The core logic involves pre-defining the selection priority of plant parts (e.g., prioritizing the identification of specific parts such as flowers, fruits, and leaves) and relying on manually designed feature extraction rules (such as fixed feature templates for texture, shape, and color) for classification decisions. However, such methods have some technical limitations: Firstly, the key discriminative features of plants vary significantly across different species, and real-world shooting scenarios often involve issues such as target parts being occluded, incomplete, or subject to complex lighting conditions, making it impossible for predefined part selection priorities to dynamically adapt to the specific content of the image. Secondly, manually designed feature extraction rules lack the ability to mine complex semantic information in plant images, making it difficult to cope with the variability of plant morphology at different growth stages and under different environments.

[0004] The aforementioned defects directly lead to insufficient generalization ability of existing classification models, making them prone to classification bias in real natural scenes and making it difficult to further improve classification accuracy. At the same time, the models lack the ability to self-diagnose their own classification results, cannot identify difficult samples with low confidence, and have no continuous optimization mechanism, resulting in the recurrence of similar errors, which seriously restricts the reliable application of plant image classification technology in real-world scenarios. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a plant image classification method that solves the problems of poor generalization ability, low classification accuracy, lack of self-diagnosis of classification uncertainty and continuous optimization mechanism in existing plant image classification technology, which leads to repeated classification errors of difficult samples.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0007] On the one hand, the present invention provides a plant image classification method, comprising the following steps:

[0008] S1. Obtain images of the plants to be classified;

[0009] S2. The plant image is subjected to feature extraction through a pre-trained feature extraction network to obtain multiple image features containing different levels of abstraction;

[0010] S3. The multiple image features are fused to obtain fused features. The fusion process includes: generating weights corresponding to each image feature based on the multiple image features, and performing a weighted summation of the multiple image features based on the weights.

[0011] S4. Based on the fused features, generate the classification probability of the plant image belonging to each of the multiple plant categories;

[0012] S5. Determine the uncertainty measure of the classification result based on the classification probability;

[0013] S6. If the uncertainty metric meets the preset conditions, the plant image is sent to the annotation terminal to request manual labeling, and the feature extraction network is updated based on the received manual labeling and the plant image.

[0014] Furthermore, in step S2, feature extraction is performed on the plant image using a pre-trained feature extraction network, including: extracting image features from the shallow, medium, and deep layers of the feature extraction network, respectively. The image features extracted by the shallow network are low-level features including edge or color texture features; the image features extracted by the medium network are medium-level features including local shape feature points; and the image features extracted by the deep network are high-level features including global morphology or semantic context. The abstraction levels of the low-level, medium-level, and high-level features increase sequentially.

[0015] Furthermore, in step S2, the feature extraction network employs a deep convolutional neural network (CNN).

[0016] Furthermore, in step S3, generating weights corresponding to each image feature based on the multiple image features includes:

[0017] The multiple image features are then stitched together;

[0018] The concatenated features are compressed and activated in the channel dimension to generate a channel attention weight vector. The element values ​​in the weight vector are positively correlated with the importance of the corresponding image features.

[0019] Furthermore, the process of compressing and incentivizing the concatenated features to generate channel attention weight vectors includes: first, performing global average pooling on the concatenated features to obtain channel description vectors; then, performing dimensionality reduction and dimensionality increase incentivizing operations through two fully connected layers, and generating the channel attention weight vectors through the Sigmoid activation function.

[0020] Furthermore, in step S5, determining the uncertainty measure of the classification result based on the classification probability includes:

[0021] Calculate the information entropy of the classification probability and use the information entropy as a measure of uncertainty.

[0022] Furthermore, in step S6, the method for determining whether the uncertainty measure value meets the preset conditions includes:

[0023] If the uncertainty metric is greater than the preset threshold, then the uncertainty metric is determined to meet the preset condition.

[0024] Furthermore, in step S6, the feature extraction network is updated, including:

[0025] The plant images and their corresponding manually labeled entries are added as new samples to the training dataset.

[0026] The feature extraction network was fine-tuned using the expanded training dataset.

[0027] Secondly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described plant image classification method.

[0028] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described plant image classification method.

[0029] The beneficial effects of this invention are:

[0030] (1) Improved classification accuracy and generalization ability:

[0031] This invention employs a deep convolutional neural network to automatically extract shallow, medium, and deep multi-layer image features. The shallow layer captures basic information such as edges and textures, the medium layer represents component features such as local shapes, and the deep layer mines global morphology and semantic context, comprehensively covering key discriminative information of plant images. Simultaneously, it combines an attention mechanism to generate adaptive weights, enhancing highly discriminative features and suppressing secondary features, dynamically focusing on the core region of the image. This effectively avoids the problem that fixed location selection priorities cannot adapt to complex scenes, significantly improving the model's classification accuracy and generalization ability in real natural environments.

[0032] (2) Achieve continuous iterative optimization of the model:

[0033] This invention introduces an uncertainty quantification evaluation mechanism based on information entropy, which can automatically identify difficult samples with low classification confidence and accurately locate the model's capability boundaries. Through an active learning loop, such samples are pushed to an annotation terminal to obtain accurate expert labels, and then the model parameters are fine-tuned with new samples, achieving continuous iteration of identifying difficult samples, supplementing with high-quality data, and optimizing model performance. This mechanism addresses model shortcomings with minimal manual annotation costs, solving the problems of traditional methods' inability to self-diagnose errors and lack of continuous optimization capabilities leading to the recurrence of similar errors. With long-term use, classification performance can be continuously improved. Attached Figure Description

[0034] Figure 1 This is a flowchart of the plant image classification method in an embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram of multi-level feature extraction in an embodiment of the present invention.

[0036] Figure 3 This is a flowchart of the feature fusion process based on the attention mechanism in an embodiment of the present invention.

[0037] Figure 4 This is a schematic diagram of the active learning closed loop in an embodiment of the present invention. Detailed Implementation

[0038] This invention aims to provide a plant image classification method that addresses the problems of poor generalization ability, low classification accuracy, and lack of self-diagnosis of classification uncertainty and continuous optimization mechanisms in existing plant image classification technologies, leading to repeated misclassification of difficult samples. The core idea is to extract shallow, medium, and deep multi-level features from plant images using a pre-trained CNN model, adaptively fuse these features using an attention mechanism to accurately represent plant information, quantify uncertainty based on the information entropy of classification probabilities, and then manually label and supplement the training set of difficult samples with high uncertainty to fine-tune the model, thereby improving classification accuracy and generalization ability.

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] This embodiment provides a plant image classification method that automatically mines multi-dimensional semantic features of plant images through deep neural networks, replacing manually designed rules and improving adaptability to complex scenes. It also leverages an attention mechanism to dynamically focus on key discriminative information, enhancing the accuracy of feature representation. Furthermore, by introducing uncertainty quantification and an active learning closed loop, it achieves continuous model iteration with minimal manual cost, ensuring long-term optimization of classification performance. For detailed implementation procedures, please refer to [link to implementation details]. Figure 1 This includes the following steps:

[0041] S1. Plant Image Acquisition and Preprocessing:

[0042] The core objective of this step is to acquire standardized input images, eliminate irrelevant interference factors, and lay the foundation for subsequent feature extraction. Specifically, raw digital images of the plants to be classified are acquired using image acquisition devices such as smartphones, field monitoring cameras, and scanners. The acquisition scenarios can cover natural growing environments, greenhouse cultivation scenarios, etc., and the image formats support common formats such as JPG and PNG, ensuring compatibility with the practical needs of various acquisition terminals.

[0043] Next, standardization preprocessing is performed on the acquired raw images: First, the image size is uniformly scaled to a fixed size (e.g., 256x256 pixels) to ensure the efficiency of subsequent processing; second, image normalization is performed to convert the pixel values ​​from the range of 0-255 to between 0-1, or mean and variance standardization is performed to eliminate interference caused by different lighting conditions and improve the robustness of the model.

[0044] S2. Multi-level image feature extraction:

[0045] This step extracts features at different levels of abstraction using a pre-trained deep convolutional neural network (CNN) to comprehensively cover the key discriminative information of plant images. Specifically, a ResNet50 model pre-trained on the ImageNet dataset or a large plant-specific dataset can be selected as the feature extraction network. This model has the characteristics of moderate depth, strong feature representation ability, and excellent generalization performance, and can be effectively transferred to plant image classification tasks.

[0046] The process of using this feature extraction network to extract image features from different depth levels is described in [link to documentation]. Figure 2 The details are as follows:

[0047] Shallow feature extraction: Low-level features (feature map F1) are extracted from the conv1 or layer1 layers of ResNet50. These features focus on basic image information, including edges, corners, color textures and basic gradient features, which can accurately capture intuitive attributes such as plant surface material and color.

[0048] Intermediate feature extraction: Intermediate features (feature map F2) are extracted from the layer 2 layer of ResNet50. These features are composed of low-level features and can characterize the local shape and organ parts of the plant, such as the serrated outline of the leaves and the morphological structure of the petals.

[0049] Deep feature extraction: High-level features (feature map F3) are extracted from layer 4 of ResNet50. These features are highly abstract semantic features that can reflect the global morphology, overall outline and spatial context relationship between different organs of the plant.

[0050] The extracted shallow, medium, and deep features are all in the form of feature maps, and their number of channels is consistent with the output of the corresponding network layer.

[0051] S3. Feature fusion based on attention mechanism:

[0052] This step achieves efficient fusion of multi-level features through adaptive weight allocation, strengthening key features, suppressing redundant information, and addressing the problem of poor fusion results caused by differences in the importance of different features. For details, please refer to the feature fusion process. Figure 3 This includes the following processes:

[0053] Feature preprocessing: Since the spatial dimensions of the shallow, medium and deep feature maps are different, they need to be unified first: bilinear interpolation upsampling is used for the deep and medium feature maps, and adaptive average pooling is used for the shallow feature maps. Finally, all feature maps are adjusted to a uniform spatial dimension to ensure the feasibility of subsequent splicing and weighting operations.

[0054] Feature stitching: The three-layer feature maps with uniform size are stitched together along the channel dimension to form a comprehensive feature block.

[0055] Adaptive Weight Generation: The concatenated composite feature block is input into a Squeeze-and-Excitation (SE) module. This module first compresses the feature map of each channel into a scalar using global average pooling (squeeze operation), resulting in a channel description vector. Then, this vector is passed through two fully connected layers (excitation operation). The first fully connected layer reduces the dimensionality, and the second layer increases the dimensionality. Finally, a channel attention weight vector between 0 and 1 is generated using the Sigmoid activation function. Each value in this weight vector represents the importance of the corresponding channel feature.

[0056] The method for generating weights is not limited to the SE module. For example, a self-attention mechanism can also be used: first, multiple image features are serialized, and an attention weight matrix is ​​generated by calculating the correlation between the query, key, and value. The elements in this matrix represent the strength of the correlation between different image feature regions, thus serving as the basis for weighting. Alternatively, weights can be generated by performing a dot product between a simple learnable parameter vector and the features, followed by Softmax normalization.

[0057] Weighted fusion and feature aggregation: The generated attention weight vector is multiplied channel by channel with the concatenated integrated feature block to enhance the features of high-importance channels and suppress the features of low-importance channels. Then, global average pooling is performed on the weighted feature map to aggregate it into a fixed-dimensional fusion feature vector, which retains key discrimination information and reduces the computational complexity of subsequent operations.

[0058] The weighted summation operation can be performed at the following two levels:

[0059] Level 1: Channel Weighting. This involves using the channel attention weight vector generated by the SE module and performing channel-by-channel multiplication directly with the concatenated feature blocks (i.e., multiplying the feature map of each channel by a scalar weight). This process is called weighting. Subsequently, all the weighted feature maps can be directly added together to obtain a fused feature map, which can then be flattened or pooled to obtain the feature vector.

[0060] Level 2: Spatial Weighting. If a self-attention mechanism is used, the generated weight matrix can be used to weight the importance of different positions (i.e., different image feature regions) in the feature sequence, and then the weighted sequence is summed and pooled to obtain the final feature vector.

[0061] S4. Classification probability generation:

[0062] This step predicts the probability of plant categories based on the fused feature vector, providing a foundation for subsequent uncertainty assessment. Specifically, the fused feature vector obtained in step S3 is input into a fully connected layer classifier. The number of output nodes of this classifier is the same as the number of target plant categories N (N is a positive integer, such as 1000), followed by a Softmax activation function to normalize the output into a probability distribution vector. .in, This indicates that the image belongs to the first... The predicted probability of plant-like organisms, and satisfying the following conditions: and This probability distribution vector This is the final "classification probability". This probability distribution intuitively reflects the model's confidence level in each class.

[0063] S5. Calculation of uncertainty measure:

[0064] This step quantifies the reliability of the model's classification results using information entropy, accurately identifying problematic samples and providing a basis for triggering active learning. Specifically, to evaluate the reliability of the classification results generated in step S4, the classification probability distribution vector is used as a basis. Calculate its information entropy as a measure of uncertainty. The calculation formula is as follows:

[0065] ;

[0066] in, The calculated uncertainty measure is a non-negative real number. The higher the value, the more uncertain the model's classification result for the current image; The lower the value, the more confident the model is. Represents the classification probability distribution vector The Middle The element that the image is predicted to be is the th element. The probability of a class. This represents a traversal index, which is a category number, and its value ranges from... (Total number of categories). Represented by natural constant The base is the natural logarithm function. In other embodiments of the present invention, a base-2 or base-10 logarithm function may also be used, both of which measure the uncertainty of the probability distribution. The summation symbol represents summation over all categories. The calculation results are summed.

[0067] Information entropy value The higher the entropy value, the more uniform the probability distribution (i.e., the model believes that the probability of it belonging to many classes is about the same), and the less certain the model is about its prediction results; the lower the entropy value, the more certain the model is that it belongs to a certain specific category.

[0068] S6. Active Learning and Model Update:

[0069] This step establishes a closed loop of labeling challenging samples and fine-tuning the model by determining an uncertainty threshold, thereby continuously optimizing classification performance. Specifically, the process of active learning and model updating can be found in [link to relevant documentation]. Figure 4 ,include:

[0070] Judgment: The system presets an uncertainty threshold T (e.g., T=0.8).

[0071] Low uncertainty handling: If U <= T, the model classification result is considered highly reliable, and the class with the highest probability is directly output as the final classification result, and the process ends.

[0072] Handling high uncertainty (active learning loop): If U > T, then the following process is triggered:

[0073] a. Request manual annotation: The system automatically pushes the plant image and its current predicted probability distribution to the expert annotation platform. After reviewing the image, domain experts provide accurate plant species labels.

[0074] The technical implementation of sending data to the annotation terminal includes: the system internally calls a predefined application programming interface (API), sending image data, unique identifiers, and uncertainty values ​​as parameters to the server where the annotation terminal is located via HTTP / HTTPS network protocol. Upon receiving the data, the server stores it in a task queue database and updates and displays the task list on the web front-end or application interface used by the expert, prompting the expert to process it.

[0075] b. Model Update: The system adds the (image, expert label) pair as a new training sample to the existing training dataset. Periodically, or when a certain number of new samples have accumulated, the model fine-tuning process is initiated. This involves using the current model as a starting point and the expanded dataset to update and train the parameters of the feature extraction network and classifier with a small learning rate. This allows the model to make more accurate predictions when encountering similar challenging samples in the future.

[0076] The model update strategy can be selected based on the requirements of computing resources and the degree of forgetting of old knowledge: First, as shown in this embodiment, the feature extraction network and the classifier can be fine-tuned at the same time; Second, in order to save computing resources and maintain the stability of feature extraction capability, only the parameters of the classifier layer can be updated while the weights of the feature extraction network are frozen; Third, a more complex incremental learning algorithm can be used to update the network.

[0077] Although embodiments of the present invention have been described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present invention, and all such changes and alterations shall not depart from the protection scope of the present invention.

Claims

1. A plant image classification method, characterized in that, Includes the following steps: S1. Obtain images of the plants to be classified; S2. The plant image is subjected to feature extraction through a pre-trained feature extraction network to obtain multiple image features containing different levels of abstraction; S3. The multiple image features are fused to obtain fused features. The fusion process includes: generating weights corresponding to each image feature based on the multiple image features, and performing a weighted summation of the multiple image features based on the weights. S4. Based on the fused features, generate the classification probability of the plant image belonging to each of the multiple plant categories; S5. Determine the uncertainty measure of the classification result based on the classification probability; S6. If the uncertainty metric meets the preset conditions, the plant image is sent to the annotation terminal to request manual labeling, and the feature extraction network is updated based on the received manual labeling and the plant image.

2. The plant image classification method as described in claim 1, characterized in that, In step S2, feature extraction is performed on the plant image using a pre-trained feature extraction network, including: extracting image features from the shallow, medium, and deep layers of the feature extraction network, respectively. The image features extracted by the shallow network are low-level features including edge or color texture features; the image features extracted by the medium network are medium-level features including local shape feature points; and the image features extracted by the deep network are high-level features including global morphology or semantic context. The abstraction levels of the low-level, medium-level, and high-level features increase sequentially.

3. The plant image classification method as described in claim 2, characterized in that, In step S2, the feature extraction network employs a deep convolutional neural network.

4. The plant image classification method as described in claim 1, characterized in that, In step S3, weights corresponding to each image feature are generated based on the multiple image features, including: The multiple image features are then stitched together; The concatenated features are compressed and activated in the channel dimension to generate a channel attention weight vector. The element values ​​in the weight vector are positively correlated with the importance of the corresponding image features.

5. The plant image classification method as described in claim 4, characterized in that, The process of compressing and activating the concatenated features to generate channel attention weight vectors includes: first, performing a global average pooling compression operation on the concatenated features to obtain a channel description vector; then, performing dimensionality reduction and dimensionality increase activation operations through two fully connected layers, and generating the channel attention weight vectors through a Sigmoid activation function.

6. The plant image classification method as described in claim 1, characterized in that, In step S5, the uncertainty metric of the classification result is determined based on the classification probability, including: Calculate the information entropy of the classification probability and use the information entropy as a measure of uncertainty.

7. The plant image classification method as described in claim 6, characterized in that, In step S6, the determination method for whether the uncertainty measure value meets the preset conditions includes: If the uncertainty metric is greater than the preset threshold, then the uncertainty metric is determined to meet the preset condition.

8. A plant image classification method as described in any one of claims 1-7, characterized in that, In step S6, the feature extraction network is updated, including: The plant images and their corresponding manually labeled entries are added as new samples to the training dataset. The feature extraction network was fine-tuned using the expanded training dataset.

9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the plant image classification method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the plant image classification method as described in any one of claims 1 to 8.

Images