Image recognition method and device, electronic equipment and storage medium

Abstract

The application provides an image recognition method and device, electronic equipment and storage medium, wherein the method comprises: acquiring an image to be recognized; performing local feature extraction on the image to be recognized to obtain local features; constructing a global feature representation of the image based on the correlation between each block in the local features and the position information of each block to obtain the global feature representation; and performing recognition on the global feature representation to obtain an image recognition result. The image recognition method and device, electronic equipment and storage medium provided by the application can construct a global feature representation of the image based on the correlation between each block in the local features and the position information of each block, and the global feature representation obtained thereby has the representation ability of local subtle features and the representation ability of global semantic information. Image recognition based on the global feature representation can reduce misclassification and improve the accuracy of recognition.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an image recognition method, apparatus, electronic device, and storage medium. Background Technology

[0002] In agricultural production, crop diseases are a major factor causing a decline in crop yield and quality, directly impacting agricultural economic benefits. To ensure high-quality and high-yield crops, accurate and timely detection and treatment of crop diseases have become an indispensable part of agricultural production.

[0003] Traditional methods for identifying crop diseases mainly rely on manual labor, which is not only costly and time-consuming, but also requires the personnel to have a certain level of disease knowledge. Although existing deep learning methods can effectively reduce labor costs, they have consistently performed poorly in crop disease identification, and their identification efficiency and accuracy need to be improved. Summary of the Invention

[0004] This invention provides an image recognition method, apparatus, electronic device, and storage medium to address the shortcomings of poor efficiency and accuracy in image recognition in the prior art.

[0005] This invention provides an image recognition method, comprising:

[0006] Acquire the image to be recognized;

[0007] Local features are extracted from the image to be identified to obtain local features;

[0008] Based on the correlation between each patch in the local features and the positional information of each patch, a global feature representation of the image to be identified is constructed to obtain the global feature representation;

[0009] The global feature representation is then identified to obtain the image recognition result.

[0010] According to the image recognition method provided by the present invention, the step of constructing a global feature representation of the image to be recognized based on the correlation between each patch in the local features and the positional information of each patch, to obtain the global feature representation, includes:

[0011] Based on the correlation between each patch in the local features and the positional information of each patch, a global feature representation of the image to be identified is constructed to obtain a feature sequence;

[0012] Based on the importance of each tile feature in the feature sequence, feature selection is performed on the feature sequence to obtain the global feature representation.

[0013] According to the image recognition method provided by the present invention, the step of selecting features from the feature sequence based on the importance of each patch feature in the feature sequence includes:

[0014] Based on the correlation between every two feature tiles in the feature sequence, the importance score of each feature tile is determined.

[0015] Feature selection is performed on the feature sequence based on the importance scores of each tile feature.

[0016] According to the image recognition method provided by the present invention, the feature selection based on the importance scores of the features of each image patch includes:

[0017] Based on the importance score, a first number of fixed tile features are selected from each tile feature;

[0018] Based on the number of initial tile features and the cumulative distribution information corresponding to the importance scores, a first number of dynamic tile features are selected from the tile features.

[0019] Feature selection is performed on the feature sequence based on the first number of fixed tile features and the second number of dynamic tile features.

[0020] According to the image recognition method provided by the present invention, the step of performing feature selection on the feature sequence based on the importance of each patch feature in the feature sequence to obtain the global feature representation includes:

[0021] Based on the correlation between each patch and the position information of each patch in the previous global feature representation, a global feature representation of the image to be identified is constructed to obtain the current feature sequence;

[0022] Based on the importance of each tile feature in the current feature sequence, feature selection is performed on the current feature sequence to obtain the current global feature representation;

[0023] The current global feature representation is used as the previous global feature representation, and the process of reconstructing the global feature representation of the image is repeated until the number of times the global feature representation of the constructed image is executed reaches a preset number.

[0024] According to the image recognition method provided by the present invention, the step of extracting local features from the image to be recognized to obtain local features, constructing a global feature representation of the image to be recognized based on the correlation between each patch in the local features and the position information of each patch, obtaining a global feature representation, and recognizing the global feature representation to obtain an image recognition result includes:

[0025] The image to be recognized is input into the image recognition model to obtain the image recognition result output by the image recognition model;

[0026] The image recognition model is trained based on sample images and their labels.

[0027] According to the image recognition method provided by the present invention, the image to be recognized is a crop image, and the image recognition result is a crop disease recognition result.

[0028] The present invention also provides an image recognition device, comprising:

[0029] An image acquisition unit is used to acquire the image to be recognized.

[0030] The local feature extraction unit is used to extract local features from the image to be identified, thereby obtaining local features;

[0031] A global feature representation construction unit is used to construct a global feature representation of the image to be identified based on the correlation between each patch in the local features and the position information of each patch, thereby obtaining a global feature representation;

[0032] The recognition unit is used to recognize the global feature representation to obtain the image recognition result.

[0033] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image recognition method as described above.

[0034] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image recognition method as described above.

[0035] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image recognition method as described above.

[0036] The image recognition method, apparatus, electronic device, and storage medium provided by this invention construct a global feature representation of the image to be recognized based on the correlation between each patch in the local features and the positional information of each patch. The resulting global feature representation has the ability to represent local subtle features and also has the ability to represent global semantic information. Image recognition based on this global feature representation can reduce misclassification and improve the accuracy of recognition. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0038] Figure 1 This is one of the flowcharts illustrating the image recognition method provided by the present invention;

[0039] Figure 2 This is one of the flowcharts illustrating step 130 in the image recognition method provided by the present invention;

[0040] Figure 3 This is one of the flowcharts illustrating the feature selection method provided by the present invention;

[0041] Figure 4 This is the second flowchart illustrating the feature selection method provided by the present invention;

[0042] Figure 5 This is the third flowchart illustrating the feature selection method provided by the present invention;

[0043] Figure 6 This is a schematic diagram of the structure of the image recognition model provided by the present invention;

[0044] Figure 7 This is the second flowchart illustrating the image recognition method provided by the present invention;

[0045] Figure 8 This is a schematic diagram of the image recognition device provided by the present invention;

[0046] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] In existing schemes, crop disease identification methods can generally be divided into the following two types:

[0049] (1) Traditional methods for identifying crop diseases

[0050] Traditional methods for identifying crop diseases mainly rely on manual labor. This involves observing collected crop disease samples manually or analyzing them with the aid of relevant equipment, combined with knowledge of the diseases, to determine whether the crops have been affected and, if so, the type of disease.

[0051] This method requires the personnel to have a certain level of disease knowledge, which is quite demanding for ordinary farmers, making it difficult to achieve good results in identifying crop diseases. While it can achieve good results for technicians with professional knowledge, it undoubtedly involves high labor costs, low efficiency, and is not suitable for widespread application.

[0052] (2) Using deep learning methods for identification

[0053] Most deep learning methods are based on the idea of ​​image classification. They use convolutional neural networks to extract features from crop disease images and then classify the images based on the extracted features to achieve the goal of automatic identification of crop diseases.

[0054] Compared to traditional identification methods, this approach can effectively reduce labor costs and quickly identify the occurrence and type of crop diseases. However, it has consistently performed poorly in crop disease identification. The main reason is that, compared to conventional image classification tasks, crop diseases exhibit diversity and similarity. Existing methods lack the ability to extract discriminative subtle features and are easily influenced by factors such as crop leaf shape and background, learning redundant features and leading to misidentification of crop diseases, thus resulting in poor accuracy.

[0055] Based on this, the present invention provides an image recognition method that extracts local features of the image to be recognized and constructs a global feature representation of the image based on the correlation between each patch in the local features and the positional information of each patch. The resulting global feature representation has the ability to represent local subtle features and also has the ability to represent global semantic information of the image. Image recognition based on this global feature can reduce misclassification and improve the accuracy of recognition.

[0056] It should be noted that the image recognition method provided by the present invention can be applied not only to the disease recognition of crops, but also to other scenarios with diversity and similarity between categories, such as facial expression recognition, medical image recognition, or text recognition, etc. The embodiments of the present invention do not make specific limitations in this regard.

[0057] The technical solution of the image recognition method of the present invention will be described in detail below with reference to the accompanying drawings. Figure 1This is one of the flowcharts illustrating the image recognition method provided by this invention. This method can be applied to terminals, servers, or systems including both terminals and servers, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be a standalone server or a server cluster composed of multiple servers, or it can be a node in a blockchain. Figure 1 As shown, the image recognition method provided in this embodiment of the invention may include the following steps:

[0058] Step 110: Obtain the image to be recognized.

[0059] Specifically, the image to be identified can be any image that requires image recognition, such as images of crops, faces, or medical images. There must be at least one image to be identified.

[0060] This invention does not limit the method of acquiring the image to be identified. The image to be identified can be scanned by a scanner, captured by a document camera, mobile device, or downloaded from the internet, received from a device, or even an image from a video. For example, the image to be identified can be an image imported by the user from another device. For instance, the user imports an image from a Universal Serial Bus (USB) flash drive into the electronic device, allowing the electronic device to acquire that image, i.e., the electronic device acquires the image to be identified.

[0061] Step 120: Extract local features from the image to be recognized to obtain local features.

[0062] Specifically, considering the diversity and similarity of crop diseases, for example, in identifying crop diseases, it is necessary to pay attention to the discriminative subtle features, i.e., local features, in order to further improve the accuracy of disease identification. These local features reflect the discriminative subtle features in the image that require special attention.

[0063] To obtain local features, this can be achieved by extracting local features from the image to be recognized. Furthermore, this can be achieved through a local feature extraction module. Considering that Convolutional Neural Networks (CNNs) are excellent at extracting local features from images, this local feature extraction module can specifically be a CNN module. The local feature extraction module can include multiple cascaded sub-modules, each of which includes a convolutional layer, a normalization layer, and an activation layer. Specifically, the convolutional layer performs convolution operations on the image to be recognized; the normalization layer performs layer normalization on the image features obtained from the convolution to prevent gradient explosion and gradient vanishing; and the activation layer introduces nonlinear factors, mapping the image features to a high-dimensional nonlinear region.

[0064] It should be noted that other models besides the CNN module can also be used for local feature extraction, such as the scale-invariant feature transform (SIFT) algorithm, etc., which can be flexibly selected according to actual needs.

[0065] Step 130: Based on the correlation between patches in the local features and the positional information of each patch, construct a global feature representation of the image to be recognized, and obtain the global feature representation.

[0066] Specifically, considering that local features can characterize subtle features that are discriminative, but are not powerful enough to represent the global features of the image to be recognized, a global feature representation of the image can be further constructed based on the local features.

[0067] To obtain a global feature representation, it can be constructed by building a global feature representation of the image to be recognized based on the correlation between patches in the local features and the positional information of each patch. The construction of the global feature representation of the image can be achieved through a global feature representation construction module.

[0068] Preferably, the global feature representation construction module can be an attention-based encoding module that performs tensor flattening on local features, dividing them into multiple equally sized patches. The position information of each patch can be a positional code calculated using a formula, or it can be obtained by encoding the position of each patch using a trained positional encoder.

[0069] The global feature representation building block can include one or more concatenated sub-modules. Each sub-module consists of a multi-head attention (MSA) with residual structure and a multilayer perceptron (MLP). Local features are standardized using layer norm (LN) before being input into these two modules.

[0070] The local features and positional information of each image patch are input into the global feature representation construction module. When constructing the global feature representation of the image to be recognized, the global feature representation construction module can capture the long-distance representation of the features based on the correlation between each image patch, so that the obtained global feature representation has the ability to represent the image globally.

[0071] Preferably, considering that the global feature representation may contain patches that are not important for image recognition, in order to further reduce the computational load of the model, feature filtering can be performed on each patch in the global feature representation, and the global feature representation can be determined based on the feature sequence after feature filtering.

[0072] Step 140: Recognize the global feature representation to obtain the image recognition result.

[0073] Specifically, after obtaining the global feature representation, a classification layer can be used to identify the global feature representation to obtain the image recognition result.

[0074] For example, when the image to be identified is related to crops, the above steps can be used to obtain a global feature representation of the crop image, and disease identification can be performed based on the global feature representation to identify whether the crop has a disease and the type of disease.

[0075] When the image to be identified is a medical image, a global feature representation of the medical image can be obtained, and lesion type identification can be performed based on the global feature representation to identify the lesion type in the medical image.

[0076] The method provided in this invention constructs a global feature representation of the image to be recognized based on the correlation between each patch in the local features and the positional information of each patch. The resulting global feature representation has the ability to represent local subtle features, and at the same time has the ability to represent the global semantic information of the image. Image recognition based on this global feature representation can reduce misclassification and improve the accuracy of recognition.

[0077] Based on the above embodiments, Figure 2 This is one of the flowcharts illustrating step 130 in the image recognition method provided by the present invention, such as... Figure 2 As shown, step 130 specifically includes:

[0078] Step 131: Based on the correlation between each patch in the local features and the positional information of each patch, construct the global feature representation of the image to be recognized to obtain the feature sequence;

[0079] Step 132: Based on the importance of each patch feature in the feature sequence, perform feature selection on the feature sequence to obtain a global feature representation.

[0080] Specifically, a Tokens-to-Tokens Vision Transformer (T2T-ViT model) can be used to construct a global feature representation of the image. The local features of each image patch, the positional information of each image patch, and the classification token are input into the T2T-ViT model to obtain the feature sequence (Token sequence) corresponding to the image to be identified, which is used as the global feature representation.

[0081] When using the Transformer network to construct a global feature representation of an image, the network will iteratively learn the feature representation by associating any two tokens in the sequence with each other through the self-attention mechanism in the network structure, given a given number of token input sequences. In this way, the complexity of the model is quadratic with the number of input tokens. Especially when the image input is large, the computational load of the model increases dramatically, resulting in significant resource consumption and time cost.

[0082] To address this, the present invention proposes a feature selection method that selects features based on the importance of each tile feature in the feature sequence, retaining tile features with higher importance and deleting tile features with relatively lower importance.

[0083] The feature sequence includes categorical features and patch features, where the importance of each patch feature is relative to the categorical features. The higher the correlation between any patch feature and the categorical feature, the more important that patch feature is for classifying the input image category, at least at the current stage. Therefore, the higher the importance of that patch feature, the greater the probability that it will be retained during feature selection. Conversely, the lower the importance of any patch feature, the lower the probability that it will be retained during feature selection.

[0084] The method provided in this invention selects features based on the importance of each patch feature, reduces the number of patch features, thereby reducing computational costs, avoiding the impact of redundant patch features on feature extraction, further enhancing feature representation capabilities, and thus improving the accuracy of subsequent image recognition.

[0085] Based on any of the above embodiments Figure 3 This is one of the flowcharts illustrating the feature selection method provided by the present invention, such as... Figure 3 As shown, step 132 specifically includes:

[0086] Step 132-1: Determine the importance score of each feature based on the correlation between every two feature pairs in the feature sequence;

[0087] Step 132-2: Based on the importance scores of each patch feature, perform feature selection on the feature sequence.

[0088] Specifically, the importance of each tile feature can be measured by an importance score. The importance score of each tile feature can be obtained by the correlation between every two tile features in the feature sequence. The correlation between every two tile features in the feature sequence can be determined by a self-attention matrix.

[0089] Assume the self-attention matrix output after the transformer layer is A, and the number of tokens is m+1, where 1 represents the first classification token in the feature sequence. The self-attention matrix represents the correlation between every two tile features. The importance score of each tile feature can be calculated using the following formula:

[0090]

[0091] In formula (1), a 1,i and a 1,j Let h represent the elements in the 1st row and i-th column of the self-attention matrix A, and the elements in the 1st row and j-th column, respectively; j The score is the feature score of the j-th patch.

[0092] Based on the importance scores of each patch, some patch features can be removed from the self-attention matrix according to their importance scores. For example, patch features with lower importance scores can be removed and those with higher importance scores can be retained, thus achieving feature selection.

[0093] Based on any of the above embodiments Figure 4 This is a second schematic flowchart of the feature selection method provided by the present invention, as shown below. Figure 4 As shown, step 132-2 specifically includes:

[0094] Steps 132-21: Based on importance scores, select a first number of fixed patch features from each patch feature;

[0095] Steps 132-22: Based on the number of initial tile features and the cumulative distribution information corresponding to the importance scores, select a second number of dynamic tile features from each tile feature;

[0096] Steps 132-23: Feature selection is performed based on a first number of fixed tile features and a second number of dynamic tile features.

[0097] Specifically, considering that the construction of global feature representation and feature selection may occur multiple times, if only the highest-scoring patch features are selected and retained, the discarded low-scoring patch features are not necessarily unimportant for the final classification. Different patch features may play different roles and represent different feature meanings at different stages. If a patch feature is directly deleted simply because its score is low in a certain intermediate stage, then the deleted patch feature cannot participate in the subsequent construction of global feature representation, but it may play an important role in a later stage, which may affect the final recognition result.

[0098] Based on this, this invention proposes an adaptive dynamic selection method that combines fixed tile feature selection and dynamic tile feature selection.

[0099] Fixed tile features are selected by ranking the importance scores of tile features from highest to lowest, and then selecting a certain number of high-scoring tile features (n). For example, 3-7 tile features can be selected as the first number of fixed tile features based on their scores from highest to lowest.

[0100] Dynamic tile features are selected based on the importance scores of each tile feature, so that high-scoring tile features are more likely to be retained, while low-scoring tile features are less likely to be retained.

[0101] The specific implementation strategy involves calculating the cumulative distribution function corresponding to the feature scores of the image patches, using the following formula:

[0102]

[0103] In formula (2), j = 2 indicates that the first classification feature is excluded, and the selection starts from the second feature. After obtaining the cumulative distribution function (CDF), the patch feature selection function can be obtained from its inverse form, as shown in the following formula:

[0104] η(α)=CDF -1 (α) (3)

[0105] The value of α is between 0 and 1. Specifically, a number α is randomly selected from a uniform distribution between 0 and 1, η(α) is calculated, and the nearest integer is selected as the index of the selected tile feature. This operation is repeated R times. During these R times, a tile feature may be selected multiple times, resulting in an actual number of dynamic tile features of size r, where r ≤ R.

[0106] This shows that the number of dynamic tile features is not only related to the importance score of each tile, but also to the number of initial tile features. In practice, the value of R can usually be set to 70% of the number of input tile features in the initial stage.

[0107] Finally, the selected patch features are all the selected patch features in n and r. After removing duplicates, the classification features are added to obtain the global feature representation after feature selection.

[0108] The feature selection method based on fixed and dynamic tile features provided in this invention adaptively and dynamically selects tile features. This method is applied after each transformer layer for feature selection, which not only removes redundant feature information and enhances feature representation capabilities, but also acts as a pruning mechanism for the network, thereby reducing computational costs and constructing a lightweight network structure.

[0109] Based on any of the above embodiments Figure 5 This is the third flowchart illustrating the feature selection method provided by the present invention, as shown below. Figure 5 As shown, step 132 specifically includes:

[0110] Step 132-3: Based on the correlation between each patch and the positional information of each patch in the previous global feature representation, construct the global feature representation of the image to be recognized to obtain the current feature sequence;

[0111] Step 132-4: Based on the correlation between every two patches in the current feature sequence, perform feature selection on the current feature sequence to obtain the global feature representation of the current time.

[0112] Step 132-5: Use the current global feature representation as the previous global feature representation, and return to construct the global feature representation of the image until the number of times the global feature representation of the image is constructed reaches the preset number.

[0113] Specifically, in order to make the constructed global feature representation more accurate in terms of global representation capability, the construction of global feature representation and feature selection can be performed multiple times, and feature selection is performed after each execution of global feature representation construction, until the number of times global feature representation construction is performed reaches the preset number.

[0114] In some embodiments, the number of times the global feature representation is constructed can be 12. Therefore, the global feature representation construction module can include 12 layers, each layer being used to construct the global feature representation for the current iteration. Correspondingly, the feature selection module also includes 12 layers, each layer being used to perform feature selection for the current iteration.

[0115] Understandably, local features are input into the first layer of the global feature representation construction module. After the first global feature representation is constructed, the first feature sequence is obtained. The first feature sequence is then input into the first layer of the feature selection module. After feature selection, the first global feature representation is obtained. The first global feature representation is then input into the second layer of the global feature representation construction module. After the second global feature representation is constructed, the second feature sequence is obtained. The second feature sequence is then input into the second layer of the feature selection module. After feature selection, the second global feature representation is obtained. This process is repeated 12 times until the final global feature representation is obtained.

[0116] The method provided in this invention obtains the final global feature representation after constructing and selecting multiple layers of global feature representation. Feature selection is performed after each global feature representation construction, which can not only remove redundant feature information and enhance the representation ability of key features, but also play a role in pruning the network, thereby reducing computational costs and constructing a lightweight network structure.

[0117] Based on any of the above embodiments, steps 110-140 may specifically include:

[0118] The image to be recognized is input into the image recognition model, and the image recognition result output by the image recognition model is obtained;

[0119] The image recognition model is trained based on sample images and their labels.

[0120] Specifically, the image recognition method provided in this embodiment of the invention can be implemented through an image recognition model. After obtaining the image to be recognized, the image to be recognized can be preprocessed, including scale adjustment, normalization and other operations. Then, the image to be recognized is input into the trained image recognition model. The model inferences to obtain the scores of each preset category, and the category with the highest score is selected as the recognition result.

[0121] Before this, an initial recognition model can be constructed, and the initial recognition model can be trained based on sample images and the labels of the sample images to obtain a trained image recognition model.

[0122] In some embodiments, the image to be identified is a crop image, and the image recognition result is a crop disease identification result.

[0123] Figure 6 This is a schematic diagram of the image recognition model provided by the present invention, as shown below. Figure 6 As shown, the image recognition model includes a local feature extraction module, which is used to extract local features from the image to be recognized, and obtain local features. The local feature extraction module can be a CNN module.

[0124] It also includes a global feature representation construction module, which is used to construct a global feature representation of the image to be recognized based on the correlation between each patch in the local features and the position information of each patch, so as to obtain a global feature representation. The global feature representation construction module can be a Transformer encoding module.

[0125] It also includes a recognition module, which is used to recognize the global feature representation and obtain the image recognition result.

[0126] Preferably, the image recognition model may further include a feature selection module, used to select features from the feature sequence based on the importance of each patch feature in the feature sequence, to obtain a global feature representation. The feature selection method can be found in the description of the above embodiments.

[0127] The CNN module consists of five CNN-blocks concatenated to extract features from the input crop disease image at different levels. The Transformer encoding module comprises 12 Transformer Layers to construct a global feature representation of the image. After each Transformer Layer, a feature selection module selects extracted features from the token dimension, retaining key features and removing redundant information. Finally, the classification layer (recognition module) yields the final classification result.

[0128] The CNN module consists of five CNN-blocks, each including convolutional layers, normalization layers, and activation layers. The first four CNN-blocks have the same structure, using convolution operations with a kernel size of 3×3 and a stride of 2 to extract local features at different levels from the input image. The last CNN-block uses a convolution operation with a kernel size of 1×1 and a stride of 1 to enhance information fusion between channels and adjust the number of channels for subsequent use. The input image is first resized to a fixed size of 448*448*3 before being input into the CNN module for feature extraction, outputting a feature vector of size 28*28*768.

[0129] The feature vector of size 28*28*768 obtained by the CNN module is compressed to 784*768 by the Flatten operation, and the position encoding vector and classification token are added before being input into the Transformer encoding module.

[0130] The Transformer encoding module consists of 12 Transformer Layers. Correspondingly, the feature selection module has 12 feature selection layers, with each Transformer Layer followed by a feature selection layer.

[0131] The Transformer Layer consists of a Multi-Head Attention (MSA) module with residual structures and a Multilayer Perceptron (MLP). Data is normalized using Layer Norm (LN) before being input into these two modules. Unlike CNNs, which only construct relationships between neighboring pixels, the Visual Transformer captures long-range representations of features, possessing a more robust ability to construct global image representations. The output of the final Transformer layer is then processed by a classification head to obtain the final classification result.

[0132] Based on this, the constructed model is trained. The training process may include the following:

[0133] 1) Data Acquisition and Labeling: Collect a large number of sample images to establish a database. For example, a camera can be used to photograph areas affected by crop diseases. When shooting, a close-up viewpoint centered on the diseased area can be selected to collect RGB images of different disease types. After screening and organizing the sample images and removing invalid data, each image is labeled.

[0134] 2) Data Processing: The labeled data is divided into training and test sets in a 9:1 ratio. Preprocessing is performed on the training set data, including scaling, random cropping, color adjustment, and horizontal flipping.

[0135] 3) Model Training: The model is trained using the training set. The Binary Cross-Entropy Loss (BCE loss) function is used during training. The formula for the BCE loss function is:

[0136]

[0137] In formula (4), m is the number of sample images, n is the number of disease categories, and p(x i,j ) represents the true value, q(x) i,j The predicted value is denoted as . Adjusting the weighting coefficients of the loss function based on the number of samples in each category of the training data can address the problem of imbalanced training samples, accelerate model convergence, and improve model performance.

[0138] 4) Model Performance Testing and Iterative Optimization: Train the model using the training set until it converges. Test the trained model and continuously iterate and optimize it to meet performance requirements. At this point, an image recognition model is obtained, which can be used to identify diseases in crop images.

[0139] The method provided in this invention integrates CNN networks and visual Transform networks, enabling the model to extract local image features at different levels while possessing the ability to represent global images. This improves the ability to extract discriminative subtle features in image classification and reduces the likelihood of model misclassification.

[0140] Furthermore, this invention calculates the correlation between other tokens and the classification token based on the self-attention matrix, and adaptively and dynamically selects tokens according to the correlation. This method is applied after each transformer layer for feature selection, which not only removes redundant feature information and enhances feature representation capabilities, but also acts as a pruning mechanism for the network, thereby reducing computational costs and constructing a lightweight network structure.

[0141] Figure 7 This is a second schematic flowchart of the image recognition method provided by the present invention, as shown below. Figure 7 As shown, the model is trained based on the constructed image recognition model, sample images and labels, and the trained image recognition model is used to identify diseases in crop images.

[0142] The image recognition device provided by the present invention is described below. The image recognition device described below can be referred to in correspondence with the image recognition method described above.

[0143] Based on any of the above embodiments Figure 8 This is a schematic diagram of the image recognition device provided by the present invention, as shown below. Figure 8 As shown, the image recognition device includes:

[0144] Image acquisition unit 810 is used to acquire the image to be recognized;

[0145] The local feature extraction unit 820 is used to extract local features from the image to be identified to obtain local features;

[0146] The global feature representation construction unit 830 is used to construct a global feature representation of the image to be identified based on the correlation between each patch in the local features and the position information of each patch, thereby obtaining a global feature representation;

[0147] The recognition unit 840 is used to recognize the global feature representation to obtain the image recognition result.

[0148] The image recognition device provided in this invention constructs a global feature representation of the image to be recognized based on the correlation between each patch in the local features and the positional information of each patch. The resulting global feature representation has the ability to represent local subtle features, and at the same time has the ability to represent the global semantic information of the image. Image recognition based on this global feature representation can reduce misclassification and improve the accuracy of recognition.

[0149] Based on any of the above embodiments, the global feature representation construction unit is further used for:

[0150] Based on the correlation between each patch in the local features and the positional information of each patch, a global feature representation of the image to be identified is constructed to obtain a feature sequence;

[0151] Based on the importance of each tile feature in the feature sequence, feature selection is performed on the feature sequence to obtain the global feature representation.

[0152] Based on any of the above embodiments, the global feature representation construction unit is further used for:

[0153] Based on the correlation between every two feature tiles in the feature sequence, the importance score of each feature tile is determined.

[0154] Feature selection is performed on the feature sequence based on the importance scores of each tile feature.

[0155] Based on any of the above embodiments, the global feature representation construction unit is further used for:

[0156] Based on the importance score, a first number of fixed tile features are selected from each tile feature;

[0157] Based on the number of initial tile features and the cumulative distribution information corresponding to the importance scores, a second number of dynamic tile features are selected from each tile feature.

[0158] Feature selection is performed on the feature sequence based on the first number of fixed tile features and the second number of dynamic tile features.

[0159] Based on any of the above embodiments, the global feature representation construction unit is further used for:

[0160] Based on the correlation between each patch and the position information of each patch in the previous global feature representation, a global feature representation of the image to be identified is constructed to obtain the current feature sequence;

[0161] Based on the importance of each tile feature in the current feature sequence, feature selection is performed on the current feature sequence to obtain the current global feature representation;

[0162] The current global feature representation is used as the previous global feature representation, and the process of reconstructing the global feature representation of the image is repeated until the number of times the global feature representation of the constructed image is executed reaches a preset number.

[0163] Based on any of the above embodiments, a model recognition unit is further included, for:

[0164] The image to be recognized is input into the image recognition model to obtain the image recognition result output by the image recognition model;

[0165] The image recognition model is trained based on sample images and their labels.

[0166] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include a processor 910, a communication interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute an image recognition method, which includes: acquiring an image to be recognized; extracting local features from the image to be recognized to obtain local features; constructing a global feature representation of the image to be recognized based on the correlation between patches in the local features and the positional information of each patch to obtain a global feature representation; and recognizing the global feature representation to obtain an image recognition result.

[0167] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0168] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the image recognition method provided by the above methods. The method includes: acquiring an image to be recognized; extracting local features from the image to be recognized to obtain local features; constructing a global feature representation of the image to be recognized based on the correlation between each patch in the local features and the position information of each patch to obtain a global feature representation; and recognizing the global feature representation to obtain an image recognition result.

[0169] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image recognition method provided by the above methods. The method includes: acquiring an image to be recognized; extracting local features from the image to be recognized to obtain local features; constructing a global feature representation of the image to be recognized based on the correlation between patches in the local features and the positional information of the patches to obtain a global feature representation; and recognizing the global feature representation to obtain an image recognition result.

[0170] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0171] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image recognition method, characterized in that, include: Acquire the image to be recognized; Local features are extracted from the image to be identified to obtain local features; Based on the correlation between each patch in the local features and the positional information of each patch, a global feature representation of the image to be identified is constructed to obtain the global feature representation; The global feature representation is identified to obtain the image recognition result; Based on the correlation between patches in the local features and the positional information of each patch, a global feature representation of the image to be identified is constructed, resulting in the global feature representation, including: Based on the correlation between each patch in the local features and the positional information of each patch, a global feature representation of the image to be identified is constructed to obtain a feature sequence; Based on the importance of each tile feature in the feature sequence, feature selection is performed on the feature sequence to obtain a global feature representation; The step of selecting features from the feature sequence based on the importance of each tile feature in the feature sequence to obtain a global feature representation includes: Based on the correlation between each patch and the position information of each patch in the previous global feature representation, a global feature representation of the image to be identified is constructed to obtain the current feature sequence; Based on the importance of each tile feature in the current feature sequence, feature selection is performed on the current feature sequence to obtain the global feature representation of the current time. The current global feature representation is used as the previous global feature representation, and the process of reconstructing the global feature representation of the image is repeated until the number of times the global feature representation of the constructed image is executed reaches a preset number.

2. The image recognition method according to claim 1, characterized in that, The feature selection based on the importance of each tile feature in the feature sequence includes: Based on the correlation between every two feature tiles in the feature sequence, the importance score of each feature tile is determined. Feature selection is performed on the feature sequence based on the importance scores of each tile feature.

3. The image recognition method according to claim 2, characterized in that, The feature selection based on the importance scores of each tile feature includes: Based on the importance score, a first number of fixed tile features are selected from each tile feature; Based on the number of initial tile features and the cumulative distribution information corresponding to the importance scores, a second number of dynamic tile features are selected from each tile feature. Feature selection is performed on the feature sequence based on the first number of fixed tile features and the second number of dynamic tile features.

4. The image recognition method according to any one of claims 1-3, characterized in that, The process involves extracting local features from the image to be identified, obtaining local features, constructing a global feature representation of the image based on the correlation between patches in the local features and the positional information of each patch, obtaining the global feature representation, and then identifying the global feature representation to obtain the image recognition result, including: The image to be recognized is input into the image recognition model to obtain the image recognition result output by the image recognition model; The image recognition model is trained based on sample images and their labels.

5. The image recognition method according to claim 4, characterized in that, The image to be identified is a crop image, and the image recognition result is a crop disease identification result.

6. An image recognition device, characterized in that, include: An image acquisition unit is used to acquire the image to be recognized. The local feature extraction unit is used to extract local features from the image to be identified, thereby obtaining local features; A global feature representation construction unit is used to construct a global feature representation of the image to be identified based on the correlation between each patch in the local features and the position information of each patch, thereby obtaining a global feature representation; The recognition unit is used to recognize the global feature representation to obtain the image recognition result; Based on the correlation between patches in the local features and the positional information of each patch, a global feature representation of the image to be identified is constructed, resulting in the global feature representation, including: Based on the correlation between each patch in the local features and the positional information of each patch, a global feature representation of the image to be identified is constructed to obtain a feature sequence; Based on the importance of each tile feature in the feature sequence, feature selection is performed on the feature sequence to obtain a global feature representation; The step of selecting features from the feature sequence based on the importance of each tile feature in the feature sequence to obtain a global feature representation includes: Based on the correlation between each patch and the position information of each patch in the previous global feature representation, a global feature representation of the image to be identified is constructed to obtain the current feature sequence; Based on the importance of each tile feature in the current feature sequence, feature selection is performed on the current feature sequence to obtain the global feature representation of the current time. The current global feature representation is used as the previous global feature representation, and the process of reconstructing the global feature representation of the image is repeated until the number of times the global feature representation of the constructed image is executed reaches a preset number.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the image recognition method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image recognition method as described in any one of claims 1 to 5.

Images