Agricultural crop classification method and system based on adaptive masked attention network

Abstract

This invention relates to the field of artificial intelligence technology, specifically to an agricultural crop classification method and system based on an adaptive masked attention network. The method includes: acquiring image data of a target region; adaptively filtering redundant information in the image data and enhancing effective features to obtain a first feature; generating a second feature based on a balanced attention weight at different scales within the first feature; and identifying crops in the target region based on the second feature. This invention can improve the accuracy of crop identification in complex farmland environments.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an agricultural crop classification method and system based on an adaptive masked attention network. Background Art

[0002] Agricultural crop identification mainly involves identifying and classifying crops in farmland, which helps farmland managers better understand farmland conditions and further improve crop management efficiency and agricultural production quality.

[0003] Traditionally, agricultural crop identification mainly relies on manual observation and experience-based decision-making by agricultural workers, which is time-consuming, laborious and inefficient. Automatic classification of agricultural crop images based on machine learning can greatly improve recognition efficiency. However, on the one hand, agricultural crops are easily obscured by complex farmland environments, resulting in the inability of machine learning models to accurately extract effective features of agricultural crops. On the other hand, there are often many different types of agricultural crops with very similar appearances in complex farmland environments, resulting in the inability of machine learning models to accurately extract subtle distinguishing features of agricultural crops. Therefore, at this stage, the agricultural crop image classification method based on machine learning has a low recognition accuracy rate for agricultural crops in complex farmland environments. Summary of the invention

[0004] In view of this, the purpose of an embodiment of the present invention is to provide an agricultural crop classification method and system based on an adaptive masked attention network to improve the recognition accuracy of existing agricultural crop classification methods based on neural networks in complex farmland environments.

[0005] In a first aspect, an embodiment of the present invention provides an agricultural crop classification method based on an adaptive masked attention network, comprising:

[0006] Acquire image data of the target area;

[0007] The image data is input into an agricultural crop recognition model, and a recognition result of the agricultural crops is output, wherein the agricultural crop recognition model includes an adaptive masking module and a fair attention module, the adaptive masking module is used to adaptively filter redundant information of the image data and enhance effective features to obtain a first feature, the fair attention module is used to generate a second feature based on a balance of attention weights of different scales in the first feature, and the agricultural crop recognition model recognizes the agricultural crops in the target area based on the second feature.

[0008] Optionally, the adaptively filtering redundant information of the image data includes:

[0009] Acquire a feature mask of the image data;

[0010] Redundant information of the image data is filtered based on the feature mask and effective features are enhanced to generate the first feature.

[0011] Optionally, the calculation formula for obtaining the feature mask of the image data includes:

[0012] ;

[0013] Among them, x is the image data, y represents the feature value of x after the convolution operation conv and the softmax function, Mask(x) represents the feature mask of the image data x, conv(•) represents the convolution operation, softmax(•) represents the softmax function, sign(•) represents the sign function, and λ is a learnable parameter in the network.

[0014] Optionally, filtering redundant information of the image data based on the feature mask and enhancing effective features to generate the first feature includes:

[0015] Performing a dot product operation on the feature map of the image data based on the feature mask to generate a valid feature map;

[0016] The effective features of the effective feature map are extracted using a residual network, and the effective features are added to the feature map to obtain the first features.

[0017] Optionally, generating the second feature based on attention weight balance of different scales in the first feature includes:

[0018] generating an attention weight map of the first feature;

[0019] Suppressing the maximum weight in the attention weight map and balancing the attention weights of different scales to generate a balanced weight map;

[0020] The second feature is generated based on the equalization weight map.

[0021] Optionally, generating the attention weight map of the first feature includes:

[0022] Processing the first feature with a linear layer to generate a three-dimensional feature space consisting of feature vectors of three dimensions;

[0023] In the three-dimensional feature space, the product of the first-dimensional feature vector and the transposed second-dimensional feature vector is processed with a softmax function to obtain the attention weight map.

[0024] Optionally, the calculation formula for suppressing the maximum weight in the attention weight map and balancing the attention weights of different scales to generate a balanced weight map includes:

[0025] Among them, F is the attention weight map, MN(F) means that F is processed by the maximum normalization function to obtain the balanced weight map, Max(•) is used to obtain the maximum eigenvalue in F, and softmax(•) represents the softmax function.

[0026] Optionally, generating the second feature based on the equalization weight map includes:

[0027] Multiplying the third dimension feature vector of the three-dimensional feature space by the equalization weight map;

[0028] The product is input into the linear layer and added to the first feature to obtain the second feature.

[0029] In a second aspect, an embodiment of the present invention provides an agricultural crop classification device based on an adaptive masked attention network, comprising:

[0030] An acquisition module, used for acquiring image data of a target area;

[0031] The recognition module is used to input the image data into an agricultural crop recognition model and output a recognition result of the agricultural crop, wherein the agricultural crop recognition model includes an adaptive masking module and a fair attention module, the adaptive masking module is used to adaptively filter redundant information of the image data and enhance effective features to obtain a first feature, the fair attention module is used to generate a second feature based on a balance of attention weights of different scales in the first feature, and the agricultural crop recognition model recognizes the agricultural crops in the target area based on the second feature.

[0032] In a third aspect, an embodiment of the present invention provides an agricultural crop classification system based on an adaptive mask attention network, comprising a memory and a processor,

[0033] The memory is used to store computer programs;

[0034] The processor is used to implement the method described in the first aspect of the present invention when executing the computer program.

[0035] The embodiments of the present invention include the following beneficial effects: the embodiment filters redundant information of the environment in the farmland image and enhances the effective features of the crops through the adaptive masking module, so that the crops and the environment can be distinguished more effectively; at the same time, the fair attention module balances the attention weights of different scales, so that fine-grained features can be more effectively extracted to identify similar crops between different categories. The combination of the two greatly improves the recognition accuracy in complex farmland environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 is a flow chart of an agricultural crop classification method based on an adaptive mask attention network provided by an embodiment of the present invention;

[0038] Figure 2 is a schematic diagram of the structure of an adaptive masking module in some embodiments of the present invention;

[0039] Figure 3 is a schematic diagram of the structure of a fairness attention module in some embodiments of the present invention;

[0040] Figure 4 is a schematic diagram of an agricultural crop classification device based on an adaptive mask attention network according to an embodiment of the present invention;

[0041] Figure 5 is a schematic diagram of an agricultural crop classification system based on an adaptive mask attention network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0045] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0046] The block diagrams shown in the accompanying drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware charging modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0047] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0048] Figure 1 A flowchart of a method for agricultural crop classification based on an adaptive masked attention network provided by an embodiment of the present invention is applied to an agricultural crop classification server based on an adaptive masked attention network. The method comprises the following steps:

[0049] S110, acquiring image data of the target area;

[0050] Specifically, the image data may be photos taken by image acquisition devices such as cameras, webcams, and drones, or may be videos taken by video acquisition devices such as drones and video cameras.

[0051] S120, inputting the image data into an agricultural crop recognition model, and outputting recognition results of the agricultural crops, wherein the agricultural crop recognition model includes an adaptive masking module and a fair attention module, the adaptive masking module is used to adaptively filter redundant information of the image data and enhance effective features to obtain a first feature, the fair attention module is used to generate a second feature based on a balance of attention weights of different scales in the first feature, and the agricultural crop recognition model recognizes the agricultural crops in the target area based on the second feature.

[0052] In natural environments, crops are often hidden in complex backgrounds. Most current recognition models tend to focus on effective components to identify effective features. However, in complex environments, it is difficult for recognition models to distinguish effective components from the background. Instead of focusing on the target, the present invention hopes to improve performance by abandoning noise through the network. To this end, an adaptive mask (AM) module is developed to distinguish crops from agricultural environments, filter redundant information and further enhance effective features. The structure of the adaptive masking module is shown in Figure 2. Figure 2 shown.

[0053] like Figure 2 As shown, the adaptive masking module can be expressed as:

[0054] ;

[0055] Among them, Res and Mask represent the residual operation and the calculation of Mask respectively. The specific steps are as follows:

[0056] First, the original image (I) is input into the Mask module to obtain the feature mask, where the first convolutional layer is used to generate the feature map (F), and then the second one is based on F to generate the feature mask (M) after softmax:

[0057] ;

[0058] Among them, sign(•), softmax(•) and conv(•) represent the sign function, softmax function and convolution layer respectively, while λ is a learnable parameter in the network, and y represents the eigenvalue of x after the convolution operation conv and softmax function.

[0059] Second, point generation is performed on the feature map (F) and mask map (M) to generate a valid feature map (N), where redundant information is filtered by the mask and the target components are preserved.

[0060] Finally, the feature map (N) is transferred to the Res block to obtain the output feature map (K), and the original feature map (F) is added to the valid feature map (N) to generate the final feature map K.

[0061] After distinguishing the environment from crops and extracting effective features of crops, the adaptive masking module inputs the effective features into the fair attention (FA) module to identify subtle distinguishing features between different types of crops.

[0062] Another factor that makes crop identification difficult is that crops are relatively similar to each other. The present invention adopts an attention mechanism to distinguish similar classifications by modeling fine-grained features. Although the attention mechanism can help the network focus on the target component, excessive attention to specific features will cause the network to focus on learning content while ignoring other feature components, resulting in a decrease in network performance. In this case, the attention mechanism should be guided by a balanced mode to learn effective information of the target. To this end, the fair attention (FA) module aims to identify similar crops between different categories by modeling fine-grained features, where maximum normalization (MN) is proposed to model attention weights of different scales while suppressing the maximum weight. The structural diagram of the fair attention module is shown in Figure 3 As shown. The algorithm framework of the fair attention module can be expressed as:

[0063] ;

[0064] Among them, MN and softmax represent maximum normalization and softmax function respectively, and X is the input feature. The specific process is as follows:

[0065] First, the input feature map X is processed through a linear layer to generate a three-dimensional feature space, namely U, V, and W.

[0066] Then, the feature map U is multiplied by the transposed V to obtain the attention weight map (F) after the softmax function.

[0067] Next, the feature map is fed into a Max Normalization (MN) block to produce balanced attention weights:

[0068] ;

[0069] Here, softmax(•) indicates that the softmax function Max(•) is used to obtain the maximum value of each value in the rows and columns of the feature map F.

[0070] Finally, the spatial feature map W is multiplied by F to obtain the filtered feature map R, which is input into the linear layer and added to the input feature map X to obtain the final feature map O.

[0071] After distinguishing the features of crops and the environment, and modeling the balance of attention weights at different scales in the features, the common crop classification model based on adaptive mask attention network can be used for identification based on the features. Because the present invention can filter the redundant information of the environment in the farmland image, it can more effectively distinguish crops from the environment and balance the attention weights at different scales, so that fine-grained features can be more effectively extracted to identify similar crops between different categories. The combination of the two greatly improves the recognition accuracy in complex farmland environments.

[0072] One embodiment of the present invention is to analyze the Figure 1 The method described in S110-S120 was verified.

[0073] The indicators for evaluating classification performance include: accuracy (Acc), precision (Pre), recall (Rec) and F1-score (F1);

[0074] , ;

[0075] , ;

[0076] Among them, TP is the correctly classified positive category mispredicted positive category, TN is the true negative sample, and FN is the false negative category.

[0077] The prediction performance comparison of different models is shown in Table 1. In Table 1, AMA-Net is the prediction method to be protected by the present invention. As can be seen from Table 1, AMA-Net leads in all four evaluation indicators, demonstrating Figure 1 The methods described in S110-S120 have excellent prediction performance. The classification models used for comparison include: EfficientNet, ResNet, DenseNet, VggNet and GhostNet.

[0078] Model Accuracy(%) recall(%) F1 score (%) EfficientNet 72.54 78.28 73.34 ResNet 76.47 81.07 76.56 DenseNet 82.35 85.45 82.62 VggNet 82.35 87.16 82.25 GhostNet 84.31 89.06 84.72 AMA-Net 88.23 91.35 88.25

[0079] Figure 4 is a schematic diagram of a crop identification device according to some embodiments of the present invention. Figure 4 As shown, the crop identification device includes an acquisition module 410, a feature enhancement module 420 and an identification module 430. In some embodiments of the present disclosure, the crop identification function is performed by a crop identification server.

[0080] An acquisition module 410 is used to acquire image data of a target area;

[0081] A feature enhancement module 420 is used to adaptively filter redundant information of the image data and enhance effective features, obtain a first feature, and generate a second feature based on the attention weights of different scales in the first feature;

[0082] The identification module 430 is used to identify the crops in the target area based on the second feature.

[0083] One embodiment of the present invention provides an agricultural crop classification system based on an adaptive masked attention network. Figure 5 As shown, the agricultural crop classification system based on the adaptive mask attention network includes a memory 520 and a processor 510, wherein the memory 520 is used to store a computer program; and the processor 510 is used to implement when executing the computer program. Figure 1 The method described in S110-S120.

[0084] In summary, the agricultural crop classification method and system based on the adaptive mask attention network provided by the embodiments of the present invention can more effectively distinguish between crops and the environment by filtering out redundant information of the environment in the farmland image and enhancing the effective features of the crops through the adaptive masking module; at the same time, the attention weights of different scales are balanced through the fair attention module, so that fine-grained features can be more effectively extracted to identify similar crops between different categories. The combination of the two greatly improves the recognition accuracy in complex farmland environments.

[0085] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the charging modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0086] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional charging modules / units in the systems and devices may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0087] It should be understood that in this application, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. "At least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or plural.

[0088] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0089] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0090] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for agricultural crop classification based on adaptive masked attention network, characterized in that: include: Acquire image data of the target area; Input the image data into an agricultural crop recognition model, and output recognition results of the agricultural crops, wherein the agricultural crop recognition model includes an adaptive masking module and a fair attention module, the adaptive masking module is used to adaptively filter redundant information of the image data and enhance effective features to obtain a first feature, the fair attention module is used to generate a second feature based on a balance of attention weights of different scales in the first feature, and the agricultural crop recognition model recognizes the agricultural crops in the target area based on the second feature; The generating of the second feature based on the attention weight balance of different scales in the first feature includes: generating an attention weight map of the first feature; Suppressing the maximum weight in the attention weight map and balancing the attention weights of different scales to generate a balanced weight map; generating the second feature based on the equalization weight map; The generating of the attention weight map of the first feature comprises: Processing the first feature with a linear layer to generate a three-dimensional feature space consisting of feature vectors of three dimensions; In the three-dimensional feature space, a softmax function is used to process the product of the first-dimensional feature vector and the transposed second-dimensional feature vector to obtain the attention weight map; The calculation formula for suppressing the maximum weight in the attention weight map and balancing the attention weights of different scales to generate a balanced weight map includes: ； Wherein, F is the attention weight map, MN(F) means that F is processed by the maximum normalization function to obtain the balanced weight map, Max(•) is used to obtain the maximum eigenvalue in F, and softmax(•) represents the softmax function; Generating the second feature based on the equalization weight map includes: Multiplying the third dimension feature vector of the three-dimensional feature space by the equalization weight map; The product is input into the linear layer and added to the first feature to obtain the second feature.

2. The method according to claim 1, characterized in that: The adaptive filtering of redundant information of the image data includes: Acquire a feature mask of the image data; Redundant information of the image data is filtered based on the feature mask and effective features are enhanced to generate the first feature.

3. The method according to claim 2, characterized in that: The calculation formula for obtaining the feature mask of the image data includes: ； Among them, x is the image data, y represents the feature value of x after the convolution operation conv and the softmax function, Mask(x) represents the feature mask of the image data x, conv(•) represents the convolution operation, softmax(•) represents the softmax function, sign(•) represents the sign function, and λ is a learnable parameter in the network.

4. The method according to claim 3, characterized in that: The filtering redundant information of the image data based on the feature mask and enhancing effective features to generate the first feature comprises: Performing a dot product operation on the feature map of the image data based on the feature mask to generate a valid feature map; The effective features of the effective feature map are extracted using a residual network, and the effective features are added to the feature map to obtain the first features.

5. An agricultural crop classification device based on an adaptive masked attention network, characterized in that: include: An acquisition module, used for acquiring image data of a target area; a recognition module, configured to input the image data into an agricultural crop recognition model, and output a recognition result of the agricultural crop, wherein the agricultural crop recognition model includes an adaptive masking module and a fair attention module, wherein the adaptive masking module is configured to adaptively filter redundant information of the image data and enhance effective features to obtain a first feature, and the fair attention module is configured to generate a second feature based on a balance of attention weights of different scales in the first feature, and the agricultural crop recognition model recognizes the agricultural crops in the target area based on the second feature; The generating of the second feature based on the attention weight balance of different scales in the first feature includes: generating an attention weight map of the first feature; Suppressing the maximum weight in the attention weight map and balancing the attention weights of different scales to generate a balanced weight map; generating the second feature based on the equalization weight map; The generating of the attention weight map of the first feature comprises: Processing the first feature with a linear layer to generate a three-dimensional feature space consisting of feature vectors of three dimensions; In the three-dimensional feature space, a softmax function is used to process the product of the first-dimensional feature vector and the transposed second-dimensional feature vector to obtain the attention weight map; The calculation formula for suppressing the maximum weight in the attention weight map and balancing the attention weights of different scales to generate a balanced weight map includes: ； Wherein, F is the attention weight map, MN(F) means that F is processed by the maximum normalization function to obtain the balanced weight map, Max(•) is used to obtain the maximum eigenvalue in F, and softmax(•) represents the softmax function; Generating the second feature based on the equalization weight map includes: Multiplying the third dimension feature vector of the three-dimensional feature space by the equalization weight map; The product is input into the linear layer and added to the first feature to obtain the second feature.

6. An agricultural crop classification system based on adaptive masked attention network, characterized in that: including memory and processor, The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 4 when executing the computer program.

Images