A fruit disease detection method and system based on global memory and local contrast

Abstract

The application relates to the field of image processing, in particular to a fruit disease detection method and system based on global memory and local contrast. The method comprises the following steps: obtaining to-be-processed features of a fruit to-be-detected image, inputting the to-be-processed features into a plurality of cascaded global memory encoders after channel mixing and dimension reduction; each global memory encoder obtains modulated features through channel and spatial attention mechanism in turn, then generates first and second attention features through window self-attention mechanism and adaptive contrast perception gated attention mechanism respectively, and outputs after fusion, nonlinear transformation and feature integration; and the output features of the last-stage encoder are recognized to obtain the fruit disease condition. Compared with the prior art, the local context dependence is modeled through the window self-attention mechanism, the global historical information is modeled through the gating technology of the adaptive contrast perception gated attention mechanism, and the fruit disease is identified by fusing the two, so that the accuracy is greatly improved while the reasoning efficiency is ensured.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to a method and system for detecting fruit diseases based on global memory and local contrast. Background Technology

[0002] Fruits are rich in various vitamins, minerals, dietary fiber, and natural antioxidants such as polyphenols and flavonoids, possessing significant nutritional, health-promoting, and medicinal value, making them an indispensable source of healthy food in the human diet. However, the healthy development of the fruit industry is facing severe challenges, with diseases being a key factor restricting its yield, quality, and economic benefits. These diseases directly damage the leaves and fruits, leading to flower and fruit drop, reduced yield, and deterioration in quality, seriously affecting growers' profits.

[0003] In existing technologies, the diagnosis of fruit diseases relies heavily on the professional knowledge and experience of agricultural experts. While this method has some reference value, its high degree of subjectivity, dependence on expert experience, and low efficiency not only lead to inconsistent diagnostic results but also severely restrict efficient disease control and early warning.

[0004] Therefore, there is an urgent need for a method that can efficiently and accurately detect fruit diseases. Summary of the Invention

[0005] This invention provides a method and system for detecting fruit diseases based on global memory and local comparison, which can be used to efficiently and accurately detect fruit diseases.

[0006] According to a first aspect of this application, a method for detecting fruit diseases based on global memory and local comparison is provided, the method comprising:

[0007] Feature extraction is performed on the fruit image to be detected to obtain the features to be processed;

[0008] The features to be processed are subjected to channel shuffling and dimensionality reduction processing to obtain the dimensionality-reduced features;

[0009] The dimensionality reduction feature is input into several sequentially connected global memory encoders for processing; wherein the input feature of the first global memory encoder is the dimensionality reduction feature, and the input features of the other global memory encoders are the output features of the previous global memory encoder;

[0010] The output features of the last global memory encoder are obtained for identification, and the disease identification results of the fruit are obtained.

[0011] The processing of each of the global memory encoders includes:

[0012] The input features are processed sequentially through channel attention and spatial attention mechanisms to obtain modulation features;

[0013] The modulation features are processed using a window self-attention mechanism to obtain the first attention feature;

[0014] The modulation features are processed by an adaptive contrastive perception gating attention mechanism to obtain a second attention feature;

[0015] The first attention feature and the second attention feature are fused to obtain the fused feature;

[0016] The fused features are then subjected to nonlinear transformation and feature integration in sequence to obtain the output features.

[0017] Optionally, the step of performing channel shuffling and dimensionality reduction processing on the features to be processed to obtain dimensionality-reduced features includes:

[0018] The feature to be processed is divided into several channel features based on a preset channel dimension, and each channel feature is convolved to obtain the first convolution feature corresponding to each channel feature.

[0019] Perform channel shuffling on all the first convolutional features to obtain the shuffled features corresponding to each channel feature;

[0020] The shuffled features are subjected to convolutional dimensionality reduction processing to obtain the second convolutional features;

[0021] The first convolutional feature is subjected to convolutional dimensionality reduction based on the dimension of the second convolutional feature to obtain the third convolutional feature;

[0022] The second convolutional feature and the third convolutional feature corresponding to each of the channel features are fused to obtain the channel fused feature corresponding to each of the channel features;

[0023] The channel fusion features corresponding to all the channel features are spliced ​​and recombined to obtain the dimensionality reduction features.

[0024] Optionally, the step of processing the modulation features through an adaptive contrastive perception gating attention mechanism to obtain second attention features includes:

[0025] The modulation feature is divided into several non-overlapping local window features according to a preset local window.

[0026] The local window features are sorted according to the adjacency relationship of the local windows corresponding to the local window features;

[0027] According to the sorted order, each of the local window features is processed sequentially to obtain the local attention features of the local window features;

[0028] The local attention features of all the local windows are spliced ​​and recombined to obtain the second attention feature;

[0029] The step of processing the local window features to obtain the local attention features of the local window features includes:

[0030] The query vector, key vector, and value vector of the local window features are generated by linear projection.

[0031] Based on the value vector and the local window size corresponding to the local window feature, obtain the attention value vector corresponding to the local window feature;

[0032] Calculate the local enhancement attention weights corresponding to the local window features based on the query vector and the key vector;

[0033] The local attention features of the local window are obtained based on the associated attention state of the local window corresponding to the local window features, the local enhanced attention weights, the query vector, and the attention value vector; the associated attention state is generated based on the query vector and local enhanced attention weights corresponding to the previous local window of the local window.

[0034] Optionally, obtaining the attention value vector corresponding to the local window feature based on the value vector and the local window size corresponding to the local window feature includes:

[0035] The value vector is then subjected to spatial global average pooling based on the size of the local window corresponding to the local window feature, resulting in a spatially pooled value vector, represented as follows:

[0036] ;

[0037] In the formula, The spatial pooling value vector, For the first The pixels in the local window corresponding to the aforementioned local window features The value vector, The height of the local window corresponding to the aforementioned local window feature. The width of the local window corresponding to the aforementioned local window feature;

[0038] Attention weights are generated based on the spatial pooling vector, and are expressed as follows:

[0039] ;

[0040] In the formula, The attention weights, and These are the learnable weights, and These are the learnable biases, For Sigmoid activation function, ReLU represents ReLU activation function;

[0041] The attention weights are multiplied channel by channel by channel of the value vector to obtain the attention value vector corresponding to the local window features.

[0042] Optionally, obtaining the local attention features of the local window based on the associated attention state of the local window corresponding to the local window features, the local enhanced attention weights, the query vector, and the attention value vector includes:

[0043] Based on the attention state of the local window corresponding to the local window features and the first query vector, the gating information of the local window is obtained. The first query vector is obtained by splitting the query vector based on dimensions, and is represented as follows:

[0044] ;

[0045] In the formula, The gate control information, This is the first query vector. Indicates the first The associated attention state of the local window corresponding to the local window feature, specifically the first... The attention state of the local window corresponding to each of the aforementioned local window features. For learnable weights, For learnable bias, Use the Sigmoid activation function;

[0046] The attention state of the local window is generated based on the gating information, the associated attention state, and the local enhanced attention weights, and is represented as follows:

[0047] ;

[0048] In the formula, Let r be the attention state of the local window corresponding to the r-th local window feature. For the local enhanced attention weights, For learnable scaling factor, The symbol for element-wise multiplication;

[0049] The local attention features of the local window are calculated based on the attention state of the local window and the attention value vector, and are expressed as follows:

[0050] ;

[0051] In the formula, The local attention features, For the attention value vector, For learnable weights, This is a learnable bias.

[0052] Optionally, calculating the local enhancement attention weights corresponding to the local window features based on the query vector and the key vector includes:

[0053] The attention score of the local window feature is calculated based on the query vector and the key vector;

[0054] The dimension matching score of the local window features is calculated based on the preset learnable weight tensor.

[0055] Based on the attention score and dimension matching score, calculate the local enhanced attention weights corresponding to the local window features.

[0056] Optionally, the attention score includes a primary attention score and a secondary attention score;

[0057] The step of calculating the attention score of the local window features based on the query vector and the key vector includes:

[0058] The query vector is split into a first query vector and a second query vector, and the key vector is split into a first key vector and a second key vector;

[0059] The main attention score is calculated based on the first query vector and the first key vector, and is expressed as follows:

[0060] ;

[0061] In the formula, The main attention score is... This is the first query vector. This represents the transpose of the first key vector. The dimensions of the query vector and the key vector;

[0062] The secondary attention score is calculated based on the second query vector and the second key vector, and is expressed as follows:

[0063] ;

[0064] In the formula, The auxiliary attention score is... This is the second query vector. This represents the transpose of the second key vector.

[0065] Optionally, the dimension matching score includes the primary dimension matching score and the secondary dimension matching score;

[0066] Calculate the dimensionality matching score of the local window features based on a preset learnable weight tensor, including:

[0067] Learnable weight tensors corresponding to each dimension of the first query vector, the second query vector, the first key vector, and the second key vector are extracted respectively; the learnable weight tensors are obtained through pre-training.

[0068] The main dimension matching score is calculated based on the learnable weight tensors of all dimensions of the first query vector and the first key vector, and is expressed as:

[0069] ;

[0070] In the formula, Match scores for the main dimension. For the first query vector, the first The learnable weight tensor corresponding to each dimension For the first key vector, the first key vector is the first key vector. The learnable weight tensors corresponding to each dimension; C is the number of dimensions of the first query vector, the first key vector, the second query vector, and the second key vector; It is an exponential function;

[0071] The secondary dimension matching score is calculated based on the learnable weight tensors of all dimensions of the second query vector and the second key vector, and is expressed as follows:

[0072] ;

[0073] In the formula, Match scores for the secondary dimension. For the second query vector, the first The learnable weight tensor corresponding to each dimension For the second key vector, the first The learnable weight tensor corresponding to each dimension.

[0074] Optionally, calculating the local enhancement attention weights corresponding to the local window features based on the attention score and the dimension matching score includes:

[0075] Calculate the difference between the primary dimension matching score and the secondary dimension matching score, and map the difference using the Sigmoid function to obtain the fusion weight vector, expressed as:

[0076] ;

[0077] ;

[0078] In the formula, The fusion weight vector, These are the basic parameters of the global memory encoder. This refers to the position of the global memory encoder among several sequentially connected global memory encoders. Use the Sigmoid activation function;

[0079] The main attention weight is calculated based on the main attention score and expressed as follows:

[0080] ;

[0081] In the formula, Main attention weight, The relative position offset of the local window corresponding to the local window feature;

[0082] The secondary attention weight is calculated based on the secondary attention score and expressed as follows:

[0083] ;

[0084] In the formula, As a secondary attention weight;

[0085] The local enhanced attention weight is calculated based on the main attention weight, the secondary attention weight, and the fusion weight vector, and is expressed as follows:

[0086] .

[0087] According to a second aspect of this application, a fruit disease detection system based on global memory and local comparison is provided, the system comprising:

[0088] The feature extraction module is used to extract features from the fruit image to be detected and obtain the features to be processed.

[0089] The feature dimensionality reduction module is used to perform channel shuffling dimensionality reduction processing on the features to be processed to obtain dimensionality-reduced features;

[0090] The feature processing module is used to input the dimensionality reduction features into several sequentially connected global memory encoders for processing;

[0091] The feature recognition module is used to identify the output features of the last global memory encoder and obtain the disease identification results of the fruit.

[0092] The processing of each of the global memory encoders includes:

[0093] The input features are processed sequentially through channel attention and spatial attention mechanisms to obtain modulation features;

[0094] The modulation features are processed using a window self-attention mechanism to obtain the first attention feature;

[0095] The modulation features are processed by an adaptive contrastive perception gating attention mechanism to obtain a second attention feature;

[0096] The first attention feature and the second attention feature are fused to obtain the fused feature;

[0097] The fused features are then subjected to nonlinear transformation and feature integration in sequence to obtain the output features.

[0098] According to a third aspect of this application, an electronic device is provided, comprising:

[0099] Memory, used to store one or more computer programs;

[0100] A processor, when the one or more computer programs are executed by the processor, implements the fruit disease detection method based on global memory and local comparison described in the first aspect above.

[0101] According to a fourth aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the fruit disease detection method based on global memory and local comparison described in the first aspect above.

[0102] Based on any of the above aspects, the embodiments of this application provide a fruit disease detection method, system, electronic device, and computer storage medium based on global memory and local comparison. This method reduces the dimensionality of the features to be processed through channel shuffling, preserving discriminative information while reducing computational load. Furthermore, by co-modulating the reduced-dimensional features through channel attention and spatial attention mechanisms, the channel response and spatial positioning accuracy of the diseased area can be effectively enhanced. Further, by modeling local contextual dependencies through a window self-attention mechanism and modeling historical information through gating technology in an adaptive contrastive perception gating attention mechanism, and by integrating the local contextual dependencies of the window self-attention mechanism and the global historical information of the adaptive contrastive perception gating attention mechanism to identify fruit disease conditions, the accuracy of fruit disease identification is significantly improved while ensuring inference efficiency. Attached Figure Description

[0103] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0104] Figure 1 This is a flowchart illustrating the steps of the disease detection method provided in this embodiment.

[0105] Figure 2 This is a schematic diagram of the steps for obtaining dimensionality reduction features provided in this embodiment.

[0106] Figure 3 This is a schematic diagram of the steps for obtaining modulation features provided in this embodiment.

[0107] Figure 4 This is a schematic diagram of the steps for obtaining the second attention feature provided in this embodiment.

[0108] Figure 5 This is a schematic diagram illustrating the steps for calculating the local enhanced attention weights in this embodiment.

[0109] Figure 6 This is a schematic diagram of the fruit disease detection model provided in this embodiment.

[0110] Figure 7 This is a schematic diagram of the global memory encoder provided in this embodiment.

[0111] Figure 8 This is a schematic diagram of the processing flow of the adaptive contrast-gated attention unit provided in this embodiment.

[0112] Figure 9 This is a schematic diagram of the functional modules of the disease detection system provided in this embodiment.

[0113] Figure 10 This is a schematic diagram of the device structure of the electronic device provided in this embodiment. Detailed Implementation

[0114] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this application. To better illustrate the following embodiments, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product; it is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0115] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0116] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0117] Fruits are rich in various vitamins (such as vitamin C, vitamin A, and B vitamins), minerals (such as potassium, magnesium, and iron), dietary fiber, and natural antioxidants such as polyphenols and flavonoids. They possess significant nutritional, health-promoting, and medicinal value, making them an indispensable source of healthy food in the human diet. Among them, the lychee is hailed as the "Queen of Fruits." Lychees are not only nutritious, but their abundance of vitamins, minerals, and fruit polyphenols also gives them significant edible and medicinal value, making them highly popular. However, the healthy development of the fruit industry, especially the lychee industry, is facing severe challenges. Diseases are a key factor restricting yield, quality, and economic benefits. These diseases directly damage the leaves and fruit, leading to flower and fruit drop, reduced yield, and deteriorated quality, seriously affecting growers' profits.

[0118] In existing technologies, the diagnosis of fruit diseases relies heavily on the professional knowledge and experience of agricultural experts. While this method has some reference value, its high degree of subjectivity, dependence on expert experience, and low efficiency not only lead to inconsistent diagnostic results but also severely restrict efficient disease control and early warning.

[0119] This embodiment provides a technical solution that can solve the above problems. The specific implementation of this application will be described in detail below with reference to the accompanying drawings.

[0120] like Figure 1As shown in the figure, this embodiment provides a fruit disease detection method based on global memory and local comparison. The method may include the following steps:

[0121] S1: Extract features from the fruit image to be detected and obtain the features to be processed;

[0122] In this embodiment, features can be extracted from the image to be detected using a ResNet backbone network to obtain the features to be processed. The ResNet backbone network has a very mature feature extraction capability, which can effectively realize the feature extraction of the image to be detected. At the same time, the features to be processed extracted by the ResNet backbone network have higher versatility and can be effectively compatible with subsequent processing.

[0123] S2: Perform channel shuffling and dimensionality reduction processing on the features to be processed to obtain dimensionality-reduced features;

[0124] In this embodiment, as Figure 2 As shown, step S2 may include the following sub-steps:

[0125] S21: Divide the feature to be processed into several channel features based on a preset channel dimension, and perform convolution processing on each channel feature to obtain the first convolution feature corresponding to each channel feature;

[0126] In this embodiment, dividing the feature to be processed into several channel features based on a preset channel dimension can be understood as dividing the feature to be processed into channel features of several channel dimensions. For example, if the size of the feature to be processed is H×W×C, where H, W, and C are the height, width, and channel of the feature to be processed, respectively, and the channel dimension is c1, then the feature to be processed can be divided into C / c1 channel features, and the size of each channel feature is H×W×c1. Dividing the feature to be processed based on the channel dimension makes each channel feature independent of each other, which helps to separate the normal texture of the fruit from abnormal lesions.

[0127] In this embodiment, each channel feature can be processed by 1×1 convolution. 1×1 convolution can linearly combine and weight the finer-grained features in the channel features to achieve the fusion of information in the channel features and enhance the feature expression within the channel features.

[0128] S22: Perform channel shuffling on all the first convolutional features to obtain the shuffled features corresponding to each channel feature;

[0129] To overcome the information isolation problem between channel features caused by grouped convolution, this embodiment performs channel shuffling on all the convolutional features. Channel shuffling refers to shuffling and rearranging the features of each channel feature while maintaining the number of channel features, and then reassembling the rearranged features. For example, suppose there are three channel features, where the features of each channel feature are represented as [1,2,3], [4,5,6], and [7,8,9], respectively. After channel shuffling, there are still three channel features, but the features of each channel feature are represented as [3,5,7], [1,4,6], and [9,8,2], respectively. Channel shuffling is equivalent to exchanging the features in each channel feature, enabling each channel feature to learn information from the features of other channels.

[0130] S23: Perform convolutional dimensionality reduction on the shuffled features to obtain the second convolutional features;

[0131] In this embodiment, the shuffled features can be processed by k×k convolution, and the output dimension of the k×k convolution can be set to achieve dimensionality reduction of the shuffled features. Here, k can be set according to the dimension of the shuffled features. Using k×k convolution can capture richer local spatial features and contextual information through a larger receptive field, while channel dimensionality reduction is achieved by setting the output dimension of the k×k convolution.

[0132] S24: Perform convolution dimensionality reduction on the first convolution feature based on the dimension of the second convolution feature to obtain the third convolution feature;

[0133] S25: Perform feature fusion on the second convolutional feature and the third convolutional feature corresponding to each of the channel features to obtain the channel fusion feature corresponding to each of the channel features;

[0134] In order to preserve the original feature information, in this embodiment, the first convolutional feature can be subjected to 1×1 convolution dimensionality reduction to obtain the third convolutional feature. The third convolutional feature is then fused with the second convolutional feature. The shuffled feature information is combined with the original feature information, which ensures that while learning complex abstract features, the key details and contextual information in the original feature information are preserved, and can more comprehensively represent the semantic content of the input feature to be processed.

[0135] S26: The channel fusion features corresponding to all the channel features are spliced ​​and recombined to obtain the dimensionality reduction features.

[0136] S3: Input the dimensionality reduction features into several sequentially connected global memory encoders for processing;

[0137] In this embodiment, the global memory encoder is pre-trained. It can be understood that the global memory encoder uses the dimensionality reduction feature as its input feature or the output feature of the previous global memory encoder as its input feature. That is, among the several sequentially connected global memory encoders, the input feature of the first global memory encoder is the dimensionality reduction feature, and the input features of the other global memory encoders are the output features of the previous global memory encoder.

[0138] In this embodiment, the processing of each of the global memory encoders may include the following steps:

[0139] S31: The input features are processed sequentially through channel attention mechanism and spatial attention mechanism to obtain modulation features;

[0140] In this embodiment, as Figure 3 As shown, step S31 may include the following sub-steps:

[0141] S311: The input features are processed through a channel attention mechanism to obtain channel attention features;

[0142] S312: The channel attention features are fused with the input features to obtain attention fusion features;

[0143] S313: The attention fusion features are processed through a spatial attention mechanism to obtain spatial attention features;

[0144] S314: The attention fusion feature and the spatial attention feature are fused to obtain the modulation feature.

[0145] Wherein, the feature fusion of the channel attention feature and the input feature is performed by multiplying the channel attention feature and the input feature element by element; the feature fusion of the attention fusion feature and the spatial attention feature is performed by multiplying the attention fusion feature and the spatial attention feature element by element.

[0146] Understandably, in this embodiment, based on the synergistic effect of the channel attention mechanism and the spatial attention mechanism, discriminative features are screened from the channel dimension and potential disease areas are located from the spatial dimension, respectively, providing more accurate feature representations for subsequent processing.

[0147] S32: The modulation features are processed by a window self-attention mechanism to obtain the first attention feature;

[0148] In this embodiment, step S32 may include:

[0149] The modulation feature is divided into several non-overlapping local features according to a preset attention window, and each local feature is processed by a self-attention mechanism to obtain the local self-attention feature corresponding to each local feature. Then, all local self-attention features are spliced ​​and recombined based on the division of the local features to obtain the first attention feature.

[0150] S33: The modulation features are processed by an adaptive contrastive perception gating attention mechanism to obtain a second attention feature;

[0151] In this embodiment, as Figure 4 As shown, step S33 may include the following sub-steps:

[0152] S331: Divide the modulation feature into several non-overlapping local window features according to a preset local window;

[0153] In this embodiment, step S331 can be represented as:

[0154] ;

[0155] In the formula, The modulation feature, For the local window features, The height of the local window corresponding to the aforementioned local window feature. The width of the local window corresponding to the aforementioned local window feature; (·) is the window partitioning function.

[0156] S332: Sort the local window features according to the adjacency relationship of the local windows corresponding to the local window features;

[0157] S333: Process each of the local window features in the sorted order to obtain the local attention features of the local window features;

[0158] S334: Concatenate and reassemble the local attention features of all the local windows to obtain the second attention feature;

[0159] In this embodiment, by sorting the local window features based on the adjacency relationship of the local windows and processing each local window feature in turn, the local window features can be processed in combination with the spatial information of the previous local window feature, so as to realize information transmission between local windows and improve the accuracy of fruit disease detection.

[0160] In one implementation, the step S333, which involves processing the features of each local window to obtain the local attention features of the local window features, may include the following sub-steps:

[0161] S3331: Generate the query vector, key vector, and value vector of the local window features through linear projection;

[0162] In this embodiment, the local window features can be processed using a learnable linear projection matrix and a learnable bias, as follows:

[0163] ;

[0164] In the formula, For the query vector, For the key vector, For the value vector, For learnable linear projection matrices, This is a learnable bias.

[0165] In one implementation, after obtaining the query vector and the key vector, the query vector can be split into a first query vector and a second query vector based on the average dimension of the query vector, and the key vector can be split into a first key vector and a second key vector based on the average dimension of the key vector. This splitting of the query vector and the key vector supports multi-dimensional feature interactions. It is understood that if the dimension of the query vector is 2c, then the dimension of the first query vector and the second query vector is c; similarly, if the dimension of the key vector is 2c, then the dimension of the first key vector and the majority of the second key vectors is c.

[0166] S3332: Based on the value vector and the local window size corresponding to the local window feature, obtain the attention value vector corresponding to the local window feature;

[0167] In this embodiment, step S333 may include:

[0168] First, the value vector is subjected to spatial global average pooling based on the size of the local window corresponding to the local window feature, resulting in a spatially pooled value vector, represented as:

[0169] ;

[0170] In the formula, The spatial pooling value vector, For the first The pixels in the local window corresponding to the aforementioned local window features The value vector, The height of the local window corresponding to the aforementioned local window feature. The width of the local window corresponding to the aforementioned local window feature;

[0171] Then, attention weights are generated based on the spatial pooling vector, expressed as:

[0172] ;

[0173] In the formula, and These are the learnable weights, and These are the learnable biases, For Sigmoid activation function, ReLU represents ReLU activation function;

[0174] Next, the attention weights are multiplied channel by channel by channel of the value vector to obtain the attention value vector corresponding to the local window feature.

[0175] Understandably, by performing spatial global average pooling on the value vector according to the size of the local window corresponding to the local window features, the weights of channel attention can be effectively learned. Then, by multiplying it channel by channel with the original value vector to strengthen the disease-related feature channels and suppress noise, the accuracy of disease identification can be improved.

[0176] S3333: Calculate the local enhanced attention weights corresponding to the local window features based on the query vector and the key vector;

[0177] In this embodiment, as Figure 5 As shown, step S3333 may include:

[0178] A1: Calculate the attention score of the local window feature based on the query vector and the key vector;

[0179] In this embodiment, the attention score may include a primary attention score and a secondary attention score. Furthermore, as described above, the query vector is divided into a first query vector and a second query vector, and the key vector is divided into a first key vector and a second key vector, to enable feature interaction from different perspectives. Therefore, the primary attention score and secondary attention score can be calculated using the first query vector and the first key vector, and the second query vector and the second key vector, respectively, thereby achieving feature interaction through the primary and secondary attention scores. Step A1 may then include:

[0180] The main attention score is calculated based on the first query vector and the first key vector, and is expressed as follows:

[0181] ;

[0182] In the formula, The main attention score is... This is the first query vector. This represents the transpose of the first key vector. The dimensions of the query vector and the key vector;

[0183] The secondary attention score is calculated based on the second query vector and the second key vector, and is expressed as follows:

[0184] ;

[0185] In the formula, The auxiliary attention score is... This is the second query vector. This represents the transpose of the second key vector.

[0186] A2: Calculate the dimension matching score of the local window features based on the preset learnable weight tensor;

[0187] In this embodiment, step A2 may include:

[0188] Learnable weight tensors corresponding to each dimension of the first query vector, the second query vector, the first key vector, and the second key vector are extracted respectively; the learnable weight tensors are obtained through pre-training.

[0189] The main dimension matching score is calculated based on the learnable weight tensors of all dimensions of the first query vector and the first key vector, and is expressed as:

[0190] ;

[0191] In the formula, Match scores for the main dimension. For the first query vector, the first The learnable weight tensor corresponding to each dimension For the first key vector, the first key vector is the first key vector. The learnable weight tensors corresponding to each dimension; C is the number of dimensions of the first query vector, the first key vector, the second query vector, and the second key vector; It is an exponential function;

[0192] The secondary dimension matching score is calculated based on the learnable weight tensors of all dimensions of the second query vector and the second key vector, and is expressed as follows:

[0193] ;

[0194] In the formula, Match scores for the secondary dimension. For the second query vector, the first The learnable weight tensor corresponding to each dimension For the second key vector, the first The learnable weight tensor corresponding to each dimension.

[0195] Understandably, in this embodiment, attention calculation is divided into primary attention calculation and secondary attention calculation, and the primary dimension matching score and secondary dimension matching score of the primary attention calculation and secondary attention calculation are calculated respectively to capture the dimensional correlation between the query vector and the key vector. Then, the primary dimension matching score and secondary dimension matching score are amplified by an exponential function and mapped to the positive number domain to enhance the distinguishability of small matching degree differences.

[0196] A3: Calculate the local enhanced attention weights corresponding to the local window features based on the attention score and dimension matching score.

[0197] In this embodiment, step A3 may include:

[0198] Calculate the difference between the primary dimension matching score and the secondary dimension matching score, and map the difference using the Sigmoid function to obtain the fusion weight vector, expressed as:

[0199] ;

[0200] ;

[0201] In the formula, The fusion weight vector, These are the basic parameters of the global memory encoder. This refers to the position of the global memory encoder among several sequentially connected global memory encoders. Use the Sigmoid activation function;

[0202] The main attention weight is calculated based on the main attention score and expressed as follows:

[0203] ;

[0204] In the formula, Main attention weight, The relative position offset of the local window corresponding to the local window feature;

[0205] The secondary attention weight is calculated based on the secondary attention score and expressed as follows:

[0206] ;

[0207] In the formula, As a secondary attention weight;

[0208] The local enhanced attention weight is calculated based on the main attention weight, the secondary attention weight, and the fusion weight vector, and is expressed as follows:

[0209] ;

[0210] S3334: Obtain the local attention features of the local window based on the associated attention state of the local window corresponding to the local window features, the local enhanced attention weight, the query vector, and the attention value vector;

[0211] In this embodiment, the associated attention state is generated based on the query vector corresponding to the previous local window and the local enhanced attention weights; therefore, step S3334 may include:

[0212] Based on the associated attention state of the local window corresponding to the local window features and the first query vector, the gating information of the local window is obtained, represented as follows:

[0213] ;

[0214] In the formula, The gate control information, This is the first query vector. Indicates the first The associated attention state of the local window corresponding to the local window feature, specifically the first... The attention state of the local window corresponding to each of the aforementioned local window features. For learnable weights, For learnable bias;

[0215] The attention state of the local window is generated based on the gating information, the associated attention state, and the local enhanced attention weights, and is represented as follows:

[0216] ;

[0217] In the formula, Let r be the attention state of the local window corresponding to the r-th local window feature. For learnable scaling factor, The symbol for element-wise multiplication;

[0218] The local attention features of the local window are calculated based on the attention state of the local window and the attention value vector, and are expressed as follows:

[0219] ;

[0220] In the formula, The local attention features, For the attention value vector, For learnable weights, This is a learnable bias.

[0221] Understandably, fruit disease symptoms are complex and highly similar, easily confused with natural leaf textures, leading to difficulties in detecting small targets and insufficient extraction of fine-grained features. To construct long-distance dependencies between disease features, in this embodiment, a state attention unit is set for each local window in the global memory encoder. This state attention unit outputs the attention state of the attention query for the corresponding local window. Since the features of each local window are processed sequentially in this embodiment, the global memory encoder can adjust the current key attention weight of the local window based on the disease key attention weight information learned from the previous local window (historical local window). Furthermore, by calculating the gating information Z of the local window, a linear transformation is performed on the query vector of the local window, mapping it to the [0,1] interval, thereby controlling the attention state output by the current state attention unit of the current local window. In this process, it's necessary to determine how much of the attention state of historical local windows needs to be retained and how much of the local enhancement attention weight of the current local window needs to be retained, thereby achieving adaptive adjustment of the attention state. Understandably, when the gating information is close to 1, the global memory encoder primarily relies on the attention state of historical local windows; when the gating information is close to 0, the global memory encoder pays more attention to the local enhancement attention weight of the current local window.

[0222] In one embodiment, before the step of dividing the modulation feature into several non-overlapping local window features according to a preset local window, the following step may be included:

[0223] The modulation feature is divided into several modulation channel features based on a preset modulation channel dimension, and each modulation channel feature is convolved to obtain the channel convolution feature corresponding to the modulation channel feature.

[0224] Based on the dimension of the modulation features, the convolutional features of each channel are fused by pointwise convolution to obtain convolutional fusion features;

[0225] The convolutional fusion feature and the modulation feature are added together to obtain the enhanced modulation feature;

[0226] The enhanced modulation features are used to perform the step of dividing the modulation features into several non-overlapping local window features according to a preset local window.

[0227] S34: Fuse the first attention feature and the second attention feature to obtain the fused feature;

[0228] In this embodiment, the feature fusion of the first attention feature and the second attention feature is to add the first attention feature and the second attention feature element by element.

[0229] S35: The fused features are sequentially subjected to nonlinear transformation and feature integration to obtain the output features;

[0230] S4: Obtain the output features of the last global memory encoder for identification, and obtain the fruit disease identification results.

[0231] This embodiment also provides a fruit disease detection model based on global memory and local comparison. The fruit disease detection model is obtained through pre-training. In this embodiment, the above steps S1-S4 can be performed through the fruit disease detection model 100.

[0232] In one implementation, such as Figure 6 As shown, the fruit disease detection model 100 may include a feature extraction unit 110, a mixed washing interactive dimensionality reduction unit 120, an encoding network layer 130, and a decoding network layer 140 connected in sequence.

[0233] The feature extraction unit 110 is used to perform step S1, and the specific content of the feature extraction unit 110 can be found in the description of step S1 above; the shuffling interaction dimensionality reduction unit 120 is used to perform step S2, and the specific content of the shuffling interaction dimensionality reduction unit 120 can be found in the description of step S2 above; the decoding network layer 140 is used to perform step S4, and the specific content of the decoding network layer 140 can be found in the description of step S4 above.

[0234] In this embodiment, the coding network layer 130 includes a plurality of sequentially connected global memory encoders as described above. The coding network layer 130 can be used to perform the above step S3. The specific content of the coding network layer 130 can be found in the description of the above step S3.

[0235] Furthermore, in one implementation, such as Figure 7As shown, the global memory encoder includes a channel attention unit 131, a spatial attention unit 132, a first stitching unit 133, a second stitching unit 134, a third stitching unit 135, a window self-attention unit 136, an adaptive contrast gating attention unit 137, and a multilayer perceptron 138.

[0236] Specifically, the channel attention unit 131 is used to execute step S311, that is, to process the input features through a channel attention mechanism to obtain channel attention features. The specific details of the channel attention unit 131 can be found in the description of step S311 above. The first splicing unit 133 is used to execute step S312, that is, to fuse the channel attention features with the dimensionality reduction features to obtain attention fusion features. The specific details of the first splicing unit 133 can be found in the description of step S312 above. The spatial attention unit 132 is used to execute step S313, that is, to process the attention fusion features through a spatial attention mechanism to obtain spatial attention features. The specific details of the spatial attention unit 132 can be found in the description of step S313 above. The second splicing unit 134 is used to execute step S314, that is, to fuse the attention fusion features with the spatial attention features to obtain the modulation features. The specific details of the second splicing unit 134 can be found in the description of step S314 above.

[0237] The window self-attention unit 136 is used to perform step S32, that is, to process the modulation feature through the window self-attention mechanism to obtain the first attention feature. The specific content of the window self-attention unit 136 can be found in the description of step S32 above.

[0238] The third splicing unit 135 is used to perform step S34, that is, to fuse the first attention feature and the second attention feature to obtain the fused feature. The specific content of the third splicing unit 135 can be found in the description of step S34 above.

[0239] The multilayer perceptron 138 is used to perform step S35, that is, to sequentially perform nonlinear transformation and feature integration on the fused features to obtain output features. The specific content of the multilayer perceptron 138 can be found in the description of step S35 above.

[0240] The adaptive contrast-gated attention unit 137 is used to execute step S33 and the sub-steps in step S33. The processing flow of the adaptive contrast-gated attention unit 137 is as follows: Figure 8As can be understood, the specific details of the adaptive contrast-gated attention unit 137 can be found in the description of step S33 and its sub-steps.

[0241] like Figure 9 As shown, this application embodiment also provides a fruit disease detection system based on global memory and local comparison. Optionally, the fruit disease detection system may include:

[0242] The feature extraction module 11 is used to extract features from the image of the fruit to be detected and obtain the features to be processed.

[0243] In this embodiment, the feature extraction module 11 can be used to perform... Figure 1 For a detailed description of the feature extraction module 11 shown in step S1, please refer to the description of step S1.

[0244] Feature dimensionality reduction module 12 is used to perform channel shuffling dimensionality reduction processing on the features to be processed to obtain dimensionality-reduced features;

[0245] In this embodiment, the feature dimensionality reduction module 12 can be used to perform... Figure 1 For a detailed description of the feature dimensionality reduction module 12 shown in step S2, please refer to the description of step S2.

[0246] Feature processing module 13 is used to input the dimensionality reduction features into several sequentially connected global memory encoders for processing;

[0247] In this embodiment, the feature processing module 13 can be used to perform... Figure 1 For a detailed description of the feature processing module 13 shown in step S3, please refer to the description of step S3.

[0248] The feature recognition module 14 is used to recognize the output features of the last global memory encoder and obtain the disease recognition results of the fruit.

[0249] In this embodiment, the feature recognition module 14 can be used to perform... Figure 1 For a detailed description of the feature recognition module 14 shown in step S4, please refer to the description of step S4.

[0250] This application provides an electronic device with the following structure: Figure 10 As shown.

[0251] The electronic device includes a memory 21, a processor 22, a communication module 23, and an input / output interface 24, etc. Optionally, the memory 21, the processor 22, the communication module 23, and the input / output interface 24 can be connected and communicate with each other through a bus 25.

[0252] The memory 21 is used to store one or more computer programs and to transfer the code of the computer programs to the processor 22; when the one or more computer programs are executed by the processor 22, a fruit disease detection method based on global memory and local comparison in this application embodiment is implemented.

[0253] Optionally, the electronic device can be connected to a network via communication module 23 to communicate with other devices, such as terminals or servers, to achieve data interaction. The electronic device can be various forms of digital computers, exemplarily such as desktop computers, servers, workbenches, mainframes, or other types of computers. The electronic device can also be various forms of mobile terminals, exemplarily such as smartphones, tablets, wearable devices (such as helmets, glasses, watches, etc.), and other similar mobile terminals.

[0254] Optionally, the electronic device can connect to required input / output devices, such as a keyboard or display device, via the input / output interface 24. The electronic device itself may have a display device, and other display devices can also be connected externally via the input / output interface 24. Optionally, a storage device, such as a hard disk, can also be connected via the input / output interface 24 to store data from the electronic device, read data from the storage device, or store data from the storage device in the memory 21. It is understood that the input / output interface 24 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 24 can be a component of the electronic device or an external device connected to the electronic device when needed.

[0255] Optionally, the memory 21 may be a volatile memory and / or a non-volatile memory. The volatile memory may be a random access memory, etc., and the non-volatile memory may be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, or a flash memory, etc.

[0256] Optionally, the computer program stored in the processor 22 can be divided into one or more modules, which are stored in the memory 21 and executed by the processor 22 to perform the method provided in this embodiment. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device.

[0257] Optionally, the processor 22 can be various general-purpose and / or dedicated processing components with processing and computing capabilities. Some examples of the processor 22 include, but are not limited to, a central processing unit, a graphics processing unit, a digital signal processor, various dedicated artificial intelligence computing chips, various processors running machine learning model algorithms, and can also be any suitable controller, microcontroller, processor, etc. The processor 22 executes the various methods and processes of this embodiment, exemplarily, such as a fruit disease detection method based on global memory and local comparison according to an embodiment of this application.

[0258] Optionally, the bus 25 may include a path for transmitting information. Depending on its function, the bus 25 may be divided into an address bus, a data bus, a control bus, etc.

[0259] In an optional implementation, this application embodiment also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods described in the above-described method embodiments. Part or all of the computer program may be loaded and / or installed on the memory 21 of an electronic device. When the computer program is executed by the processor 22, one or more steps of a fruit disease detection method based on global memory and local comparison, according to an embodiment of this application, can be performed.

[0260] Optionally, the computer-readable storage medium may be a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, etc.

[0261] Obviously, the above embodiments of this application are merely examples for clearly illustrating the technical solution of this application, and are not intended to limit the specific implementation of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of this application should be included within the protection scope of the claims of this application.

Claims

1. A method for detecting fruit diseases based on global memory and local comparison, characterized in that, The method includes: Feature extraction is performed on the fruit image to be detected to obtain the features to be processed; The features to be processed are subjected to channel shuffling and dimensionality reduction processing to obtain the dimensionality-reduced features; The dimensionality reduction feature is input into several sequentially connected global memory encoders for processing; wherein the input feature of the first global memory encoder is the dimensionality reduction feature, and the input features of the other global memory encoders are the output features of the previous global memory encoder; The output features of the last global memory encoder are obtained for identification, and the disease identification results of the fruit are obtained. The processing of each of the global memory encoders includes: The input features are processed sequentially through channel attention and spatial attention mechanisms to obtain modulation features; The modulation features are processed using a window self-attention mechanism to obtain the first attention feature; The modulation features are processed by an adaptive contrastive perception gating attention mechanism to obtain a second attention feature; The first attention feature and the second attention feature are fused to obtain the fused feature; The fused features are then subjected to nonlinear transformation and feature integration in sequence to obtain the output features; The process of processing the modulation features through an adaptive contrastive perception gated attention mechanism to obtain second attention features includes: The modulation feature is divided into several non-overlapping local window features according to a preset local window. The local window features are sorted according to the adjacency relationship of the local windows corresponding to the local window features; According to the sorted order, each of the local window features is processed sequentially to obtain the local attention features of the local window features; The local attention features of all the local windows are spliced ​​and recombined to obtain the second attention feature; The step of processing the local window features to obtain the local attention features of the local window features includes: The query vector, key vector, and value vector of the local window features are generated by linear projection. Based on the value vector and the local window size corresponding to the local window feature, obtain the attention value vector corresponding to the local window feature; Calculate the local enhancement attention weights corresponding to the local window features based on the query vector and the key vector; The local attention features of the local window are obtained based on the associated attention state of the local window corresponding to the local window features, the local enhanced attention weights, the query vector, and the attention value vector; the associated attention state is generated based on the query vector and local enhanced attention weights corresponding to the previous local window of the local window.

2. The fruit disease detection method based on global memory and local comparison according to claim 1, characterized in that, The process of performing channel shuffling and dimensionality reduction on the features to be processed to obtain dimensionality-reduced features includes: The feature to be processed is divided into several channel features based on a preset channel dimension, and each channel feature is convolved to obtain the first convolution feature corresponding to each channel feature. Perform channel shuffling on all the first convolutional features to obtain the shuffled features corresponding to each channel feature; The shuffled features are subjected to convolutional dimensionality reduction processing to obtain the second convolutional features; The first convolutional feature is subjected to convolutional dimensionality reduction based on the dimension of the second convolutional feature to obtain the third convolutional feature; The second convolutional feature and the third convolutional feature corresponding to each of the channel features are fused to obtain the channel fused feature corresponding to each of the channel features; The channel fusion features corresponding to all the channel features are spliced ​​and recombined to obtain the dimensionality reduction features.

3. The fruit disease detection method based on global memory and local comparison according to claim 1, characterized in that, The step of obtaining the attention value vector corresponding to the local window feature based on the value vector and the local window size corresponding to the local window feature includes: The value vector is then subjected to spatial global average pooling based on the size of the local window corresponding to the local window feature, resulting in a spatially pooled value vector, represented as follows: ； In the formula, The spatial pooling value vector, For the first The pixels in the local window corresponding to the aforementioned local window features The value vector, The height of the local window corresponding to the aforementioned local window feature. The width of the local window corresponding to the aforementioned local window feature; Attention weights are generated based on the spatial pooling vector, and are expressed as follows: ； In the formula, The attention weights, and These are the learnable weights, and These are the learnable biases, For Sigmoid activation function, ReLU represents ReLU activation function; The attention weights are multiplied channel by channel by channel of the value vector to obtain the attention value vector corresponding to the local window features.

4. The fruit disease detection method based on global memory and local comparison according to claim 1, characterized in that, The step of obtaining the local attention features of the local window based on the associated attention state of the local window corresponding to the local window features, the local enhanced attention weight, the query vector, and the attention value vector includes: Based on the attention state of the local window corresponding to the local window features and the first query vector, the gating information of the local window is obtained. The first query vector is obtained by splitting the query vector based on dimensions, and is represented as follows: ； In the formula, The gate control information, This is the first query vector. Indicates the first The associated attention state of the local window corresponding to the local window feature, specifically the first... The attention state of the local window corresponding to each of the aforementioned local window features. For learnable weights, For learnable bias, Use the Sigmoid activation function; The attention state of the local window is generated based on the gating information, the associated attention state, and the local enhanced attention weights, and is represented as follows: ； In the formula, Let r be the attention state of the local window corresponding to the r-th local window feature. For the local enhanced attention weights, For learnable scaling factor, The symbol for element-wise multiplication; The local attention features of the local window are calculated based on the attention state of the local window and the attention value vector, and are expressed as follows: ； In the formula, The local attention features, For the attention value vector, For learnable weights, This is a learnable bias.

5. The fruit disease detection method based on global memory and local comparison according to claim 1, characterized in that, The step of calculating the local enhancement attention weights corresponding to the local window features based on the query vector and the key vector includes: The attention score of the local window feature is calculated based on the query vector and the key vector; The dimension matching score of the local window features is calculated based on the preset learnable weight tensor. Based on the attention score and dimension matching score, calculate the local enhanced attention weights corresponding to the local window features.

6. The fruit disease detection method based on global memory and local comparison according to claim 5, characterized in that, The attention score includes the primary attention score and the secondary attention score; The step of calculating the attention score of the local window features based on the query vector and the key vector includes: The query vector is split into a first query vector and a second query vector, and the key vector is split into a first key vector and a second key vector; The main attention score is calculated based on the first query vector and the first key vector, and is expressed as follows: ； In the formula, The main attention score is... This is the first query vector. This represents the transpose of the first key vector. The dimensions of the query vector and the key vector; The secondary attention score is calculated based on the second query vector and the second key vector, and is expressed as follows: ； In the formula, The auxiliary attention score is... This is the second query vector. This represents the transpose of the second key vector.

7. The fruit disease detection method based on global memory and local comparison according to claim 6, characterized in that, The dimension matching score includes the primary dimension matching score and the secondary dimension matching score; Calculate the dimensionality matching score of the local window features based on a preset learnable weight tensor, including: Learnable weight tensors corresponding to each dimension of the first query vector, the second query vector, the first key vector, and the second key vector are extracted respectively; the learnable weight tensors are obtained through pre-training. The main dimension matching score is calculated based on the learnable weight tensors of all dimensions of the first query vector and the first key vector, and is expressed as: ； In the formula, Match scores for the main dimension. For the first query vector, the first The learnable weight tensor corresponding to each dimension For the first key vector, the first key vector is the first key vector. The learnable weight tensors corresponding to each dimension; C is the number of dimensions of the first query vector, the first key vector, the second query vector, and the second key vector; It is an exponential function; The secondary dimension matching score is calculated based on the learnable weight tensors of all dimensions of the second query vector and the second key vector, and is expressed as follows: ； In the formula, Match scores for the secondary dimension. For the second query vector, the first The learnable weight tensor corresponding to each dimension For the second key vector, the first The learnable weight tensor corresponding to each dimension.

8. The fruit disease detection method based on global memory and local comparison according to claim 7, characterized in that, The step of calculating the local enhancement attention weights corresponding to the local window features based on the attention score and dimension matching score includes: Calculate the difference between the primary dimension matching score and the secondary dimension matching score, and map the difference using the Sigmoid function to obtain the fusion weight vector, expressed as: ； ； In the formula, The fusion weight vector, These are the basic parameters of the global memory encoder. This refers to the position of the global memory encoder among several sequentially connected global memory encoders. Use the Sigmoid activation function; The main attention weight is calculated based on the main attention score and expressed as follows: ； In the formula, Main attention weight, The relative position offset of the local window corresponding to the local window feature; The secondary attention weight is calculated based on the secondary attention score and expressed as follows: ； In the formula, As a secondary attention weight; The local enhanced attention weight is calculated based on the main attention weight, the secondary attention weight, and the fusion weight vector, and is expressed as follows: 。 9. A fruit disease detection system based on global memory and local comparison, characterized in that, The system includes: The feature extraction module is used to extract features from the fruit image to be detected and obtain the features to be processed. The feature dimensionality reduction module is used to perform channel shuffling dimensionality reduction processing on the features to be processed to obtain dimensionality-reduced features; The feature processing module is used to input the dimensionality reduction features into several sequentially connected global memory encoders for processing; wherein the input feature of the first global memory encoder is the dimensionality reduction feature, and the input features of the other global memory encoders are the output features of the previous global memory encoder; The feature recognition module is used to acquire the output features of the last global memory encoder for recognition, and obtain the disease recognition results of the fruit; The processing of each of the global memory encoders includes: The input features are processed sequentially through channel attention and spatial attention mechanisms to obtain modulation features; The modulation features are processed using a window self-attention mechanism to obtain the first attention feature; The modulation features are processed by an adaptive contrastive perception gating attention mechanism to obtain a second attention feature; The first attention feature and the second attention feature are fused to obtain the fused feature; The fused features are then subjected to nonlinear transformation and feature integration in sequence to obtain the output features; The process of processing the modulation features through an adaptive contrastive perception gated attention mechanism to obtain second attention features includes: The modulation feature is divided into several non-overlapping local window features according to a preset local window. The local window features are sorted according to the adjacency relationship of the local windows corresponding to the local window features; According to the sorted order, each of the local window features is processed sequentially to obtain the local attention features of the local window features; The local attention features of all the local windows are spliced ​​and recombined to obtain the second attention feature; The step of processing the local window features to obtain the local attention features of the local window features includes: The query vector, key vector, and value vector of the local window features are generated by linear projection. Based on the value vector and the local window size corresponding to the local window feature, obtain the attention value vector corresponding to the local window feature; Calculate the local enhancement attention weights corresponding to the local window features based on the query vector and the key vector; The local attention features of the local window are obtained based on the associated attention state of the local window corresponding to the local window features, the local enhanced attention weights, the query vector, and the attention value vector; the associated attention state is generated based on the query vector and local enhanced attention weights corresponding to the previous local window of the local window.

Images