Method for detecting disease and insect pest crop image symptom region by using angle similarity based on rotation transformer mechanism

Abstract

The invention relates to a disease and insect pest crop image symptom region detection method based on a rotation transformer mechanism and using angle similarity. Compared with the prior art, the defect that detection of an irregular disease and insect pest crop image symptom region is difficult to achieve is overcome. The method comprises the following steps: acquiring a symptom region image of crops damaged by diseases and pests; constructing a symptom region detection model of crops damaged by diseases and pests; training a symptom region detection model of the crops damaged by diseases and pests; acquiring a disease and pest-damaged crop symptom region image to be detected; and obtaining a disease and pest-damaged crop symptom region detection result. According to the method, high-quality query is obtained according to angle similarity on the basis of a rotation transform mechanism, and more accurate symptom region detection of crops damaged by diseases and pests is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of pest and disease image recognition, and in particular to a method for detecting symptom areas of pest and disease crop images based on angle similarity based on a rotation transformer mechanism. Background Art

[0002] In recent years, although there are some methods for detecting symptom areas of pest-infested crops based on image recognition, there are still some drawbacks. Traditional image recognition methods usually rely on manually designed features, which may not be robust and accurate enough for complex pest and disease symptoms. In addition, existing detection methods do not perform well when dealing with crop images at different angles, and it is difficult to accurately detect the area where the symptoms are located. Rapid and accurate identification of symptom areas of pest-infested crops is beneficial to the timely implementation of prevention and control measures and reduce the losses caused by pests and diseases to agricultural production. Symptom areas of pest-infested crops in the field do not occur regularly. For example, the damage of rice leaf rollers is characterized by intersection, density, and slenderness. Figure 2 As shown, sheath blight may show characteristics such as elongated and arranged, e.g. Figure 3 shown.

[0003] Therefore, it is of great practical value to develop a method that can adapt to complex natural environmental conditions and accurately detect symptom areas of pest-infested crops. Summary of the invention

[0004] The purpose of the present invention is to solve the defect that it is difficult to realize symptom area detection of irregular pest and disease crop images in the prior art, and to provide a method for detecting symptom area of ​​pest and disease crop images based on a rotation transformer mechanism and utilizing angle similarity to solve the above problem.

[0005] In order to achieve the above object, the technical solution of the present invention is as follows:

[0006] A method for detecting symptom areas of pest and disease crop images based on a rotation transformer mechanism and utilizing angle similarity comprises the following steps:

[0007] Acquiring symptom area images of crops damaged by pests and diseases: acquiring symptom area images of crops damaged by pests and diseases in the field, and forming a symptom area image dataset of crops damaged by pests and diseases;

[0008] Constructing a symptom area detection model for pest-infested crops: The symptom area detection model for pest-infested crops is set to include a local feature modeling network, a global feature modeling network, a high-quality query screening network, a detail feature modeling network, and a classification regression network;

[0009] Training of the symptom area detection model for pest-damaged crops: The symptom area detection model for pest-damaged crops is trained using the pest-damaged symptom area image dataset;

[0010] Acquire the image of the symptom area of ​​the crop to be detected by pests and diseases: acquire the image of the symptom area of ​​the crop to be detected by pests and diseases;

[0011] Obtaining the detection results of the symptom areas of pest-infested crops: inputting the images of the symptom areas of pest-infested crops to be detected into the trained symptom area detection model of pest-infested crops to obtain the classification and positioning results of the symptom areas of pest-infested crops.

[0012] The construction of the symptom area detection model for crops damaged by pests and diseases comprises the following steps:

[0013] The first part of the symptom area detection model for pest-damaged crops is set as a local feature modeling network, the second part is a global feature modeling network, the third part is a high-quality query screening network, the fourth part is a detail feature modeling network, and the fifth part is a classification regression network;

[0014] A local feature modeling network is set up. The local feature modeling network is used to locally model the features of the symptom area of ​​the pest-damaged crops in the entire image. The input of the local feature modeling network is the symptom area image of the pest-damaged crops in the field, and the output is a multi-layer local feature map of the symptom area image of the pest-damaged crops in the field. The local feature vector corresponding to each feature space position in the multi-layer local feature map can only be seen in a partial area range of the original image.

[0015] The local feature modeling network is set to be built based on the residual network. After the first convolution with a kernel size of 7*7 and a stride of 2, a pooling operation with a pooling range of 3*3 and stride=2 is used, which is composed of a convolutional neural network layer, a batch normalization layer, and an activation function layer.

[0016] Setting a global feature modeling network: The global feature modeling network obtains global modeling information of the symptom area image of the pest-infested crop based on the multi-layer local feature map. The input of the global feature modeling network is the multi-layer local feature map, and the output of the global feature modeling network is the multi-layer global feature map.

[0017] The global feature modeling network adopts the Transformer encoder structure. The global feature modeling network consists of 6 encoding modules with the same structure stacked in sequence.

[0018] Each encoding module consists of a multi-head deformable attention structure and a feed-forward network. The multi-head deformable attention structure has residual connections, and both the multi-head deformable attention structure and the feed-forward network are followed by Layer Norm operations.

[0019] The global feature modeling network first performs a global modeling operation on the multi-layer features obtained by the local feature modeling network, and uses the sine-cosine function combined with the specific spatial position of the feature vector to generate a spatial position code for the feature vector. At the same time, a learnable layer position code is set according to the position of the layer. The spatial position code and the layer position code are added to form a position code. The local feature vector and the position code are combined to obtain a local feature vector with a position code. All local feature vectors in the multi-layer local feature map with position code are input into the encoding module, and finally fused and mapped into a global feature vector to form a global feature map.

[0020] Set up a high-quality query screening network: The input of the high-quality query screening network is a multi-layer global feature map. The high-quality query screening network first removes the query vectors corresponding to some boxes with large overlap based on the maximum suppression method with a rotation intersection ratio of 0.8 for the multi-layer global feature map, and then removes the query vectors corresponding to boxes with small overlap but small angle range difference through angle similarity. Finally, k1 unique query vectors are obtained. During the training process, the k2=1.5*k1 query vectors with the highest scores and k3 positive and negative sample annotation query vectors generated according to the annotation of the own data set are selected. The global feature vector composed of these query vectors is used to initialize the query vector of the detail feature modeling network to assist the training of the detail feature modeling network. During the test process, the auxiliary training branch is no longer used;

[0021] Set up a detail feature modeling network to model the detail features of the global feature vector. The detail feature modeling network contains 6 decoding modules. Each decoding module has the same structure. Each decoding module consists of a multi-head self-attention, a multi-head deformable attention and a forward feedforward network. Both the multi-head self-attention and the multi-head deformable attention have residual connections. All three parts are connected with a Layer Norm operation.

[0022] The input of the detail feature modeling network is k1 unique query vectors, k2=1.5*k1 high-scoring queries, and a set of k3 positive and negative sample annotated query vectors created based on the annotated instance labels. The number of output queries of the detail feature modeling network is k1+k2+k3, where the k2 high-scoring queries and k3 annotated queries only assist the training of the network and are no longer used during the test process.

[0023] A classification regression network is set to map the output query of the detail feature modeling network to the type and specific location coordinates of the symptom area of ​​pest-infested crops. The classification regression network includes a main network head and a secondary network head. The secondary network head only exists in the training process. Both the main network head and the secondary network head include two sub-networks, classification and regression. The inputs of the classification sub-network and the regression sub-network are the output queries of the detail feature modeling network. For each detail query, a forward feedforward network is passed to output the predicted category and specific location coordinates of the target box to be detected.

[0024] The training of the symptom area detection model for pest-damaged crops comprises the following steps:

[0025] The symptom area image dataset of pests and diseases is input into the local feature modeling network. The local feature modeling network uses the residual network to obtain the local feature map of the symptom area image of pests and diseases. Feature maps with different receptive field sizes are obtained through different numbers of layers, including: C3 feature map with 512 dimensions and stride=8, C4 feature map with 1024 dimensions and stride=16, and C5 feature map with 2048 dimensions and stride=32.

[0026] The C3, C4, and C5 feature maps are then subjected to convolution with a kernel size of 1*1 and a channel number of 256, and group normalization operations to achieve the purpose of consistent channel numbers, and local feature maps M1, M2, and M3 of w1*h1*256, w2*h2*256, and w3*h3*256 are obtained respectively. In addition, the C5 feature map is subjected to convolution with a kernel size of 3*3, stride=2, and a channel number of 256, and group normalization operations to obtain a local feature map M4 of w4*h4*256, which is also kept consistent with the number of channels of M1, M2, and M3, and M1, M2, M3, and M4 with consistent channel numbers and inconsistent sizes are obtained, a total of 4 layers of local feature maps, and the input image receptive field range represented by the feature vector at each feature space position is different;

[0027] The local feature maps M1, M2, M3, and M4 are input into the global feature modeling network and enter the 6 encoding modules respectively.

[0028] In the encoding module, the input of the multi-head deformable attention structure includes query, value and reference point position Ref; the query is obtained by adding the local image features and the position encoding features. The position encoding features are obtained by encoding the spatial position of each feature point through the sine-cosine function plus the learnable layer position encoding set by the position of the layer. The number of channels of the two is the same, and the position encoding covers the spatial position information of the feature point and the layer information where the feature point is located;

[0029] The query and value of the multi-head deformable attention structure are the same. The query of the multi-head deformable attention structure of the first encoding module is a vector set arranged according to the spatial position and the number of layers, totaling The number of reference points is Among them, the i-th layer reference point p q The position is p q =(p qx ,p qy ),in Normalize the reference point positions of different layers according to the effective positions Then map the effective positions of different layers to other layers to generate

[0030]

[0031] If only one layer is used, the specific formula of the multi-head deformable attention structure is expressed as:

[0032]

[0033] If multiple layers are used, the specific formula of the multi-head deformable attention structure is expressed as:

[0034]

[0035] in

[0036] z q represents the query feature vector, is the position coordinate of the reference point normalized according to the effective position, It is the position coordinates generated by mapping the effective positions of different layers to other layers, W m and W m 'All are fully connected operations from 256 dimensions to 256 dimensions, A mlqk Yes q The weight of the sampling point obtained by the full connection operation and the softmax operation, Δp mlqk Yes q The offset of the sampling point obtained by the full connection operation, is the feature vector of the offset point on the feature map of the lth layer corresponding to the reference point, M is the number of heads in the multi-head deformable attention structure, L is the number of layers of the feature map, K is the number of offset points selected in each layer, The query is obtained through the multi-head deformable attention structure, and then the final global feature vector is obtained through residual connection;

[0037] The global feature vector is normalized by the Layer Norm layer and then input into the feedforward network;

[0038] The feedforward network is mainly composed of two fully connected layers, a dropout layer and a ReLu activation function layer. The vector dimension increases from 256 to 1024 after the first fully connected layer, followed by a ReLu activation function layer and a dropout layer, and then decreases from 1024 to 256 after the second fully connected layer, followed by a dropout layer operation.

[0039] Then the vector obtained after passing through a Layer Norm layer is the output query of the first encoding module;

[0040] After 6 encoding modules with the same settings, the global feature vector output by the global feature modeling network is obtained. Finally, the output of the global feature modeling network is the same size as the input, totaling query vector;

[0041] The high-quality query screening network will screen out all spatial feature vectors of different layers output by the global feature modeling network to obtain high-quality and large-scale initial queries as the detailed feature modeling network.

[0042] First, we remove query vectors corresponding to boxes with large overlaps based on the maximum suppression method with a rotational intersection-union ratio of 0.8 to obtain discriminative queries, where the rotational NMS threshold is 0.8. Then, we remove query vectors corresponding to boxes with small intersection-union ratios but small angle range differences based on angle similarity, and finally obtain k1 unique query vectors.

[0043] During the training process, the k2=1.5*k1 high-scoring query vectors with the highest scores and the k3 positive and negative sample annotated query vectors generated according to the annotation of the own data set are selected to assist the training of the detail feature modeling network. During the testing process, the auxiliary training branch is no longer used. Finally, the k1 unique query vectors, k2 high-scoring query vectors and k3 positive and negative sample annotated query vectors together constitute the query input Q of the detail feature modeling network. In addition, the k1 unique rotation reference frame, the 1.5*k1 high-scoring rotation reference frame and the k3 positive and negative sample rotation reference frames together constitute the rotation reference frame B of the detail feature modeling network.

[0044] The input query of the detail feature modeling network is the k1 unique query vectors output by the high-quality query screening network, the k2=1.5*k1 query vectors with the highest scores, and the k3 positive and negative sample annotation queries created according to the labeled instance labels of the dataset. The six decoding modules are stacked in sequence. The output of the previous module is the query input of the multi-head self-attention in the next module. The query, key and value of the multi-head self-attention in each module are the same. The input query of the multi-head deformable attention of each module is the query output of the multi-head self-attention mechanism in the current module. The value of the multi-head deformable attention is the output of all layers of the global feature modeling network.

[0045] The multi-head self-attention is set to focus on the relationship between the targets in the symptom area of ​​the pest-damaged crops. Through the self-attention structure, the global dependency between targets is captured, allowing each target to understand the contextual information of all other targets in the image.

[0046] The input of the multi-head self-attention structure includes query, keyword and value. The keyword and value of the self-attention structure are generally the same as the query.

[0047] The input of the first decoding module is the sum of the query Q and the position code. The position code is obtained by encoding the rotated reference frame B through the sine-cosine function. The query Q is composed of k1 unique queries, k2 high-scoring queries and k3 positive and negative sample annotation queries. The rotated reference frame B is composed of the corresponding k1 unique rotated reference frames, 1.5*k1 high-scoring rotated reference frames and k3 positive and negative sample rotated reference frames.

[0048] The specific formulas of the self-attention structure and the multi-head self-attention structure are respectively expressed as:

[0049]

[0050] MSA(Q,K,V)=Concat(Attn({QW Q} m ,{KW K} m ,{VW V} m ))W out ,

[0051] m∈1,...,M

[0052] The keyword K and value V of the self-attention structure are the same as the query Q. Attention represents self-attention, and d k is the dimension of K, which is equivalent to normalizing the attention. MSA stands for multi-head self-attention, and W Q ,W K ,W V ,W out is a learnable parameter, m is the index of the multi-head, and M is the number of heads in the multi-head self-attention;

[0053] The output after the multi-head self-attention structure has a residual connection, and then passes through the Layer Norm layer to enter the multi-head deformable attention structure of the current module;

[0054] The value in the multi-head deformable attention structure in the detail feature modeling network is the output of the global feature modeling network, totaling The query input in the structure is the output of the self-attention in the current decoding module, and the number during training is k1+k2+k3;

[0055] The output of the multi-head deformable attention structure has a residual connection, and then passes through the Layer Norm layer and enters the feedforward network structure of the current module;

[0056] At this point, the first decoding module ends, and the training of the detail feature modeling network is repeated six times;

[0057] The classification branch of the classification regression network uses a fully connected operation to use the output query vector of the detail feature modeling network to generate the specific category of the rotation detection box. The regression branch uses a fully connected operation to use the output query vector to generate the specific position of the rotation detection box. The loss of unique queries during training is k1 unique target queries. The 1vs1 HungarianAssigner label assignment method is used to calculate the classification loss, regression loss, and intersection-over-union loss. The classification loss uses FocalLoss, the regression loss uses L1Loss, and the intersection-over-union loss uses GioULoss.

[0058] The auxiliary loss consists of two parts. The first part is k2 high-score queries, which use the 1vs4 TopkHungarianAssigner label assignment method to calculate the classification loss and intersection-over-union loss. The classification loss uses QualityFocalLoss, and the intersection-over-union loss uses GioULoss. The second part is k3 positive and negative sample annotation queries, which use the 1vsG label assignment method to calculate the classification loss, regression loss, and intersection-over-union loss. The three losses are the same as those of unique queries. G is the number of groups. According to the number of labeled instances n, the number of groups is obtained according to G=ceil(100 / n). The sum of all losses finally obtained is used to optimize the model through the AdamW optimizer.

[0059] The method of obtaining the detection result of the image of the symptom area of ​​the crop damaged by pests and diseases comprises the following steps:

[0060] The symptom area image of the pests and diseases to be detected is input into the local feature modeling network. The local feature modeling network uses the residual network to obtain the local feature map of the symptom area image of the pests and diseases, and obtains the local feature maps M1, M2, M3, and M4 of w1*h1*256, w2*h2*256, w3*h3*256, and w4*h4*256 respectively;

[0061] The local feature maps M1, M2, M3, and M4 are input into the global feature modeling network. After 6 encoding modules with the same settings, the global feature query vector output by the global feature modeling network is obtained. Finally, the output of the global feature modeling network is the same size as the input, totaling A multi-layer global feature map composed of query vectors;

[0062] The high-quality query screening network will screen out all spatial feature vectors of different layers output by the global feature modeling network based on angle similarity through the maximum suppression method to select high-quality initial queries for the detail feature modeling network. First, the query vectors corresponding to some boxes with large overlaps are removed based on the maximum suppression method with a rotation intersection-union ratio of 0.8 to obtain discriminative queries. Then, the query vectors corresponding to boxes with small overlaps but small angle range differences are removed through angle similarity, and finally a unique rotated reference box and the corresponding query vector are obtained.

[0063] The input query of the detail feature modeling network is the k1 unique query vectors output by the high-quality query screening network, where the input value of the multi-head deformable attention in the encoding module is the global feature modeling network query vectors, and finally output k1 unique query vectors for detail feature modeling;

[0064] Finally, k1 rotation detection frames are obtained through the classification regression network. The classification branch uses a fully connected operation to use the query vector to generate the specific category of the rotation detection frame, and the regression branch uses a fully connected operation to use the query vector to generate the specific position of the rotation detection frame.

[0065] Beneficial Effects

[0066] The method for detecting symptom areas of pest-infested crop images based on a rotating transformer mechanism and utilizing angle similarity obtains high-quality queries based on angle similarity based on a rotating transformer mechanism compared to the prior art, thereby achieving more accurate symptom area detection of pest-infested crops. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 is a method sequence diagram of the present invention;

[0068] Figure 2 A map of regions marked with damage symptoms of rice leaf roller in the prior art;

[0069] Figure 3 A regional diagram for marking symptoms of sheath blight in the prior art. DETAILED DESCRIPTION

[0070] In order to have a further understanding and recognition of the structural features and the effects achieved by the present invention, a preferred embodiment and accompanying drawings are used for detailed description as follows:

[0071] like Figure 1As shown, the method for detecting symptom areas of pest and disease crop images based on the rotation transformer mechanism and utilizing angle similarity of the present invention comprises the following steps:

[0072] The first step is to obtain symptom area images of pest-infested crops: obtain symptom area images of pest-infested crops in the field to form a symptom area image dataset of pest-infested crops.

[0073] The second step is to build a symptom area detection model for pest-infested crops: the symptom area detection model for pest-infested crops is set to include a local feature modeling network, a global feature modeling network, a high-quality query screening network, a detail feature modeling network and a classification regression network.

[0074] Here, the entire network can realize an end-to-end detection structure. First, the disadvantages of the detection structure based on convolutional neural networks, which requires setting the initial candidate frame and reprocessing the detection results, are eliminated. Then, the entire detection architecture performs precise rotation detection on the symptom area of ​​pest-infested crops to achieve precise positioning, which is more conducive to the judgment of the occurrence level. Finally, the high-quality query screening network uses angle similarity to solve the problem of repeated frames caused by field intersections and symptom discontinuity, thereby improving the detection effect.

[0075] (1) The first part of the symptom area detection model for pest-infested crops is set as the local feature modeling network, the second part is the global feature modeling network, the third part is the high-quality query screening network, the fourth part is the detail feature modeling network, and the fifth part is the classification regression network.

[0076] (2) Setting a local feature modeling network, which is used to locally model the features of the symptom area of ​​the crop infested by pests and diseases in the entire image, wherein the input of the local feature modeling network is an image of the symptom area of ​​the crop infested by pests and diseases in the field, and the output of the local feature modeling network is a multi-layer local feature map of the image of the symptom area of ​​the crop infested by pests and diseases in the field, wherein the local feature vector corresponding to each feature space position in the multi-layer local feature map can only be seen in a partial area range of the original image;

[0077] The local feature modeling network is set to be built based on the residual network. After the first convolution with a kernel size of 7*7 and a stride of 2, a pooling operation with a pooling range of 3*3 and stride=2 is used. It is composed of a repeated convolutional neural network layer, a batch normalization layer, and an activation function layer.

[0078] (3) Setting a global feature modeling network: The global feature modeling network obtains global modeling information of the symptom area image of the pest-infested crop based on the multi-layer local feature map. The input of the global feature modeling network is the multi-layer local feature map, and the output of the global feature modeling network is the multi-layer global feature map.

[0079] The global feature modeling network adopts the Transformer encoder structure. The global feature modeling network consists of 6 encoding modules with the same structure stacked in sequence.

[0080] Each encoding module consists of a multi-head deformable attention structure and a feed-forward network. The multi-head deformable attention structure has residual connections, and both the multi-head deformable attention structure and the feed-forward network are followed by Layer Norm operations.

[0081] The global feature modeling network first performs a global modeling operation on the multi-layer features obtained by the local feature modeling network, and uses the sine-cosine function combined with the specific spatial position of the feature vector to generate a spatial position code for the feature vector. At the same time, a learnable layer position code is set according to the position of the layer. The spatial position code and the layer position code are added to form a position code. The local feature vector and the position code are combined to obtain a local feature vector with a position code. All local feature vectors in the multi-layer local feature map with position code are input into the encoding module, and finally fused and mapped into a global feature vector to form a global feature map.

[0082] (4) Setting up a high-quality query screening network: The input of the high-quality query screening network is a multi-layer global feature map. The high-quality query screening network first removes the query vectors corresponding to the boxes with large overlap based on the maximum suppression method with a rotation intersection ratio of 0.8 for the multi-layer global feature map, and then removes the query vectors corresponding to the boxes with small overlap but small angle range difference through angle similarity. Finally, k1 unique query vectors are obtained. During the training process, the k2=1.5*k1 query vectors with the highest scores and k3 positive and negative sample annotated query vectors generated according to the annotation of its own data set are selected. The global feature vector composed of these query vectors is used to initialize the query vector of the detail feature modeling network to assist the training of the detail feature modeling network. During the testing process, the auxiliary training branch is no longer used.

[0083] (5) Setting a detail feature modeling network to perform detail feature modeling on the global feature vector. The detail feature modeling network includes six decoding modules. Each decoding module has the same structure. Each decoding module consists of a multi-head self-attention, a multi-head deformable attention, and a feed-forward network. Both the multi-head self-attention and the multi-head deformable attention have residual connections. All three parts are followed by a Layer Norm operation.

[0084] The input of the detail feature modeling network is k1 unique query vectors, k2=1.5*k1 high-scoring queries, and a set of k3 positive and negative sample labeled query vectors created based on the labeled instance labels. The number of output queries of the detail feature modeling network is k1+k2+k3, where the k2 high-scoring queries and k3 labeled queries only assist in the training of the network, and are no longer used during the testing process.

[0085] (6) A classification and regression network is set up to map the output query of the detail feature modeling network into the type and specific location coordinates of the symptom area of ​​the pest-infested crop. The classification and regression network includes a main network head and a secondary network head. The secondary network head only exists in the training process. Both the main network head and the secondary network head include two sub-networks, namely, classification and regression. The inputs of the classification sub-network and the regression sub-network are the output query of the detail feature modeling network. For each detail query, after a forward feedforward network, the predicted category and specific location coordinates of the target box to be detected are output.

[0086] The third step is to train the symptom area detection model for pest-infested crops: use the symptom area image dataset of pest-infested crops to train the symptom area detection model for pest-infested crops.

[0087] (1) Inputting the symptom area image dataset of pests and diseases into the local feature modeling network, the local feature modeling network uses the residual network to obtain the local feature map of the symptom area image of pests and diseases, and obtains feature maps with different receptive field sizes through different numbers of layers, including: C3 feature map with 512 dimensions and stride = 8, C4 feature map with 1024 dimensions and stride = 16, and C5 feature map with 2048 dimensions and stride = 32;

[0088] The C3, C4, and C5 feature maps are then subjected to convolution with a kernel size of 1*1 and a channel number of 256, and group normalization operations to achieve the purpose of consistent channel numbers, and local feature maps M1, M2, and M3 of w1*h1*256, w2*h2*256, and w3*h3*256 are obtained respectively. In addition, the C5 feature map is subjected to convolution with a kernel size of 3*3, stride=2, and a channel number of 256, and group normalization operations to obtain a local feature map M4 of w4*h4*256, which also keeps the number of channels consistent with M1, M2, and M3, and obtains M1, M2, M3, and M4 with consistent channel numbers but inconsistent sizes, for a total of 4 layers of local feature maps. The receptive field range of the input image represented by the feature vector at each feature space position is different.

[0089] (2) The local feature maps M1, M2, M3, and M4 are input into the global feature modeling network and enter the six encoding modules respectively.

[0090] A1) In the encoding module, the input of the multi-head deformable attention structure includes query, value and reference point position Ref; the query is obtained by adding the local image features and the position encoding features. The position encoding features are obtained by encoding the spatial position of each feature point through the sine-cosine function plus the learnable layer position encoding set by the position of the layer. The number of channels of the two is the same, and the position encoding covers the spatial position information of the feature point and the layer information where the feature point is located;

[0091] The query and value of the multi-head deformable attention structure are the same. The query of the multi-head deformable attention structure of the first encoding module is a vector set arranged according to the spatial position and the number of layers, totaling The number of reference points is Among them, the i-th layer reference point p q The position is p q =(p qx ,p qy ),in Normalize the reference point positions of different layers according to the effective positions Then map the effective positions of different layers to other layers to generate

[0092]

[0093] If only one layer is used, the specific formula of the multi-head deformable attention structure is expressed as:

[0094]

[0095] If multiple layers are used, the specific formula of the multi-head deformable attention structure is expressed as:

[0096]

[0097] in

[0098] z q represents the query feature vector, is the position coordinate of the reference point normalized according to the effective position, It is the position coordinates generated by mapping the effective positions of different layers to other layers, W m and W m 'All are fully connected operations from 256 dimensions to 256 dimensions, A mlqk Yes q The weight of the sampling point obtained by the full connection operation and the softmax operation, Δp mlqk Yes q The offset of the sampling point obtained by the full connection operation, is the feature vector of the offset point on the feature map of the lth layer corresponding to the reference point, M is the number of heads in the multi-head deformable attention structure, L is the number of layers of the feature map, K is the number of offset points selected in each layer, The query is obtained through the multi-head deformable attention structure, and then the final global feature vector is obtained through residual connection;

[0099] A2) The global feature vector is normalized by the Layer Norm layer and then input into the feedforward network;

[0100] A3) The feedforward network is mainly composed of two fully connected layers, and there is also dropout

[0101] After the first fully connected layer, the vector dimension increases from 256 to 1024, followed by the ReLu activation function layer and the dropout layer, and then after the second fully connected layer, the vector dimension decreases from 1024 back to 256, followed by the dropout layer operation;

[0102] A4) The vector obtained after passing through a Layer Norm layer is the output query of the first encoding module;

[0103] After 6 encoding modules with the same settings, the global feature vector output by the global feature modeling network is obtained. Finally, the output of the global feature modeling network is the same size as the input, totaling query vector.

[0104] (3) The high-quality query screening network will screen out all spatial feature vectors of different layers output by the global feature modeling network, and select those with high quality differences as the initial queries of the detail feature modeling network.

[0105] First, we remove query vectors corresponding to boxes with large overlaps based on the maximum suppression method with a rotational intersection-union ratio of 0.8 to obtain discriminative queries, where the rotational NMS threshold is 0.8. Then, we remove query vectors corresponding to boxes with small intersection-union ratios but small angle range differences based on angle similarity, and finally obtain k1 unique query vectors.

[0106] During the training process, the k2=1.5*k1 high-scoring query vectors with the highest scores and k3 positive and negative sample annotated query vectors generated according to the annotations of their own data set are selected to assist in the training of the detail feature modeling network. During the testing process, the auxiliary training branch is no longer used. Finally, the k1 unique query vectors, k2 high-scoring query vectors and k3 positive and negative sample annotated query vectors together constitute the query input Q of the detail feature modeling network. In addition, the k1 unique rotated reference frame, 1.5*k1 high-scoring rotated reference frame and k3 positive and negative sample rotated reference frames together constitute the rotated reference frame B of the detail feature modeling network.

[0107] (4) The input query of the detail feature modeling network is the k1 unique query vectors output by the high-quality query screening network, the k2=1.5*k1 query vectors with the highest scores, and the k3 positive and negative sample annotation queries created according to the annotated instance labels of the dataset. The six decoding modules are stacked in sequence. The output of the previous module is the query input of the multi-head self-attention in the next module. The query, key and value of the multi-head self-attention in each module are the same. The input query of the multi-head deformable attention of each module is the query output of the multi-head self-attention mechanism in the current module. The value of the multi-head deformable attention is the output of all layers of the global feature modeling network.

[0108] B1) Multi-head self-attention is set to focus on the relationship between targets in the symptom area of ​​pest-damaged crops. Through the self-attention structure, the global dependency between targets is captured, so that each target can understand the contextual information of all other targets in the image;

[0109] The input of the multi-head self-attention structure includes query, keyword and value. The keyword and value of the self-attention structure are generally the same as the query.

[0110] The input of the first decoding module is the sum of the query Q and the position code. The position code is obtained by encoding the rotated reference frame B through the sine-cosine function. The query Q is composed of k1 unique queries, k2 high-scoring queries and k3 positive and negative sample annotation queries. The rotated reference frame B is composed of the corresponding k1 unique rotated reference frames, 1.5*k1 high-scoring rotated reference frames and k3 positive and negative sample rotated reference frames.

[0111] The specific formulas of the self-attention structure and the multi-head self-attention structure are respectively expressed as:

[0112]

[0113] MSA(Q,K,V)=Concat(Attn({QW Q} m ,{KW K} m ,{VW V} m ))W out ,

[0114] m∈1,...,M

[0115] The keyword K and value V of the self-attention structure are the same as the query Q. Attention represents self-attention, and d k is the dimension of K, which is equivalent to normalizing the attention. MSA stands for multi-head self-attention, and W Q ,W K ,WV ,W out is a learnable parameter, m is the index of the multi-head, and M is the number of heads in the multi-head self-attention;

[0116] The output after the multi-head self-attention structure has a residual connection, and then passes through the Layer Norm layer to enter the multi-head deformable attention structure of the current module;

[0117] B2) The value in the multi-head deformable attention structure in the detail feature modeling network is the output of the global feature modeling network, totaling The query input in the structure is the output of the self-attention in the current decoding module, and the number during training is k1+k2+k3;

[0118] The output of the multi-head deformable attention structure has a residual connection, and then passes through the Layer Norm layer and enters the feedforward network structure of the current module;

[0119] B3) At this point, the first decoding module ends and is repeated six times to complete the training of the detail feature modeling network.

[0120] (5) The classification branch of the classification regression network uses a fully connected operation to use the output query vector of the detail feature modeling network to generate the specific category of the rotation detection box. The regression branch uses a fully connected operation to use the output query vector to generate the specific position of the rotation detection box. The loss of unique queries during training is k1 unique target queries. The 1vs1 HungarianAssigner label assignment method is used to calculate the classification loss, regression loss, and intersection-over-union loss. The classification loss uses FocalLoss, the regression loss uses L1Loss, and the intersection-over-union loss uses GiO LOSS.

[0121] The auxiliary loss consists of two parts. The first part is k2 high-score queries, which use the 1vs4 TopkHungarianAssigner label assignment method to calculate the classification loss and intersection-over-union loss. The classification loss uses QualityFocalLoss, and the intersection-over-union loss uses GioULoss. The second part is k3 positive and negative sample annotation queries, which use the 1vsG label assignment method to calculate the classification loss, regression loss, and intersection-over-union loss. The three losses are the same as those of unique queries. G is the number of groups. According to the number of labeled instances n, the number of groups is obtained according to G=ceil(100 / n). The sum of all losses finally obtained is used to optimize the model through the AdamW optimizer.

[0122] The fourth step is to obtain the symptom area image of the crop to be detected by pests and diseases: obtain the symptom area image of the crop to be detected by pests and diseases.

[0123] The fifth step is to obtain the detection results of the symptom areas of pest-infested crops: the images of the symptom areas of pest-infested fields to be detected are input into the trained symptom area detection model of pest-infested crops to obtain the classification and positioning results of the symptom areas of pest-infested fields.

[0124] (1) The symptom area image of the pest and disease to be detected is input into the local feature modeling network. The local feature modeling network uses the residual network to obtain the local feature map of the symptom area image of the pest and disease, and obtains the local feature maps M1, M2, M3, and M4 of w1*h1*256, w2*h2*256, w3*h3*256, and w4*h4*256 respectively.

[0125] (2) The local feature maps M1, M2, M3, and M4 are input into the global feature modeling network. After six encoding modules with the same settings, the global feature query vector output by the global feature modeling network is obtained. Finally, the output of the global feature modeling network is the same size as the input, totaling A multi-layer global feature map composed of query vectors.

[0126] (3) The high-quality query screening network will select all spatial feature vectors of different layers output by the global feature modeling network based on angular similarity through the maximum suppression method to select high-quality initial queries for the detail feature modeling network. First, the query vectors corresponding to the boxes with large overlap are removed based on the maximum suppression method with a rotation intersection-union ratio of 0.8 to obtain discriminative queries. Then, the query vectors corresponding to the boxes with small overlap but small angular range difference are removed through angular similarity. Finally, a unique rotated reference box and the corresponding query vector are obtained.

[0127] (4) The input query of the detail feature modeling network is the k1 unique query vectors output by the high-quality query screening network, where the input value of the multi-head deformable attention in the encoding module is the global feature modeling network. query vectors, and finally outputs k1 unique query vectors for detail feature modeling.

[0128] (5) Finally, k1 rotation detection frames are obtained through the classification and regression network. The classification branch uses a fully connected operation to use the query vector to generate the specific category of the rotation detection frame, and the regression branch uses a fully connected operation to use the query vector to generate the specific position of the rotation detection frame.

[0129] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for detecting symptom areas of pest and disease crop images based on a rotation transformer mechanism using angle similarity, characterized in that: The following steps are involved: 11) Obtaining symptom area images of crops damaged by pests and diseases: obtaining symptom area images of crops damaged by pests and diseases in the field to form a symptom area image dataset of crops damaged by pests and diseases; 12) Constructing a symptom area detection model for pest-infested crops: setting the symptom area detection model for pest-infested crops to include a local feature modeling network, a global feature modeling network, a high-quality query screening network, a detail feature modeling network, and a classification regression network; 13) Training of symptom area detection model for pest-damaged crops: using the symptom area image dataset of pest-damaged crops to train the symptom area detection model for pest-damaged crops; 14) Obtaining images of symptom areas of crops to be detected that are damaged by pests and diseases: obtaining images of symptom areas of crops to be detected that are damaged by pests and diseases in the field; 15) Obtaining the detection results of the symptom areas of pest-infested crops: Inputting the images of the symptom areas of pest-infested crops to be detected into the trained symptom area detection model of pest-infested crops, the classification and positioning results of the symptom areas of pest-infested crops are obtained.

2. The method for detecting symptom areas of pest and disease crop images based on the rotation transformer mechanism and utilizing angle similarity according to claim 1, characterized in that: The construction of the symptom area detection model for crops damaged by pests and diseases comprises the following steps: 21) The first part of the symptom area detection model for pest-damaged crops is set as a local feature modeling network, the second part is a global feature modeling network, the third part is a high-quality query screening network, the fourth part is a detail feature modeling network, and the fifth part is a classification regression network; 22) Setting a local feature modeling network, the local feature modeling network is used to locally model the features of the symptom area of ​​the pest-damaged crops in the whole image, the input of which is the symptom area image of the pest-damaged crops in the field, and the output is a multi-layer local feature map of the symptom area image of the pest-damaged crops in the field, and the local feature vector corresponding to each feature space position in the multi-layer local feature map can only be seen in a partial area range of the original image; The local feature modeling network is set to be built based on the residual network. After the first convolution with a kernel size of 7*7 and a stride of 2, a pooling operation with a pooling range of 3*3 and stride=2 is used, which is composed of a convolutional neural network layer, a batch normalization layer, and an activation function layer. 23) Setting a global feature modeling network: The global feature modeling network obtains global modeling information of the symptom area image of the pest-infested crop according to the multi-layer local feature map, the input of the global feature modeling network is the multi-layer local feature map, and the output of the global feature modeling network is the multi-layer global feature map; The global feature modeling network adopts the Transformer encoder structure. The global feature modeling network consists of 6 encoding modules with the same structure stacked in sequence. Each encoding module consists of a multi-head deformable attention structure and a feed-forward network. The multi-head deformable attention structure has residual connections, and both the multi-head deformable attention structure and the feed-forward network are followed by Layer Norm operations. The global feature modeling network first performs a global modeling operation on the multi-layer features obtained by the local feature modeling network, and uses the sine-cosine function combined with the specific spatial position of the feature vector to generate a spatial position code for the feature vector. At the same time, a learnable layer position code is set according to the position of the layer. The spatial position code and the layer position code are added to form a position code. The local feature vector and the position code are combined to obtain a local feature vector with a position code. All local feature vectors in the multi-layer local feature map with position code are input into the encoding module, and finally fused and mapped into a global feature vector to form a global feature map. 24) Setting a high-quality query screening network: The input of the high-quality query screening network is a multi-layer global feature map. The high-quality query screening network first removes the query vectors corresponding to a portion of the boxes with large overlap based on the maximum suppression method with a rotation intersection ratio of 0.8 for the multi-layer global feature map, and then removes the query vectors corresponding to the boxes with small overlap but small angle range difference through angle similarity, and finally obtains k1 unique query vectors. During the training process, the k2=1.5*k1 query vectors with the highest scores and the k3 positive and negative sample annotated query vectors generated according to the annotation of the own data set are selected. The global feature vector composed of these query vectors is used to initialize the query vector of the detail feature modeling network to assist the training of the detail feature modeling network. During the testing process, the auxiliary training branch is no longer used; 25) Setting a detail feature modeling network for modeling the detail features of the global feature vector. The detail feature modeling network includes 6 decoding modules. Each decoding module has the same structure. Each decoding module consists of a multi-head self-attention, a multi-head deformable attention and a forward feedforward network. Both the multi-head self-attention and the multi-head deformable attention have residual connections. All three parts are connected with a Layer Norm operation. The input of the detail feature modeling network is k1 unique query vectors, k2=1.5*k1 high-scoring queries, and a set of k3 positive and negative sample annotated query vectors created based on the annotated instance labels. The number of output queries of the detail feature modeling network is k1+k2+k3, where the k2 high-scoring queries and k3 annotated queries only assist the training of the network and are no longer used during the test process. 26) A classification regression network is set to map the output query of the detail feature modeling network to the type and specific location coordinates of the symptom area of ​​the pest-infested crop. The classification regression network includes a main network head and a secondary network head. The secondary network head only exists in the training process. The main network head and the secondary network head both include two sub-networks, namely, classification and regression. The inputs of the classification sub-network and the regression sub-network are both the output query of the detail feature modeling network. For each detail query, a forward feedforward network is passed to output the predicted category and specific location coordinates of the target box to be detected.

3. The method for detecting symptom areas of crop images of pests and diseases based on the rotation transformer mechanism using angle similarity according to claim 1, characterized in that: The training of the symptom area detection model for pest-damaged crops comprises the following steps: 31) Inputting the symptom area image dataset of pests and diseases into the local feature modeling network, the local feature modeling network uses the residual network to obtain the local feature map of the symptom area image of pests and diseases, and obtains feature maps with different receptive field sizes through different numbers of layers, including: C3 feature map with 512 dimensions and stride=8, C4 feature map with 1024 dimensions and stride=16, and C5 feature map with 2048 dimensions and stride=32; The C3, C4, and C5 feature maps are then subjected to convolution with a kernel size of 1*1 and a channel number of 256, and group normalization operations to achieve the purpose of consistent channel numbers, and local feature maps M1, M2, and M3 of w1*h1*256, w2*h2*256, and w3*h3*256 are obtained respectively. In addition, the C5 feature map is subjected to convolution with a kernel size of 3*3, stride=2, and a channel number of 256, and group normalization operations to obtain a local feature map M4 of w4*h4*256, which also keeps the number of channels consistent with M1, M2, and M3, and obtains M1, M2, and M3 with consistent channel numbers and inconsistent sizes. 、 M4, a total of 4 layers of local feature maps, the input image receptive field range represented by the feature vector at each feature space position is different; 32) Input the local feature maps M1, M2, M3, and M4 into the global feature modeling network and enter the 6 encoding modules respectively. 321) In the encoding module, the input of the multi-head deformable attention structure includes a query, a value, and a reference point position Ref; the query is obtained by adding the local image feature and the position encoding feature, the position encoding feature is obtained by encoding the spatial position of each feature point through the sine-cosine function plus the learnable layer position encoding set by the position of the layer, the number of channels of the two is the same, and the position encoding covers the spatial position information of the feature point and the layer information where the feature point is located; The query and value of the multi-head deformable attention structure are the same. The query of the multi-head deformable attention structure of the first encoding module is a vector set arranged according to the spatial position and the number of layers, totaling The number of reference points is Among them, the i-th layer reference point p q The position is p q =(p qx ,p qy ),in Normalize the reference point positions of different layers according to the effective positions Then map the effective positions of different layers to other layers to generate If only one layer is used, the specific formula of the multi-head deformable attention structure is expressed as: If multiple layers are used, the specific formula of the multi-head deformable attention structure is expressed as: in z q represents the query feature vector, is the position coordinate of the reference point normalized according to the effective position, It is the position coordinates generated by mapping the effective positions of different layers to other layers, W m and W m 'All are fully connected operations from 256 dimensions to 256 dimensions, A mlqk Yes q The weight of the sampling point obtained by the full connection operation and the softmax operation, Δp mlqk Yes q The offset of the sampling point obtained by the full connection operation, is the feature vector of the offset point on the feature map of the lth layer corresponding to the reference point, M is the number of heads in the multi-head deformable attention structure, L is the number of layers of the feature map, K is the number of offset points selected in each layer, The query is obtained through the multi-head deformable attention structure, and then the final global feature vector is obtained through residual connection; 322) The global feature vector is normalized by the Layer Norm layer and then input into the feedforward network; 323) The feedforward network is mainly composed of two fully connected layers, and there are also dropout layers and ReLu activation function layers. The vector dimension increases from 256 to 1024 after the first fully connected layer, followed by the ReLu activation function layer and the dropout layer, and then the vector dimension decreases from 1024 to 256 after the second fully connected layer, followed by the dropout layer operation; 324) Then the vector obtained after passing through a Layer Norm layer is the output query of the first encoding module; After 6 encoding modules with the same settings, the global feature vector output by the global feature modeling network is obtained. Finally, the output of the global feature modeling network is the same size as the input, totaling query vector; 33) The high-quality query screening network will screen out all spatial feature vectors of different layers output by the global feature modeling network, and select those with high quality differences as the initial queries of the detail feature modeling network. First, we remove query vectors corresponding to boxes with large overlaps based on the maximum suppression method with a rotational intersection-union ratio of 0.8 to obtain discriminative queries, where the rotational NMS threshold is 0.

8. Then, we remove query vectors corresponding to boxes with small intersection-union ratios but small angle range differences based on angle similarity, and finally obtain k1 unique query vectors. During the training process, the k2=1.5*k1 high-scoring query vectors with the highest scores and the k3 positive and negative sample annotated query vectors generated according to the annotation of the own data set are selected to assist the training of the detail feature modeling network. During the testing process, the auxiliary training branch is no longer used. Finally, the k1 unique query vectors, k2 high-scoring query vectors and k3 positive and negative sample annotated query vectors together constitute the query input Q of the detail feature modeling network. In addition, the k1 unique rotation reference frame, the 1.5*k1 high-scoring rotation reference frame and the k3 positive and negative sample rotation reference frames together constitute the rotation reference frame B of the detail feature modeling network. 34) The input query of the detail feature modeling network is the k1 unique query vectors output by the high-quality query screening network, the k2=1.5*k1 query vectors with the highest scores, and the k3 positive and negative sample annotation queries created according to the annotated instance labels of the data set. The 6 decoding modules are stacked in sequence, and the output of the previous module is the query input of the multi-head self-attention in the next module. The query, key and value of the multi-head self-attention in each module are the same. The input query of the multi-head deformable attention of each module is the query output of the multi-head self-attention mechanism in the current module. The value of the multi-head deformable attention is the output of all layers of the global feature modeling network; 341) The multi-head self-attention is set to focus on the relationship between the targets in the symptom area of ​​the pest-damaged crops. Through the self-attention structure, the global dependency between the targets is captured, so that each target can understand the contextual information of all other targets in the image; The input of the multi-head self-attention structure includes query, keyword and value. The keyword and value of the self-attention structure are generally the same as the query. The input of the first decoding module is the sum of the query Q and the position code. The position code is obtained by encoding the rotated reference frame B through the sine-cosine function. The query Q is composed of k1 unique queries, k2 high-scoring queries and k3 positive and negative sample annotation queries. The rotated reference frame B is composed of the corresponding k1 unique rotated reference frames, 1.5*k1 high-scoring rotated reference frames and k3 positive and negative sample rotated reference frames. The specific formulas of the self-attention structure and the multi-head self-attention structure are respectively expressed as: MSA(Q,K,V)=Concat(Attn({QW Q } m ,{KW K } m ,{VW V } m ))W out , m∈1,...,M The keyword K and value V of the self-attention structure are the same as the query Q. Attention represents self-attention, and d k is the dimension of K, which is equivalent to normalizing the attention. MSA stands for multi-head self-attention, and W Q ,W K ,W V ,W out is a learnable parameter, m is the index of the multi-head, and M is the number of heads in the multi-head self-attention; The output after the multi-head self-attention structure has a residual connection, and then passes through the Layer Norm layer to enter the multi-head deformable attention structure of the current module; 342) The value in the multi-head deformable attention structure in the detail feature modeling network is the output of the global feature modeling network, totaling The query input in the structure is the output of the self-attention in the current decoding module, and the number during training is k1+k2+k3; The output of the multi-head deformable attention structure has a residual connection, and then passes through the Layer Norm layer and enters the feedforward network structure of the current module; 343) At this point, the first decoding module ends, and the training of the detail feature modeling network is repeated six times; 35) The classification branch of the classification regression network uses a fully connected operation to use the output query vector of the detail feature modeling network to generate the specific category of the rotation detection box. The regression branch uses a fully connected operation to use the output query vector to generate the specific position of the rotation detection box. The loss of the unique query during the training process is k1 unique target queries. The 1vs1 HungarianAssigner label assignment method is used to calculate the classification loss, regression loss and intersection-over-union loss. The classification loss uses FocalLoss, the regression loss uses L1Loss loss, and the intersection-over-union loss uses GioULoss; The auxiliary loss consists of two parts. The first part is k2 high-score queries, which use the 1vs4 TopkHungarianAssigner label assignment method to calculate the classification loss and intersection-over-union loss. The classification loss uses QualityFocalLoss, and the intersection-over-union loss uses GioULoss. The second part is k3 positive and negative sample annotation queries, which use the 1vsG label assignment method to calculate the classification loss, regression loss, and intersection-over-union loss. The three losses are the same as those of unique queries. G is the number of groups. According to the number of labeled instances n, the number of groups is obtained according to G=ceil(100 / n). The sum of all losses finally obtained is used to optimize the model through the AdamW optimizer.

4. The method for detecting symptom areas of pest and disease crop images based on a rotation transformer mechanism using angle similarity according to claim 1, characterized in that: The method of obtaining the detection result of the image of the symptom area of ​​the crop damaged by pests and diseases comprises the following steps: 41) Inputting the symptom area image of the pest to be detected into the local feature modeling network, the local feature modeling network uses the residual network to obtain the local feature map of the symptom area image of the pest to be detected, and obtaining the local feature maps M1, M2, M3, and M4 of w1*h1*256, w2*h2*256, w3*h3*256, and w4*h4*256 respectively; 42) Input the local feature maps M1, M2, M3, and M4 into the global feature modeling network, and obtain the global feature query vector output by the global feature modeling network after passing through 6 encoding modules with the same settings. Finally, the output of the global feature modeling network is the same size as the input, totaling A multi-layer global feature map composed of query vectors; 43) The high-quality query screening network will screen out all spatial feature vectors of different layers output by the global feature modeling network based on angle similarity through the maximum suppression method to select high-quality initial queries with large differences as the detail feature modeling network. First, based on the maximum suppression method with a rotation intersection-union ratio of 0.8, a portion of query vectors corresponding to boxes with large overlaps are removed to obtain discriminative queries. Then, through angle similarity, query vectors corresponding to boxes with small overlaps but small angle range differences are removed, and finally a unique rotation reference box and the corresponding query vector are obtained. 44) The input query of the detail feature modeling network is the k1 unique query vectors output by the high-quality query screening network, where the input value of the multi-head deformable attention in the encoding module is the global feature modeling network query vectors, and finally output k1 unique query vectors for detail feature modeling; 45) Finally, k1 rotation detection frames are obtained through the classification regression network. The classification branch uses a fully connected operation to use the query vector to generate the specific category of the rotation detection frame, and the regression branch uses a fully connected operation to use the query vector to generate the specific position of the rotation detection frame.

Images