A High-Precision Rice Map Segmentation Method Based on Multi-Head Attention Mechanism and Temporal Fusion

Abstract

This invention relates to a high-precision rice paddy map segmentation method based on multi-head attention mechanism and temporal fusion, comprising: collecting image data of rice paddies from multiple periods; performing data preprocessing; constructing a high-precision agricultural map temporal segmentation model; training the high-precision agricultural map temporal segmentation model; and inputting the image to be detected and auxiliary period images of the same plot in the test set into the trained high-precision agricultural map temporal segmentation model to generate segmentation results of rice and background in the image to be detected. This invention can fuse features of the same farmland from multiple periods, overcoming the limitations of single-frame perception and improving the accuracy of rice paddy map segmentation algorithms. In the decoder part, by introducing a temporal fusion module, the specific features of rice can be preserved and fused with features of the current period. This fusion method helps optimize the semantic segmentation of the map in the current period and improves the recognition accuracy of rice paddy areas.

Description

Technical Field

[0001] This invention relates to the field of crop planting area extraction technology, and in particular to a multi-period high-precision rice map segmentation method based on multi-head attention mechanism and temporal fusion. Background Technology

[0002] Extracting crop planting area is a fundamental task in remote sensing yield estimation and a prerequisite for automated equipment trajectory planning. However, current technologies present several challenges: First, farm map creation typically relies on manual annotation, a time-consuming and inefficient process. To reduce manual workload, more intelligent methods for automated map creation are needed. Second, traditional image segmentation algorithms are usually based on expert systems, requiring extensive prior knowledge and manually designed rules. However, in complex agricultural map segmentation scenarios, these rules and prior knowledge often differ, leading to time-consuming, labor-intensive, and subjective processes. Therefore, it is necessary to find algorithms more suitable for agricultural map segmentation that can automatically learn and adapt to the characteristics of different farms.

[0003] Furthermore, to improve the generalization ability of deep learning models, a large and comprehensive dataset is required for training. However, crops have unique phenological characteristics, and crops at different growth stages exhibit significant differences. Relying solely on remote sensing imagery data from a single period is often insufficient for accurate crop identification; data from different periods is typically required as a training set. Simultaneously, current algorithms, based on single-period data, are susceptible to the influence of illumination and morphological changes during the growth stage. Therefore, how to utilize multi-period data and improve the robustness of algorithms is a pressing issue that needs to be addressed in the field of agricultural remote sensing segmentation. Summary of the Invention

[0004] To address the issues of low accuracy and efficiency in extracting crop planting areas, this invention aims to provide a multi-period high-precision rice map segmentation method based on multi-head attention mechanism and temporal fusion, which improves the accuracy of rice map segmentation algorithms, saves GPU memory, and enhances the accuracy of rice region identification.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a multi-period high-precision rice map segmentation method based on multi-head attention mechanism and temporal fusion, the method comprising the following sequential steps:

[0006] (1) Data collection: Collect images of paddy fields from multiple periods;

[0007] (2) Data preprocessing: Based on the latitude and longitude of the collected rice paddy images, the collected rice paddy images are segmented and labeled in sequence to obtain the rice time series dataset. The rice time series dataset is divided into training set and test set.

[0008] (3) Constructing a high-precision agricultural map temporal segmentation model: A high-precision agricultural map temporal segmentation model is constructed using an encoder and a decoder;

[0009] (4) Training the high-precision agricultural map time-series segmentation model: simultaneously input images of the same plot at different times from the same period in the training set into the high-precision agricultural map time-series segmentation model for training;

[0010] (5) Rice and background region segmentation: The images to be detected in the test set and the auxiliary images of the same plots are input into the trained high-precision agricultural map time-series segmentation model to generate the segmentation results of rice and background in the images to be detected.

[0011] The step (1) specifically refers to: using a drone to collect images, with the drone flying at an altitude between 100 and 150 meters; the multiple periods include the tillering stage, jointing stage, booting stage, and grain-filling stage of rice.

[0012] Step (2) specifically includes the following steps:

[0013] (2a) Use ArcGIS software to align the latitude and longitude of the same area images of different resolutions collected by the UAV, and then crop them to obtain the required rice area;

[0014] (2b) Use ArcGIS software to annotate the images collected by the UAV and divide the targets in the images into two categories: agriculture and background;

[0015] (2c) Use latitude and longitude to set up a grid, create a grid on the images and corresponding annotation data collected by the UAV to accurately divide each grid area, and crop it to obtain images and annotation data of the same area at different times;

[0016] (2d) A rice time series dataset was created using plots divided by latitude and longitude as indexes. Each plot contains images of four stages: tillering stage, jointing stage, booting stage, and grain filling stage.

[0017] (2e) The rice time series dataset is divided into training set and test set in a ratio of 4:1.

[0018] Step (3) specifically includes the following steps:

[0019] (3a) The encoder consists of a patch encoder and four Transformer coding layers. The specific steps of the encoder to extract multi-scale features are as follows:

[0020] (3a1) Randomly select the image to be detected I1 in the training set, and at the same time, obtain the time series image set of the plots in the rice time series dataset using the plots in the image to be detected I1 as the index. Randomly select the auxiliary period image I2 from the time series image set of the plots. The image to be detected I1 and the auxiliary period image I2 are scaled to the same scale after data augmentation. The image to be detected I1 and the auxiliary period image I2 are sequentially processed by the encoder to extract multi-scale features. The auxiliary period image I2 does not go through backpropagation.

[0021] (3a2) The auxiliary period image I2 is processed by the patch encoder. The auxiliary period image I2 is first divided into multiple uniformly sized patch regions. Then, the patch encoder uses a 7×7 convolution kernel to perform a convolution operation on the auxiliary period image I2, where 7 is the size of the patch region. The auxiliary period image I2 is converted into a 32×32 image representation. Then, the 32×32 image is normalized to obtain the patch feature P2.

[0022] (3a3) The Transformer encoding layer consists of a high-efficiency multi-head attention module, a hybrid feedforward network module, and a patch fusion module. The Transformer encoding layer is used to extract multi-scale features. The processing steps of each Transformer encoding layer are as follows:

[0023] First, the patch feature P2 is normalized, and then attention is calculated using a high-efficiency multi-head attention module:

[0024]

[0025] Where Q is the query, K is the key, V is the key value, d is the number of heads in the multi-head attention mechanism, T is the matrix transpose, and Softmax is the activation function;

[0026] The high-efficiency multi-head attention module consists of a multi-head attention layer and a linear layer for computing the attention representation of the input and projecting the result onto a specific dimension through a non-linear mapping. Before computing the attention representation, the linear layer is used to reduce the dimensionality of the input, and then the attention representation is normalized again, as shown in the following formula:

[0027]

[0028]

[0029] Where (N, C) is the dimension of K, N is the original sequence length of K, and C is the number of channel layers of K; It is the key after the change of dimensions. The dimension is (N / R, C) R), where R is the scaling factor; It is to convert the dimension of K to Dimensions It is Number of channels C R is remapped to C;

[0030] Finally, feedforward neural network processing using a hybrid feedforward network module generates the top-down view feature B2. 4 B2 4 The subscript 2 is the image number, B2 4 The superscript 4 is the Transformer coding layer number;

[0031] (3a4) The four Transformer encoding layers are the first Transformer encoding layer, the second Transformer encoding layer, the third Transformer encoding layer, and the fourth Transformer encoding layer. Steps (3a2) and (3a3) are repeated for each Transformer encoding layer, and this is repeated four times. Each time, the number of channels representing the feature is doubled, and the size of the image is halved.

[0032] The top-down view feature B2 of the first Transformer coding layer output. 1 B2 1 It is a 32-channel feature representation with a size of 128×128; B2 1 The subscript 2 is the image number, and the superscript 1 is the Transformer encoding layer number;

[0033] The second Transformer coding layer outputs an overhead view feature B2. 2 B2 2 It is a feature representation with 64 channels and a size of 64×64;

[0034] The top-down view feature B2 of the third Transformer coding layer output. 3 B2 3 It is a feature representation with 160 channels and a size of 32×32;

[0035] The fourth Transformer encoding layer output top-down view feature B2 4 B2 4 It is a feature representation with 256 channels and a size of 16×16;

[0036] The above describes the specific steps of the encoder on the auxiliary image I2, which outputs four feature maps of different resolutions, temporarily stored in the first multi-scale feature map sequence {B2}. 1 B2 2 B2 3 B2 4Subsequently, the image to be detected, I1, undergoes the same steps described above to generate a second multi-scale feature map sequence, {B1}. 1 B1 2 B1 3 B1 4};

[0037] (3b) The temporal fusion module of the decoder will combine the first multi-scale feature map sequence {B2} 1 B2 2 B2 3 B2 4} and the second multi-scale feature map sequence {B1 1 B1 2 B1 3 B1 4} Fusion, to obtain the multi-scale fused feature map sequence {B1} of the image to be detected I1. 1’ B1 2’ B1 3’ B1 4’};

[0038] First multi-scale feature map sequence {B2} 1 B2 2 B2 3 B2 4}、Second multi-scale feature map sequence {B1 1 B1 2 B1 3 B1 4 The eight feature maps in the image are divided into four groups based on the image resolution, B1 1 B2 1 B1 2 B2 2 B1 3 B2 3 B1 4 B2 4 The top-down view features of the image to be detected at different scales (I1) are fused with the top-down view features of the corresponding scale of the auxiliary image (I2) to update the feature map, where B1... 1 It is the top-down view feature of the first Transformer encoding layer of the image to be detected (I1), B1 2 It is the top-down view feature of the second Transformer encoding layer of the image to be detected (I1), B1. 3 It is the top-down view feature of the third Transformer encoding layer of the image to be detected (I1), B1 4 It is an overhead view feature of the fourth Transformer coding layer of the image to be detected (I1).

[0039] The temporal fusion module is used to fuse data from multiple periods. It consists of two compression and extraction modules. The compression and extraction modules dynamically fuse data from multiple periods at different scales. The compression and extraction modules consist of an upsampling module, global pooling, a fully connected layer, a sigmoid activation function, and a fusion convolution.

[0040] (3b2) The top-down view features of the fourth Transformer encoding layer of the image to be detected, I1, are B1. 4 Feature map B1 is obtained by upsampling to 1 / 4 the resolution of the original image. 41 Feature map B1 41 Features are aggregated through global pooling, and then the feature dimensions are remapped through two fully connected layers. Finally, the feature map B1 is obtained through the Sigmoid activation function. 42 B1 41 and B1 42 Multiply to regenerate feature map B1 41 ;

[0041] (3b3) Auxiliary period image I2 Top view feature of the fourth Transformer coding layer B2 4 After the same operation in step (3b2), feature map B2 is obtained. 41 ;

[0042] (3b4) Feature map B1 41 and B2 41 The data from multiple periods are stitched together and fused through a convolutional layer to generate a new top-down view feature B1 of the image to be detected, I1. 4’ ;

[0043] (3b5)B1 1 B2 1 B1 2 B2 2 B1 3 B2 3 B1 4 B2 4 The four sets of time-series features will be processed through steps (3b2), (3b3), and (3b4) respectively, and then fused with the overhead view feature B2 from the auxiliary period image I2. 1 B2 2 B2 3 B2 4 Top-view feature B1 of image I1 to be detected 1 B1 2 B1 3 B1 4 The image I1 is updated using an upsampling algorithm, and its B1 value is adjusted accordingly. 1’ B1 2’ B1 3’B1 4’ The resolution is 1 / 4 of the original image;

[0044] (3b6) The top-down view feature B1 generated in step (3b5) 1’ B1 2’ B1 3’ B1 4’ The data is concatenated and then passed through a convolutional layer with 2 output channels, a kernel size of (1, 1), and a stride of (1, 1). The output feature map is then activated by a Sigmoid function to obtain the final predicted probability map, which is used for semantic segmentation tasks.

[0045] Through the above steps, the high-precision agricultural map time-series segmentation model is completed.

[0046] Step (5) specifically refers to: comparing the segmentation result with the labeled true value, and using the average intersection-union ratio and average pixel precision to evaluate the segmentation result.

[0047] As can be seen from the above technical solution, the beneficial effects of the present invention are as follows: First, by introducing a Transformer coding layer, the present invention can fuse the features of the same farmland in multiple periods. Compared with the traditional single-frame perception method, this multi-period fusion can make up for the limitations of single-frame perception and improve the accuracy of rice map segmentation algorithm. Second, in the present invention, the data of the auxiliary period is only propagated forward without additional storage, which greatly saves the use of video memory, making the algorithm more efficient and scalable, while maintaining accuracy. In the decoder part, by introducing a temporal fusion module, the specific features of rice can be preserved and fused with the features of the current period. This fusion method helps to optimize the semantic segmentation of the map in the current period and improve the recognition accuracy of rice areas. Attached Figure Description

[0048] Figure 1 This is a flowchart of the method of the present invention;

[0049] Figure 2 This is a schematic diagram of the overall structure of a high-precision agricultural map time-series segmentation model;

[0050] Figure 3 This is a schematic diagram of the encoder for a high-precision agricultural map temporal segmentation model;

[0051] Figure 4 This is a schematic diagram of the decoder for a high-precision agricultural map temporal segmentation model;

[0052] Figure 5 This is a diagram illustrating the segmentation effect of the present invention. Detailed Implementation

[0053] like Figure 1As shown, a multi-period high-precision rice map segmentation method based on multi-head attention mechanism and temporal fusion is characterized by the following sequential steps:

[0054] (1) Data collection: Collect images of paddy fields from multiple periods;

[0055] (2) Data preprocessing: Based on the latitude and longitude of the collected rice paddy images, the collected rice paddy images are segmented and labeled in sequence to obtain the rice time series dataset. The rice time series dataset is divided into training set and test set.

[0056] (3) Constructing a high-precision agricultural map temporal segmentation model: A high-precision agricultural map temporal segmentation model is constructed using an encoder and a decoder;

[0057] (4) Training the high-precision agricultural map time-series segmentation model: simultaneously input images of the same plot at different times from the same period in the training set into the high-precision agricultural map time-series segmentation model for training;

[0058] (5) Rice and background region segmentation: The images to be detected in the test set and the auxiliary images of the same plots are input into the trained high-precision agricultural map time-series segmentation model to generate the segmentation results of rice and background in the images to be detected.

[0059] Step (1) specifically refers to: using a drone for image acquisition, with the drone flying at an altitude between 100 and 150 meters; the multiple periods include the tillering stage, jointing stage, booting stage, and grain-filling stage of rice. Using drones to collect image data of rice paddies, covering observations at different time periods, captures the changing process of rice growth. Drones provide high-resolution and wide-coverage image data, which helps improve the accuracy of map segmentation.

[0060] Step (2) specifically includes the following steps:

[0061] (2a) Use ArcGIS software to align the latitude and longitude of the same area images of different resolutions collected by the UAV, and then crop them to obtain the required rice area;

[0062] (2b) Use ArcGIS software to annotate the images collected by the UAV and divide the targets in the images into two categories: agriculture and background;

[0063] (2c) Use latitude and longitude to set up a grid, create a grid on the images and corresponding annotation data collected by the UAV to accurately divide each grid area, and crop it to obtain images and annotation data of the same area at different times;

[0064] (2d) A rice time series dataset was created using plots divided by latitude and longitude as indexes. Each plot contains images of four stages: tillering stage, jointing stage, booting stage, and grain filling stage.

[0065] (2e) The rice time series dataset is divided into training set and test set in a ratio of 4:1.

[0066] like Figure 2 As shown, step (3) specifically includes the following steps:

[0067] like Figure 3 As shown, (3a) the encoder consists of a patch encoder and four Transformer encoding layers. The specific steps of the encoder to extract multi-scale features are as follows:

[0068] (3a1) Randomly select the image to be detected I1 in the training set, and at the same time, obtain the time series image set of the plots in the rice time series dataset using the plots in the image to be detected I1 as the index. Randomly select the auxiliary period image I2 from the time series image set of the plots. The image to be detected I1 and the auxiliary period image I2 are scaled to the same scale after data augmentation. The image to be detected I1 and the auxiliary period image I2 are sequentially processed by the encoder to extract multi-scale features. The auxiliary period image I2 does not go through backpropagation.

[0069] (3a2) The auxiliary period image I2 is processed by the patch encoder. The auxiliary period image I2 is first divided into multiple uniformly sized patch regions. Then, the patch encoder uses a 7×7 convolution kernel to perform a convolution operation on the auxiliary period image I2, where 7 is the size of the patch region. The auxiliary period image I2 is converted into a 32×32 image representation. Then, the 32×32 image is normalized to obtain the patch feature P2.

[0070] (3a3) The Transformer encoding layer consists of a high-efficiency multi-head attention module, a hybrid feedforward network module, and a patch fusion module. The Transformer encoding layer is used to extract multi-scale features. The processing steps of each Transformer encoding layer are as follows:

[0071] First, the patch feature P2 is normalized, and then attention is calculated using a high-efficiency multi-head attention module:

[0072]

[0073] Where Q is the query, K is the key, V is the key value, d is the number of heads in the multi-head attention mechanism, T is the matrix transpose, and Softmax is the activation function;

[0074] The high-efficiency multi-head attention module consists of a multi-head attention layer and a linear layer for computing the attention representation of the input and projecting the result onto a specific dimension through a non-linear mapping. Before computing the attention representation, the linear layer is used to reduce the dimensionality of the input, and then the attention representation is normalized again, as shown in the following formula:

[0075]

[0076]

[0077] Where (N, C) is the dimension of K, N is the original sequence length of K, and C is the number of channel layers of K; It is the key after the change of dimensions. The dimension is (N / R, C) R), where R is the scaling factor; It is to convert the dimension of K to Dimensions It is Number of channels C R is remapped to C;

[0078] Finally, feedforward neural network processing using a hybrid feedforward network module generates the top-down view feature B2. 4 B2 4 The subscript 2 is the image number, B2 4 The superscript 4 is the Transformer coding layer number;

[0079] (3a4) The four Transformer encoding layers are the first Transformer encoding layer, the second Transformer encoding layer, the third Transformer encoding layer, and the fourth Transformer encoding layer. Steps (3a2) and (3a3) are repeated for each Transformer encoding layer, and this is repeated four times. Each time, the number of channels representing the feature is doubled, and the size of the image is halved.

[0080] The top-down view feature B2 of the first Transformer coding layer output. 1 B2 1 It is a 32-channel feature representation with a size of 128×128; B2 1 The subscript 2 is the image number, and the superscript 1 is the Transformer encoding layer number;

[0081] The second Transformer coding layer outputs an overhead view feature B2. 2 B2 2 It is a feature representation with 64 channels and a size of 64×64;

[0082] The top-down view feature B2 of the third Transformer coding layer output. 3 B2 3 It is a feature representation with 160 channels and a size of 32×32;

[0083] The fourth Transformer encoding layer output top-down view feature B2 4 B2 4 It is a feature representation with 256 channels and a size of 16×16;

[0084] The above describes the specific steps of the encoder on the auxiliary image I2, which outputs four feature maps of different resolutions, temporarily stored in the first multi-scale feature map sequence {B2}. 1 B2 2 B2 3 B2 4 Subsequently, the image to be detected, I1, undergoes the same steps described above to generate a second multi-scale feature map sequence, {B1}. 1 B1 2 B1 3 B1 4};

[0085] like Figure 4 As shown in Figure (3b), the temporal fusion module of the decoder will combine the first multi-scale feature map sequence {B2} 1 B2 2 B2 3 B2 4} and the second multi-scale feature map sequence {B1 1 B1 2 B1 3 B1 4} Fusion, to obtain the multi-scale fused feature map sequence {B1} of the image to be detected I1. 1’ B1 2’ B1 3’ B1 4’};

[0086] First multi-scale feature map sequence {B2} 1 B2 2 B2 3 B2 4}、Second multi-scale feature map sequence {B1 1 B1 2 B1 3 B1 4 The eight feature maps in the image are divided into four groups based on the image resolution, B1 1 B2 1 B1 2 B2 2 B1 3 B23 B1 4 B2 4 The top-down view features of the image to be detected at different scales (I1) are fused with the top-down view features of the corresponding scale of the auxiliary image (I2) to update the feature map, where B1... 1 It is the top-down view feature of the first Transformer encoding layer of the image to be detected (I1), B1 2 It is the top-down view feature of the second Transformer encoding layer of the image to be detected (I1), B1. 3 It is the top-down view feature of the third Transformer encoding layer of the image to be detected (I1), B1 4 It is an overhead view feature of the fourth Transformer coding layer of the image to be detected (I1).

[0087] The temporal fusion module is used to fuse data from multiple periods. It consists of two compression and extraction modules. The compression and extraction modules dynamically fuse data from multiple periods at different scales. The compression and extraction modules consist of an upsampling module, global pooling, a fully connected layer, a sigmoid activation function, and a fusion convolution.

[0088] (3b2) The top-down view features of the fourth Transformer encoding layer of the image to be detected, I1, are B1. 4 Feature map B1 is obtained by upsampling to 1 / 4 the resolution of the original image. 41 Feature map B1 41 Features are aggregated through global pooling, and then the feature dimensions are remapped through two fully connected layers. Finally, the feature map B1 is obtained through the Sigmoid activation function. 42 B1 41 and B1 42 Multiply to regenerate feature map B1 41 ;

[0089] (3b3) Auxiliary period image I2 Top view feature of the fourth Transformer coding layer B2 4 After the same operation in step (3b2), feature map B2 is obtained. 41 ;

[0090] (3b4) Feature map B1 41 and B2 41 The data from multiple periods are stitched together and fused through a convolutional layer to generate a new top-down view feature B1 of the image to be detected, I1. 4’ ;

[0091] (3b5)B1 1 B2 1 B1 2 B2 2 B13 B2 3 B1 4 B2 4 The four sets of time-series features will be processed through steps (3b2), (3b3), and (3b4) respectively, and then fused with the overhead view feature B2 from the auxiliary period image I2. 1 B2 2 B2 3 B2 4 Top-view feature B1 of image I1 to be detected 1 B1 2 B1 3 B1 4 The image I1 is updated using an upsampling algorithm, and its B1 value is adjusted accordingly. 1’ B1 2’ B1 3’ B1 4’ The resolution is 1 / 4 of the original image;

[0092] (3b6) The top-down view feature B1 generated in step (3b5) 1’ B1 2’ B1 3’ B1 4’ The data is concatenated and then passed through a convolutional layer with 2 output channels, a kernel size of (1, 1), and a stride of (1, 1). The output feature map is then activated by a Sigmoid function to obtain the final predicted probability map, which is used for semantic segmentation tasks.

[0093] Through the above steps, the high-precision agricultural map time-series segmentation model is completed.

[0094] Transformer can capture long-range dependencies and effectively model temporal features, thereby improving the accuracy of rice map segmentation.

[0095] Step (5) specifically refers to comparing the segmentation result with the labeled ground truth, and using the average intersection-union ratio and average pixel precision to evaluate the segmentation result, as shown in Table 1.

[0096]

[0097] like Figure 5 As shown, Segformer is the currently common method, while Atfsformer is the method proposed in this invention. Both methods are used to detect data during the jointing stage, and the tillering stage is used to help Atfsformer segment the jointing stage data. It can be seen that Atfsformer performs better than Segformer.

[0098] In summary, this invention, by introducing a Transformer coding layer, can fuse features of the same farmland across multiple periods. Compared to traditional single-frame perception methods, this multi-period fusion overcomes the limitations of single-frame perception and improves the accuracy of rice map segmentation algorithms. In this invention, data from auxiliary periods is propagated forward without additional storage, significantly saving GPU memory usage and making the algorithm more efficient and scalable while maintaining accuracy. In the decoder section, by introducing a temporal fusion module, the specific features of rice can be preserved and fused with features from the current period. This fusion method helps optimize the semantic segmentation of the map in the current period and improves the recognition accuracy of rice areas.

Claims

1. A multi-head attention mechanism and time fusion-based multi-period high-precision rice map segmentation method, characterized by: The method comprises the following steps in sequence: (1) Collecting data: collecting picture data of rice fields in multiple periods; (2) Preprocessing data: according to the longitude and latitude of the collected picture data of the rice field, the collected picture data of the rice field is sequentially segmented and labeled to obtain a rice time series data set, and the rice time series data set is divided into a training set and a test set; (3) Constructing a high-precision agricultural map time series segmentation model: using an encoder and a decoder to jointly construct a high-precision agricultural map time series segmentation model; Specifically comprising the following steps: (3a) The encoder is composed of a patch encoder and four Transformer encoding layers, and the specific steps of the encoder extracting multi-scale features are as follows: (3a1) Randomly selecting a to-be-detected picture I1 in the training set, and obtaining a time series picture set of the plot from the rice time series data set with the plot in the to-be-detected picture I1 as an index, and randomly selecting an auxiliary period picture I2 from the time series picture set in the plot; the to-be-detected picture I1 and the auxiliary period picture I2 are scaled to the same size after data enhancement, and the to-be-detected picture I1 and the auxiliary period picture I2 are sequentially extracted multi-scale features by the encoder, and the auxiliary period picture I2 does not undergo back propagation; (3a2) The auxiliary period picture I2 is processed by the patch encoder, the auxiliary period picture I2 is first divided into multiple patch regions of uniform size, then the patch encoder uses a convolution kernel with a size of 7x7 to perform convolution operation on the auxiliary period picture I2, converts the auxiliary period picture I2 into a picture representation with a size of 32x32, and then normalizes the 32x32 picture to obtain patch features P2; (3a3) the four Transformer encoding layers are respectively a first Transformer encoding layer, a second Transformer encoding layer, a third Transformer encoding layer and a fourth Transformer encoding layer, each Transformer encoding layer repeatedly executes step (3a2) and step (3a3), and the repetition is 4 times, the channel number of the feature representation is doubled each time, the size of the picture is halved each time, and the feature maps of the auxiliary period picture I2 and the to-be-detected picture I1 at 14 different resolutions are respectively output as a first multi-scale feature map sequence {B2 1 , B2 2 , B2 3 , B2 4} and a second multi-scale feature map sequence {B1 1 , B1 2 , B1 3 , B1 4}; (3b) the timing fusion module of the decoder fuses the first multi-scale feature map sequence {B2 1 , B2 2 , B2 3 , B2 4} and the second multi-scale feature map sequence {B1 1 , B1 2 , B1 3 , B1 4} to obtain a multi-scale fused feature map sequence {B1 1’ , B1 2’ , B1 3’ , B1 4’} of the picture I1 to be detected, and the construction of the high-precision agricultural map timing segmentation model is completed. (4) Training the high-precision agricultural map time series segmentation model: inputting the pictures of different periods of the same plot in the training set into the high-precision agricultural map time series segmentation model for training; (5) Rice and background region segmentation: inputting the to-be-detected picture and the auxiliary period picture of the same plot in the test set into the trained high-precision agricultural map time series segmentation model to generate the segmentation result of the rice and the background of the to-be-detected picture.

2. The multi-head attention mechanism and temporal fusion-based multi-period high-precision rice map segmentation method according to claim 1, characterized in that: The step (1) specifically refers to: using a UAV to collect images, and the flight height of the UAV is between 100 and 150 meters; the multiple periods include the tillering period, the jointing period, the booting period and the filling period of rice.

3. The multi-head attention mechanism and temporal fusion-based multi-period high-precision rice map segmentation method according to claim 1, characterized in that: The step (2) specifically comprises the following steps: (2a) using ArcGIS software to align the pictures of the same area collected by the UAV in longitude and latitude, and then cutting to obtain the required rice area; (2b) using ArcGIS software to label the pictures collected by the UAV, and dividing the targets in the pictures into two categories, namely agriculture and background; (2c) using longitude and latitude to set a grid, creating a grid to accurately divide each grid area on the pictures collected by the UAV and the corresponding labeled data, and cutting to obtain pictures and labeled data of the same area in different periods; (2d) establishing a rice time series data set with the plot divided by longitude and latitude as an index, and one plot contains pictures of four periods: tillering period, jointing period, booting period and filling period. (2e) The rice time-series dataset is divided into training set and test set in the ratio of 4:

1.

4. The multi-head attention mechanism and temporal fusion based multi-period high-precision rice map segmentation method according to claim 1, characterized in that: The step (5) is specifically to compare the segmentation result with the labeled true value, and use the average intersection over union and the average pixel accuracy to evaluate the segmentation result.

5. The multi-head attention mechanism and temporal fusion based multi-period high-precision rice map segmentation method according to claim 1, characterized in that: In (3a), the Transformer encoding layer is composed of an efficient multi-head attention module, a hybrid feedforward network module and a patch fusion module, and is used to extract multi-scale features. The processing steps of each Transformer encoding layer are as follows: First, the patch feature P2 is normalized, and then the efficient multi-head attention module is used for attention calculation: Where Q is the query, K is the key, V is the key value, d is the number of multi-head attention mechanism heads, T is the matrix transpose, and Softmax is the activation function. The efficient multi-head attention module includes a multi-head attention layer and a linear layer, which is used to calculate the attention representation of the input and project the result to a specific dimension through nonlinear mapping; before calculating the attention representation, the input is reduced in dimension using the linear layer, and then the attention representation is normalized again. The specific formula is as follows: Wherein, (N, C) is the dimension of K, N is the original sequence length of K, and C is the number of channel layers of K; is the key after dimension change, the dimension of K is (N / R, C R), and R is a scaling factor; is the dimension of K converted to the dimension of K, is the dimension of K converted to the channel number C R of K is remapped to C; Finally, the hybrid feedforward network module is used for feedforward neural network processing to generate the overhead map feature.

6. The multi-head attention mechanism and temporal fusion based multi-period high-precision rice map segmentation method according to claim 1, characterized in that: The four feature maps with different resolutions in (3a3) are: The first Transformer encoding layer outputs the bird's eye view feature B2 1 , B2 1 is a 32-channel feature representation of size 128x128; B2 1 The subscript 2 is the picture number, and the superscript 1 is the Transformer encoding layer number; The second Transformer encoding layer outputs the overhead map features B2 2 , B2 2 is a 64-channel feature representation of size 64 x 64; The third Transformer encoding layer outputs the bird's eye view features B2 3 , B2 3 is a 160-channel feature representation of size 32x32; The fourth Transformer encoding layer outputs the bird's eye view feature B2 4 , B2 4 is a 256-channel feature representation of size 16x16; The above are the specific steps of the auxiliary period picture I2 passing through the encoder, which outputs four feature maps of different resolutions, temporarily saved into the first multi-scale feature map sequence {B2 1 , B2 2 , B2 3 , B2 4}. Then, the to-be-detected picture I1 also passes through the above steps to generate the second multi-scale feature map sequence {B1 1 , B1 2 , B1 3 , B1 4}.

7. The multi-head attention mechanism and temporal fusion based multi-period high-precision rice map segmentation method according to claim 1, characterized in that: In (3b), the construction of the high-precision agricultural map time-series segmentation model adopts the following steps: (3b1) the first multi-scale feature map sequence {B2 1 , B2 2 , B2 3 , B2 4}, the second multi-scale feature map sequence {B1 1 , B1 2 , B1 3 , B1 4} are divided into 4 groups on the basis of image resolution, (B1 1 , B2 1 ); (B1 2 , B2 2 ); (B1 3 , B2 3 ); (B1 4 , B2 4 ); the aerial view features of the to-be-detected picture I1 on different scales are fused with the aerial view features of the auxiliary picture I2 corresponding to the scale to update the feature maps, wherein B1 1 is the aerial view feature of the first layer of the Transformer encoding layer of the to-be-detected picture I1, B1 2 is the aerial view feature of the second layer of the Transformer encoding layer of the to-be-detected picture I1, B1 3 is the aerial view feature of the third layer of the Transformer encoding layer of the to-be-detected picture I1, and B1 4 is the aerial view feature of the fourth layer of the Transformer encoding layer of the to-be-detected picture I1. The time-series fusion module is used to fuse multi-period data, which is composed of two compression extraction modules. The multi-period data is dynamically fused at different scale levels through the compression extraction module. The compression extraction module is composed of an up-sampling module, a global pooling, a fully connected layer, a Sigmoid activation function and a fusion convolution. (3b2) the bird's eye view feature B1 of the fourth Transform coding layer of the picture to be detected I1 4 The feature map B1 is obtained by upsampling operation to the original picture size 1 / 4 resolution 41 ; the feature map B1 41 The features are aggregated by global pooling, and the feature dimension is remapped by two consecutive fully connected layers, and finally the feature map B1 is obtained by Sigmoid activation function 42 ; B1 41 and B1 42 are multiplied to regenerate the feature map B1 41 ; (3b3) A bird's eye view feature B2 of the fourth Transformer encoding layer of the auxiliary period picture I2 4 The same operation as step (3b2) is performed to obtain the feature map B2 41 ; (3b4) the feature map B1 41 and B2 41 are spliced together to generate a new bird's eye view feature B1 4’ of the detection picture I1 through a convolutional layer (3b5) 4 groups of timing features will be respectively operated in steps (3b2), (3b3), (3b4), and the auxiliary period picture I2 overhead view feature B2 is fused 1 , B2 2 , B2 3 , B2 4 The overhead view feature B1 of the to-be-detected picture I1 is updated 1 , B1 2 , B1 3 , B1 4 After the up-sampling algorithm, the resolution of B1 of the to-be-detected picture I1 is 1 / 4 of the original picture 1’ , B1 2’ , B1 3’ , B1 4’ ​ (3b6) the generated overhead view feature B1 of step (3b5) is spliced 1’ , B1 2’ , B1 3’ , B1 4’ is spliced, and finally a convolution layer is performed, the number of output channels is 2, and the output feature map is subjected to a Sigmoid activation function to obtain a final prediction probability map for a semantic segmentation task.

Images