A semantic segmentation method and system for medium and low resolution time-series satellite images

Abstract

This application provides a semantic segmentation method and system for medium- and low-resolution time-series satellite imagery. The method includes determining the target time series of each pixel in a target area; and extracting temporal features based on the target time series of each pixel to obtain the temporal feature h of each pixel. T Based on the target time series of each pixel, spatial features are extracted to obtain the spatial feature x of each pixel. T For each pixel, based on the corresponding temporal feature h T and spatial features x T Perform feature fusion memorization to obtain the corresponding feature fusion sequence r. T ; Feature fusion sequence r based on each pixel T Feature stitching is performed to obtain a target feature map; based on the target feature map, the corresponding segmentation result is output. This method can improve the utilization rate of medium- and low-resolution satellite imagery in land classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, in particular to a semantic segmentation method and system for medium and low resolution time-series satellite images. BACKGROUND

[0002] Remote sensing image classification is a process of classifying each pixel according to the different information features of each type of ground object in the image, and then matching the corresponding information of the actual ground object, so as to realize classification. Among them, semantic segmentation according to different information features of ground objects is the first step and a very key step. As an important part of the ground surface, land cover plays an important role in maintaining the stability of the ecological system. Improving the accuracy of land cover mapping has very important application value in the production and application of various industries such as ecology, climate, and agriculture. Therefore, as accurately as possible semantic segmentation of remote sensing satellite images can not only provide an important reference standard for remote sensing image classification, but also provide a basis for subsequent planning and construction work.

[0003] At present, in the process of remote sensing image segmentation, in order to solve the problem of insufficient classification accuracy based on single feature description, image features are often fused to retain effective feature information to the greatest extent and improve the accuracy and reliability of semantic segmentation. However, this method is more suitable for high-resolution images or photo images. Compared with high-resolution images with more detailed information, medium and low resolution satellite images lack more detailed texture, color, line and other detailed information. However, this type of image has a wider field of view and more extensive semantic information. Therefore, how to segment medium and low resolution satellite images to improve the utilization rate of medium and low resolution satellite images in land classification is the direction of focus of the present application. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a semantic segmentation method and system for medium and low resolution time-series satellite images, which can improve the utilization rate of medium and low resolution satellite images in land classification.

[0005] The embodiments of the present application also provide a semantic segmentation method for medium and low resolution time-series satellite images, comprising the following steps:

[0006] S1, determining a target time sequence of each pixel point in a target land block;

[0007] S2, respectively performing time-series feature extraction based on the target time sequence of each pixel point to obtain a time-series feature h of each pixel point T ;

[0008] S3, respectively performing spatial feature extraction based on the target time sequence of each pixel point to obtain a spatial feature x of each pixel pointT ;

[0009] S4, for each pixel point, respectively based on the corresponding time sequence feature h T , and the spatial feature x T Memory feature fusion, get the corresponding feature fusion sequence r T ;

[0010] S5, based on the feature fusion sequence r T of each pixel point, feature splicing is carried out, and the target feature map is obtained.

[0011] S6, based on the target feature map, output the corresponding segmentation result.

[0012] In a second aspect, the embodiments of the present application also provide a semantic segmentation system for low and medium resolution time sequence satellite images, the system comprises a preprocessing module, a time sequence feature extraction module, a spatial feature extraction module, a feature fusion memory module, a feature splicing module and a semantic segmentation module, wherein:

[0013] The preprocessing module is used to determine the target time sequence of each pixel point in the target plot.

[0014] The time sequence feature extraction module is used to extract the time sequence feature of each pixel point based on the target time sequence of each pixel point, and obtain the time sequence feature h T of each pixel point.

[0015] The spatial feature extraction module is used to extract the spatial feature of each pixel point based on the target time sequence of each pixel point, and obtain the spatial feature x T of each pixel point.

[0016] The feature fusion memory module is used to perform feature fusion memory for each pixel point based on the corresponding time sequence feature h T and the spatial feature x T , and obtain the corresponding feature fusion sequence r T .

[0017] The feature splicing module is used to perform feature splicing based on the feature fusion sequence r T of each pixel point, and obtain the target feature map.

[0018] The semantic segmentation module is used to output the corresponding segmentation result based on the target feature map.

[0019] Thirdly, embodiments of this application also provide a readable storage medium, which includes a semantic segmentation method program for medium- and low-resolution time-series satellite imagery. When the semantic segmentation method program for medium- and low-resolution time-series satellite imagery is executed by a processor, it implements the steps of a semantic segmentation method for medium- and low-resolution time-series satellite imagery as described in any of the preceding claims.

[0020] As described above, the semantic segmentation method, system, and readable storage medium for medium- and low-resolution time-series satellite imagery provided in this application firstly determines the target time series of each pixel in the target land parcel. Secondly, based on the target time series of each pixel, the concept of memory cells is used to fully extract key and effective temporal features in the time dimension. Thirdly, based on the target time series of each pixel, spatial features are extracted using a multilayer perceptron, which is more suitable for medium- and low-resolution satellite imagery, to obtain the spatial features of each pixel. The powerful learning ability of the multilayer perceptron is utilized to improve the prediction accuracy of spatial features. Fourthly, for each pixel, feature fusion and memory are performed based on the corresponding temporal and spatial features. This achieves deep feature extraction while ensuring the integrity of feature information and possessing feature memory capabilities, resulting in better feature extraction capabilities. Finally, feature stitching is performed based on the feature fusion sequence of each pixel to obtain a target feature map. Based on the obtained target feature map, high-precision classification and extraction of land features are achieved, effectively improving the utilization rate of medium- and low-resolution satellite imagery in land classification.

[0021] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a semantic segmentation method for medium- and low-resolution temporal satellite imagery provided in this application embodiment;

[0024] Figure 2 A schematic diagram of the generation structure of candidate memory cells;

[0025] Figure 3 A network structure schematic diagram of a multilayer perceptron;

[0026] Figure 4 A structure schematic diagram of a semantic segmentation system for medium and low resolution time-series satellite images provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0028] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0029] Please refer to Figure 1 , Figure 1 is a flowchart of a semantic segmentation method for medium and low resolution time-series satellite images in some embodiments of the present application. The method comprises the following steps:

[0030] Step S1, determining a target time series of each pixel point in a target land parcel.

[0031] Step S2, based on the target time series of each pixel point, respectively performing time-series feature extraction to obtain a time-series feature h T of each pixel point.

[0032] Step S3, based on the target time series of each pixel point, respectively performing spatial feature extraction to obtain a spatial feature x T of each pixel point.

[0033] Step S4, for each pixel point, respectively performing feature fusion and memory based on the corresponding time-series feature h T and spatial feature x T to obtain a corresponding feature fusion sequence r T .

[0034] Step S5, fusing features of each pixel point based on the sequence r T Step S4, performing feature splicing based on the target feature map to obtain a target feature map.

[0035] Step S6, outputting a corresponding segmentation result based on the target feature map.

[0036] As can be seen from the above, the semantic segmentation method for medium and low resolution time series satellite images disclosed in the present application first determines the target time sequence of each pixel point in the target land block; secondly, based on the target time sequence of each pixel point, the concept of memory cells is used to fully extract key effective time sequence features in the time dimension; thirdly, based on the target time sequence of each pixel point, spatial feature extraction is performed by using a multilayer perceptron more suitable for medium and low resolution satellite images to obtain spatial features of each pixel point, and the powerful learning ability of the multilayer perceptron is used to improve the prediction accuracy of the spatial features; fourthly, for each pixel point, feature fusion memory is performed based on the corresponding time sequence features and spatial features, which realizes deep feature extraction while ensuring the completeness of the feature information and has feature memory capability and good feature extraction capability; finally, feature splicing is performed based on the feature fusion sequence of each pixel point to obtain a target feature map, and high-precision classification and extraction of ground objects are realized based on the obtained target feature map, which effectively improves the utilization rate of medium and low resolution satellite images in land classification.

[0037] In one embodiment, in step S1, the determination of the target time sequence of each pixel point in the target land block comprises:

[0038] Step S11, acquiring a time series satellite image label set of the target land block.

[0039] Step S12, for each pixel point in the target land block, based on the time series satellite image label set, a plurality of time series satellite images corresponding to different times are determined, and the plurality of time series satellite images are combined in chronological order to obtain a corresponding target time sequence.

[0040] Based on steps S11-S12, it is necessary to explain that in the current embodiment, the land block information is first read, wherein the land block information includes land block configuration information, ID, grouping, etc., and also includes image information. Then, according to the satellite image label data corresponding to the read land block information, the pixel organizes the satellite image label data (organizes the pixels into a time sequence, and then arranges the label data in the same order), and ensures consistency with the time sequence of the pixels.

[0041] In one embodiment, in step S12, the target time sequence is obtained by the following steps:

[0042] Step S121, for each pixel point, the associated multiple time sequence satellite images are reorganized in time sequence to obtain the corresponding initial time sequence.

[0043] Step S122, compare the length of each initial time sequence to obtain the maximum length.

[0044] Step S123, when determining that the length of the corresponding initial time sequence is less than the maximum length, zero padding is performed on the sequence part with insufficient length of the corresponding initial time sequence, so that the length of each initial time sequence remains consistent, and the corresponding target time sequence is obtained.

[0045] Based on steps S121-S123, it is necessary to explain that in the current embodiment, the length of the time sequence of each pixel processed is kept consistent, because the amount of image data collected by different plots in the data set is not within the same range, and the image data of all plots is normalized to the same length, and the insufficient part is filled with 0, which can facilitate subsequent data processing.

[0046] In one embodiment, in step S2, for the pixel point p i , when extracting the time sequence feature h i of the pixel point p T , the method comprises:

[0047] Step S21, the selected time sequence satellite image a0 in the first time step t0 is processed by the sigmoid function, and the features with processing results of 1 are retained to generate the initial memory cell C0.

[0048] Specifically, in the process of processing by the sigmoid function, each band information included in the image is processed by the sigmoid function, and the feature part with a processing result of 1 is retained to obtain the initial memory cell C0.

[0049] Step S22, the time sequence satellite image a1 selected in sequence in the next time step t1 is processed by the sigmoid function, and the features with processing results of 1 are retained to generate the first candidate memory cell

[0050] Specifically, the candidate memory cell uses the tanh function with a value range of [-1, 1] as the activation function, and specifically, the first candidate memory cell is calculated as follows:

[0051]

[0052] Wherein, W C , bC The coefficients for the process of screening candidate memory cells are denoted by x1, which is the input data for the process of screening candidate memory cells, namely the input time-series satellite image a1.

[0053] Step S23, combine the initial memory cell C0 and the first candidate memory cell. Generate the first target memory cell C1 at the current time step t1, and pass the state information h1 of the first target memory cell C1 to the external state.

[0054] Specifically, in the current embodiment, the flow of information in the hidden state can be controlled by an input gate, a forget gate, and an output gate with element values ​​in the range [0,1]. This is generally achieved by using element-wise multiplication (symbolized as ⊙). It should be noted that the calculation of the first target memory cell C1 at the current time step t1 combines the information of the memory cell generated in the previous time step and the candidate memory cell generated in the current time step, and controls the flow of information through the forget gate and the input gate.

[0055] It should be noted that in the current embodiment, the feature information with a result of 1 is passed out through the sigmoid function. The output gate controls the flow of information from the memory cell to the hidden state, and the tanh function ensures that the hidden state element values ​​are between -1 and 1. It is important to note that when the output gate is approximately 1, the memory cell information is passed to the hidden state for use by the output layer; when the output gate is approximately 0, the memory cell information is retained only by the memory cell itself.

[0056] like Figure 2 As shown, the forget gate controls whether information from the memory cells of the previous time step is passed to the current time step, while the input gate controls how the input of the current time step flows into the memory cells of the current time step through the candidate memory cells. If the forget gate is always approximately equal to 1 and the input gate is always approximately equal to 0, then past memory cells will be preserved over time and passed to the current time step. This design can address the gradient decay problem in recurrent neural networks and better capture dependencies between time steps that are far apart in a time series.

[0057] Step S24: In the next time step t2, the next time-series satellite image a2 selected in sequence is processed by the sigmoid function, and the features with a processing result of 1 are retained to generate the second candidate memory cell in the current time step t2.

[0058] Step S25: Combine the first target memory cell C1 and the second candidate memory cell. Generate the second target memory cell C2 at the current time step t2, and pass the state information h2 of the second target memory cell C2 to the external state.

[0059] Specifically, steps S24-S25 can be understood in the light of the foregoing, and will not be described in detail here.

[0060] Step S26, steps S22-S25 are executed in a loop, and when the last one of the time sequence satellite images a i is processed, the tth target memory cell C t is obtained. t The state information h t of the tth target memory cell C t is passed to the external state.

[0061] Step S27, integrate the obtained external state information to obtain the time sequence feature h i of the pixel point p T ={h1,h2,...,h t}.

[0062] Based on steps S21-S27, it should be noted that the forget gate, input gate and output gate used in the current embodiment use sigmoid function as the activation function, and in generating the candidate memory, the hyperbolic tangent function tanh is used as the activation function. It is worth noting that these two activation functions are saturated, that is, in the case of input reaching a certain value, the output will not change significantly. If a non-saturated activation function is used, such as ReLU function, it will be difficult to achieve the effect of gating.

[0063] In the above embodiment, fusion extraction can be performed in the time dimension, improving the feature extraction quality.

[0064] In one of the embodiments, in step S4, for the pixel point p i in the target land, based on the time sequence feature h i of the pixel point p T and the spatial feature x T , the feature fusion memory is performed to obtain the feature fusion sequence r i of the pixel point p T , including:

[0065] Step S41, generate a memory cell C A0 , and take h1 in the time sequence feature h T as the initial state of the memory cell C A0 .

[0066] Step S42, pass the spatial feature x T into the memory cell C A0 for feature fusion memory, and pass the spatial feature xT performing processing to generate a corresponding candidate memory cell

[0067] Step S43, combining the memory cell C A0 and the candidate memory cell to generate a corresponding target memory cell C A1 and passing state information r1 of the target memory cell C A1 to an external state.

[0068] Step S44, sequentially updating state information of the memory cell C T based on extracting {h2,...,h t} from the time-series feature h A0 , and performing steps S42-S43 to perform feature fusion memory each time the state information is updated.

[0069] Step S45, integrating the obtained external state information to obtain a feature fusion sequence r i of the pixel point p T ={r1,r2,...,r t}.

[0070] The above embodiment can realize deep feature extraction and has feature memory capability, and has better feature extraction capability.

[0071] In one of the embodiments, in step S3, for the pixel point p i in the target plot, when extracting the spatial feature x i of the pixel point p T , the method comprises:

[0072] Step S31, using a multilayer perceptron to process the input target time series to obtain the spatial feature x i of the pixel point p T , wherein the multilayer perceptron comprises an input layer, a hidden layer and an output layer, the input layer is used to receive input data, the hidden layer is used to learn feature representation, and the output layer is used to generate final output result, each neuron in the hidden layer and the output layer has an activation function for introducing a nonlinear mapping.

[0073] Specifically, the multilayer perceptron (MLP, Multilayer Perceptron) is a kind of feedforward artificial neural network model, which maps multiple input data sets to a single output data set. For example Figure 3As shown, the multi-layer perceptron includes an input layer (the leftmost), a hidden layer (the middle two layers), and an output layer (the rightmost), and the processing process includes a forward process and a reverse process, wherein the reverse process is generally used for training, and the flow of the forward process is input, then the first layer of neurons is operated (multiplied by the weight, added by the bias, and activated once), the output is obtained, then the output of the first layer is taken as the input of the second layer, the operation is performed, the output of the second layer is obtained, and the operation of the output layer is performed to obtain the result.

[0074] The above embodiment predicts the spatial features of each pixel point through the multi-layer perceptron, has high accuracy, and has more accurate detection performance.

[0075] In one of the embodiments, in step S6, the corresponding segmentation result is output based on the target feature map, including:

[0076] In step S61, the target feature map is input into a 3*3 convolution kernel for a first convolution operation, and the obtained convolution result is activated once through a ReLU function to obtain an initial decoding feature map.

[0077] In step S62, the initial decoding feature map is input into a 3*3 convolution kernel for a second convolution operation, and the obtained convolution result is activated twice through a ReLU function to obtain a target decoding feature map.

[0078] In step S63, the target decoding feature map is mapped to classification based on a 1*1 convolution kernel, and the corresponding segmentation result is output.

[0079] Specifically, in the current implementation, the feature sequence of each pixel obtained through feature fusion and memory is spliced into a feature map, and then 3*3 convolution operations are repeated twice, wherein an activation function ReLU is used after each convolution to avoid gradient disappearance and reduce overfitting. Finally, a 1*1 convolution is performed to map the feature map obtained through the two 3*3 convolution operations to classification, and finally the segmentation result is output.

[0080] To verify the effectiveness of the semantic segmentation of the medium and low resolution time series satellite image proposed in the application, the PASTIS (Panoptic Agricultural Satellite Time Series) data set is selected for experiment, wherein the data set is collected by a Sentinel-2 satellite, has 2468 multispectral image sequences, each image has 10 channels, and the spatial range is 128*128; and each observation point has a corresponding label, and there are 19 categories.

[0081] In another aspect, the application also selects pixel accuracy (PA), class average pixel accuracy (MPA), intersection over union (IoU), and mean intersection over union (MIoU) as evaluation indexes, and the calculation formulas of the above evaluation indexes include:

[0082]

[0083]

[0084]

[0085]

[0086] Finally, the application also performs a comparative experiment on the feature fusion method proposed in the application and the method of using the time sequence feature alone, the method of using the spatial feature alone, and the existing UNET and Fast R-CNN, and the experimental results are shown in Table 1 as follows:

[0087] Table 1

[0088] Method PA MPA IoU MIoU Temporal feature + spatial feature 81.9 83.1 63.6 62.4 Temporal feature 78.1 79.8 60.1 59.9 Spatial feature 78.5 80.3 59.8 60.4 UNET 79.2 80.7 61.3 60.6 FastR-CNN 78.9 80.5 60.8 59.7

[0089] Based on the above experimental results, it can be known that the feature fusion method disclosed in the application can show good detection effects in various evaluation indexes, can be suitable for semantic segmentation of low-resolution time sequence satellite remote sensing images, and solves the problem of low utilization rate of time sequence remote sensing images caused by insufficient texture information of low-resolution images.

[0090] Please refer to Figure 4 The application discloses a semantic segmentation system for low-resolution time sequence satellite images, which comprises a preprocessing module, a time sequence feature extraction module, a spatial feature extraction module, a feature fusion memory module, a feature splicing module and a semantic segmentation module, wherein:

[0091] The preprocessing module is used to determine the target time sequence of each pixel point in the target land.

[0092] The time sequence feature extraction module is used to respectively perform time sequence feature extraction based on the target time sequence of each pixel point, to obtain the time sequence feature h T of each pixel point.

[0093] The spatial feature extraction module is used to respectively perform spatial feature extraction based on the target time sequence of each pixel point, to obtain the spatial feature x T of each pixel point.

[0094] The feature fusion memory module is used to respectively based on the corresponding time sequence feature h T and the spatial feature xT perform feature fusion memory to obtain a corresponding feature fusion sequence r T .

[0095] The feature splicing module is configured to perform feature splicing based on the feature fusion sequence r of each pixel point to obtain a target feature map. T

[0096] The semantic segmentation module is configured to output a corresponding segmentation result based on the target feature map.

[0097] In one of the embodiments, the modules in the system are further configured to perform the method in any of the optional implementation manners of the above embodiments.

[0098] As can be seen from the above, the semantic segmentation system for medium and low resolution time-series satellite images disclosed in the present application first determines the target time sequence of each pixel point in the target land parcel; secondly, based on the target time sequence of each pixel point, the concept of memory cells is adopted to fully extract key effective time-series features in the time dimension; thirdly, based on the target time sequence of each pixel point, spatial feature extraction is performed by using a multilayer perceptron more suitable for medium and low resolution satellite images to obtain the spatial feature of each pixel point, and the powerful learning ability of the multilayer perceptron is used to improve the prediction accuracy of the spatial feature; fourthly, for each pixel point, feature fusion memory is performed based on the corresponding time-series feature and spatial feature to realize deep feature extraction while ensuring the completeness of the feature information, and the feature memory capability is provided to have better feature extraction capability; finally, feature splicing is performed based on the feature fusion sequence of each pixel point to obtain a target feature map, and high-precision classification and extraction of ground objects are realized based on the obtained target feature map, thereby effectively improving the utilization rate of medium and low resolution satellite images in land classification.

[0099] ​The embodiment of the present application provides a readable storage medium, and the computer program is executed by the processor to execute the method in any optional implementation manner of the above embodiment. The storage medium can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk or an optical disk.

[0100] The readable storage medium is used for the following purposes. Firstly, the target time sequence of each pixel point in the target plot is determined. Secondly, based on the target time sequence of each pixel point, the concept of memory cells is used to fully extract key effective time sequence features in the time dimension. Thirdly, based on the target time sequence of each pixel point, the spatial feature of each pixel point is obtained by using a multilayer perceptron more suitable for medium and low resolution satellite images to extract spatial features, and the powerful learning ability of the multilayer perceptron is used to improve the prediction accuracy of the spatial feature. Fourthly, for each pixel point, the corresponding time sequence feature and the spatial feature are used for feature fusion memory, so as to realize deep feature extraction, ensure the integrity of the feature information, have the feature memory ability, and have better feature extraction ability. Finally, the feature splicing is performed based on the feature fusion sequence of each pixel point to obtain a target feature map, and based on the obtained target feature map, high-precision classification and extraction of ground objects are realized, and the utilization rate of medium and low resolution satellite images in land classification is effectively improved.

[0101] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and another division mode can be used in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0102] In addition, the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0103] Furthermore, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0104] In this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations.

[0105] The above is only an embodiment of the present application and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A semantic segmentation method for medium- and low-resolution temporal satellite imagery, characterized in that, Includes the following steps: S1. Determine the target time series of each pixel in the target plot; S2. Based on the target time series of each pixel, perform temporal feature extraction to obtain the temporal features of each pixel. h T ; S3. Based on the target time series of each pixel, perform spatial feature extraction to obtain the spatial features of each pixel. x T ; S4. For each pixel, based on the corresponding temporal features... h T and spatial characteristics x T Perform feature fusion memorization to obtain the corresponding feature fusion sequence. r T For each pixel p i ,include: S41, Generating memory cells C A0 and time series characteristics h T ={ h 1, h 2,..., h t In} h 1 as the memory cell C A0 The initial state; S42. The corresponding spatial features x T The memory cells are then fed into. C A0 In performing feature fusion memory, through sigmoid Functions on the spatial features x T Processing is performed to generate corresponding candidate memory cells. ; S43, Combine the memory cells C A0 and the candidate memory cells Generate corresponding target memory cells C A1 and the target memory cells C A1 Status information r 1. Pass the state to the outside; S44, based sequentially on temporal features h T Extracted { h 2,..., h t }, update the memory cells C A0 The status information is updated each time, and steps S42 to S43 are executed to perform feature fusion and memorization. S45. Integrate the obtained external state information to obtain the pixel points. p i Feature fusion sequence r T ={ r 1, r 2,..., r t }; S5, Feature fusion sequence based on each pixel r T Feature concatenation is performed to obtain the target feature map; S6. Based on the target feature map, output the corresponding segmentation result.

2. The method according to claim 1, characterized in that, In step S1, determining the target time series of each pixel in the target plot includes: S11. Obtain the time-series satellite image tag set for the target site; S12. For each pixel in the target plot, based on the time-series satellite image tag set, determine multiple time-series satellite images corresponding to different times, and combine the multiple time-series satellite images in chronological order to obtain the corresponding target time series.

3. The method according to claim 2, characterized in that, In step S12, the target time series is obtained through the following steps: S121. For each pixel, reassemble the associated multiple time-series satellite images in chronological order to obtain the corresponding initial time series. S122. Compare the lengths of each of the initial time series to obtain the maximum length; S123. When it is determined that the length of the corresponding initial time series is less than the maximum length, zero values ​​are filled in the sequence portion of the corresponding initial time series that is not long enough, so that the length of each initial time series is consistent, and the corresponding target time series is obtained.

4. The method according to claim 1, characterized in that, In step S2, the pixels in the target plot are... p i In extracting pixels p i Temporal characteristics h T When, the method includes: S21, the first time step t In 0, the selected time-series satellite imagery a 0 passed sigmoid The function processes the data and retains the feature where the processing result is 1, generating the initial memory cell. C 0; S22, Next time step t In section 1, the time-series satellite images selected in sequence a 1 through sigmoid The function processes the data and retains the feature where the processing result is 1, generating the current time step. t The first candidate memory cell under 1 ; S23, Combine the initial memory cells C 0 and the first candidate memory cell Generate the current time step t The first target memory cell under 1 C 1, and the first target memory cell C Status information of 1 h 1. Pass the state to the outside; S24, Next time step t In step 2, the next time-series satellite image selected in sequence. a 2. Proceed sigmoid The function processes the data and retains the feature where the processing result is 1, generating the current time step. t The second candidate memory cell under 2 ; S25, Combine the first target memory cell C 1 and the second candidate memory cell Generate the current time step t The second target memory cell under 2 C 2, and the second target memory cell C 2. Status information h 2. Pass the state to the outside; S26. Repeat steps S22 to S25, and continue processing until all pixels are processed. p i The last time-series satellite image a t , obtained the t Target memory cells C t At that time, the first t Target memory cells C t Status information h t Pass the state to the outside; S27. Integrate the obtained external state information to obtain the pixel points. p i Temporal characteristics h T ={ h 1, h 2,..., h t } 5. The method according to claim 1, characterized in that, In step S3, the pixels in the target plot are... p i In extracting pixels p i Spatial features x T When, the method includes: S31. A multilayer perceptron is used to process the input target time series to obtain pixel points. p i Spatial features x T The multilayer perceptron includes an input layer, a hidden layer, and an output layer. The input layer is used to receive input data, the hidden layer is used to learn feature representations, and the output layer is used to generate the final output result. Each neuron in the hidden layer and the output layer has an activation function to introduce a nonlinear mapping.

6. The method according to claim 1, characterized in that, In step S6, the step of outputting the corresponding segmentation result based on the target feature map includes: S61. Input the target feature map into 3 The first convolution operation is performed using three convolution kernels, and the resulting convolution is then processed through... ReLU The function performs one activation to obtain the initial decoded feature map; S62, Input the initial decoded feature map into 3 A second convolution operation is performed using three convolution kernels, and the resulting convolution result is then processed through... ReLU The function undergoes secondary activation to obtain the target decoded feature map; S63, based on 1 1. The convolutional kernel maps the target decoded feature map to the classification, and outputs the corresponding segmentation result.

7. A semantic segmentation system for medium- and low-resolution temporal satellite imagery, characterized in that, The system includes a preprocessing module, a temporal feature extraction module, a spatial feature extraction module, a feature fusion and memory module, a feature concatenation module, and a semantic segmentation module, wherein: The preprocessing module is used to determine the target time series of each pixel in the target plot; The temporal feature extraction module is used to extract temporal features based on the target time series of each pixel, thereby obtaining the temporal features of each pixel. h T ; The spatial feature extraction module is used to extract spatial features based on the target time series of each pixel, thereby obtaining the spatial features of each pixel. x T ; The feature fusion memory module is used to perform a feature fusion for each pixel based on the corresponding temporal features. h T and spatial characteristics x T Perform feature fusion memorization to obtain the corresponding feature fusion sequence. r T For each pixel p i This includes: generating memory cells C A0 and time series characteristics h T ={ h 1, h 2,..., h t In} h 1 as the memory cell C A0 The initial state; the corresponding spatial features x T The memory cells are then fed into. C A0 In performing feature fusion memory, through sigmoid Functions on the spatial features x T Processing is performed to generate corresponding candidate memory cells. Combine the memory cells C A0 and the candidate memory cells Generate corresponding target memory cells C A1 and the target memory cells C A1 Status information r 1. Pass the state to the external state; then, based on the temporal features... h T Extracted { h 2,..., h t }, update the memory cells C A0 Each time the state information is updated, a feature fusion and memory process is performed; the obtained external state information is integrated to obtain the pixel. p i Feature fusion sequence r T ={ r 1, r 2,..., r t }; The feature splicing module is used to fuse the feature sequence of each pixel. r T Feature concatenation is performed to obtain the target feature map; The semantic segmentation module is used to output the corresponding segmentation result based on the target feature map.

8. A readable storage medium, characterized in that, The readable storage medium includes a semantic segmentation method program for medium- and low-resolution time-series satellite imagery. When the semantic segmentation method program for medium- and low-resolution time-series satellite imagery is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images