Method, device, equipment and medium for detecting changes of last-year end library and current-year image

Abstract

The invention provides a last year end library and current year image change detection method. The last year end library and current year image change detection method comprises the steps of obtaining a last year end library and a current year image containing various land types and various land type codes; the last year end library is subjected to vector rasterization processing, then the last year end library and the current year image are input into a trained multi-modal change detection model for change detection, a change grid is obtained, and the multi-modal change detection model comprises an end-of-year library encoder comprising a one-hot encoding module and an embedded layer, an image encoder and a change detection network; and performing grid vectorization processing on the change grid to obtain a change detection result. According to the method, change detection is carried out on the last-year end library and the current-year image by using the multi-modal change detection module, change detection is automatically carried out on remote sensing data of different modals, the workload of manpower can be greatly reduced, and the extraction efficiency of remote sensing monitoring pattern spots is effectively improved.

Description

Technical Field

[0001] The present invention relates to the fields of remote sensing information processing and artificial intelligence technology, and in particular to a method for detecting changes between a database at the end of the previous year and images of the current year. Background Art

[0002] The annual change survey of land and resources refers to the periodic monitoring and management of land and resources by recording and tracking the types of land features and their changes within a certain area of ​​each region.

[0003] A key task in the annual change survey is extracting remote sensing monitoring patches. This extraction relies on change detection between the year-end database and the current year's remote sensing imagery. Because the year-end database is vector data and represents a different modality from remote sensing imagery, existing methods for extracting remote sensing monitoring patches rely primarily on manual comparison. This involves manually comparing the year-end database with the current year's remote sensing imagery and using the change detection results as the remote sensing monitoring patches. This results in a high workload and low extraction efficiency. Summary of the Invention

[0004] In response to the above problems, the present invention provides a method, device, electronic device and computer-readable storage medium for detecting changes in remote sensing data of two different modalities: year-end database and image.

[0005] According to a first aspect, a method for detecting changes between images stored at the end of the previous year and images of the current year is provided, the method comprising:

[0006] S1: Acquire the year-end database of the previous year and the current year's images, wherein the year-end database of the previous year contains multiple land class codes and multiple land classes, and the multiple land class codes are used to indicate the multiple land classes;

[0007] S2: performing vector rasterization processing on the year-end library of the previous year to obtain the year-end library raster of the previous year;

[0008] S3: Inputting the previous year's end-of-year grid and the current year's image into a trained multimodal change detection model for change detection to obtain a change grid; wherein the multimodal change detection model includes a year-end-of-year encoder, an image encoder, and a change detection network; the year-end-of-year encoder includes a one-hot encoding module and an embedding layer;

[0009] S4: performing raster vectorization processing on the change grid to obtain change detection results for the library at the end of the previous year and the image of the current year.

[0010] According to a second aspect, a device for detecting changes between images stored at the end of the previous year and images of the current year is provided, the device comprising:

[0011] An acquisition module is used to acquire the year-end database of the previous year and the images of the current year, wherein the year-end database of the previous year contains multiple land category codes and multiple land categories, and the multiple land category codes are used to indicate the multiple land categories;

[0012] A conversion module, configured to perform vector rasterization processing on the year-end library of the previous year to obtain a raster of the year-end library of the previous year;

[0013] A multimodal change detection module is configured to input the previous year's end-of-year grid and the current year's image into a trained multimodal change detection model for encoding and change detection processing to obtain a change grid; wherein the multimodal change detection model includes a year-end-of-year encoder, an image encoder, and a change detection network; and the year-end-of-year encoder includes a one-hot encoding module and an embedding layer;

[0014] The inverse conversion module is used to perform raster vectorization processing on the change grid to obtain the change detection results for the library at the end of the previous year and the image of the current year.

[0015] According to a third aspect, an electronic device is provided, the electronic device comprising:

[0016] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the above-mentioned method for detecting changes between the library at the end of the previous year and the image of the current year are implemented.

[0017] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for detecting changes between the library at the end of the previous year and the images of the current year are implemented.

[0018] The beneficial effects of the present invention include at least:

[0019] 1. The present invention uses a multimodal change detection model to perform change detection on the year-end database of the previous year in vector format and the current year image in raster format, thereby automatically performing change detection on remote sensing data in two different modalities, vector and raster, greatly reducing the workload of manpower and effectively improving the extraction efficiency of remote sensing monitoring spots.

[0020] 2. The present invention realizes the encoding of the year-end database of the previous year in vector form by using a year-end database encoder including a one-hot encoding module and an embedding layer, and has a simple network structure and low resource consumption;

[0021] 3. The present invention trains the multimodal change detection model through three stages: pre-training, supervised learning, and reinforcement learning. It achieves the alignment of the year-end library encoder and the image encoder, and improves the accuracy of the multimodal change detection model, thereby improving the accuracy of change detection. In addition, the reinforcement learning stage adopts the sampling-free GRPO reinforcement learning algorithm to reduce resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 Schematic diagram of the flow of the method for detecting changes between the image library at the end of the previous year and the image of the current year according to the present invention;

[0024] Figure 2 Schematic diagram of the workflow of the multimodal change detection model of the present invention;

[0025] Figure 3 This is a schematic structural diagram of a year-end library encoding module of the present invention;

[0026] Figure 4 Schematic diagram of the training process of the multimodal change detection model of the present invention;

[0027] Figure 5 It is a structural diagram of the device for detecting changes between the image library at the end of the previous year and the image of the current year according to the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present invention.

[0029] Figure 1 The figure shows a flow chart of a method for detecting changes between the image library at the end of the previous year and the image of the current year provided by the present invention. The method includes:

[0030] S1: Acquire the year-end database of the previous year and the current year's images, wherein the year-end database of the previous year contains multiple land class codes and multiple land classes, and the multiple land class codes are used to indicate the multiple land classes;

[0031] S2: performing vector rasterization processing on the year-end library of the previous year to obtain the year-end library raster of the previous year;

[0032] S3: Inputting the previous year's end-of-year grid and the current year's image into a trained multimodal change detection model for change detection to obtain a change grid; wherein the multimodal change detection model includes a year-end-of-year encoder, an image encoder, and a change detection network; the year-end-of-year encoder includes a one-hot encoding module and an embedding layer;

[0033] S4: performing raster vectorization processing on the change grid to obtain change detection results for the library at the end of the previous year and the image of the current year.

[0034] According to an embodiment of the present invention, the step S1 further includes: obtaining land change data and preprocessing it to obtain the year-end database of the previous year and the images of the current year.

[0035] Land change data includes data from the previous year's end database and original current year imagery. The previous year's end database is vector data, stored in a .shp file format, and records the type of every inch of land within the jurisdiction from the previous year. Specifically, the previous year's end database contains multiple land types and multiple land type codes. Multiple land type codes are used to indicate multiple land types. Land type codes are in string format and represented by DLBM fields. For example, the previous year's end database contains 12 primary land types and 60 secondary land types, corresponding to 12 primary land type codes and 60 secondary land type codes.

[0036] Table 1 shows an example of land categories and land category codes in the database at the end of the previous year. As shown in Table 1, the various land categories in the database at the end of the previous year include multiple primary land categories and multiple secondary land categories, and each secondary land category belongs to a primary land category. Accordingly, the various land category codes include multiple primary land category codes and multiple secondary land category codes. For example, "garden" is a primary land category with a primary land category code of "02," "tea garden" is a secondary land category with a secondary land category code of "05," and the primary land category to which "tea garden" belongs is "garden."

[0037] Table 1 Example of land type and land type coding information in the database at the end of the previous year

[0038]

[0039] After acquiring land change data, the original current-year imagery is too large, so preprocessing is required to obtain data of appropriate size and uniform format. Specifically, the original current-year imagery is cropped into slices of uniform size to obtain the current-year imagery. For example, the original current-year imagery can be cropped based on the patch boundaries in the previous year's year-end database, and resampled to fill in the boundaries and achieve a uniform size, resulting in multiple current-year images. Optionally, the slice size is 512*512.

[0040] According to one embodiment of the present invention, step S2 includes:

[0041] Use the Vector to Raster tool to convert the previous year's end database into a vector raster to obtain the previous year's end database raster. The previous year's end database raster is the previous year's end database data in raster form.

[0042] Converting vector-to-raster data files from SHP format to raster format allows alignment of the previous year's end imagery with the current year's imagery for subsequent processing and analysis. For example, vector-to-raster conversion tools are available in ArcGIS.

[0043] The present invention utilizes a trained multimodal change detection model to automatically perform change detection on the library grid at the end of the previous year and the image of the current year.

[0044] Next, step S3 is described in detail.

[0045] Figure 2 A schematic diagram of the workflow of the multimodal change detection model is shown. Figure 2 ,The multimodal change detection model includes a year-end library encoder, an image encoder and a change ,detection network, where the year-end library encoder includes a one-hot encoding ,module and an embedding layer.

[0046] Continue to see Figure 2 In step S3, the end-of-year library grid and the current year image are input into the trained multimodal change detection model for change detection, and the obtained change grid includes:

[0047] S31: Input the year-end library grid of the previous year into the year-end library encoder, use the one-hot encoding module to one-hot encode the multiple land class codes to obtain multiple land class IDs, and use the embedding layer to vector map the multiple land class IDs to obtain the feature vector of the year-end library of the previous year.

[0048] Specifically, the previous year-end database raster is input into the year-end database encoder. The one-hot encoding module in the year-end database encoder performs one-hot encoding on the multiple land class codes in the year-end database raster to obtain multiple land class IDs. The embedding layer in the year-end database encoder then performs vector mapping on the multiple land class IDs to obtain the previous year-end database feature vector. The one-hot encoding module refers to a unique hot encoding, which is used to convert discrete numerical values ​​into binary vectors.

[0049] The present invention achieves encoding of the year-end library data of the previous year in vector format by using a year-end library encoder including a one-hot encoding module and an embedding layer. At the same time, it avoids the use of a complex network structure and does not need to perform context understanding and information filtering. Only a simple network structure and a concise processing flow are used to achieve vector feature extraction, with low resource consumption.

[0050] In one feasible implementation, change detection can be performed on the first-level land class code or the second-level land class code, that is, a one-hot encoding module is used to one-hot encode one of the first-level land class code or the second-level land class code to obtain multiple first-level land class IDs or multiple second-level land class IDs, and an embedding layer is used to perform vector mapping on the multiple first-level land class IDs or multiple second-level land class IDs to obtain the library feature vector at the end of the previous year.

[0051] In practical applications, first-level land classification codes are easier to learn, but using only first-level land classification codes is too coarse-grained to meet actual business needs. Second-level land classification codes provide more detailed classification, but using only second-level land classification codes will lose common information between land classifications. Therefore, in a preferred embodiment, the first-level and second-level land classification codes can be processed simultaneously and merged to obtain the feature vector of the database at the end of the previous year.

[0052] In a preferred embodiment, two channels are set in the one-hot encoding module of the year-end library encoder, and two embedding layers are set accordingly to process the primary land category code and the secondary land category code respectively and simultaneously.

[0053] The secondary land class code and the primary land class code to which it belongs are regarded as a pair of land class codes. The multiple land class codes in the database at the end of the previous year include multiple pairs of primary land class codes and secondary land class codes. Step S31 includes:

[0054] Use the two channels of the one-hot encoding module to simultaneously perform one-hot encoding on each pair of first-level land class codes and second-level land class codes to obtain multiple pairs of first-level land class IDs and second-level land class IDs.

[0055] Specifically, Figure 3 A schematic diagram of the year-end library coding module structure of the present invention is shown. Figure 3 , the first channel and the second channel are set in the one-hot encoding module. For each pair of first-level land class code and second-level land class code, the first channel is used to one-hot encode the first-level land class code to obtain the first-level land class id, and the second channel is used to one-hot encode the second-level land class code to obtain the second-level land class id.

[0056] Two embedding layers are used to simultaneously perform vector mapping processing on the first-level land class ID and the corresponding second-level land class ID, and multiple pairs of first-level land class vectors and second-level land class vectors are obtained.

[0057] Specifically, the embedding layer includes a first embedding layer and a second embedding layer. For each pair of first-level land class id and second-level land class id, the first embedding layer is used to perform vector mapping processing on the first-level land class id to obtain a first-level land class vector, and the second embedding layer is used to perform vector mapping processing on the second-level land class id to obtain a second-level land class vector.

[0058] Each pair of the first-level land class vector and the second-level land class vector is concatenated to obtain the feature vector of the library at the end of the previous year. Specifically, each pair of the first-level land class vector and the second-level land class vector is concatenated in a one-to-one correspondence.

[0059] The present invention sets two channels in the one-hot encoding module to simultaneously process the first-level land category codes and second-level land category codes of the database at the end of the previous year, and correspondingly sets two embedding layers for vector mapping. The obtained feature vector of the database at the end of the previous year can not only reflect the similarities between land features under the same first-level land category and retain prior knowledge, but also accurately distinguish the differences between second-level land categories, thereby improving the accuracy of change detection results.

[0060] S32: Inputting the image of this year into the image encoder, encoding the image of this year using the image encoder, and obtaining a feature vector of the image of this year.

[0061] Specifically, the current year's image is input into an image encoder, which encodes the semantic features and texture features of the current year's image to obtain a feature vector for the current year's image. The image encoder can employ a convolutional neural network or other network type, as long as it can extract image features, and this is not limited in the present invention.

[0062] S33: Inputting the library feature vector at the end of the previous year and the image feature vector of the current year into the change detection network for detection to obtain the change grid.

[0063] Specifically, the library feature vector at the end of the previous year and the image feature vector of this year are input into a change detection network, and the change detection network is used to perform change detection on the library feature vector at the end of the previous year and the image feature vector of this year, and a change grid is output.

[0064] The change detection network can include a feature comparison layer and a multilayer perceptron (MLP). The feature comparison layer calculates the difference and absolute value of the feature vector of the previous year's end library and the feature vector of the current year's image. The MLP then processes the feature vector to output a change raster. The change raster is the change detection result in raster format.

[0065] The present invention realizes automatic change detection of remote sensing data in two different modes, vector and raster, by using a multimodal change detection model, which greatly reduces the human workload and effectively improves the extraction efficiency of sensing monitoring spots.

[0066] Next, the training of a multimodal change detection model is described in detail based on an embodiment of the present invention.

[0067] Figure 4The schematic diagram of the multimodal change detection model training process of the present invention is shown in FIG. Figure 4 As shown in Figure 2, the training process of the above multimodal change detection model includes:

[0068] Step a: pre-training the year-end library encoder and the image encoder in the multimodal change detection model to obtain a primary multimodal change detection model.

[0069] Specifically, a pre-training set and a seed land category code are obtained, and the pre-training set and the seed land category code are used to pre-train the year-end library encoder and the image encoder in the multimodal change detection model to obtain a pre-trained year-end library encoder and a pre-trained image encoder, and the pre-trained year-end library encoder, the pre-trained image encoder and the change detection network are used as a primary multimodal change detection model.

[0070] The pre-training set includes multiple pairs of unchanged year-end library samples and unchanged current year image samples. After obtaining and pre-processing land change data, the current year's images are compared with the previous year's year-end library. The current year's images with no land classification changes and their corresponding previous year-end library samples are used as pairs of unchanged previous year-end library samples and unchanged current year image samples. Multiple pairs of unchanged previous year-end library samples and unchanged current year image samples are used to construct the pre-training set.

[0071] In one feasible method, the seed land categories are all secondary land categories and special land categories contained in the annual change survey data, or all primary land categories and special land categories. Accordingly, the seed land category codes include all secondary land category codes and special land category codes, or all primary land category codes and special land category codes. Assuming that all secondary land categories or all primary land categories are n types, then the seed land categories and seed land category codes are n+1 types.

[0072] Special land classes refer to land classes not covered in the annual change survey data. They are used to represent areas that were not covered in the library samples at the end of the previous year. For example, at the edge of the image or the edge of the jurisdiction, it is impossible to guarantee that every pixel in the slice has a corresponding land class and land class code. Special land classes and special land class codes can be used to represent them. Special land class codes are represented by PAD.

[0073] In a preferred embodiment, the seed land categories are all secondary land categories and their corresponding primary land categories and special land categories contained in the annual change survey data. Accordingly, the seed land category codes include all land category code pairs and special land category codes consisting of secondary land category codes and primary land category codes corresponding to secondary land category codes.

[0074] For example, assuming that there are n types of secondary land categories and m types of primary land categories, m≤n, and m and n are integers greater than 1, then the corresponding seed land category codes include land category code pairs consisting of n types of secondary land category codes and their corresponding primary land category codes, as well as special land category codes, totaling n+1 types, where n is 60, for example.

[0075] The pre-training steps include:

[0076] (1) Input n+1 seed land class codes into the year-end database encoder for encoding to obtain n+1 seed land class feature vectors. At the same time, input the unchanged image samples of this year into the image encoder for encoding to obtain the unchanged image feature vectors.

[0077] Assuming that the total number of pixels in an image is k, the invariant image feature vector obtained by the image encoder includes k pixel feature vectors.

[0078] (2) After normalizing the n+1 seed land class feature vectors, the inner products between the n+1 normalized seed land class feature vectors and each pixel feature vector are calculated to obtain the image predicted land class vector.

[0079] Specifically, for each pixel, the inner product between each normalized seed land class feature vector and the pixel feature vector is calculated to obtain n+1 inner product values, which form the pixel prediction land class vector. The k pixel prediction land class vectors constitute the image prediction land class vector.

[0080] (3) The true land class vector of the image is obtained from the unchanged year-end library samples corresponding to the unchanged current year image samples. Based on the image predicted land class vector and the image true land class vector, the pre-training loss value is calculated using the soft-max loss function, and the parameters of the year-end library encoder and the image encoder are adjusted according to the pre-training loss value.

[0081] The ground-class labels for each pixel in the image are recorded in the unchanged year-end database, from which the true ground-class vector of the image is derived. The soft-max loss function has been documented in related art and will not be further described here.

[0082] Considering that some land types have similarities and some image features may exist simultaneously in different land types, preferably, the pre-training process uses label smoothing to increase robustness.

[0083] (4) The cycle continues until the year-end library encoder and the image encoder converge, and the pre-trained year-end library encoder and the pre-trained image encoder are obtained.

[0084] Model convergence conditions include but are not limited to the loss value reaching the preset value and the number of training times reaching the set number.

[0085] By pre-training the encoder using a pre-training set, this method can train the embedding layer of the year-end database encoder and align the image encoder with the year-end database encoder. By maximizing the inner product between image features and land class code features in the year-end database, pre-training forces the multimodal change detection model to learn the correspondence between land class code embeddings and image features, providing prior knowledge for subsequent tasks. Using a soft-max loss function, it can achieve rapid alignment of the image encoder with the year-end database encoder.

[0086] Step b: training the primary multimodal change detection model using a supervised learning algorithm to obtain an intermediate multimodal change detection model.

[0087] Specifically, a training set and a reference patch are obtained, and the primary multimodal change detection model is trained using the training set and the reference patch with a supervised learning algorithm to obtain an intermediate multimodal change detection model.

[0088] The training set consists of multiple pairs of samples from the previous year's end database and images from the current year's changes. After obtaining and preprocessing land change data, the current year's images are compared with the previous year's end database. The current year's images with land class changes and their corresponding previous year's end database are used as pairs of samples from the previous year's end database and the current year's change image. Multiple pairs of these samples are used to construct the training set.

[0089] Reference patches are nationally issued remote sensing imagery monitoring patches. Using the SHP format, they record the areas of change identified by comparing the current year's imagery with the corresponding year-end database. As you can see, reference patches record the true values ​​of the changed areas. Reference patches are in vector format and must be rasterized to generate the true value change raster before use.

[0090] The steps of training using a supervised learning algorithm include: vectorizing the samples of the library at the end of the previous year and inputting them into the library encoder at the end of the previous year to obtain a feature vector of the library at the end of the previous year; inputting the image of the current year into the image encoder to obtain a feature vector of the image of the current year; inputting the feature vector of the library at the end of the previous year and the feature vector of the image of the current year into the change detection network for change detection to obtain a predicted change grid; obtaining a reference patch and vectorizing it to obtain a true change grid, calculating a supervised loss value based on the predicted change grid and the true change grid using a supervised loss function, and adjusting the parameters of the primary multimodal change detection model according to the supervised loss value; and continuously looping until the primary multimodal change detection model converges to obtain an intermediate multimodal change detection model.

[0091] The supervision loss function has been described in the related art and can be selected as needed, and the present invention does not impose any limitation on this.

[0092] According to a preferred embodiment, during the training phase using a supervised learning algorithm, the embedding layer of the year-end library encoder is frozen.

[0093] The present invention can obtain a basic model with certain change detection capabilities by using reference patches and training sets to perform supervised training on a primary multimodal change detection model.

[0094] Step c: training the intermediate multimodal change detection model using a reinforcement learning algorithm to obtain the trained multimodal change detection model.

[0095] Specifically, a training set and a reference patch are obtained, and the intermediate multimodal change detection model is trained using a sampling-free Group Relative Policy Optimization (GRPO) reinforcement learning algorithm using the training set and the reference patch to obtain a trained multimodal change detection model.

[0096] Let the intermediate multimodal change detection model be model θ, and copy model θ to get the reference model θ ref , the steps of training using the sampling-free GRPO reinforcement learning algorithm include:

[0097] (1) Input the samples of the library at the end of the previous year and the image samples of the current year into the model θ and the reference model θ ref , for each pixel s i Output the predicted change probability o respectively i and the reference change probability o iref .

[0098] Specifically, each pixel s in the image sample of this year will be changed i As an independent generation task, the model θ and the reference model θ ref For pixel s i Perform change detection and output the predicted probability o respectively i and the reference probability o iref Where i∈[1,k], k is a positive integer, representing the total number of pixels in the image samples of the current year.

[0099] The result of the change detection task is selected from "change" or "unchanged", so the predicted probability o i Represents pixel s i "Change" or "Unchange".

[0100] (2) Obtain the reference patch and perform vector rasterization to obtain the true value change grid. According to the true value change grid and the predicted probability o i and the reference probability o iref , and get the corresponding reward ri , prediction correct probability and reference correct probability

[0101] Specifically, the true value change grid includes each pixel s i Whether the pixel true value g changes i , compared with the predicted change probability o i and pixel truth g i , judge the predicted change probability o i The result is "correct" or "wrong", which determines the corresponding reward. The reward for "correct" is defined as 1, and the reward for "wrong" is defined as 0.

[0102] Probability of correct prediction Represents pixel s i The predicted probability o i and pixel true value g i The probability of the same result is obtained by comparing the predicted probability o i and pixel true value g i For example, if o i The corresponding pixel true value g i If it means "change", Equal to o i If o i The corresponding pixel true value g i If it means "unchanged", Equal to 1-o i . Similarly, refer to the correct probability Represents pixel s i The reference probability o iref and pixel true value g i The probability of the same result is obtained by comparing the predicted probability o iref and pixel true value g i get.

[0103] (3) According to the reward r i and the probability of correct prediction Pixel point s is calculated using the improved advantage function i Positive advantages and negative advantages

[0104] The improved advantage function is:

[0105]

[0106] Among them, r i 、 s is the pixel i The reward and prediction correct probability, i∈[1,k].

[0107] The reward will be i Substituting 1 into the improved advantage function yields a positive advantage The reward will be i Substituting 0 into the improved advantage function yields a negative advantage

[0108] The improved advantage function is obtained by improving the advantage function in the GRPO reinforcement learning algorithm based on the characteristic that the change detection task has only two results.

[0109] (4) Based on positive advantages and negative advantages The enhanced loss value is calculated using the improved loss function, and the parameters of the mid-level multimodal change detection model are adjusted according to the enhanced loss value.

[0110] The improved loss function is:

[0111]

[0112] in, and Represents pixel s respectively i The predicted correct probability, reference correct probability, positive advantage and negative advantage of , β is a hyperparameter, i∈[1,k], k is a positive integer.

[0113] The improved loss function is obtained by improving the loss function in the GRPO reinforcement learning algorithm based on the characteristic that the change detection task has only two results.

[0114] When adjusting the parameters of the intermediate multimodal change detection model according to the reinforcement loss value, the parameters of the model θ are updated using the gradient descent and backpropagation method, and the KL divergence is regularized. When updating the parameters of the model θ, only the correct prediction probability is calculated. gradient.

[0115] In the sampling-free GRPO reinforcement learning algorithm, there are only model θ and reference model θ ref , there is no old model for sampling, the actual sampling step is exempted, and the positive advantage and negative advantage of the pixel point can be directly calculated according to the improved advantage function, which omits the sampling cost and effectively reduces the consumption of computing power.

[0116] (5) The cycle continues until the intermediate multimodal change detection model converges and a trained multimodal change detection model is obtained.

[0117] The model θ is trained continuously according to steps (1) to (4) using the training set and the reference patch until the model θ converges and the training is stopped. The convergence conditions include but are not limited to: the number of training times reaches the preset number and the reinforcement loss value reaches the threshold.

[0118] The present invention uses a reinforcement learning algorithm for post-training based on the intermediate multimodal change detection model obtained by supervised learning, which can further improve the model accuracy.

[0119] The present invention trains a multimodal change detection model through three stages: pre-training, supervised learning, and reinforcement learning. The pre-training embedding layer is trained and the image encoder is aligned with the year-end library encoder. The supervised learning method can quickly acquire the change detection capability. On this basis, the reinforcement learning method is used to further improve the model accuracy, thereby improving the accuracy of change detection.

[0120] In one embodiment of the present invention, a device for detecting changes between a library at the end of the previous year and images of the current year is provided.

[0121] Figure 5 The schematic diagram of the structure of the device for detecting changes between the image at the end of the previous year and the image of the current year is shown in FIG. Figure 5 A device 500 for detecting changes between images from the end of the previous year and the current year, comprising:

[0122] An acquisition module 501 is configured to acquire a database at the end of the previous year and images of the current year, wherein the database at the end of the previous year contains multiple land class codes and multiple land classes, and the multiple land class codes are used to indicate the multiple land classes;

[0123] The conversion module 502 is used to perform vector rasterization processing on the year-end library of the previous year to obtain the year-end library raster of the previous year;

[0124] A multimodal change detection module 503 is configured to input the previous year's end-of-year grid and the current year's image into a trained multimodal change detection model for encoding and change detection processing to obtain a change grid; wherein the multimodal change detection model includes a year-end-of-year encoder, an image encoder, and a change detection network; and the year-end-of-year encoder includes a one-hot encoding module and an embedding layer.

[0125] The inverse conversion module 504 is used to perform raster vectorization processing on the change grid to obtain the change detection results for the library at the end of the previous year and the image of the current year.

[0126] An embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the aforementioned method embodiments when executing the program. Furthermore, the electronic device further comprises: at least one input device; at least one output device. The above-mentioned memory, processor, input device, and output device are connected via a bus. Specifically, the input device may be a camera, a touch panel, a physical button, a mouse, or the like. Specifically, the output device may be a display screen. The memory may be a high-speed random access memory (RAM) memory, or a non-volatile memory, such as a disk memory. The memory is used to store a set of executable program codes, and the processor is coupled to the memory.

[0127] An embodiment of the present invention further provides a computer-readable storage medium, which may be provided in the electronic device in each of the above embodiments, and the computer-readable storage medium may be the electronic device in the aforementioned embodiment. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of the aforementioned method embodiments. Furthermore, the computer-storable medium may also be various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0128] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0129] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for detecting changes between the image library at the end of the previous year and the image of the current year, characterized in that: The method comprises: S1: Acquire the year-end database of the previous year and the current year's images, wherein the year-end database of the previous year contains multiple land class codes and multiple land classes, and the multiple land class codes are used to indicate the multiple land classes; S2: performing vector rasterization processing on the year-end library of the previous year to obtain the year-end library raster of the previous year; S3: Inputting the previous year's end-of-year grid and the current year's image into a trained multimodal change detection model for change detection to obtain a change grid; wherein the multimodal change detection model includes a year-end-of-year encoder, an image encoder, and a change detection network; the year-end-of-year encoder includes a one-hot encoding module and an embedding layer; S4: performing raster vectorization processing on the change grid to obtain change detection results for the library at the end of the previous year and the image of the current year.

2. The method according to claim 1, characterized in that The change detection is performed by inputting the library grid at the end of the previous year and the image of the current year into the trained multimodal change detection model to obtain the change grid, which includes: S31: Input the year-end library grid of the previous year into the year-end library encoder, use the one-hot encoding module to one-hot encode the multiple land class codes to obtain multiple land class IDs, and use the embedding layer to vector map the multiple land class IDs to obtain the feature vector of the year-end library of the previous year; S32: Inputting the image of this year into the image encoder, and using the image encoder to encode the image of this year to obtain a feature vector of the image of this year; S33: Inputting the library feature vector at the end of the previous year and the image feature vector of the current year into the change detection network for detection to obtain the change grid.

3. The method according to claim 2, characterized in that The multiple land class codes include multiple pairs of primary land class codes and secondary land class codes, the one-hot encoding module includes two channels, the embedding layer includes two, and S31 includes: Use the two channels of the one-hot encoding module to simultaneously perform one-hot encoding on the first-level land class code and the corresponding second-level land class code, and obtain multiple pairs of first-level land class IDs and second-level land class IDs; Use two embedding layers to simultaneously perform vector mapping processing on the first-level land class ID and the corresponding second-level land class ID, and obtain multiple pairs of first-level land class vectors and second-level land class vectors; Each pair of the first-level land classification vector and the second-level land classification vector is concatenated to obtain the characteristic vector of the library at the end of the previous year.

4. The method according to any one of claims 1 to 3, characterized in that The training process of the multimodal change detection model includes: Pre-training the year-end library encoder and the image encoder in the multimodal change detection model to obtain a primary multimodal change detection model; Training the primary multimodal change detection model using a supervised learning algorithm to obtain an intermediate multimodal change detection model; The intermediate multimodal change detection model is trained using a reinforcement learning algorithm to obtain the trained multimodal change detection model.

5. The method according to claim 4, characterized in that The pre-training of the year-end library encoder and the image encoder in the multimodal change detection model comprises: Obtain a pre-training set and a seed land class code, and use the pre-training set and the seed land class code to jointly pre-train the year-end library encoder and the image encoder in the multimodal change detection model; wherein, the pre-training set includes multiple pairs of unchanged year-end library samples of the previous year and unchanged image samples of the current year, and the seed land class code includes all land class code pairs composed of secondary land class codes and primary land class codes corresponding to the secondary land class codes, and special land class codes.

6. The method according to claim 4, characterized in that The step of training the primary multimodal change detection model using a supervised learning algorithm includes: A training set and a reference patch are obtained, and the primary multimodal change detection model is trained using the training set and the reference patch with a supervised learning algorithm; wherein the training set includes multiple pairs of library samples at the end of the previous year and image samples of the current year.

7. The method according to claim 4, characterized in that The training of the intermediate multimodal change detection model using a reinforcement learning algorithm includes: A training set and a reference patch are obtained, and the intermediate multimodal change detection model is trained using the training set and the reference patch by adopting a sampling-free group relative strategy optimization reinforcement learning algorithm; wherein the training set includes multiple pairs of library samples at the end of the previous year and image samples of the current year.

8. A device for detecting changes between the image library at the end of the previous year and the image of the current year, characterized in that: The device comprises: An acquisition module is used to acquire the year-end database of the previous year and the images of the current year, wherein the year-end database of the previous year contains multiple land category codes and multiple land categories, and the multiple land category codes are used to indicate the multiple land categories; A conversion module, configured to perform vector rasterization processing on the year-end library of the previous year to obtain a raster of the year-end library of the previous year; A multimodal change detection module is configured to input the previous year's end-of-year grid and the current year's image into a trained multimodal change detection model for encoding and change detection processing to obtain a change grid; wherein the multimodal change detection model includes a year-end-of-year encoder, an image encoder, and a change detection network; and the year-end-of-year encoder includes a one-hot encoding module and an embedding layer; The inverse conversion module is used to perform raster vectorization processing on the change grid to obtain the change detection results for the library at the end of the previous year and the image of the current year.

9. An electronic device, characterized in that: The electronic device comprises: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which implements the steps of the method according to any one of claims 1 to 7 when executed by a processor.

Images