Change detection patch optimization method, electronic device, and computer product

Abstract

The application discloses a change detection polygon optimization method, electronic equipment and computer products. The method first acquires slice data of change detection polygon results, including cutting out a position area of a polygon from a first time phase image, a second time phase image and a mask image. Then, the slice data is selectively labeled. Part of the slice data is labeled based on active learning technology to obtain label information of the part of the slice data. Then, the slice data with the label information is input into a refinement model to calculate performance data of the slice data. Whether the model meets the standard is judged. If the model does not meet the standard, the model is trained based on the slice data and the label information. The above steps of selecting and judging are repeated until the model meets the standard or all the slice data labeling is completed. Finally, the optimized change detection polygon result is obtained based on the refinement model, the slice data and the label information. The application can reduce the false alarm rate on the basis of high recall rate, reduce the verification workload and optimize specific business.

Description

Technical Field

[0001] This invention relates to the technical fields of natural resource change monitoring operations, artificial intelligence neural network models, transfer learning and active learning, and in particular to a method, electronic equipment and computer product for optimizing existing change detection patches to improve accuracy and reduce the workload of subsequent verification. Background Technology

[0002] With the ever-changing demands for remote sensing applications, the rapid development of high-resolution remote sensing data, the accumulation of geographical knowledge, and the development of artificial intelligence technology, automated and intelligent natural resource monitoring has become possible. One of the key technological capabilities is intelligent detection of natural resource change patches.

[0003] In the prior art, the following patent documents may have touched on this issue, such as CN118968338A, CN116051437A, CN117056856A, CN112990109A, etc.

[0004] Although numerous studies have shown that the most effective methods for detecting changes in natural resources are based on deep neural networks, limitations such as unstable accuracy and poor generalization greatly restrict their widespread application. This is mainly due to two factors: the diversity of ground features and the incompleteness of the remote sensing data used for training.

[0005] The diversity of land features is reflected in the complex morphological characteristics (for example, due to the differences in geographical environment in different agricultural regions, the shape and size of cultivated land plots vary greatly) and phenological characteristics (for example, the same plot may carry multiple crop types over time, with complex growth patterns and cultivation practices).

[0006] The incompleteness of remote sensing data used for training is reflected in the fact that, due to the diverse scales and complex modalities of remote sensing data, it is easily affected by factors such as clouds, rain, fog, and sensor conditions. The data used to train deep neural network models cannot cover all time periods, geographical areas, and the full range of variations of the aforementioned influencing factors. Remote sensing data covering only a limited time, space, and spectral bands cannot characterize the dynamic changes of natural features with complex morphological and phenological characteristics, leading to a significant challenge in comprehensively understanding natural resource features across all time periods and dimensions.

[0007] In existing intelligent change patch detection technologies, deep neural network models are typically trained on historical open-source / proprietary datasets. However, the amount of data in historical datasets is always limited, covering only certain regions, phenological conditions, data characteristics, application scenarios / requirements, etc., and cannot cover the various personalized situations that arise in actual business processes.

[0008] Therefore, the actual operational performance of existing intelligent change detection technologies is unsatisfactory, particularly in terms of a high false alarm rate while maintaining a high recall rate, which fails to effectively reduce the workload of subsequent change detection work. A solution is needed that can optimize the change detection results output by existing change detection systems by utilizing actual data and verification processes from current business operations. Summary of the Invention

[0009] The main objective of this invention is to provide a dual-temporal change detection patch optimization method, electronic device, and computer product to solve the problems of unstable accuracy, poor generalization, high false alarm rate, and inability to effectively handle personalized situations in actual business in existing natural resource change detection technologies.

[0010] Based on the first main aspect of the present invention, a method for optimizing dual-temporal change detection patches is provided, characterized by comprising the following steps:

[0011] Step 1: Obtain slice data of change detection patch results, including locating each patch that is determined to have changed, and cropping out a specific size area of ​​the patch location from the first time phase image, the second time phase image and the mask image to form slice data;

[0012] Step 2: Selectively label the slice data to obtain labeling information for a portion of the slice data; this includes selecting a portion of the slice data from all unlabeled slice data for labeling, and obtaining information on whether the patch regions in the selected slice data have changed between two time phases;

[0013] Step 3: Based on the slice data with annotation information, obtain the performance data of the refinement model, including using the slice data as input to the refinement model, the model outputting a prediction result of whether the patch area in the slice data has changed, and calculating the performance data of the refinement model based on the model's prediction result for the slice data and the annotation information of the slice data.

[0014] Step 4: Based on the performance data, determine whether the refined model meets the requirements. If it does, proceed to step 7; otherwise, proceed to step 5.

[0015] Step 5: Based on the slice data and annotation information, train the refined model, including unsupervised training of the refined model using the slice data and supervised training of the refined model using the slice data with annotation information;

[0016] Step 6: Determine whether all slice data has been labeled. If all slice data has been labeled, proceed to step 7; otherwise, proceed to step 2.

[0017] Step 7: Based on the refined model, slice data and annotation information, obtain the optimized change detection patch results.

[0018] In some embodiments, as a further preferred embodiment, the change detection patch result in step 1 includes a registered first phase image, a second phase image, and a mask image. The mask image distinguishes changed pixels from unchanged pixels by different values, and each independent connected region formed by the changed pixels corresponds to a change patch.

[0019] In some embodiments, as a further preferred option, when cropping the patch to form slice data in step 1, each patch corresponds to one slice data, and each slice data includes a slice of the first temporal image, a slice of the second temporal image, and a slice of the mask image.

[0020] In some embodiments, as a further preferred option, the refinement model in step 3 is a pre-trained deep neural network model, whose input is slice data and whose output is a prediction of whether the patch region in the input slice data has changed.

[0021] In some embodiments, as a further preferred embodiment, the optimized change detection patch result in step 7 includes:

[0022] 1) Based on the tile data with annotation information, output all changed patches labeled as changed;

[0023] 2) Based on the refined model, predictions are made for all unlabeled slice data, and all predicted changes are output as changes. The two output slices constitute the optimized change detection slice results.

[0024] In some embodiments, as a further preferred embodiment, step 2 involves selecting a portion of the unlabeled slice data for labeling using active learning techniques, including:

[0025] 1) Feed all unlabeled tile data into the refinement model to obtain the predicted probability of whether the patch area in all unlabeled tile data has changed;

[0026] 2) Obtain the performance data of the current refined model. In the first active learning iteration, the refined model has no performance data. In subsequent iterations, the performance data calculated in step 3 is used.

[0027] 3) Using an active learning method, select a portion of the slice data from all unlabeled slice data based on the information obtained in 1) and 2).

[0028] In some embodiments, as a further preferred embodiment, during the first active learning iteration, a portion of the unlabeled slice data is randomly selected; in subsequent iterations, let the number of unlabeled slice data be M, and the number of slice data to be selected be 3*N, then the method for selecting slice data is as follows:

[0029] 1) Sort all unlabeled slice data in ascending order according to the entropy value of the predicted probability obtained by the refined model and the probability value of being judged as positive.

[0030] 2) Randomly select N slice data from the unlabeled slice data whose entropy values ​​are sorted in the interval [start, M], and determine the value of start as follows: Let Fb represent a normalized performance index value of the refined model. First, let start be equal to the integer value of the product of Fb and M. Then, if M-start is less than the integer value B of the product of parameter a and N, let start be equal to MB. Otherwise, keep the original value of start and set parameter a to 2.

[0031] 3) Select n1 slice data from the front end of the slice data sequence sorted by positive class probability value, and select 2N-n1 slice data from the end of the sequence, where n1 is equal to the integer value of the product of R and 2N, and R is the proportion coefficient of positive class slice data in the slice data labeled in the previous active learning iteration or in the cumulative labeled slice data.

[0032] Based on the second main aspect of the present invention, the present invention provides a method for monitoring changes in natural resources, comprising: obtaining information on changes in natural resources by using change patches obtained by the aforementioned change detection patch optimization method.

[0033] Based on the third main aspect of the present invention, the present invention provides a method for providing map patch optimization services based on existing change detection map patch results, including: providing map patch optimization services to natural resource monitoring business parties using the aforementioned change detection map patch optimization method, the services including providing optimization results of existing map patch results, providing a refinement model that can be used to optimize existing map patch results, and embedding the aforementioned method of the present invention into the business party's map patch verification workflow, etc.

[0034] Based on a fourth key aspect of the present invention, the present invention provides an apparatus for optimizing change detection patches, comprising at least:

[0035] The slice acquisition module is configured to acquire slice data of the change detection patch results based on the existing change detection patch results.

[0036] The slice annotation module is configured to selectively annotate the slice data to obtain annotation information for a portion of the slice data.

[0037] The performance calculation module is configured to obtain the performance data of the refined model based on the slice data with labeled information.

[0038] The model training module is configured to train and refine the model based on the sliced ​​data and annotation information.

[0039] The result output module is configured to obtain optimized change detection patch results based on the refined model, slice data, and annotation information.

[0040] In the above-described apparatus, the functions of each module can be implemented through hardware or through hardware executing corresponding software. The aforementioned hardware or software includes one or more modules corresponding to the functions described above. In one possible design, the apparatus includes a memory and a processor, wherein the memory stores one or more computer instructions supporting the apparatus in executing the corresponding methods described above, and the processor is configured to execute the computer instructions stored in the memory. The apparatus may also include a communication interface for communication with other devices or communication networks.

[0041] Based on a fifth principal aspect of the invention, the invention provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described in any of the preceding aspects.

[0042] Based on a sixth principal aspect of the present invention, the present invention provides a computer-readable storage medium for storing computer instructions used by any of the above-described devices, which, when executed by a processor, are used to implement the method described in any of the above-described aspects.

[0043] Based on the seventh principal aspect of the present invention, the present invention provides a computer program product comprising computer instructions which, when executed by a processor, are used to implement the methods described in any of the preceding aspects.

[0044] Compared with the prior art, the present invention has the following advantages and effects:

[0045] (1) Compared with existing technologies, this invention can reduce the false alarm rate of changed feature results while maintaining high recall. To ensure high recall, existing technologies have a high false alarm rate for the changed feature results. This invention optimizes the changed feature results of existing technologies, and the optimized output of changed feature results consists of two parts: first, the features confirmed to be changed through annotation. Because some erroneous features are filtered out during the annotation process, the false alarm rate is effectively reduced; second, the features that have not been confirmed through annotation but are judged to be changed by the refinement model. The refinement model is required to meet performance requirements in terms of recall and false alarm rate during iterative training. If the performance requirements are not met, it is not allowed to participate in the output of the final feature results. Therefore, the false alarm rate of the features judged to be changed is also lower than that of existing feature results.

[0046] (2) Compared with the prior art, the present invention can significantly reduce the workload of the business party in verifying map features. The present invention filters out some erroneous map features in the results of the prior art, reducing the false alarm rate. Therefore, the number of clue map features that the business party needs to verify will be significantly reduced, thereby significantly reducing the workload of verification.

[0047] (3) Existing methods mainly use open-source data and historical business data when training models, without performing performance tuning for each specific business; while the present invention mainly uses the specific business data to be processed when training models, and performs performance tuning for the current specific business. This difference allows the present invention to optimize the output results of existing methods when applied to specific business, thereby adapting to various personalized change detection business tasks.

[0048] (4) The more times the same / similar business is carried out, the smaller the average annotation workload required to implement this invention. The refined model obtained by implementing this invention in one business can be used as the initial refined model when implementing this invention in subsequent same / similar business. If the initial model already meets the performance requirements, there is no need to continue annotating slice data; if the initial model does not yet meet the performance requirements, usually only a smaller amount of data needs to be annotated than that in the previous business to meet the performance requirements. In addition, as the number of the same / similar business increases, the accumulated slice data and labeling information also increase, and the performance of the initial refined model trained using this data also improves, thereby reducing the need to annotate new data.

[0049] (5) Existing methods separate the model training process from the specific map patch verification process, and their model optimization process is also unsuitable for inclusion within the specific map patch verification process. However, the model training and optimization process of this invention is suitable for integration with the specific map patch verification process. One reason is that this invention optimizes for personalized business processes, requiring integration with the specific map patch verification process. Another reason is that the labeling process for slice data in this invention overlaps with the function of the specific map patch verification process; that is, the labeling of slice data in this invention can be directly completed using the specific map patch verification process. A third reason is that this invention uses an active learning iterative approach to complete model training, minimizing the required labeling workload, which aligns with the goal of minimizing the verification workload in the specific map patch verification process.

[0050] (6) This invention can cooperate with existing change detection methods / systems for mutual benefit. Business parties / users do not need to abandon or stop implementing their existing change detection methods / systems. This invention optimizes the output patch results of existing change detection methods / systems.

[0051] (7) In the implementation of the present invention, if the slice data is labeled using the specific map patch verification business process, the map patches to be verified are selected by the method of the present invention. Attached Figure Description

[0052] Figure 1 A flowchart of a change detection patch optimization method in one embodiment of the present invention is shown;

[0053] Figure 2 This diagram illustrates an application scenario of the change detection patch optimization service in one embodiment of the present invention. Detailed Implementation

[0054] Hereinafter, exemplary embodiments of this patent will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of the exemplary embodiments have been omitted from the drawings.

[0055] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and do not preclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0056] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0057] The details of the embodiments of this disclosure are described in detail below through specific examples.

[0058] Figure 1 A flowchart illustrating a change detection patch optimization method according to an embodiment of this disclosure is shown. Figure 1 As shown, the method for optimizing change detection patches includes the following steps:

[0059] Step 1: Obtain slice data of the change detection patch results. This includes locating each patch identified as having changed, and cropping a specific-size region of the patch's location from the first time-phase image, the second time-phase image, and the mask image to form slice data. Each patch corresponds to one slice data set, and each slice data set contains a slice of the first time-phase image, a slice of the second time-phase image, and a slice of the mask image.

[0060] In this embodiment, the change detection patch result includes a registered first temporal image, a second temporal image, and a mask image. In the mask image, changed pixels and unchanged pixels are distinguished by different values, and each independent connected region formed by the changed pixels corresponds to a change patch. The change detection patch result can be a change detection patch result obtained using existing methods and systems for the current service data (first temporal image data and second temporal image data), or it can be a historical change detection patch result.

[0061] Step 2: Selectively label the slice data to obtain labeling information for a portion of the slice data. This includes selecting a portion of the slice data from all unlabeled slice data for labeling, and obtaining information on whether the patch regions in the selected slice data changed between two time phases.

[0062] In this embodiment, the process of selecting a subset of slice data from all unlabeled slice data using active learning techniques includes:

[0063] (1) Input all unlabeled slice data into the refinement model to obtain the predicted probability of whether the patch area in all unlabeled slice data has changed;

[0064] (2) Obtain the performance data of the current refinement model (the refinement model has no performance data in the first active learning iteration; in subsequent iterations, take the performance data calculated in step 3).

[0065] (3) Using an active learning method, select a subset of slice data from all unlabeled slice data based on the information obtained in (1) and (2). The active learning method may use all or part of the information in (1) and (2) or not use this information.

[0066] In this embodiment, the refinement model is a pre-trained deep neural network model. Its input is slice data (including a first temporal image slice, a second temporal image slice, and a mask image slice), and its output is a prediction of whether the patch region in the input slice data has changed.

[0067] In this embodiment, during the first active learning iteration, a portion of unlabeled slice data is randomly selected. In subsequent iterations, let the number of unlabeled slice data be M, and the number of slice data to be selected be 3*N. The method for selecting slice data is as follows:

[0068] (1) Sort all unlabeled slice data in ascending order according to the entropy value of the predicted probability obtained by the refined model and the probability value of being judged as positive (i.e., changing);

[0069] (2) Randomly select N slice data from the unlabeled slice data whose entropy values ​​are sorted in the interval [start,M].

[0070] In this embodiment, the value of start is determined as follows: Let Fb represent a normalized performance index value of the refined model. First, set start equal to the integer value of the product of Fb and M. Then, if M-start is less than the integer value B of the product of parameter a and N, set start equal to MB; otherwise, keep the original value of start. In this embodiment, parameter a is set to 2. In other embodiments, a can be set to other values ​​within the range [1, M / N].

[0071] (3) Select n1 slice data from the front end of the slice data sequence sorted by positive class probability value, and select 2*N-n1 slice data from the end of the sequence. Wherein, n1 is equal to the integer value of the product of R and 2*N, and R is the proportion coefficient of positive class slice data in the slice data labeled in the previous active learning iteration, or in the cumulative labeled slice data.

[0072] It should be noted that the slice data selection method in this embodiment tends to maintain a balance between the number of positive and negative class slice data to avoid the adverse consequences that imbalanced datasets may have on model training. At the same time, the better the performance of the refined model, the more the slice data selection method tends to select slice data that is difficult to distinguish (high entropy value) to further improve the performance of the refined model. In addition, the specific implementation of this slice data selection method is dynamically adjusted according to the proportion of positive class in the labeled slice data / cumulative labeled slice data in the previous active learning iteration and the performance of the refined model, which has a high degree of adaptability.

[0073] Step 3: Based on the labeled slice data, obtain the performance data of the refinement model. This includes using the slice data as input to the refinement model, the model outputting a prediction result on whether the patch regions in the slice data have changed, and calculating the performance data of the refinement model based on the model's prediction result for the slice data and the labeling information of the slice data.

[0074] In this embodiment, the calculation of the performance data of the refined model includes: calculating the training performance parameters of the refined model based on the slice data with labeled information that participated in the training of the refined model; and calculating the test performance parameters of the refined model based on the slice data with labeled information that did not participate in the training of the refined model.

[0075] In this embodiment, the performance parameters used are recall, precision, and Fβ coefficient. Other performance parameters, such as AUC, can also be used.

[0076] Step 4: Determine whether the refined model meets the requirements based on the performance data. If it does, proceed to Step 7; otherwise, proceed to Step 5.

[0077] Step 5: Train the refined model based on the sliced ​​data and annotation information. This includes unsupervised training of the refined model using the sliced ​​data and supervised training of the refined model using the sliced ​​data with annotation information. Unsupervised training using unlabeled data and supervised training using labeled data are well-known techniques in the industry and will not be elaborated upon here.

[0078] Step 6: Determine if all slice data has been labeled. If all slice data has been labeled, proceed to Step 7; otherwise, proceed to Step 2.

[0079] Step 7: Based on the refined model, slice data, and annotation information, obtain the optimized change detection patch results. This includes:

[0080] (1) Based on the slice data with annotation information, output all changed patches labeled as changed;

[0081] (2) Based on the refined model, predictions are made for all unlabeled slice data, and the predicted changes in all patch patterns are output. The two output patches constitute the optimized change detection patch pattern results.

[0082] In some embodiments, the results of the patch in (1) can be used together with the results of the patch in other unlabeled slice data as the final output; or based on the refinement model, all slice data can be predicted, and then all the changed patches predicted to be changed can be output as the final result.

[0083] In some embodiments, existing active learning methods (such as entropy-based sample selection methods) can be used to select a subset of samples from all unlabeled slice data for labeling. It is understood that this disclosure is not limited to using entropy-based sample selection methods; any other sample selection method may be used.

[0084] Based on this embodiment, change detection patch results can be obtained for specific business operations. Compared with existing technologies, the change detection patches obtained by the method in this embodiment maintain a high recall rate while having a lower false alarm rate. These patch results can be used directly or after business verification for monitoring and analyzing changes in natural resources.

[0085] Based on this embodiment, a neural network model (refinement model) can be obtained to optimize existing change detection patch data for specific business operations. This model can provide patch optimization services for natural resource change detection business operators.

[0086] The method and refinement model training process of this embodiment can also be embedded into the natural resource change monitoring business process to achieve iterative mutual promotion between refinement model assisting manual verification and manual verification assisting refinement model optimization training.

[0087] An optimization device for change detection patches based on an implementation of this disclosure includes:

[0088] (1) The slice acquisition module is configured to acquire slice data of the change detection patch results based on existing change detection patch data.

[0089] (2) The slice annotation module is configured to selectively annotate the slice data to obtain annotation information for some slice data.

[0090] (3) The performance calculation module is configured to obtain the performance data of the refined model based on the slice data with the labeled information.

[0091] (4) The model training module is configured to train and refine the model based on the slice data and annotation information.

[0092] (5) The result output module is configured to obtain the optimized change detection patch results based on the refined model, slice data and annotation information.

[0093] The above functions are implemented through a server. The server's memory stores one or more computer instructions that support the execution of the corresponding methods described above by the aforementioned apparatus. The server's processor is configured to execute the computer instructions stored in the memory. The server is configured with a communication interface for communicating with other devices and communication networks. In other embodiments, the corresponding software can also be implemented through other hardware.

[0094] An electronic device based on one implementation of this disclosure is a server, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the various steps of the aforementioned change detection patch optimization method, including operations such as acquiring tile data, selective labeling, performance calculation, model training, and result output, thereby optimizing the change detection patch results. The server interacts with other related devices through its communication interface, such as receiving raw patch result data from the change detection server and transmitting the optimized results to a patch service server / computer, etc.

[0095] Based on one implementation of this disclosure, a computer-readable storage medium is a hard disk, used to store computer instructions used in any aspect of the present invention. When the processor reads and executes these computer instructions in the hard disk, it can implement the above-mentioned change detection patch optimization method, complete the optimization processing of the change detection patch results, including operations such as acquisition of slice data, annotation, model training and result generation, thereby improving the accuracy of change detection patches, reducing the false alarm rate, reducing the workload of subsequent verification, and adapting to personalized business needs.

[0096] One implementation of this disclosure provides a computer program product written in Python, containing computer instructions. When this program product runs on a server or other device, and its computer instructions are executed by the processor, it can implement each step of the aforementioned change detection patch optimization method. For example, it acquires slice data according to a set process, selects slice data for annotation using active learning technology, calculates and refines model performance data, trains the model, and finally outputs the optimized change detection patch results. This enables collaborative work with the entire change detection patch optimization system, improving the efficiency and accuracy of natural resource change monitoring operations.

[0097] Figure 2 This diagram illustrates an application scenario of a change detection patch optimization service according to an embodiment of the present disclosure. Figure 2 As shown, the change detection server sends the change detection patch result data obtained based on the existing method to the refinement server. The refinement server optimizes the existing change detection patch result data and transmits the optimized result to the patch business server / computer.

[0098] During the optimization process of existing change detection patch results, the refinement server transmits the tile data to the annotation server. The annotation server then distributes the tile data to general internet terminals and patch verification terminals, retrieves the annotation information from each terminal, and finally sends the aggregated annotation information back to the refinement server. The terminal devices for annotating tiles can be desktop computers, laptops, tablets, or mobile phones, etc.

[0099] Any aspects of this invention not described in detail are well-known to those skilled in the art.

[0100] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for optimizing dual-temporal change detection patches, characterized in that, Includes the following steps: Step 1: Obtain slice data of change detection patch results, including locating each patch that is determined to have changed, and cropping out a specific size area of ​​the patch location from the first time phase image, the second time phase image and the mask image to form slice data; Step 2: Selectively label the slice data to obtain labeling information for some slice data; This includes selecting a portion of unlabeled slice data from all unlabeled slice data for labeling, and obtaining information on whether the patch regions in the selected slice data have changed between two time phases; Step 3: Based on the tile data with annotation information, obtain the performance data of the refinement model, including using the tile data as input to the refinement model, the model outputting the prediction result of whether the patch area in the tile data has changed, and calculating the performance data of the refinement model based on the model's prediction result for the tile data and the annotation information of the tile data. Step 4: Based on the performance data, determine whether the refined model meets the requirements. If it does, proceed to step 7; otherwise, proceed to step 5. Step 5: Based on the slice data and annotation information, train the refined model, including unsupervised training of the refined model using the slice data and supervised training of the refined model using slice data with annotation information; Step 6: Determine whether all slice data has been labeled. If all slice data has been labeled, proceed to step 7; otherwise, proceed to step 2. Step 7: Based on the refined model, slice data, and annotation information, obtain the optimized change detection patch results; In step 1, when the patch is cropped to form slice data, each patch corresponds to one slice data, and each slice data includes a slice of the first time phase image, a slice of the second time phase image, and a slice of the mask image. In step 2, an active learning technique is used to select a subset of unlabeled slice data for labeling, including: 1) Feed all unlabeled tile data into the refinement model to obtain the predicted probability of whether the patch regions in all unlabeled tile data have changed; 2) Obtain the performance data of the current refined model. In the first active learning iteration, the refined model has no performance data. In subsequent iterations, the performance data calculated in step 3 is used. 3) Using an active learning method, select a portion of the slice data from all unlabeled slice data based on the information obtained in 1) and 2).

2. The dual-temporal change detection patch optimization method according to claim 1, characterized in that, The change detection patch result in step 1 includes a registered first phase image, a second phase image, and a mask image. In the mask image, different values ​​are used to distinguish changed pixels from unchanged pixels. Each independent connected region formed by the changed pixels corresponds to a change patch.

3. The dual-temporal change detection patch optimization method according to claim 1, characterized in that, The refinement model mentioned in step 3 is a pre-trained deep neural network model, whose input is slice data and whose output is a prediction of whether the patch regions in the input slice data have changed.

4. The dual-temporal change detection patch optimization method according to claim 1, characterized in that, The optimized change detection patch results described in step 7 include: 1) Based on the tile data with annotation information, output all changed patches labeled as changed; 2) Based on the refined model, prediction is performed on all slice data without annotation information, and all predicted changes are output as changes. The two output patches constitute the optimized change detection patch results.

5. The method for optimizing dual-temporal change detection patches according to claim 1, characterized in that, In the first active learning iteration, a portion of unlabeled slice data is randomly selected; in subsequent iterations, the number of unlabeled slice data is set to... M The number of slices to be selected is 3 N The method for selecting slice data is as follows: 1) Sort all unlabeled slice data in ascending order according to the entropy value of the predicted probability obtained by the refined model and the probability value of being classified as positive. 2) Sort by entropy value. [start, M] Randomly selected from unlabeled slices of data in the interval N Each slice of data is determined as follows: start Values: Fb A normalized performance metric representing the refinement model is first set to... start equal Fb and M The integer value of the product, then, if M-start Less than parameter a and N The integer value of the product B Then let start equal MB Otherwise keep start Original value, parameter a Set to 2; 3) Select from the front end of the slice data sequence sorted by positive class probability value n1 Each slice of data is selected from the end of the sequence. 2N-n1 Data slices, among which... n1 equal R and 2N The integer value of the product of , R The proportion of positive class slice data in the slice data labeled in the previous active learning iteration, or in the cumulative labeled slice data.

6. An apparatus for optimizing change detection patches using the method described in any one of claims 1-5, characterized in that, Includes the following modules: The slice acquisition module is configured to acquire slice data of the change detection patch results based on the existing change detection patch results; The slice annotation module is configured to selectively annotate the slice data to obtain annotation information for a portion of the slice data; The performance calculation module is configured to obtain performance data for the refined model based on sliced ​​data with labeled information. The model training module is configured to train and refine the model based on the sliced ​​data and annotation information; The result output module is configured to obtain optimized change detection patch results based on the refined model, slice data, and annotation information.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein, The processor executes the computer program to implement the dual-temporal change detection patch optimization method according to any one of claims 1-5.

8. A computer program product comprising computer instructions which, when executed by a processor, are used to implement the dual-temporal change detection patch optimization method according to any one of claims 1-5.

Images