Image change detection parallel decoding method and system based on block-level joint training

Abstract

The application relates to a kind of image change detection parallel decoding methods and systems based on block level joint training, relate to remote sensing image change detection technical field, the method is jointly encoded to the input dual-phase image, combined with text query, constructs condition vector. With condition vector as context, in language decoder, use autoregressive mode to generate block sequence unit by unit, and encode into change block sequence. Using the mode of double-flow joint training, the unit-by-unit prediction ability and parallel prediction ability of language decoder are trained, and the language decoder is further optimized in combination with geometric consistency loss. In practical application, the target dual-phase image obtained is inferred in parallel / unit by unit using the language decoder, and the vector file obtained by change detection is output. It can be seen that the application realizes end-to-end output of multiple types of change graph, improves the accuracy and efficiency of change detection.

Description

Technical Field

[0001] This application relates to the field of remote sensing image change detection technology, and in particular to a parallel decoding method and system for image change detection based on block-level joint training. Background Technology

[0002] Remote sensing image change detection is a core technology in fields such as routine monitoring of natural resources, detection of illegal construction, and disaster assessment. Remote sensing technology can be used to dynamically monitor the conversion of arable land to non-agricultural or non-grain uses, enabling the timely detection and prevention of illegal land use. Existing remote sensing image change detection methods are mainly divided into three categories:

[0003] The first category is pixel-level change detection methods, represented by BIT (Binary Change Detection with Transformers), SNUNet (Dense Nested Attention Network), and ChangeFormer. These methods calculate feature difference maps between preceding and following temporal images, determining pixel-by-pixel whether a change has occurred and outputting a binary change mask. However, this method suffers from severe fragmentation in practical applications. A complete newly added building area (500m²) may be broken down into 5-8 disconnected fragmented patches (10-100m² in area), requiring complex post-processing such as morphological dilation-erosion, connected component analysis, and area filtering to form meaningful change patches. In farmland conversion monitoring, the measured fragmentation rate (number of fragmented patches <50m² / total number of patches) for pixel-level methods reaches 15-25%.

[0004] The second category is the patch-level change detection method: first, semantic segmentation is used to extract patches (such as farmland and building plots) from previous and subsequent time phases, and then the patch attributes (category, area, shape) are compared to determine whether there has been a change. This method relies on segmentation quality. In actual tests on the CLCD dataset, segmentation errors (boundary offset > 2 pixels) can easily cause about 8% of changes at farmland boundaries to be missed, and it cannot detect small changes outside the segmentation range.

[0005] The third category is target detection-based change detection methods. These methods transform change detection into the problem of detecting new targets, and can only detect new types of changes. They cannot handle the two common change patterns in monitoring the non-agriculturalization of arable land: disappearance (arable land has been leveled but not yet developed) and transformation (arable land has been converted into fish ponds / orchards or other types).

[0006] Furthermore, the aforementioned methods all have shortcomings at the output level: they cannot simultaneously output the bounding box of the changed region, the change type (new / disappear / transformation), and geometric attributes (area, perimeter) in a single inference, requiring multi-stage cascaded processing (change detection → change classification → area calculation → vectorization), which is inefficient and prone to error accumulation. In large-area farmland monitoring, the number of changed patches can reach thousands, and serially outputting each patch individually is extremely inefficient. Summary of the Invention

[0007] This application provides a parallel decoding method and system for image change detection based on block-level joint training. It introduces a block-structured output representation dedicated to change detection to achieve unified end-to-end output of multi-type change patches; introduces bidirectional attention to ensure geometric consistency of prediction and improve change detection accuracy; and combines parallel prediction to improve inference speed. It effectively solves the technical problems of fragmented patches, single change type and low output efficiency in existing remote sensing change detection technologies.

[0008] Firstly, this application provides a parallel decoding method for image change detection based on block-level joint training, including:

[0009] In the constructed joint coding module, joint feature encoding is performed on dual-temporal image samples to obtain joint features, and multimodal processing is performed in combination with the query vector sequence of the text to obtain condition vectors;

[0010] Using the conditional vector as input, the language decoder generates a block sequence unit by unit using an autoregressive approach. After encoding using the block sequence organization rules, a change block sequence is obtained, which includes change blocks and end blocks that characterize image changes.

[0011] A dual-stream joint training mechanism is constructed based on the next unit prediction stream and the parallel prediction stream to train the language decoder;

[0012] In the next unit prediction stream training, the unit-by-unit prediction capability is trained based on the condition vector and the corresponding real block sequence, and in the parallel prediction stream training, the parallel prediction capability is trained based on the condition vector and the changed block sequence to obtain the two-stream loss.

[0013] Based on the characteristics of block-structured output, a geometric consistency loss is designed, and combined with the two-stream loss, the total loss is determined to optimize the language decoder.

[0014] The trained language decoder is used to infer the acquired target dual-temporal images and output vector files.

[0015] Secondly, this application provides an image change detection parallel decoding system, comprising:

[0016] The joint encoding module is used to perform joint feature encoding on dual-temporal image samples to obtain joint features, and to perform multimodal processing in combination with the query vector sequence of the text to obtain condition vectors;

[0017] The change detection block structured output module is used to generate a block sequence unit by unit in the language decoder using the condition vector as input and the block sequence organization rules after encoding to obtain a change block sequence, which includes change blocks and end blocks that characterize image changes.

[0018] A dual-stream joint training module is used to construct a dual-stream joint training mechanism based on the next-unit prediction stream and the parallel prediction stream to train the language decoder. In the training of the next-unit prediction stream, the unit-by-unit prediction capability is trained based on the condition vector and the corresponding real block sequence, and in the training of the parallel prediction stream, the parallel prediction capability is trained based on the condition vector and the changed block sequence to obtain the dual-stream loss.

[0019] The geometric consistency loss calculation module is used to design the geometric consistency loss based on the characteristics of block-structured output, combine it with the two-stream loss to determine the total loss, and optimize the language decoder.

[0020] The variable patch vectorization output module is used to perform inference on the acquired target dual-temporal image using the trained language decoder and output a vector file.

[0021] In summary, this application constructs a joint feature encoding module to perform joint feature encoding on the input dual-temporal images, obtaining joint features. Simultaneously, it combines the query vector sequence from the text query to form a condition vector. Using the condition vector as context input, the language decoder generates block sequences unit by unit using an autoregressive approach. These block sequences are then encoded according to organization rules to obtain a change block sequence, achieving unified end-to-end output of multi-type change patches. Subsequently, the next-unit prediction stream and the parallel prediction stream are combined to establish a dual-stream joint training mechanism. This dual-stream joint training method trains both the unit-by-unit prediction capability and the parallel prediction capability of the language decoder, simultaneously balancing inference speed and detection accuracy. Based on the dual-stream loss, geometric consistency loss is further incorporated to optimize the language decoder. In practical applications, the language decoder performs parallel / unit-by-unit inference on the acquired target dual-temporal images, outputting vector files obtained from change detection. This solves the technical problems of fragmented patches, single change types, and low output efficiency in existing remote sensing change detection technologies. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A technical roadmap for a parallel decoding method for remote sensing image change detection based on block-level joint training, provided for embodiments of this application;

[0025] Figure 2 This is a structural block diagram of a parallel decoding system for remote sensing image change detection provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] To facilitate understanding of the embodiments of this application, further explanations and descriptions will be provided below in conjunction with the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application.

[0028] The image change detection parallel decoding method based on block-level joint training provided in this application embodiment may specifically include the following steps:

[0029] Step 110: In the constructed joint coding module, joint feature coding is performed on the dual-temporal image samples to obtain joint features, and multimodal processing is performed in combination with the query vector sequence of the text to obtain condition vectors.

[0030] Reference Figure 1 As shown, this embodiment can construct a joint coding module for dual-temporal image features. This joint coding module can consist of a visual encoder for feature extraction, a cross-attention fusion layer, and a text encoder. Each sample in the dual-temporal image sample consists of a pair of remote sensing images, namely, a previous temporal image and a subsequent temporal image, referred to as previous and subsequent temporal images.

[0031] In the joint encoding module, feature extraction is performed on the input dual-temporal image samples to obtain joint features containing features from both preceding and following temporal images, which serve as the encoding result. Simultaneously, the user-input text query is processed to form a query vector sequence. The encoding result and the query vector sequence are then fused using multimodal methods to form a conditional vector driving the subsequent change block decoding.

[0032] In an optional embodiment, the above-described joint feature encoding of dual-temporal image samples to obtain joint features may include: inputting the dual-temporal image samples into a weight-sharing visual encoder; using the visual encoder to divide each image in the dual-temporal image samples into a sequence of tiles, encoding the spatial, textural, and semantic features of various objects in the images, and outputting multi-scale visual features; constructing positive and negative query relationships for the multi-scale visual features according to the preceding and following temporal phases; and capturing the relationships and differences between preceding and following temporal phases in a cross-attention fusion layer based on the positive and negative query relationships, analyzing the attention in the positive and negative directions, and obtaining joint features.

[0033] Combination Figure 1 The registered previous temporal images (denoted as...) can be... ) and post-temporal images (denoted as The images are input into the Moon-ViT visual encoder (based on the ViT-Large architecture, patch size=16) with shared weights. The visual encoder divides each image into a sequence of 16×16 pixels, and outputs multi-scale visual features after 12 layers of Transformer encoding.

[0034] Among them, the multi-scale visual features are composed of corresponding encoded features of the preceding and following temporal images, that is, including the preceding temporal features (denoted as...). ) and post-phase characteristics (denoted as ).in, , , The length of the tile sequence. , The spatial location, texture structure, and semantic category information of objects in the preceding and following temporal images were encoded respectively.

[0035] Subsequently, cross-attention joint encoding is performed. Existing attention fusion methods typically fuse features extracted from multiple images, which is not suitable for capturing differences between preceding and following time phases. To accurately capture the correspondence and difference information between preceding and following time phases, this application constructs a bidirectional cross-attention fusion layer. (The text then repeats the previous sentence about features from previous time phases.) For query, post-temporal features Using key and value pairs to form a positive query relationship; simultaneously, subsequent time phase characteristics. For querying, previous time phase features Using the key value, a reverse query relationship is formed, thus creating a forward and reverse query relationship.

[0036] Attention is calculated for the query relation to obtain the attention result; simultaneously, reverse attention is calculated for the inverse query relation to obtain the corresponding attention result. The attention results from both directions are added together to obtain the joint feature. :

[0037] Therefore, this application adopts bidirectional cross-attention to realize position-by-position correspondence modeling of features in the preceding and following time phases, which can explicitly establish the semantic association of the same spatial location in the preceding and following time phases, and provide richer differential representations for the subsequent determination of change types.

[0038] Optionally, the above-mentioned multimodal processing of the query vector sequence of the combined text to obtain the condition vector may include: taking the acquired change detection instruction as a text query; segmenting the text query to obtain a word sequence, and mapping the word sequence to a query vector sequence through a text encoder; performing multimodal alignment and fusion of the joint features and the query vector sequence through cross-attention to map the change region retrieval capability of the text query to obtain the condition vector, wherein the condition vector carries visual difference information of each image and the semantic intent of the query.

[0039] This embodiment constructs a text query encoding: the user's change detection command input in natural language form is used as the text query (denoted as...). For example, "Detect the changes in all cultivated land in a county in Guangdong Province from 2023 to 2024". This text query is first segmented into a sequence of tokens by the BPE tokenizer, and then mapped into a query vector sequence by the Sentence-BERT text encoder (denoted as...). , ).

[0040] in, To query the length of the tokens. Text query. It carries the user-specified constraints, such as the detection area, time range, and types of changes of interest.

[0041] After obtaining the query vector sequence, multimodal conditional vectors can be generated by combining the joint features with the query vector sequence. Specifically, the joint features... With query vector sequence Multimodal alignment and fusion are performed through a single layer of cross-attention to optimize the query vector. For querying and combining features As the key, this enables text queries to retrieve information about related areas of change from visual features, resulting in a condition vector (denoted as ). , ):

[0042] Understandable, conditional vector Simultaneously carrying visual difference information from dual-temporal images and the semantic intent of the user query, it can serve as the initial context input for the subsequent language decoder, guiding the decoder to autoregressively generate a sequence of change blocks that meet the requirements, and a condition vector. It directly drives the generation process of subsequent change blocks.

[0043] Step 120: Using the conditional vector as input, the language decoder generates a block sequence unit by unit using an autoregressive method. After encoding using the block sequence organization rules, a variable block sequence is obtained.

[0044] The change block sequence includes change blocks and end blocks that characterize image changes.

[0045] The unit refers to the token, which can be understood as the smallest computational unit / unit for the model to process text. In this embodiment, a language decoder is constructed based on the block-structured output representation for change detection and its token sequence encoding rules. This language decoder can adopt the Transformer language decoder to transform the traditional pixel-level change mask into a discrete token (unit) sequence that can be generated autoregressively by the language decoder.

[0046] The conditional vector is input to the language decoder. During the training phase, the language decoder generates a block sequence for each token using an autoregressive approach. Iteratively predicts the next token using the probability distribution of the already predicted tokens. Before prediction, the functional block token sequence can be defined. Based on the possible changes that may occur in the preceding and following temporal images, functional blocks can be divided into five categories: newly added blocks, disappearing blocks, transition blocks, unchanged blocks, and ending blocks.

[0047] By predicting the probability distribution of the first token, the block header type is determined. Subsequent blocks are then continuously predicted until the final block is generated, thus obtaining the sequence of changed blocks.

[0048] Understandably, the change blocks in the change block sequence that represent image changes may include new blocks, disappearing blocks, transformation blocks, etc. Based on the definition of the above functional blocks, the change block sequence may also include invariant blocks.

[0049] In this specific implementation, the language decoder uses an autoregressive approach to generate block sequences unit by unit. After encoding using block sequence organization rules, a change block sequence is obtained. Specifically, this can include: constructing a language decoder based on the block structured output representation and sequence encoding rules for change detection, the language decoder including functional blocks related to change detection; defining block sequence organization rules for the complete output of the change detection scenario, limiting the spatial order of the unit block sequences for change detection, and updating the language decoder; in the language decoder, using the condition vector as the initial context, generating block sequences unit by unit using an autoregressive approach; executing the unit-by-unit block sequence generation, predicting the probability distribution of the initial unit block, sampling the block header type, predicting the load unit of the initial unit block, until the block tail marker of the initial unit block is generated, and using the initial unit block as the reference block; using the reference block and the condition vector as conditions, iteratively predicting the next unit block, until all unit blocks are predicted; and using the block sequence organization rules to spatially order the predicted unit blocks to obtain the change block sequence.

[0050] Optionally, a block-structured output representation and sequence encoding rules for change detection are defined, and a language decoder is constructed, including: defining five types of functional blocks based on the block-structured output representation of change detection to form the language decoder. The five types of functional blocks include new blocks, disappearing blocks, transformation blocks, invariant blocks, and ending blocks. Each functional block consists of a block header marker, a load unit sequence, and a block tail marker. According to the sequence encoding rules, the relevant information of land cover categories, regional change coordinate units, and encoded change areas in the new blocks are defined. Based on the definition of the new blocks, regional disappearance information is defined for the disappearing blocks according to the sequence encoding rules. Based on the definition of the new blocks, the transformation units of land cover category changes before and after encoding are defined for the transformation blocks according to the sequence encoding rules. The invariant block is defined as a change that does not meet the conditions detected within the query area. The ending block is defined as the termination of the generation of the change block sequence.

[0051] Combination Figure 1 As shown below, the change detection process will be explained in detail:

[0052] This embodiment defines five types of functional blocks, each of which consists of three parts: a block header, a payload token sequence, and a block footer. The token sequence encoding rules for each functional block are as follows:

[0053] Add a block (can be used) <add>(Token, length L=9): For example, the token sequence is <add>[category ][ ][ ][ ][ [Area S] [Perimeter P]< / add> Wherein, category c is taken from a predefined vocabulary of local land cover categories (including but not limited to buildings, cultivated land, forest land, water bodies, roads, gardens, grasslands, bare land, etc.); coordinate token (denoted as: The bounding rectangle of the changed area is represented by ) and can be encoded using a 1000-bin discretization scheme (values ​​range from [0, 1000], each bin corresponding to 0.1% of the image's normalized width / height); the area token S and perimeter token P encode the area (in m²) and perimeter (in m) of the changed area, respectively. New blocks represent ground features that were not present in the previous time phase but appeared in the later time phase.

[0054] Vanishing Block (can be used) <rm>(Token, length L=9): For example, the token sequence is <rm> [Category c][x1][y1][x2][y2][Area S][Perimeter P]< / rm> The encoding method is the same as that of the newly added block, representing ground features that existed in the previous time phase but disappeared in the later time phase (such as farmland that has been leveled but has not yet been developed).

[0055] Transformation block (can be used) <chg>(Token, length L=10): For example, the token sequence is <chg>[Previous phase category] [Post-phase category] ][ ][ ][ ][ [Area S] [Perimeter P]< / chg> Compared to newly added blocks and disappearing blocks, transition blocks additionally encode two tokens: the previous phase category c1 and the subsequent phase category c2, which are used to represent changes in land cover type (such as farmland → construction land, farmland → fishpond, etc.).

[0056] Invariant block (can be used) <unch>(Token, length L=3): token sequence is <unch> [Query Match Markers]< / unch> This indicates that no changes matching the criteria were detected within the query area.

[0057] End block (can be used) <eos>(Token, length L=3): token sequence is <eos> [Termination Marker]< / eos> This marks the end of the generation of the entire change block sequence.

[0058] Conditional vector-driven block sequence generation: This involves generating the output conditional vector... This serves as the initial context input for the Transformer language decoder. The decoder generates a block sequence token-by-token using an autoregressive approach, starting with... Given the prefix context, predict the first token (block header token). <add> / <rm> / <chg>After determining the block header type by argmax sampling based on the probability distribution of ), the process continues with... Using the generated tokens as conditions to predict subsequent payload tokens, until a block tail marker is generated, forming a change block; then... Using all generated changed blocks as conditions, continue predicting the next block header, iterating in this way until the final block is generated. <eos>until.

[0059] Block sequence organization rules: A complete output token sequence for a change detection scenario consists of several change blocks (any combination of new blocks / disappearing blocks / transformation blocks) arranged in spatial order, ending with a closing block. For example, if a scenario detects one new building, one disappearing farmland, and one transformed farmland, the output sequence would look like this: <add> ...< / add> <rm> ...< / rm> <chg> ...< / chg> <eos> ...< / eos> Total number of change blocks The total token sequence length is determined by the number of actual changes in the scene's polygons. .

[0060] Therefore, this embodiment directly drives the output of the change block sequence based on the conditional vector through the autoregressive generation mechanism of the decoder, realizing end-to-end mapping from bi-temporal images and text queries to structured change block sequences.

[0061] Step 130: Construct a dual-stream joint training mechanism based on the next unit prediction stream and the parallel prediction stream to train the language decoder.

[0062] Among them, the next-to-token prediction stream refers to the NTP stream (Next-Token Prediction); the parallel prediction stream refers to the MTP block stream (Multi-Token Prediction).

[0063] This embodiment performs dual-stream (NTP stream and MTP block stream) joint training on the language decoder, enabling the model to simultaneously possess token-level prediction accuracy and block-level parallel inference capability.

[0064] Step 140: In the next unit prediction stream training, the unit-by-unit prediction capability is trained based on the condition vector and the corresponding real block sequence, and in the parallel prediction stream training, the parallel prediction capability is trained based on the condition vector and the changed block sequence to obtain the two-stream loss.

[0065] The NTP stream trains the language decoder's token-by-token prediction capability using a given conditional vector F and the real block sequence, establishing the NTP loss function and optimizing the model. The MTP block stream trains the language decoder's parallel prediction capability using varying blocks as units, establishing the MTP loss function. Combining the NTP and MTP loss functions yields the dual-stream loss.

[0066] This embodiment trains unit-by-unit prediction capability based on the condition vector and the corresponding real block sequence, which may include: performing training in a teacher-mandated manner, and at each time step, using the generated prefix real blocks to predict the probability distribution of the next real block based on the condition vector and the corresponding real block sequence, and constructing a first loss function.

[0067] In this embodiment, the above-mentioned training of parallel prediction capability based on the condition vector and the change block sequence to obtain the two-stream loss may include: performing parallel prediction on each change block in the change block sequence as a unit; for each change block, based on the generated block header unit, using the condition vector, the block header unit, and the corresponding intra-block preceding load unit as prediction conditions; based on the prediction conditions, predicting the remaining load units in parallel to construct a second loss function; and based on the first loss function and the second loss function, defining a two-stream loss function for joint training of two streams to obtain the two-stream loss.

[0068] The following details the dual-stream training process:

[0069] Specifically, the NTP stream trains the decoder's token-by-token prediction ability using a teacher-forcing method. Given a condition vector... and the real block sequence (by composition, (Total number of tokens), NTP stream at each time step Based on the condition vector and the generated prefix token Predict the next token The probability distribution, the NTP loss function is defined as :

[0070] NTP streams guarantee the accuracy of single-token predictions, but because they need to be generated serially, one token at a time... The inference speed is relatively slow, making it difficult to meet the real-time detection needs of hundreds of changing map patches within a provincial area.

[0071] To address this, this embodiment introduces MTP block stream training, where MTP block streams are predicted in parallel on a per-block basis. For the first... A variable block (block length) After determining the block header token, the MTP stream uses a condition vector. Given the block header token and the preceding load tokens within the block, all remaining load tokens in the block are predicted in parallel. The MTP loss function is defined as follows: :

[0072] MTP block streaming parallelizes the generation of tokens within a block, allowing each change block to output all payload tokens simultaneously with only one forward propagation during inference. For blocks containing... In a scenario with multiple change blocks, MTP inference only requires... Second-block iteration, compared to NTP This token-level serial iteration improves efficiency by approximately 5 times.

[0073] Therefore, training with NTP streams alone results in high token prediction accuracy but slow inference speed (serial generation of T steps); training with MTP streams alone results in fast inference speed but insufficient modeling of inter-token dependencies within blocks, leading to inconsistencies between predictions of geometric attributes such as area and perimeter and coordinate tokens. To address this, this application employs joint training with NTP streams and MTP block streams—MTP streams ensure block-level parallelism during inference, while NTP streams provide fine-grained token-level supervision as regularization constraints, compensating for the shortcomings of MTP streams in modeling inter-token dependencies within blocks. The joint optimization of these two loss methods allows the model to maintain prediction accuracy while ensuring inference efficiency.

[0074] Step 150: Based on the characteristics of block-structured output, design geometric consistency loss, combine it with the two-stream loss, determine the total loss, and optimize the language decoder.

[0075] Specifically, this embodiment is based on the physical constraints between the different parts within the variable block. It designs a geometric consistency loss for the characteristics of block-structured output, combines the geometric consistency loss with the two-stream loss, establishes a total loss function, outputs the total loss, and optimizes the language decoder.

[0076] In the specific implementation, based on the characteristics of block-structured output, a geometrically consistent loss is designed. Combined with the two-stream loss, the total loss is determined. This can include: determining the characteristics of block-structured output based on the physical constraints between coordinate units, area units, and perimeter units within the variable block; constraining the coordinate units, area units, and perimeter units within the variable block to the same block according to the characteristics of block-structured output, and introducing normalized forms and explicit constraints to construct a geometrically consistent loss function and output the geometrically consistent loss; weighting the two-stream loss and the geometrically consistent loss to construct a total loss function and output the total loss.

[0077] Specifically, there are physical constraints between the coordinate tokens, area tokens, and perimeter tokens within the change block. That is, the area of ​​the bounding rectangle determined by the coordinate frame should be consistent with the predicted value of the area token, and the perimeter of the bounding rectangle should be consistent with the predicted value of the perimeter token. If only cross-entropy loss is used to train each token, the model may learn statistical correlation rather than physical consistency. For example, if the predicted coordinate frame area is 100m², but the area token outputs 500m², this inconsistency will lead to errors in the geometric properties of the vectorized output. Therefore, based on this physical constraint, this application clarifies the characteristics of the structured output and designs a geometric consistency loss to address these characteristics. :

[0078] in, The predicted value of the area token within the block. The predicted value of the perimeter token. and All are calculated from the block coordinate token, and , .

[0079] This geometric consistency loss is particularly applicable in this application because: ① The coordinate tokens and area / perimeter tokens of the variable block are located within the same block, which facilitates joint constraints at the block level, a structural advantage that traditional pixel-level methods do not possess; ② The normalized form (divided by) ) This allows the loss to adapt to areas of varying scales (e.g., from small illegal structures of tens of square meters to large buildings of tens of thousands of square meters), ensuring training stability; ③ Explicit constraints improve the area measurement accuracy from 91.5% (without The percentage increased to 96.3% (with) This meets the geometric accuracy requirements of natural resource monitoring operations.

[0080] Finally, a total loss function is constructed by weighted summation of NTP loss, MTP loss, and geometric consistency loss. :

[0081] in, The preferred weighting coefficients for geometric consistency loss are... .

[0082] In the training phase, this application integrates LEVIR-CD (building changes, 637 training image pairs), WHU-CD (building changes, 606 training image pairs), CLCD (farmland changes, 3794 training image pairs), and SYSU-CD (comprehensive changes, 20000 training image pairs), and uniformly labels them in a block-structured format. A two-stage training strategy is adopted: Phase 1 involves 30,000 steps of mixed training (learning rate 4×10⁻). 5 (32 A100 GPUs), Phase 2 enhanced farmland change data for 5000 steps (learning rate 1×10⁻) 5 (This is to adapt to the application needs of monitoring the non-agricultural use of arable land).

[0083] Step 160: Use the trained language decoder to infer the acquired target dual-temporal image and output a vector file.

[0084] The trained model is used to infer new dual-temporal images. MTP parallel decoding is used to generate change block sequences. After format verification, coordinate transformation and vectorization post-processing, the output is a GeoJSON vector file that can be directly used in business systems.

[0085] In practical processing, the trained language decoder is used to infer the acquired target bi-temporal imagery and output a vector file. Specifically, this may include: performing joint feature encoding on the target bi-temporal imagery in the trained language decoder to obtain a target condition vector, and performing parallel decoding based on the target condition vector to predict the target change block sequence; performing two-fold verification: performing format integrity verification on each target change block in the target change block sequence, analyzing the pairing of block header and block tail markers, and analyzing the consistency between the number of payload units and the length of the corresponding target change block type to obtain a first verification result; performing spatial rationality verification on each target change block, analyzing coordinate constraints and relative area deviations to obtain a second verification result; when the first verification result and / or the second verification result indicate that the target change block has failed verification, performing backtracking processing, performing unit-by-unit decoding based on the target condition vector to update the target change block sequence; and performing coordinate transformation and vectorization processing on the target change block sequence to output a vector file.

[0086] Specifically, for the input target dual-temporal image, inference is performed by MTP parallel decoding, and the MTP mode is used to generate a change block sequence during inference. First, the joint encoding module of the dual-temporal image features to be detected is encoded into a target condition vector, and the decoder performs the following iterations with the target condition vector as the initial context: (1) Predict the type of the first block header token ( <add> / <rm> / <chg> / <unch> / <eos>(2) If the block header is a variable block () probability distribution, the block header type is determined by argmax sampling; <add> / <rm> / <chg>Then, using the target condition vector and the block head token as conditions, predict all load tokens (category, coordinates, area, perimeter) and block tail tokens of the block in parallel. One forward propagation completes the generation of a complete change block; (3) Repeat steps (1) and (2) to continue generating the next change block until the block head is predicted as <eos>Decoding terminates when a block ends. For a scenario with N changing patches, only N+1 block-level iterations are needed to generate all changing blocks, far fewer than the number of iterations required for token-level serial decoding.

[0087] Next, block-level format verification and NTP rollback decoding are performed. Two verifications are executed for each changed block generated in parallel by MTP: (a) Format integrity verification, including verifying that the block header and block footer tags are correctly matched (e.g., ...). <add> and< / add> (a) Pairing), and the number of load tokens is consistent with the specified length of the block type, to obtain the first verification result; (b) Spatial rationality verification, including verifying whether the coordinate tokens meet the requirements. , The constraints are: coordinate values ​​∈ [0, 1000], and the area token S is related to the area of ​​the coordinate frame. relative deviation The second verification result is obtained.

[0088] Understandably, when the block header and block tail markers are correctly matched, and the number of load tokens matches the specified length for this block type, the first verification result is a pass; when the block header and block tail markers are incorrectly matched, and / or the number of load tokens does not match the specified length for this block type, the first verification result is a fail. When the coordinate tokens and coordinate values ​​meet the constraints, and the relative deviation between the area token and the coordinate frame area meets the requirements, the second verification result is a pass; otherwise, if any item fails to meet the constraints or requirements, the second verification result is a fail.

[0089] For any changed blocks that fail the verification, the process reverts to the NTP stream for token-by-token decoding. The token-level precision of the NTP stream ensures output reliability, thereby obtaining the target changed block sequence. During token-by-token decoding, the changed blocks that fail the verification can be decoded individually, or all changed blocks can be decoded; this embodiment does not impose any restrictions on this.

[0090] If all changed blocks pass the verification, there is no need to fall back to the NTP stream for re-decoding; the current target changed block sequence can be directly used as the final sequence.

[0091] Therefore, this embodiment achieves a balance between inference efficiency and output quality through the synergy of MTP fast decoding and NTP precise backoff.

[0092] Finally, coordinate transformation and vectorization post-processing are performed to convert the verified target change block sequence into geographic vector data: (1) Normalized coordinates to pixel coordinates, including , (2) Convert pixel coordinates to geographic coordinates by using the geographic transformation parameters of the image (six parameters of affine transformation under the CGCS2000 coordinate system); (3) Merge areas of the same type of change by merging adjacent areas of the same type of change (center distance < 50 pixels) to eliminate the fragmentation of the map due to image block processing; (4) Output GeoJSON vector file, each feature contains attribute fields such as geometry (boundary of change area), change_type (addition / disappearance / transformation), class_before (previous phase category), class_after (after phase category), area (area), perimeter (perimeter), which can be directly imported into the "One Map" system for natural resource survey and monitoring.

[0093] Taking the monitoring of farmland non-agriculturalization in a county in Guangdong Province (approximately 800 km²) as an example, the test data were two Gaofen-2 multispectral fusion images (spatial resolution 0.8 m, true color) from 2023 and 2024. The county includes urban expansion areas, urban-rural fringe areas, and major agricultural production areas, with diverse types of farmland non-agriculturalization changes.

[0094] (1) Data preparation: Radiometric calibration, FLAASH atmospheric correction, panchromatic multispectral fusion (Gram-Schmidt method) and geometric registration (registration error within 0.5 pixels) were performed on the two images respectively.

[0095] (2) Change detection: Input the natural language command "Detect all changes in cultivated land from 2023 to 2024", and the model outputs a sequence of change blocks including newly added blocks, disappeared blocks and transformed blocks.

[0096] (3) Results statistics: A total of 142 changed patches were detected, including 68 newly added construction land (farmland → buildings), 41 farmland disappearances (farmland was leveled), and 33 farmland conversions (farmland → fishponds / orchards, etc.). The reasoning time was about 12 minutes (including density analysis and post-processing).

[0097] (4) Accuracy verification: The accuracy performance data can be found in Table 1 below: ; Table 1

[0098] (5) Manual verification: 30% of the changed map patches (43 patches) were randomly selected for on-site verification. 39 patches were confirmed to be real changes (accuracy rate 91.2%), of which the accuracy rate for newly added buildings was 94.1%, the accuracy rate for the disappearance of cultivated land was 87.5%, and the accuracy rate for the conversion of cultivated land was 90.0%.

[0099] In summary, existing technologies suffer from shortcomings at the output level, including the inability to simultaneously output the bounding box, change type (addition / disappearance / transformation), and geometric attributes (area, perimeter) of changed regions in a single inference iteration. This necessitates multi-stage cascaded processing (change detection → change classification → area calculation → vectorization), resulting in low efficiency and error accumulation. In provincial-level farmland monitoring, the number of changed patches can reach thousands, making serial output of each patch extremely inefficient. This application proposes a block-level joint training-based parallel decoding method for change detection. First, a joint encoding module is constructed, consisting of a visual encoder, cross-attention fusion, and a text encoder. This module performs joint feature encoding on the input dual-temporal images, obtaining joint features composed of features from preceding and following temporal phases. Simultaneously, a query vector sequence is encoded from the input text query. The joint features and query vectors are combined to form a condition vector. A language decoder comprising five functional blocks is established. Using the condition vector as input, during the training phase, an autoregressive approach is used to generate block sequences unit by unit to predict the change block sequence. By combining NTP and MTP block streams, a dual-stream training mechanism is constructed. The NTP stream trains the language encoder's unit-wise prediction capability based on conditional vectors and ground truth block sequences, while the MTP block stream trains the language encoder's parallel prediction capability based on conditional vectors and change block sequences. This allows for a balance between accuracy and efficiency in end-to-end prediction. Furthermore, a geometric consistency loss is designed to optimize the language decoder, ensuring consistency of predicted geometrically relevant attributes. Finally, in practical applications, the trained model is used for inference on new bi-temporal images. MTP parallel decoding is employed to generate change block sequences, which are then format-validated. If format validation fails, inference is performed back using the NTP stream until all change blocks pass validation. The validated change blocks undergo coordinate transformation and vectorization, outputting a GeoJSON vector file directly usable in business systems. Therefore, compared to existing technologies, this application offers the following advantages:

[0100] ① Achieve end-to-end multi-type change output. Specifically, design a dedicated block-structured output representation for change detection, including five functional blocks: addition blocks, disappearance blocks, transformation blocks, invariant blocks, and end blocks. This enables unified end-to-end output of the three change types: addition, disappearance, and transformation. A single inference can simultaneously output the regions and their geometric attributes of the three change types (addition, disappearance, and transformation) without requiring multi-stage cascaded processing. The fragmentation rate is reduced from 15-25% for pixel-level methods to <3%.

[0101] ② Achieve efficient parallel decoding. Specifically, each changed region is predicted in parallel as an atomic block. In a test scenario containing 142 changed patches, the inference time is about 12 minutes (vs. traditional pixel-level change detection + post-processing 65 minutes), which is 5.4 times more efficient.

[0102] ③ Ensure the consistency of geometric attributes. Specifically, bidirectional attention within the block ensures the geometric consistency between area and perimeter prediction and coordinate prediction. The accuracy of area measurement for changing patches is better than 95%, reaching 96.3%.

[0103] ④ It meets the requirements of natural resource monitoring operations. The output GeoJSON / SHP file contains the boundary of the changed map patch, the type of change, the previous and current land cover categories, area, and perimeter. It can be directly imported into the "One Map" system for natural resource survey and monitoring, and meets the data format requirements for dynamic monitoring of territorial spatial planning.

[0104] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should know that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps may be performed in other orders or simultaneously.

[0105] like Figure 2 As shown in the figure, this application embodiment also provides an image change detection parallel decoding system 200, including:

[0106] The joint encoding module 210 is used to perform joint feature encoding on dual-temporal image samples to obtain joint features, and to perform multimodal processing in combination with the query vector sequence of the text to obtain a condition vector;

[0107] The change detection block structured output module 220 is used to generate a block sequence unit by unit in the language decoder using an autoregressive method with the condition vector as input, and after encoding using the block sequence organization rules, a change block sequence is obtained. The change block sequence includes change blocks and end blocks that characterize image changes.

[0108] The dual-stream joint training module 230 is used to construct a dual-stream joint training mechanism based on the next unit prediction stream and the parallel prediction stream to train the language decoder. In the training of the next unit prediction stream, the unit-by-unit prediction capability is trained based on the condition vector and the corresponding real block sequence, and in the training of the parallel prediction stream, the parallel prediction capability is trained based on the condition vector and the changed block sequence to obtain the dual-stream loss.

[0109] The geometric consistency loss calculation module 240 is used to design the geometric consistency loss based on the characteristics of the block structured output, combine it with the two-stream loss to determine the total loss, and optimize the language decoder.

[0110] The variable patch vectorization output module 250 is used to infer the acquired target dual-temporal image using the trained language decoder and output a vector file.

[0111] Among them, the joint encoding module mentioned above, also known as the dual-temporal feature joint encoding module, mainly includes a shared visual encoder and cross-attention fusion; the change detection block structured output module is mainly used for the management of five types of functional block templates; the change patch vectorization output module is used to realize coordinate transformation, merging of the same type, and GeoJSON output.

[0112] For example, the system deployment environment could be: Ubuntu 20.04, a single A100 GPU, supporting automated batch image processing.

[0113] It should be noted that the image change detection parallel decoding system provided in the embodiments of this application can execute the image change detection parallel decoding method based on block-level joint training provided in any embodiment of this application, and has the corresponding functions and beneficial effects of the execution method.

[0114] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0115] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.< / eos> < / chg> < / rm> < / add> < / eos> < / unch> < / chg> < / rm> < / add> < / eos> < / chg> < / rm> < / add> < / eos> < / unch> < / chg> < / rm> < / add>

Claims

1. A parallel decoding method for image change detection based on block-level joint training, characterized in that, include: In the constructed joint coding module, joint feature encoding is performed on dual-temporal image samples to obtain joint features, and multimodal processing is performed in combination with the query vector sequence of the text to obtain condition vectors; Using the conditional vector as input, the language decoder generates a block sequence unit by unit using an autoregressive approach. After encoding using the block sequence organization rules, a change block sequence is obtained, which includes change blocks and end blocks that characterize image changes. A dual-stream joint training mechanism is constructed based on the next unit prediction stream and the parallel prediction stream to train the language decoder; In the next unit prediction stream training, the unit-by-unit prediction capability is trained based on the condition vector and the corresponding real block sequence, and in the parallel prediction stream training, the parallel prediction capability is trained based on the condition vector and the changed block sequence to obtain the two-stream loss. Based on the characteristics of block-structured output, a geometric consistency loss is designed, and combined with the two-stream loss, the total loss is determined to optimize the language decoder. The trained language decoder is used to infer the acquired target dual-temporal images and output vector files.

2. The method according to claim 1, characterized in that, Joint feature encoding is performed on dual-temporal image samples to obtain joint features, including: Input the dual-temporal image samples into a weight-shared visual encoder; Using the visual encoder, each image in the dual-temporal image sample is divided into a sequence of tiles, the spatial, textural and semantic features of each object in the image are encoded, and multi-scale visual features are output. For the aforementioned multi-scale visual features, construct positive and negative query relationships according to the preceding and following time phases; Based on the positive and negative query relationship, the relationship and difference between the preceding and following time phases are captured in the cross-attention fusion layer, and the attention in the positive and negative directions is analyzed to obtain joint features.

3. The method according to claim 1, characterized in that, By combining the query vector sequence of the text with multimodal processing, a condition vector is obtained, including: The acquired change detection instructions are used as text queries; The text query is segmented into words to obtain a word sequence, and the word sequence is mapped into a query vector sequence by a text encoder; The joint features and the query vector sequence are fused using multimodal alignment through cross-attention to map the change region retrieval capability of the text query, resulting in a condition vector. The condition vector carries visual difference information of each image and the semantic intent of the query.

4. The method according to claim 1, characterized in that, In the language decoder, a unit-by-unit block sequence is generated using an autoregressive approach. After encoding using the block sequence organization rules, a variable block sequence is obtained, including: A language decoder is constructed based on the block-structured output representation and sequence encoding rules of change detection. The language decoder includes functional blocks related to change detection. For the complete output of the change detection scenario, define block sequence organization rules, limit the spatial order of the unit block sequence for change detection, and update the language decoder; In the language decoder, the conditional vector is used as the initial context, and a block sequence is generated unit by unit using an autoregressive approach; Execute a unit-by-unit block generation sequence, predict the probability distribution of the initial unit block, sample the block header type, predict the load unit of the initial unit block, until the block tail marker of the initial unit block is generated, and use the initial unit block as the reference block; Using the reference block and the conditional vector as conditions, iteratively predict the next unit block until all unit blocks are predicted. The predicted unit blocks are spatially sorted using the block sequence organization rules to obtain the change block sequence.

5. The method according to claim 4, characterized in that, Define the block-structured output representation and sequence encoding rules for change detection, and construct a language decoder, including: The block-structured output representation based on change detection defines five types of functional blocks to form a language decoder. The five types of functional blocks include new blocks, disappearing blocks, transition blocks, invariant blocks, and end blocks. Each functional block consists of a block header marker, a payload unit sequence, and a block tail marker. According to the sequence coding rules, define the land feature categories, regional change coordinate units, and relevant information about the coded change areas in the newly added block; Based on the sequence encoding rules, and on the basis of the definition of the newly added block, region disappearance information is defined for the disappearing block; Based on the sequence coding rules, and on the basis of the definition of the newly added block, a transformation unit for the change of land cover category before and after coding is defined for the transformation block; The invariant block is defined as a change that does not meet the conditions detected within the query area; the end block is defined as the termination of the generation of the change block sequence.

6. The method according to claim 1, characterized in that, Training unit-by-unit prediction capability based on the conditional vector and the corresponding real block sequence includes: Training is performed in a teacher-mandated manner. At each time step, based on the conditional vector and the corresponding real block sequence, the probability distribution of the next real block is predicted using the generated prefix real blocks, and a first loss function is constructed.

7. The method according to claim 6, characterized in that, Parallel prediction capability is trained based on the conditional vector and the changed block sequence, resulting in a two-stream loss, including: Parallel prediction is performed on each change block in the change block sequence. For each change block, the condition vector, the block head unit, and the corresponding intra-block preceding load unit are used as prediction conditions based on the generated block head unit. Based on the prediction conditions, the remaining load units are predicted in parallel, and a second loss function is constructed. Based on the first loss function and the second loss function, a dual-stream loss function is defined for dual-stream joint training, and the dual-stream loss is obtained.

8. The method according to claim 1, characterized in that, Based on the characteristics of block-structured output, a geometrically consistent loss is designed. Combined with the aforementioned two-stream loss, the total loss is determined, including: Based on the physical constraints between coordinate units, area units, and perimeter units within the variable block, the characteristics of the block-structured output are determined. Based on the characteristics of block-structured output, the coordinate units, area units, and perimeter units within the variable block are constrained to the same block, and a normalized form and explicit constraints are introduced to construct a geometrically consistent loss function and output the geometrically consistent loss. The two-stream loss and the geometrically consistent loss are weighted to construct a total loss function, and the total loss is output.

9. The method according to claim 1, characterized in that, The trained language decoder is used to infer the acquired target bi-temporal images, outputting a vector file, including: In the trained language decoder, the target dual-temporal image is subjected to joint feature encoding to obtain the target condition vector, and parallel decoding is performed based on the target condition vector to predict the target change block sequence. Perform double verification: perform format integrity verification on each target change block in the target change block sequence, analyze the pairing of block header and block tail markers, and analyze the consistency between the number of load units and the length of the corresponding target change block type to obtain the first verification result; For each target change block, spatial rationality is verified, coordinate constraints and relative area deviation are analyzed, and a second verification result is obtained. When the first verification result and / or the second verification result indicate that the target change block has failed the verification, a rollback process is performed, and unit-by-unit decoding is performed based on the target condition vector to update the target change block sequence; The target change block sequence is subjected to coordinate transformation and vectorization processing to output a vector file.

10. A parallel decoding system for image change detection, characterized in that, include: The joint encoding module is used to perform joint feature encoding on dual-temporal image samples to obtain joint features, and to perform multimodal processing in combination with the query vector sequence of the text to obtain condition vectors; The change detection block structured output module is used to generate a block sequence unit by unit in the language decoder using the condition vector as input and the block sequence organization rules after encoding to obtain a change block sequence, which includes change blocks and end blocks that characterize image changes. A dual-stream joint training module is used to construct a dual-stream joint training mechanism based on the next unit prediction stream and the parallel prediction stream to train the language decoder; In the next unit prediction stream training, the unit-by-unit prediction capability is trained based on the condition vector and the corresponding real block sequence, and in the parallel prediction stream training, the parallel prediction capability is trained based on the condition vector and the changed block sequence to obtain the two-stream loss. The geometric consistency loss calculation module is used to design the geometric consistency loss based on the characteristics of block-structured output, combine it with the two-stream loss to determine the total loss, and optimize the language decoder. The variable patch vectorization output module is used to perform inference on the acquired target dual-temporal image using the trained language decoder and output a vector file.

Images