Parallel differentiation processing method and device for image change detection
Through parallel differentiated processing methods, twin networks and feature aggregation technology are used to solve the problem of pseudo-change interference in change detection, achieving higher accuracy and robustness, and are suitable for change detection in complex scenarios.
Patent Information
- Application Number
- CN202411169099.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-29
AI Technical Summary
Existing change detection technologies are difficult to effectively distinguish between pseudo-changes and real changes, especially in complex scenarios where occlusion, shadows and background interference are present, resulting in insufficient detection accuracy and robustness.
Using parallel differentiation processing method, the twin features of the dual-time image are extracted through the twin network, combined with bottleneck structure, multi-branch attention mechanism and Transformer block, local-global feature aggregation is carried out, information representation ability is enhanced, and deep dependencies within the image are captured. .
It improves the accuracy and robustness of change detection, can more accurately identify real changes and eliminate false changes interference, and improves the speed and accuracy of detection.
Smart Images

Figure CN119048559A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to a parallel differentiation image change detection method and device. The method utilizes parallel processing technology to detect and analyze changes in an image, thereby improving the speed and accuracy of change detection. Background Art
[0002] Change detection is an important image processing technology. Its application depth and breadth continue to deepen. It is currently widely used in many fields such as remote sensing monitoring, urban planning, and environmental monitoring. Its main purpose is to identify and extract the change areas and change information in images of different periods by analyzing multi-temporal image data. In remote sensing monitoring, by comparing satellite images taken at different times, changes in the surface can be discovered in a timely manner, thus providing a scientific basis for environmental protection and natural resource management. In urban planning, change detection technology can help city managers understand urban expansion, infrastructure construction, and land use changes, and support scientific and reasonable urban planning decisions. In the field of environmental monitoring, change detection technology can be used to monitor pollution sources, assess the impact of natural disasters, track climate change, etc., providing strong support for environmental protection work. In short, exploring change detection technology has far-reaching significance in promoting sustainable social and economic development, maintaining national security, and responding to global environmental changes.
[0003] However, in the process of change detection, non-essential changes such as illumination, viewing angle, and seasons can also cause varying degrees of interference to the image. These pseudo-changes are not actual changes in the scene. Therefore, effectively distinguishing pseudo-changes from real changes is a major challenge for change detection. In addition, in complex scenes, the objects in the image may be obstructed, shadowed, and interfered with by other backgrounds, making change detection even more difficult. Whether these complex scenes can be effectively processed and the robustness of detection can be improved is an urgent problem to be solved. At present, most methods for processing change detection use features of different scales for fusion processing, but there is a lack of "one-size-fits-all" response strategies for these features, and the intrinsic properties of these features have not been fully explored, which limits the further development of change detection technology.
[0004] Therefore, in order to solve the above difficulties, we designed a parallel differentiation processing method to improve the ability of image change detection, aiming to better explore the deep dependencies within the image. This method can not only more accurately identify the real changes in the image, but also effectively eliminate the interference of pseudo changes, thereby improving the accuracy and robustness of change detection, and providing a more accurate response strategy for surface detection, urban monitoring, and environmental protection. Summary of the invention
[0005] In view of the above-mentioned technical problems, the present invention aims to propose a parallel difference processing method and device for image change detection to solve the difficulties and challenges mentioned in the above background technology part.
[0006] In a first aspect, the present invention provides a parallel difference processing method for image change detection, comprising the following steps:
[0007] S1, extract features from the input dual-time image to obtain pairs of twin features of different scales, and compare the twin features in the twin network to obtain difference features, which together with the twin features constitute the input stream of the network;
[0008] S2, for low-level input streams, a bottleneck structure is used to form a local feature aggregator to strengthen the attention to the local information of the low-level input stream to make up for the lack of detailed information;
[0009] S3, for the medium-level input stream, a multi-branch attention strategy is used to enhance the attention to twin features and difference features. The coordinated use of the local-global attention mechanism not only improves the attention to the feature information of its own level, but also further connects the low-level and high-level information for more precise adjustment;
[0010] S4, for high-level input streams, Transformer blocks are used to form a global aggregator to enhance the attention to global information in high-level information to obtain a more comprehensive information expression.
[0011] S5, the above 2-4 are performed synchronously in parallel. After aggregation within different levels, the obtained aggregated information is enhanced and fused and predicted.
[0012] In a second aspect, the present invention provides a parallel difference processing device for image change detection, comprising:
[0013] The multi-level feature extraction module is configured to extract feature information at different levels from the input dual-temporal image by deepening the depth of the convolutional neural network.
[0014] The difference feature generation module is configured to match and compare the difference features at different levels to generate the difference features between the dual time features. This feature is rich in change information and can guide the change attention process.
[0015] The low-level feature aggregation module is configured as three bottleneck residual blocks with cross-connections to enhance the representation ability of local information and output low-level aggregated features.
[0016] The mid-level feature aggregation module is configured as a multi-branch attention framework to interactively process and aggregate twin feature pairs and difference information, and reconcile high-level and low-level aggregation information.
[0017] The high-level feature aggregation module is configured as interconnected Transformer blocks to extract low-frequency information from high-level information and improve global change perception.
[0018] The enhanced fusion and prediction module is configured to perform enhanced multi-granularity representation on the aggregated information at different levels, and then perform prediction processing on the enhanced features at each level respectively.
[0019] In a third aspect, the present invention provides a set of devices, including one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation method of the first aspect.
[0020] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.
[0021] Compared with the prior art, the present invention has the following beneficial effects:
[0022] (1) The present invention uses a parallel differential processing method for image change detection, which solves the problem of multi-scale feature information unification in existing image change detection methods and realizes differential processing;
[0023] (2) The present invention adopts a parallel approach to solve the problem of low serial efficiency in feature processing, and simultaneously processes feature information of different scales to improve aggregation efficiency;
[0024] (3) The present invention adopts a "one size fits all" coping strategy to improve the applicability of the detection method to features of different scales and capture dependencies in space. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 A parallel difference processing method for image change detection. DETAILED DESCRIPTION
[0026] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.
[0027] Example 1
[0028] A parallel difference processing method and device for image change detection, the method comprising the following steps:
[0029] Step 101: The feature extraction backbone network extracts features from the dual-time image. Through convolutional layers of different depths, twin feature information pairs of different levels can be obtained. The extracted twin feature pairs are matched and compared to generate multi-scale difference features containing rich change information. These difference features and twin features together constitute the input feature stream.
[0030] Step 102: Use the bottleneck block to form a local feature aggregation module to process the low-level input feature stream, enhance the focus on the local information therein, and output aggregated low-level feature information;
[0031] Step 103: Use a global attention processing method to strengthen the input feature stream at the middle level to reconcile the feature information of the low level and the high level;
[0032] Step 104: Use the Transformer block to build a global feature aggregator to process the high-level input feature stream, extract the low-frequency information therein, and enhance the fine-grainedness of the information;
[0033] Step 105: After enhancing the output multi-level aggregation features, the features are sent to the prediction module for prediction;
[0034] Example 2
[0035] Combine the following Figure 1 The scheme in Example 1 is further introduced with the specific calculation formula, as described below:
[0036] Step 201: extracting twin feature information from the dual-time image, and matching the twin features in the twin network to obtain difference features to guide the change detection process;
[0037]
[0038] in is the input bi-temporal image, is the l-th layer feature of the i-th image pair.
[0039] First, the corresponding dual-time features After the batch normalization layer, the input is normalized on each mini-batch sample, making the network more stable to input changes;
[0040] Afterwards, these feature maps are passed through the cross-attention layer to calculate the attention weights to establish connections between inputs of different temporal states. The cross-attention layer can capture the information between different temporal features and improve the performance of the model. Applying a linear transformation to the dual-temporal input can obtain K i and V i :
[0041]
[0042] Finally, we calculate K i Mapping to V i The weighted sum of and K i The similarity between them, the final score of the cross attention layer can be expressed as:
[0043]
[0044] Then it is sent to the bottleneck layer to reduce the complexity of the structure and improve the generalization ability. The overall structure of the bottleneck layer consists of three convolutional layers and ReLU activation function, and residual connection is introduced. In addition to the bottleneck layer, skip connection is also introduced to enhance information transmission to retain the original information;
[0045] Finally, layer normalization normalizes across the feature dimensions of each sample, which helps the differential feature extraction process generalize across different samples and can reduce dependence on batch size.
[0046] The entire difference feature extraction process is expressed as:
[0047]
[0048] Where DFE(.) represents the difference feature extractor.
[0049] Step 202: For low-level features They are extracted from the shallow layers of the backbone network. In order to process these low-level features more effectively, a local feature aggregator is designed, and a bottleneck block structure is used to achieve this goal. Specifically, the bottleneck block can better capture and aggregate local feature information by compressing and expanding feature dimensions. In this way, after the low-level features are processed by the bottleneck block, the local information inside them is more effectively expressed and enhanced.
[0050] Then, these low-level features processed by the bottleneck block are combined with Perform the addition operation to obtain the feature information rich in dual time. Aggregate the dual time information to obtain:
[0051]
[0052] Among them, BR(.) represents the bottleneck block and ⑥ represents the aggregation operation. This not only integrates the feature information at different time points, but also retains and strengthens the key change information.
[0053] Step 203: For mid-level features The method adopts a method of deepening comprehensive attention perception to process. The medium-level features can not only reflect the change information at different time points more comprehensively, but also comprehensively consider the global perspective, thereby improving the quality of feature representation.
[0054] First, the feature perception ability is enhanced by introducing a comprehensive attention mechanism. The comprehensive attention mechanism can adaptively allocate attention weights, allowing the network to focus on areas with more significant changes, thereby improving the feature discrimination ability.
[0055] Next, the difference features are used to guide the processing of mid-level features. The difference features are obtained by calculating the differences of features at different time points, which can effectively highlight the change information.
[0056] Then, the guided features are processed comprehensively to obtain more comprehensive representation information. This processing method can not only capture the significant change areas, but also perform feature fusion in a global perspective to improve the completeness and accuracy of feature representation.
[0057] Finally, a more comprehensive mid-level feature representation is obtained through comprehensive attention perception and differential feature guidance.
[0058] Utilize differential features To guide To obtain more comprehensive information This process can be expressed as:
[0059]
[0060] in Represents matrix multiplication.
[0061] Step 204: For high-level features The standard Transformer block is used to try to extract the low-frequency information to solve the problem of insufficient attention to low-level information. The Transformer block can effectively capture the long-range dependencies between features through its attention mechanism, thereby enhancing the representation ability of the features.
[0062] First, the dual-time features can capture the correlation information between these features globally through the attention mechanism of the Transformer block. Second, the difference feature highlights the change information by calculating the difference of features at different time points. The Transformer block can adaptively allocate attention weights when processing these difference features, thereby better capturing the characteristics of the changed area.
[0063] Next, the processed dual-time features and difference features are aggregated to obtain In order to obtain a more comprehensive and accurate high-level feature representation. This process can be expressed as:
[0064]
[0065] TB(.) represents a standard Transformer block. This can effectively improve the representation ability of high-level features, so that it can better reflect the change information in the image. It can not only capture low-frequency information, but also perform feature fusion on a global scale to improve the effect of change detection.
[0066] Step 205: Output multi-level aggregation features The change map is obtained through the predictor. First, the low-level, medium-level and high-level features are aggregated to form a comprehensive feature representation containing multi-level information. This multi-level aggregated feature not only contains rich detail information, but also integrates global change information, thus providing a comprehensive image representation.
[0067] Next, this multi-level aggregated feature is input into the predictor, which further processes and analyzes the features and finally outputs a change map.
[0068] Example 3
[0069] A parallel difference processing device for image change detection, the device comprises: a processor and a memory. The memory stores program instructions, and the processor calls the program instructions stored in the memory to enable the device to perform the following method steps:
[0070] Extract multi-scale feature information from dual-time images, perform difference extraction on dual-time features of the same scale to obtain difference features of the corresponding scales, and the obtained dual-time features and difference features together constitute the input feature stream;
[0071]
[0072] For input feature streams of different scales, a parallel differentiation processing method is used for processing. Specifically, the local feature aggregator, intermediate feature aggregator, and global feature aggregator are used to aggregate and enhance the dual-time features and difference features of the corresponding scales to compensate for the information attention deviation between each scale and enhance the information representation ability;
[0073]
[0074] Among them, LFA(.), MFA(.), and GFA(.) represent the local feature aggregator, intermediate feature aggregator, and global feature aggregator, respectively.
[0075] For the multi-scale features after aggregation After aggregating them, they are fed into the predictor for processing, and a probability image is output, in which each pixel value indicates whether the position has changed.
[0076] During the training process, the predicted probability image and the label of the original data are used to calculate the loss function, and the model is continuously optimized by minimizing the loss function. The specific steps of this process are as follows:
[0077] First, the low-level, mid-level, and high-level features are aggregated to form a comprehensive feature representation containing multi-level information.
[0078] The aggregated multi-scale features are then fed into the predictor, which further processes them and finally outputs a change map.
[0079] Finally, the predicted probability image is compared with the label of the original data to calculate the loss function. The gradient is calculated using the back propagation algorithm, and the parameters of the model are updated using the optimization algorithm to minimize the loss function.
[0080] Repeat the above steps to gradually optimize the model. In each iteration, the model adjusts the parameters according to the value of the loss function so that the difference between the predicted result and the true label gradually decreases. During the training process, the model needs to be evaluated using the validation set to ensure the generalization ability of the model. After the training is completed, the model is finally evaluated using the test set to measure its performance in actual applications.
[0081] Unless otherwise specified, the models of the components in the embodiments of the present invention are not limited, and any device that can perform the above functions may be used.
[0082] Those skilled in the art will appreciate that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
Claims
1. A parallel difference processing method and device for image change detection, characterized in that: The following steps are involved: S1, extract features from the input dual-time image to obtain pairs of twin features of different scales, and compare the twin features in the twin network to obtain difference features, which together with the twin features constitute the input stream of the network; S2, for low-level input streams, a bottleneck structure is used to form a local feature aggregator to strengthen the attention to the local information of the low-level input stream to make up for the lack of detailed information; S3, for the medium-level input stream, a multi-branch attention strategy is used to enhance the attention to twin features and difference features. The coordinated use of the local-global attention mechanism not only improves the attention to the feature information of its own level, but also further connects the low-level and high-level information for more precise adjustment; S4, for high-level input streams, Transformer blocks are used to form a global aggregator to enhance the attention to global information in high-level information to obtain a more comprehensive information expression. S5, the above 2-4 are performed synchronously in parallel. After aggregation within different levels, the obtained aggregated information is enhanced and fused and predicted.
2. The parallel difference processing method for image change detection according to claim 1, characterized in that: The feature extraction process includes extracting dual-time features and difference features. The twin feature pairs extracted by different depth convolutional neural networks are matched and compared to generate multi-scale difference features containing rich change information. These difference features and twin features together constitute the input feature stream.
3. The parallel difference processing method for image change detection according to claim 1, characterized in that: The proposed difference feature extractor is embedded in the Siamese network to process dual-temporal features of different scales respectively; First, the corresponding dual-time features After the batch normalization layer, the input is normalized on each mini-batch sample. After that, these feature maps are passed through the cross-attention layer to calculate the attention weights to establish connections between inputs of different temporal states. Applying a linear transformation on the dual-time input can obtain K i and V i , and finally calculated K i Mapping to V i The weighted sum of and K i The similarity between .
4. The parallel difference processing method for image change detection according to claim 1, characterized in that: The proposed parallel differentiation processing approach adopts different coping strategies to process input streams of different scales simultaneously.
5. The parallel difference processing method for image change detection according to claim 4, characterized in that: The bottleneck block is used to form a local feature aggregator. The low-level features processed by the bottleneck block are Perform addition operation, so The feature information rich in dual time is obtained, and the implementation method is: Where BR(.) represents the bottleneck block and ⑥ represents the aggregation operation.
6. The parallel difference processing method for image change detection according to claim 4, characterized in that: The idea of global perception attention is used to form an intermediate feature aggregator to reconcile low-level and low-level information, and use the difference features To guide To obtain more comprehensive information This process can be expressed as: in Represents matrix multiplication.
7. The parallel difference processing method for image change detection according to claim 4, characterized in that: Transformer blocks are used to form a global feature aggregator to enhance the attention to low-frequency information. The processing process can be expressed as: Where TB(.) represents a standard Transformer block.
8. The parallel difference processing method for image change detection according to claim 1, characterized in that: The aggregated features of the parallel output are sent to the predictor for processing. First, the input parallel features are aggregated to generate comprehensive feature information, which is then processed by the predictor. The predicted intermediate probability map is used to calculate the loss function to train the model.
9. A parallel difference processing device for image change detection, characterized in that: The device comprises: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to enable the device to execute the method steps described in any one of claims 1-8.
