An image change detection method for visually camouflaged targets
By combining multi-scale semantic alignment, implicit multi-scale change reconstruction, and image-level camouflage type classification modules, the problem of insufficient accuracy and type recognition in camouflage target detection is solved. This method achieves high-precision localization and type recognition of camouflage target change areas, improving detection stability and adaptability in complex scenarios.
Patent Information
- Application Number
- CN202611130587.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-28
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies for detecting camouflaged targets suffer from low detection accuracy, insufficient accuracy in reconstructing changed regions, and inadequate ability to represent change types. In particular, environmental changes and spatial misalignment have a significant impact in complex scenarios, leading to poor detection stability and accuracy.
A combined approach, consisting of a multi-scale semantic alignment module, an implicit multi-scale change reconstruction module, and an image-level camouflage type classification module, is employed to achieve the localization of change regions and type recognition of camouflage targets through semantic alignment, spatial correction, and continuous reconstruction of bi-temporal images.
It improves the detection accuracy of changing regions of camouflaged targets and the ability to identify change types, reduces false detections and missed detections, enhances the model's environmental adaptability and detection stability in complex scenarios, and achieves joint output of changing region detection and type recognition.
Smart Images

Figure CN123071771A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to an image change detection method for visually camouflaged targets. Background Technology
[0002] In the field of computer vision, camouflaged object detection (COD) has always been a highly challenging research area. Traditional object detection mainly targets visual targets with salient features, while camouflaged targets have features such as color, texture, and shape that are highly similar to the background, cleverly blending into the environment and making them difficult to detect by traditional detection algorithms. Existing technologies mainly focus on static camouflaged object detection, using deep neural networks to extract target features from single-frame images to achieve the location and contour segmentation of camouflaged targets.
[0003] However, in practical applications, it is not only necessary to detect the location of camouflaged targets from single-frame images, but also to analyze continuous temporal images to detect changes in the camouflaged targets over time (such as the appearance, disappearance, enlargement, and shrinkage of the targets) in order to obtain accurate change information. Therefore, improving the detection accuracy of camouflaged targets has become an urgent problem to be solved. Summary of the Invention
[0004] This invention provides an image change detection method for visually camouflaged targets, which addresses the shortcomings of low detection accuracy of camouflaged targets in existing technologies.
[0005] An image change detection method for visually camouflaged targets includes: Obtain a dual-temporal phase image, which is two images acquired at different times in the same scene; The dual-temporal image is input into the camouflage image change detection model, and based on the model, the change state of the camouflage target is output; the change state includes: the camouflage target appears, disappears, grows larger, or shrinks; The camouflage image change detection model includes: a multi-scale semantic alignment module, an implicit multi-scale change reconstruction module, and an image-level camouflage type classification module; The multi-scale semantic alignment module is used to perform channel alignment and spatial alignment of the dual-temporal graph at different scales to obtain aligned dual-temporal difference features at different scales. The implicit multi-scale change reconstruction module is used to construct a continuous mapping from coordinates and features to change values based on implicit neural representation, so as to realize implicit reconstruction of the dual temporal difference features at different scales and generate multi-scale change reconstruction features. The image-level camouflage type classification module is used to predict the multi-scale change reconstruction features and finally output a pixel-level semantic change map with change type labels; the change type labels include: camouflage target appears, disappears, grows larger, and shrinks.
[0006] Furthermore, in the image change detection method for visual camouflage targets described above, the multi-scale semantic alignment module includes several semantic alignment modules with different scales; each semantic alignment module includes: a temporal feature alignment module and a differential optical flow alignment module; The timing feature alignment module is used to align the features of one phase diagram of the dual-phase diagram with the features of the other phase diagram at the channel level, thereby generating channel-level aligned dual-phase features. The differential optical flow alignment module is used to spatially distort one of the temporal features by learning the differential optical flow field between the two temporal features, to achieve pixel-level spatial alignment, and to calculate the spatially aligned two temporal differential features.
[0007] Furthermore, in the image change detection method for visually camouflaged targets described above, the multi-scale semantic alignment module includes four semantic alignment modules with different scales: a first semantic alignment module, a second semantic alignment module, a third semantic alignment module, and a fourth semantic alignment module. The scale of the first semantic alignment module is: The scale of the second semantic alignment module is: The scale of the third semantic alignment module is: The scale of the fourth semantic alignment module is: .
[0008] Furthermore, in the image change detection method for visual camouflage targets as described above, the temporal feature alignment module is used to align the features of one temporal map of the dual temporal maps with the features of the other temporal map at the channel level, thereby generating channel-level aligned dual temporal features, including: The features of the two-phase diagrams are extracted respectively to obtain the corresponding first-phase features. Features of the second phase ; The first time phase feature Second phase characteristics After being linearly mapped to different subspaces, the corresponding first enhanced features are obtained. Second Enhancement Features ; The first enhanced feature With the second enhancement feature Perform linear mappings and reshape them into query vectors respectively. Key vector Sum value vector ; By calculating the correlation between the two-phase features along the channel dimension, a channel correlation score matrix is generated; The channel-related score matrix and value vector Perform matrix multiplication to obtain the channel-dimensional aligned features. ;
[0009]
[0010]
[0011] in, , and These are the learnable parameters of three linear layers. It is a linear projection matrix. Indicates the number of heads. Indicates the channel size for each head. It is the channel dimension, j ; This represents the softmax operation; Features aligned by channel dimensions via residual join Adding the corresponding temporal features to the channel-level aligned dual-temporal features yields the final dual-temporal features.
[0012] Furthermore, in the image change detection method for visually camouflaged targets described above, the differential optical flow alignment module's processing of dual-temporal features includes: The channel-level aligned dual-temporal features are respectively input into the optical flow MLP module composed of convolutional layers and Gaussian error linear units for feature enhancement;
[0013]
[0014] The enhanced dual-temporal features are merged, and a differential optical flow field is generated through a convolutional subnetwork. ;
[0015] Based on the differential optical flow field, spatial distortion of one phase characteristic in the dual-phase model is performed to obtain the distorted position. ; = +
[0016] in, This indicates the position of any pixel in one phase of a dual-phase diagram; Through a differentiable bilinear sampling mechanism, based on the torsion position Obtain the warp features after spatial alignment ;
[0017] in, This indicates that the pixel location is within the distortion feature. The position in the middle, Indicates the position of distortion The four neighboring positions are: top left, bottom left, top right, and bottom right. It is a bilinear kernel weight on a distorted spatial grid; Calculate the twist feature The absolute difference between the two phase features is used to obtain the spatially aligned two-phase difference features. .
[0018] Furthermore, in the image change detection method for visually camouflaged targets described above, the implicit multi-scale change reconstruction module processes the dual-temporal difference features at a certain scale, including: An implicit neural representation is defined on multiple biphase difference features with different resolutions, and spatial coordinates x are assigned to each feature vector; For a query coordinate Obtain the nearest implicit code from the biphase difference features at each scale. Relative coordinates and position encoding of relative coordinates ; The implicit encoding Relative coordinates and its position code After concatenation, the data is input into a decoding function composed of multilayer perceptrons. middle; The decoding function The features at each stage are decoded and their relationships are modeled, and the output is displayed in the query coordinates. By using continuous eigenvalues at specific points, lightweight and accurate alignment and reconstruction of multiple bi-temporal difference features can be achieved, ultimately yielding multi-scale variation reconstructed features. ;
[0019] in, It is the first Layer distance Recent implicit coding, Indicates the coordinates to be queried and Relative coordinates Indicates different levels.
[0020] Furthermore, in the image change detection method for visual camouflage targets described above, the image-level camouflage type classification module's processing of the multi-scale change reconstruction features includes: Reconstructing features from the multi-scale changes Extracting the probability mask of the changing region , used to indicate whether a pixel belongs to a changing region;
[0021] Among them, among them, Represents features reconstructed from multi-scale changes Background category prediction probability obtained by pixel-level change region prediction head; x and y represent the horizontal and vertical coordinates of the current pixel in the output prediction map, and M(x,y) represents the probability mask of the corresponding pixel position belonging to the change region; The four changing states of the camouflaged target are weighted to obtain the weighted probability of the changing type. ;
[0022] The region probability mask With the probability of the change type By combining these methods and calculating pixel-by-pixel, the probability of each changing state is obtained. ;
[0023] in, This represents the probability that the pixel at coordinates (x, y) in the predicted result image belongs to the k-th type of change state; The probability of each changing state Predicted probability with background category After normalization using the Softmax function, the final output is the changed state of the camouflaged target. ;
[0024] in, This represents the predicted probability of a disguised target appearing. This represents the predicted probability that the camouflaged target will disappear. This represents the predicted probability that the disguised target appears larger. This represents the predicted probability that the disguised target will appear smaller.
[0025] The present invention has the following beneficial effects: (1) Improve the detection accuracy of camouflaged target change areas. This scheme achieves joint semantic and spatial correction of bi-temporal images through the synergistic action of a bi-temporal semantic-guided alignment module, a differential optical flow spatial alignment module, and an implicit multi-scale change reconstruction module. Specifically, the bi-temporal semantic-guided alignment module suppresses unrealistic changes caused by illumination, weather, seasonal, and background variations; the differential optical flow spatial alignment module compensates for geometric offsets between the bi-temporal images; and the implicit multi-scale change reconstruction module effectively restores edge details and small-scale target information in changed regions. These mechanisms reduce false positives and false negatives, improving the ability to locate and detect changed regions. On the VCOD-CD dataset, this scheme achieves a precision of 89.81%, a recall of 85.27%, an F1 score of 87.36%, an IoU of 78.07%, a MAE of 0.1804, and an MSE of 0.0452. Compared to existing mainstream detection methods, this scheme demonstrates superior detection performance, validating its effectiveness in changing region detection.
[0026] (2) Achieve joint output of change region detection and change type recognition. This scheme extracts global semantic information from multi-scale spatial difference features through an image-level camouflage type classification module and jointly models it with change region features. While locating change regions, it automatically identifies change types such as target appearance, disappearance, enlargement, and shrinkage, thus achieving joint output of change region detection and change type identification, improving the information representation capability of change detection results. This scheme achieves a precision of 96.45%, a recall of 95.24%, and an F1 score of 95.23% in the change type classification task, accurately identifying different types of camouflage target changes and expanding the scope from whether a change has occurred to what kind of change has occurred. (3) Improve environmental adaptability and detection stability in complex scenarios. This application constructs a collaborative processing mechanism of semantic alignment, spatial correction, continuous reconstruction, and pattern recognition, enabling the model to simultaneously cope with various interference factors such as complex backgrounds, illumination variations, viewpoint changes, and weakly camouflaged targets. Through bi-temporal feature semantic consistency constraints, spatial position correction, and continuous feature reconstruction, the impact of environmental changes and spatial misalignment on detection results is effectively reduced, thereby improving the model's environmental adaptability and detection stability in complex scenes. This scheme demonstrates superior performance in both camouflaged target change detection and change type recognition tasks, indicating that the proposed semantic alignment, spatial correction, and continuous reconstruction mechanism can effectively enhance the model's comprehensive detection capabilities in complex scenes and has engineering application value. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0028] Figure 1 This is a flowchart illustrating the image change detection method for visually camouflaged targets provided by the present invention. Figure 2 This is a schematic diagram of the camouflage image change detection model provided in this application; Figure 3 A schematic diagram of the timing feature alignment module structure provided in this application; Figure 4 This is a schematic diagram of the differential optical flow alignment module structure provided in this application; Figure 5 A schematic diagram of the implicit multi-scale change reconstruction module provided in this application. Detailed Implementation
[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0030] Current methods for detecting changes in camouflaged targets over time have the following problems: (1) Dual-temporal images are susceptible to interference from environmental changes.
[0031] In real-world scenarios, due to factors such as changes in lighting, weather, seasons, and shooting angles, images of the same target acquired at different times can exhibit significant differences in appearance. Existing change detection methods lack effective cross-temporal semantic constraint mechanisms, easily misjudging environmental changes as target changes, thus leading to poor stability of change detection results.
[0032] (2) Due to factors such as changes in shooting angle, changes in equipment posture, and slight movements of the target, there is usually a geometric offset between images acquired at different times. Most existing methods use feature difference for change analysis, which cannot effectively eliminate the influence of positional offset and is prone to producing pseudo-change regions.
[0033] (3) The accuracy of reconstruction of the changed area is insufficient.
[0034] Existing change detection models typically use bilinear interpolation or deconvolution to recover changed regions, which can easily lead to the loss of edge details and small-scale target information. For camouflaged targets, whose boundaries are highly similar to the background, these problems further reduce the accuracy of changing region localization, resulting in incomplete contour recovery.
[0035] (4) Insufficient ability to express change types.
[0036] Existing change detection models typically only output binary change masks, which can only determine whether the target has changed, but cannot further identify whether the change belongs to a specific category such as appearance, disappearance, increase or decrease. This results in limited dimensions of change information expression, making it difficult to meet the needs of real-world application scenarios for refined analysis of change types.
[0037] To solve the above problems, such as Figure 1 As shown, the present invention provides an image change detection method for visually camouflaged targets, the method comprising: Step 101: Obtain a dual-temporal phase image, which is two consecutive images in the same scene and time dimension.
[0038] Specifically, the dual-phase images include: a first phase image T1 and a second phase image T2, which are two images taken in the same scene.
[0039] Step 102: Input the dual-temporal image into the camouflage image change detection model, and output the change state of the camouflage target based on the model; the change state includes: the camouflage target appears, disappears, grows larger, and shrinks.
[0040] Figure 2 This is a schematic diagram of the camouflage image change detection model provided in this application, as shown below. Figure 2 As shown, the model includes: a multi-scale semantic alignment module, an implicit multi-scale change reconstruction module, and an image-level camouflage type classification module.
[0041] The multi-scale semantic alignment module is used to perform channel alignment and spatial alignment of the dual-temporal image at different scales, corresponding to the aligned dual-temporal difference features at different scales; the implicit multi-scale change reconstruction module is used to construct a continuous mapping from coordinates and features to change values based on implicit neural representation, so as to realize implicit reconstruction of the dual-temporal difference features at different scales, so as to generate multi-scale change reconstruction features; the image-level camouflage type classification module is used to predict the multi-scale change reconstruction features, and finally output a pixel-level semantic change map with change type labels; the change type labels include: camouflage target appears, disappears, grows larger, shrinks.
[0042] The change type labels provided in this application include: appearance, disappearance, enlargement, and shrinkage of camouflaged targets. Since appearance, disappearance, enlargement, and shrinkage all pertain to before-and-after state relationships, rather than static attributes within a single image, for example: T1 has no target, T2 has a target (appearance); T1 has a target, T2 has no target (disappearance); T2 target range is larger than T1 (enlargement); T2 target range is smaller than T1 (shrinkage).
[0043] To address the false detection problem caused by environmental changes, this invention utilizes a multi-scale semantic alignment module to achieve joint correction of bi-temporal images at both the semantic and spatial levels. On one hand, it suppresses unrealistic changes caused by variations in illumination, background, and season, improving the model's ability to perceive truly changing regions. On the other hand, it learns the differential optical flow field between features in the bi-temporal images, establishes spatial mapping relationships between features, and uses a resampling mechanism to correct the position of target regions with geometric offsets. This achieves spatial alignment of bi-temporal features, compensates for geometric offsets between the bi-temporal images, reduces spurious changes caused by geometric offsets, and solves the problem of spatial misalignment in bi-temporal images.
[0044] To address the problem of insufficient accuracy in reconstructing changed regions, this invention introduces an implicit multi-scale change reconstruction module. This module is based on implicit neural representation and learns the continuous representation of the image in coordinate space to achieve continuous reconstruction and high-precision restoration of multi-scale change features, thereby enhancing the ability to express edge details of changed regions and effectively restoring edge details and small-scale target information of changed regions.
[0045] To address the problem of insufficient ability to express change types, this invention introduces an image-level camouflage type classification module to extract global semantic information from multi-scale spatial difference features and combine it with change region features for joint modeling to identify the change types of camouflage targets, thereby achieving classification and prediction of the appearance, disappearance, enlargement, and shrinkage of camouflage targets.
[0046] In summary, the method provided by this invention, through the cascaded processing of the above three modules, can simultaneously achieve effective localization of the changing area of the camouflaged target and identification of the change type, effectively overcoming the shortcomings of the prior art in detecting changes in camouflaged targets, reducing false detections and missed detections, and improving the ability to identify change types and the accuracy of changing area detection.
[0047] Furthermore, such as Figure 2As shown, the multi-scale semantic alignment module provided in this application includes several semantic alignment modules with different scales; each semantic alignment module includes: a temporal feature alignment module and a differential optical flow alignment module. The temporal feature alignment module is used to align the features of one temporal map of a dual-temporal map with the features of the other temporal map at the channel level, thereby generating channel-level aligned dual-temporal features; the differential optical flow alignment module is used to learn the differential optical flow field between the dual-temporal features, and spatially distort one of the temporal features according to the differential optical flow field to complete pixel-level spatial alignment, and finally use absolute subtraction to obtain the dual-temporal differential features corresponding to each scale.
[0048] Specifically, in the same scene, the first and second temporal images typically contain a large amount of identical background and target semantics. However, due to factors such as illumination, season, shadows, and background perturbations, specific pixel values or feature responses may change. The multi-scale semantic alignment module provided in this application does not directly subtract the first and second temporal images pixel-by-pixel. Instead, it uses features from one temporal phase to guide the channel response of the other, making the two temporal phases semantically consistent. If a difference only arises from illumination or seasonal variations, this difference often alters the feature intensity or channel response, rather than the true semantics of the target. By modeling the correlation between the two temporal features, the shared semantic components can be preserved, suppressing irrelevant environmental differences. While the two temporal images themselves cannot directly eliminate environmental interference, they provide a necessary reference object for the subsequent differential optical flow alignment module's cross-temporal semantic alignment. For the target spatial misalignment problem, only by simultaneously obtaining the first and second temporal images can the relative displacement of the same target between the two moments be estimated. Different shooting positions, camera shake, framing errors, or changes in target pose can cause the same target to not completely overlap in the first and second temporal images. If direct difference is performed, the misaligned edges will be mistakenly identified as areas of change. The differential optical flow alignment module provided in this application takes the features of the two temporal phases as input, predicts the differential optical flow field between them, and then samples and distorts the features of one temporal phase based on this flow field to make it spatially closer to the features of the other temporal phase. Performing absolute difference after spatial alignment can reduce spurious changes caused by positional offset.
[0049] In this embodiment of the invention, the multi-scale semantic alignment module provided by this application includes four semantic alignment modules at four scales: a first semantic alignment module, a second semantic alignment module, a third semantic alignment module, and a fourth semantic alignment module. These four semantic alignment modules form a hierarchical feature extraction structure from shallow to deep. Each semantic alignment module includes a temporal feature alignment module and a differential optical flow alignment module, and performs channel alignment, spatial alignment, and differential feature extraction of dual-temporal features at the corresponding scale. For environmental interference problems such as illumination, season, and background disturbances, shallow differences can reflect the local range of the changed area, while deep differences can reflect the overall semantics of the target. Combining the two is more conducive to determining whether the change is appearing, disappearing, increasing, or decreasing.
[0050] For the problem of spatial misalignment, the manifestation of camouflaged target misalignment varies at different scales. In high-resolution features, edge and local positional shifts can be observed; in low-resolution features, it is easier to capture the overall or large-scale shifts of the target. This application uses semantic alignment modules at four scales to predict and process spatial differences at different scales, which can simultaneously take into account both fine-grained local misalignment and large-scale overall misalignment.
[0051] In this embodiment of the invention, the scale of the first semantic alignment module is: The scale of the second semantic alignment module is: The scale of the third semantic alignment module is: The scale of the fourth semantic alignment module is: .
[0052] Specifically, because camouflaged targets are highly similar to the background, truly identifiable clues are often concentrated in subtle edges, texture breaks, and local contours. If the first-level features are directly reduced to H / 8, H / 16, or even smaller, smaller targets and narrow boundaries may only have a few pixels remaining after downsampling, making it difficult to recover the lost structural information even with subsequent upsampling. Details of the network's lower-level structure are gradually lost as the number of convolutional layers increases, and are difficult to reconstruct through traditional upsampling. Therefore, H / 4×W / 4 is mainly used to preserve higher spatial resolution, providing a detailed foundation for small target detection and changing boundary localization. The H / 8 scale has a larger receptive field than H / 4 while still retaining sufficient positional details. This scale can achieve a balance between local texture and initial semantics, which is beneficial for recognizing small-to-medium scale changes and can also provide a more stable local spatial correspondence for the differential optical flow alignment module. If the second semantic alignment module directly reduces to H / 16, there will be a lack of an intermediate transition layer between H / 4 and H / 16, resulting in too rapid a scale change and potentially insufficient expression of local details and intermediate structures. The H / 16 scale further expands the receptive field, enabling the representation of the overall shape and regional structure of targets such as animals and vehicles. This layer primarily addresses the problem of shallow features focusing only on local textures and easily misinterpreting local background disturbances as target changes. It enhances the integrity of changed regions through higher-level semantics. The H / 32 scale has the largest receptive field, used to model global scene relationships and the overall semantics of the target. In high-altitude camouflage target change detection scenarios, local textures may be very similar to the camouflage target, and relying solely on high-resolution local features can easily lead to false detections. Deep, low-resolution features can combine a wider range of context to determine whether a region conforms to the overall target semantics and provide global change type information for the image-level camouflage type classification module.
[0053] H / 4 and H / 8 primarily alleviate the problem of detail and edge loss, while H / 16 and H / 32 mainly enhance the overall target structure and global semantics. This application uses channel feature alignment and spatial semantic alignment at four scales to handle environmental interference and spatial misalignment at multiple resolutions. Finally, the four-level difference features are uniformly reconstructed by the implicit multi-scale change reconstruction module, and the global change type information is extracted by the image-level camouflage type classification module, ultimately accurately detecting the change state of the camouflaged target.
[0054] Furthermore, the timing feature alignment module provided in this application specifically includes the following scheme for processing dual-phase diagrams: The features of the two-phase diagrams are extracted respectively to obtain the corresponding first-phase features. Features of the second phase The first phase feature Second phase characteristics After being linearly mapped to different subspaces, the corresponding first enhanced features are obtained. Second Enhancement Features ; the first enhanced feature Second Enhancement Features Perform linear mappings and reshape them into query vectors respectively. Key vector Sum value vector By calculating the correlation between bi-temporal features along the channel dimension, a channel correlation score matrix is generated; the channel correlation score matrix is then compared with the value vector. Perform matrix multiplication to obtain the channel-dimensional aligned features. Features aligned by channel dimensions through residual connections Adding the corresponding temporal features to the channel-level aligned dual-temporal features yields the final dual-temporal features.
[0055] Specifically, camouflaged targets are typically photographed at different times and are significantly affected by environmental changes, such as being easily interfered with by irrelevant factors like lighting angle, seasonal changes, and background disturbances. The temporal feature alignment module uses one sequence feature to guide the alignment of another sequence feature during each layer of feature extraction, thereby achieving channel-level alignment. Figure 3 A schematic diagram of the timing feature alignment module structure provided in this application is shown below. Figure 3 As shown, the second phase characteristics Guiding the first phase characteristics For example, input a pair of sequence features and ,in This indicates the sequence length. Unlike previous spatial alignment methods that use self-attention and cross-attention mechanisms, this application achieves alignment of a pair of sequence features at the channel level.
[0056] First phase characteristics Second phase characteristics After being linearly mapped to different subspaces, the corresponding first enhanced features are obtained. Second Enhancement Features ; the first enhanced feature Second Enhancement Features A common input temporal supervision feature alignment module is used to achieve channel-level feature alignment, ultimately obtaining channel-level aligned bi-temporal features, namely: the first channel aligned feature. Second channel alignment feature .
[0057] The specific processing procedure of this module is as follows: The first enhanced feature is linearly mapped, transposed, and reshaped to obtain the query vector Q. ; Indicates the number of heads. This indicates the channel size for each head; The second enhanced feature is linearly mapped, transposed, and reshaped to obtain the key vector. Sum vector V; , ; The query vector Q and the key vector The correlation scores between channels are calculated by scaling the dot product attention, thereby generating an attention map; The attention map and V are aggregated through matrix multiplication to obtain channel-dimensional aligned sequential image features. ; The calculation process is as follows:
[0058]
[0059]
[0060] in , and These are the learnable parameters of three linear projection layers. It is the channel dimension. It is a linear projection matrix. This indicates the softmax operation.
[0061] Finally, the features of the sequence images Restored to its original state through transposition, reshaping, and linear transformation. The size is then determined and connected to the first phase feature via a residual join. Adding them together, we finally obtain the first channel alignment feature with channel-level alignment. .
[0062] Similarly, the first phase characteristics Guiding the characteristics of the second phase Finally, the second channel alignment feature with channel-level alignment is obtained. .
[0063] Furthermore, the differential optical flow alignment module provided in this application processes the dual-temporal features as follows: inputting the channel-level aligned dual-temporal features into an optical flow MLP module composed of convolutional layers and Gaussian error linear units for feature enhancement; merging the enhanced dual-temporal features and generating a differential optical flow field through a convolutional sub-network. Based on the differential optical flow field, spatial distortion is applied to one phase characteristic in the dual-phase system to obtain the distorted position. ; Through a differentiable bilinear sampling mechanism, based on the torsion position Obtain the warp features after spatial alignment ; Calculate the distortion feature The absolute difference between the two phase features is used to obtain the spatially aligned two-phase difference features. .
[0064] Specifically, Figure 4 This is a schematic diagram of the differential optical flow alignment module structure provided in this application, as shown below. Figure 4 As shown, in order to solve the problem of misalignment of camouflaged targets caused by different viewing angles and shooting errors, the differential optical flow alignment module predicts pixel offset by learning the differential optical flow field and uses deformable convolution and sampling mechanisms to perform display space alignment.
[0065] Taking the differential optical flow alignment module in the first semantic alignment module as an example, the temporal feature alignment module outputs a pair of sequence temporal features (first channel alignment features). Second channel alignment feature The features are then fed into an optical flow MLP module consisting of three convolutional layers and Gaussian error linear units (GeLU) for feature enhancement, and the enhanced features are then processed. , The features are merged, and the merged features are input into a sub-network composed of convolutional layers to generate a differential optical flow field. The process can be described as follows:
[0066]
[0067]
[0068] in, This indicates a cascaded operation, where Conv is a convolutional layer with a kernel of 3.
[0069] The differential optical flow field is a two-dimensional offset predicted for each spatial location in the feature map.
[0070] Based on the differential optical flow field, spatial distortion of one phase characteristic in the dual-phase model is performed to obtain the distorted position. .
[0071] = +
[0072] Specifically, for any pixel position in the first time phase diagram T1 Position of this pixel Adding it to the differential optical flow field yields the corresponding twist position. Performing this distortion operation on all pixel locations in the feature map and then completing it using a differentiable bilinear sampling mechanism yields the complete distortion feature of the first time-phase map T1. .
[0073]
[0074] in, This indicates that the pixel location is within the distortion feature. The position in the middle, Indicates the position of distortion The four neighboring positions are: top left, bottom left, top right, and bottom right. It is a bilinear kernel weight on a distorted spatial grid.
[0075] Finally, the distortion features are calculated. Alignment features with the second channel The absolute difference between them yields the spatially aligned dual-temporal difference features. .
[0076]
[0077] in, This indicates the subtraction of absolute values.
[0078] In this embodiment, a semantic alignment module has four modules, each generating a dual-temporal difference feature. There are a total of 4 dual-time-phase difference features, namely: .
[0079] Furthermore, Figure 5 A schematic diagram of the implicit multi-scale change reconstruction module structure provided in this application is shown below. Figure 5 As shown, the implicit multi-scale change reconstruction module provided in this application processes the dual-temporal difference features at a certain scale as follows: Implicit neural representations are defined on multiple biphase difference features with different resolutions, and spatial coordinates x are assigned to each feature vector; for a query coordinate... Obtain the nearest implicit code from the biphase difference features at each scale. Relative coordinates and position encoding of relative coordinates The implicit encoding Relative coordinates and its position code After concatenation, the data is input into a decoding function composed of multilayer perceptrons. In the middle; the decoding function The features at each stage are decoded and their relationships are modeled, and the output is displayed in the query coordinates. By utilizing continuous eigenvalues at specific points, lightweight and accurate alignment and reconstruction of dual-temporal difference features can be achieved, ultimately yielding multi-scale variation reconstructed features. ; Specifically, traditional multi-scale fusion often relies on upsampling (such as bilinear upsampling) or decoder aggregation (such as deconvolution). However, bilinear upsampling, which calculates new pixel values based on linear interpolation of surrounding pixels, lacks a deep understanding of image semantics and structure, blurring the precise information learned from the feature map, especially in areas with abrupt pixel value changes such as edges, easily causing edge blurring. Deconvolution, with its weight sharing during convolution, relatively averages the processing of features at different locations, making it difficult to accurately capture and preserve the unique features of edges and structurally complex areas, and also incurring additional computational overhead. The implicit multi-scale change reconstruction module is based on implicit neural representation. By learning the continuous representation of the image in coordinate space, it constructs a continuous mapping from coordinates + features to change values, achieving precise alignment of dual temporal difference features in a lightweight manner. The implicit neural representation defines a decoding function on the discrete feature map. Typically, it is a multilayer perceptron (MLP) to obtain multi-scale variation reconstruction features S.
[0080] Taking a single feature map as an example, in the discrete feature mapping stage, each feature vector is treated as an implicit code uniformly distributed in a two-dimensional space, and a spatial coordinate is assigned to each vector. S in The feature at this location is defined as:
[0081] in, Indicates distance Recent implicit coding, It is implicit encoding. The coordinate values. Furthermore, to further improve the learning ability of the decoding function, learnable positional encoding is introduced. Therefore, the final definition of the implicit neural representation is:
[0082] in, This represents the positional encoding of relative coordinates. To achieve alignment of multiple bi-temporal difference feature maps, a continuous mapping S is defined on multi-level difference features with different resolutions. Specifically, S is... The value at that location is defined as:
[0083] in, Indicates different levels, It is the first Layer distance Recent implicit coding, express and Relative coordinates yes The coordinates.
[0084] For a given query coordinate, retrieve the implicit code that is closest to it from each layer. Relative coordinates and position encoding of relative coordinates Then, they are concatenated and input into a decoding function consisting of three multilayer perceptrons (MLPs). middle. Features at each stage are decoded, and the relationships between different stages are modeled. In this way, multi-level difference features are aligned in a lightweight manner, thereby obtaining high-quality pixel-level prediction maps.
[0085] The method provided by this invention effectively improves the recognition accuracy of camouflaged targets through an implicit multi-scale change reconstruction module. Specifically, on the one hand, traditional bilinear interpolation only uses surrounding pixels for linear calculation, which easily leads to blurred edges, loss of details in small targets, and blunting of the contours of change areas. The implicit multi-scale change reconstruction module, however, does not directly interpolate but learns feature representations in continuous coordinate space, thus more accurately recovering edge information and avoiding the information blurring caused by traditional upsampling. On the other hand, traditional methods usually unify the size first and then stitch them together. This application allows four scales to participate in continuous mapping, with different scales providing: shallow layer: local details; middle layer: target structure; deep layer: overall semantics. The MLP encoder simultaneously learns the correspondence between these scales, thus achieving true multi-scale fusion rather than simple superposition, thereby realizing continuous fusion between differential features of different scales. Furthermore, since the final generated representation is a change representation in continuous space, the edges are more continuous, the target contours are more complete, and changes in small targets are more easily preserved. This is particularly important for camouflaged targets, especially suitable for scenarios where the camouflaged target is small and the edges are not obvious, thereby improving the accuracy of change area localization. Finally, the implicit multi-scale change reconstruction module achieves lightweight and accurate alignment of multiple bi-temporal difference features, providing high-quality features for subsequent change region localization and type identification. Therefore, the image-level camouflage type classification module can directly utilize this fused feature for change region prediction and change type identification without performing complex multi-scale feature alignment again, thereby improving the accuracy of change type identification in complex scenes.
[0086] Furthermore, the image-level camouflage type classification module provided in this application processes multi-scale variation reconstruction features including: Reconstructing features from the multi-scale changes Extracting the probability mask of the changing region , used to indicate whether a pixel belongs to a changing region;
[0087] in, Represents features reconstructed from multi-scale changes The background category prediction probability is obtained by the pixel-level change region prediction head; x and y represent the horizontal and vertical coordinates of the current pixel in the output prediction map, and M(x,y) represents the probability mask of the corresponding pixel position belonging to the change region.
[0088] The predicted probabilities of the four states of the camouflaged target—appearance, disappearance, enlargement, and shrinkage—are weighted to obtain the probability of the change type. ;
[0089] The region probability mask With the probability of the change type By combining these methods and calculating pixel-by-pixel, the probability of each changing state is obtained. ;
[0090] in, This represents the probability that the pixel at coordinates (x, y) in the predicted result image belongs to the k-th type of change state; The probability of each changing state Compared with background prediction probability After normalization using the Softmax function, the final output is the changed state of the camouflaged target. ;
[0091] in, This represents the predicted probability of a disguised target appearing. This represents the predicted probability that the camouflaged target will disappear. This represents the predicted probability that the disguised target appears larger. This represents the predicted probability that the disguised target will appear smaller.
[0092] Specifically, in complex and changing scenarios, pixel-level features are insufficient to distinguish the specific types of changes in camouflaged targets. To address this, this application introduces an image-level multi-label classification network through an image-level camouflage type classification module to extract global semantic information from multi-scale spatial difference features, thereby enabling image-level recognition of camouflage change types.
[0093] Because the number of samples of different variation types in the dataset is imbalanced, directly using the probability output by the classifier will cause the model to favor the class with more samples. Therefore, the probabilities of the four variation types output by the image-level classifier should be adjusted accordingly. Introducing weights Make corrections.
[0094]
[0095] Mask the changing region and weighted probability of change type Combined, generated by pixel-by-pixel calculation and predict the probability with the background class After normalization using the Softmax function, the output pixel's semantic change probability is divided into five categories (background + four change types). This process generates a final prediction map, enabling accurate identification of multiple categories of camouflaged changes.
[0096]
[0097]
[0098] in, This represents the predicted probability of a disguised target appearing. This represents the predicted probability that the camouflaged target will disappear. This represents the predicted probability that the disguised target appears larger. This represents the predicted probability that the disguised target will appear smaller.
[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An image change detection method for visually camouflaged targets, characterized in that, Obtain a dual-temporal phase image, which is two images acquired at different times in the same scene; The dual-temporal map is input into the camouflage image change detection model, and based on the model, the change state of the camouflage target is output; The changing states include: the camouflaged target appearing, disappearing, growing larger, and shrinking; The camouflage image change detection model includes: a multi-scale semantic alignment module, an implicit multi-scale change reconstruction module, and an image-level camouflage type classification module; The multi-scale semantic alignment module is used to perform channel alignment and spatial alignment of the dual-temporal graph at different scales to obtain aligned dual-temporal difference features at different scales. The implicit multi-scale change reconstruction module is used to construct a continuous mapping from coordinates and features to change values based on implicit neural representation, so as to realize implicit reconstruction of the dual temporal difference features at different scales and generate multi-scale change reconstruction features. The image-level camouflage type classification module is used to predict the multi-scale change reconstruction features and finally output a pixel-level semantic change map with change type labels; the change type labels include: camouflage target appears, disappears, grows larger, and shrinks.
2. The image change detection method for visually camouflaged targets according to claim 1, characterized in that, The multi-scale semantic alignment module includes several semantic alignment modules with different scales; each semantic alignment module includes: a temporal feature alignment module and a differential optical flow alignment module; The timing feature alignment module is used to align the features of one phase diagram of the dual-phase diagram with the features of the other phase diagram at the channel level, thereby generating channel-level aligned dual-phase features. The differential optical flow alignment module is used to spatially distort one of the temporal features by learning the differential optical flow field between the two temporal features, to achieve pixel-level spatial alignment, and to calculate the spatially aligned two temporal differential features.
3. The image change detection method for visually camouflaged targets according to claim 1, characterized in that, The multi-scale semantic alignment module includes four semantic alignment modules with different scales: a first semantic alignment module, a second semantic alignment module, a third semantic alignment module, and a fourth semantic alignment module. The scale of the first semantic alignment module is: The scale of the second semantic alignment module is: The scale of the third semantic alignment module is: The scale of the fourth semantic alignment module is: .
4. The image change detection method for visually camouflaged targets according to claim 2, characterized in that, The timing feature alignment module is used to align the features of one timing map with the features of the other timing map at the channel level, thereby generating channel-level aligned dual-timing features, including: The features of the two-phase diagrams are extracted respectively to obtain the corresponding first-phase features. Features of the second phase ; The first time phase feature Second phase characteristics After being linearly mapped to different subspaces, the corresponding first enhanced features are obtained. Second Enhancement Features ; The first enhanced feature With the second enhancement feature Perform linear mappings and reshape them into query vectors respectively. Key vector Sum value vector ; By calculating the correlation between the two-phase features along the channel dimension, a channel correlation score matrix is generated; The channel-related score matrix and value vector Perform matrix multiplication to obtain the channel-dimensional aligned features. ; in, , and These are the learnable parameters of three linear layers. It is a linear projection matrix. Indicates the number of heads. Indicates the channel size for each head. It is the channel dimension, j ; This represents the softmax operation; Features aligned by channel dimensions via residual join Adding the corresponding temporal features to the channel-level aligned dual-temporal features yields the final dual-temporal features.
5. The image change detection method for visually camouflaged targets according to claim 3, characterized in that, The differential optical flow alignment module processes dual-temporal features including: The channel-level aligned dual-temporal features are respectively input into the optical flow MLP module composed of convolutional layers and Gaussian error linear units for feature enhancement; The enhanced dual-temporal features are merged, and a differential optical flow field is generated through a convolutional subnetwork. ; Based on the differential optical flow field, spatial distortion of one phase characteristic in the dual-phase model is performed to obtain the distorted position. ; = + in, This indicates the position of any pixel in one phase of a dual-phase diagram; Through a differentiable bilinear sampling mechanism, based on the torsion position Obtain the warp features after spatial alignment ; in, This indicates that the pixel location is within the distortion feature. The position in the middle, Indicates the position of distortion The four neighboring positions are: top left, bottom left, top right, and bottom right. It is a bilinear kernel weight on a distorted spatial grid; Calculate the twist feature The absolute difference between the two phase features is used to obtain the spatially aligned two-phase difference features. .
6. The image change detection method for visually camouflaged targets according to claim 3, characterized in that, The implicit multi-scale change reconstruction module processes the dual-temporal difference features at a certain scale through the following steps: An implicit neural representation is defined on multiple biphase difference features with different resolutions, and spatial coordinates x are assigned to each feature vector; For a query coordinate Obtain the nearest implicit code from the biphase difference features at each scale. Relative coordinates and position encoding of relative coordinates ; The implicit encoding Relative coordinates and its position code After concatenation, the data is input into a decoding function composed of multilayer perceptrons. middle; The decoding function The features at each stage are decoded and their relationships are modeled, and the output is displayed in the query coordinates. By using continuous feature values at specific points, lightweight and accurate alignment and reconstruction of multiple bi-temporal difference features can be achieved, ultimately yielding multi-scale variation reconstructed features. ; in, It is the first Layer distance Recent implicit coding, Indicates the coordinates to be queried and Relative coordinates Indicates different levels.
7. The image change detection method for visually camouflaged targets according to claim 3, characterized in that, The image-level camouflage type classification module processes the multi-scale change reconstruction features as follows: Reconstructing features from the multi-scale changes Extracting the probability mask of the changing region , used to indicate whether a pixel belongs to a changing region; Among them, among them, Represents features reconstructed from multi-scale changes Background category prediction probability obtained by pixel-level change region prediction head; x and y represent the horizontal and vertical coordinates of the current pixel in the output prediction map, and M(x,y) represents the probability mask of the corresponding pixel position belonging to the change region; The four changing states of the camouflaged target are weighted to obtain the weighted probability of the changing type. ; The region probability mask With the probability of the change type By combining these methods and calculating pixel-by-pixel, the probability of each changing state is obtained. ; in, This represents the probability that the pixel at coordinates (x, y) in the predicted result image belongs to the k-th type of change state; The probability of each changing state Predicted probability with background category After normalization using the Softmax function, the final output is the changed state of the camouflaged target. ; in, This represents the predicted probability of a disguised target appearing. This represents the predicted probability that the camouflaged target will disappear. This represents the predicted probability that the disguised target appears larger. This represents the predicted probability that the disguised target will appear smaller.
