A multi-scale remote sensing image feature cooperative hierarchical processing method and system

Abstract

The present application relates to the technical field of remote sensing image, and particularly relates to a multi-scale remote sensing image feature cooperative hierarchical processing method and system. The present application extracts multi-scale features of each image source after registration, scale alignment and radiation normalization of multi-source remote sensing images, and then transforms the features of different scale levels to a target grid to form an integrated expression retaining the dimensions of image source and scale. On this basis, cross-scale consistency constraints are applied to adjacent scale features corresponding to the same geographical position, so that each scale feature is mutually corrected, thereby eliminating the problem of mutual contradiction between features of different scales and improving the consistency and cooperative representation ability of cross-scale features.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image technology, specifically relating to a method and system for collaborative hierarchical processing of multi-scale remote sensing image features. Background Technology

[0002] In satellite remote sensing applications, it is often necessary to fuse multi-source remote sensing images from different sensors with varying spatial, spectral, and temporal resolutions. Images from different sources often have overlapping areas, known as overlapping regions. When fusing multi-scale images, it is necessary to connect image information from different scales within these overlapping regions.

[0003] Existing multi-scale remote sensing image processing methods typically extract and process image features at each scale independently, and then perform simple stitching or overlay in overlapping areas. This processing method results in insufficient spatial complementarity in overlapping areas, uneven and unnatural transitions, and sudden brightness changes, gaps, or ghosting at the stitching or fusion points. At the same time, since the features at each scale are not transformed to the same grid, and the feature differences at the same location are not compared and corrected accordingly, the features between different scales may contradict each other.

[0004] To address the above problems, this invention proposes a collaborative hierarchical processing method and system based on multi-scale remote sensing image features. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, the present invention aims to provide a method and system for collaborative hierarchical processing of multi-scale remote sensing image features.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0007] A collaborative hierarchical processing method based on multi-scale remote sensing image features includes:

[0008] The multi-source remote sensing images are registered, scale-aligned, and radiometrically normalized, and overlapping areas are identified. Multi-scale features of each image source are extracted, so that each feature record retains the image source identifier, scale level, feature tensor, and effective area mask.

[0009] Features at different scale levels are transformed to the target grid to form an integrated expression that preserves the image source dimension and scale dimension. On the basis of the integrated expression, consistency constraints are applied to the adjacent scale features at the corresponding target grid position to form a collaborative representation record.

[0010] Complementary weighting is applied to the collaborative representation records of different image sources within the overlapping area to form a fusion feature pyramid; the fusion feature pyramid is then processed step by step in order of scale hierarchy from coarse to fine and the results are output.

[0011] Preferably, the registration of multi-source remote sensing images includes: obtaining the coordinate transformation between images through feature point matching or mutual information, and using resampling to achieve scale alignment; if the registration quality does not meet the preset conditions, the order of the transformation model is reduced or the image is marked as not suitable for fusion.

[0012] Preferably, transforming features at different scale levels to the target grid includes: upsampling or downsampling adjacent levels with different spatial resolutions to unify them to the target grid, and forming a joint representation by channel splicing or element-wise weighting; when the number of channels is inconsistent, unifying the channel dimension is achieved through linear projection or channel selection.

[0013] Preferably, applying consistency constraints to adjacent scale features of the corresponding target grid location on top of the integrated representation includes: calculating the distance between adjacent scale layer features of the same target grid location as the consistency error; correcting or not correcting the current layer features based on the comparison result of the consistency error and a preset threshold; and cyclically processing from the coarsest layer to the second layer, while keeping the finest layer unchanged.

[0014] Preferably, the complementary weighting of the collaborative representation records of different image sources within the overlapping area and the step-by-step processing of the fusion feature pyramid in a scale hierarchy from coarse to fine include: the weights of the two image sources within the overlapping area vary complementaryly along their spatial positions, and the sum of the weights at the same position is 1. The weights are determined based on the distance from the position to the effective boundary of each image source, and the complementary weighting is performed at each scale level. Then, a coarse-level processing state is generated using the coarsest fusion feature, and the previous layer processing state is upsampled and refined together with the current layer fusion feature, progressing step by step until the finest scale level.

[0015] This invention also discloses a multi-scale remote sensing image feature collaborative hierarchical processing system, comprising:

[0016] The preprocessing unit is used to register, scale-align, and radiometrically normalize multi-source remote sensing images, and to determine overlapping areas.

[0017] The multi-scale feature extraction unit is used to extract multi-scale features from each image source, so that each feature record retains the image source identifier, scale level, feature tensor and effective area mask;

[0018] An integrated representation unit is used to transform features at different scale levels to the target grid, forming an integrated representation that preserves both the image source dimension and the scale dimension.

[0019] The collaborative representation unit is used to impose consistency constraints on the adjacent scale features of the corresponding target grid position on top of the integrated representation, forming a collaborative representation record;

[0020] The overlapping area complementary unit is used to perform complementary weighting on the collaborative representation records of different image sources within the overlapping area to form a fusion feature pyramid.

[0021] The hierarchical processing unit is used to process the fused feature pyramid step by step in order of scale from coarse to fine and output the results.

[0022] And a data storage unit, used to persistently store intermediate results in the processing chain, for each unit to read and write back in the order of information flow.

[0023] Preferably, the preprocessing unit is configured to: obtain the coordinate transformation between images through feature point matching or mutual information, and achieve scale alignment by resampling; if the registration quality does not meet the preset conditions, the order of the transformation model is reduced or the image is marked as not eligible for fusion.

[0024] Preferably, the integrated expression unit is configured to: upsample or downsample adjacent levels with different spatial resolutions to the target grid, and form a joint expression by splicing by channel or weighting element by element; when the number of channels is inconsistent, the channel dimension is unified by linear projection or channel selection.

[0025] Preferably, the collaborative representation unit is configured to: calculate the distance between adjacent scale layer features at the same target grid location as the consistency error; correct or not correct the current layer features based on the comparison result of the consistency error and a preset threshold; process layer by layer from the coarsest layer to the second layer in a loop, while keeping the finest layer unchanged.

[0026] Preferably, the overlapping area complementary unit is configured such that the weights of the two image sources vary complementaryly along the spatial position and the sum of the weights at the same position is 1. The weights are determined based on the distance from the position to the effective area boundary of each image source, and the complementary weighting is performed at each scale level.

[0027] The hierarchical processing unit is configured to generate a coarse-level processing state using the coarsest layer fusion features, then upsample the previous layer processing state and refine it together with the current layer fusion features, progressing layer by layer until the finest scale level.

[0028] Beneficial effects

[0029] 1. This invention extracts multi-scale features from each image source after registering, scale-aligning, and radiometrically normalizing multi-source remote sensing images. Then, it transforms the features at different scale levels to the target grid to form an integrated expression that preserves the image source dimension and scale dimension. On this basis, it applies cross-scale consistency constraints to adjacent scale features at corresponding geographical locations, so that the features at each scale can be mutually corrected, thereby eliminating the problem of mutual contradiction between features at different scales and improving the consistency and collaborative representation ability of cross-scale features.

[0030] 2. This invention performs spatially gradual complementary weighting on the collaborative representation features of different image sources after cross-scale consistency constraint correction within the overlapping area determined during the image registration stage. This causes the weight of image source A to decrease from 1 to 0, the weight of image source B to increase from 0 to 1, and the sum of the weights at the same position to be 1. This achieves a natural transition of multi-source image features within the overlapping area, avoids brightness abrupt changes, gaps, or ghosting at the stitching or fusion points, and improves the spatial continuity and visual quality of the fusion result.

[0031] 3. This invention processes the fusion feature pyramid formed by complementary weighting of overlapping regions in a hierarchical order from coarse to fine scale. First, a preliminary processing state is generated at the coarsest scale. Then, the processing state of the previous layer is upsampled and refined together with the fusion feature of the current layer. This process is carried out layer by layer until the finest scale is reached, at which point the final result is output. This allows the output features to retain both global structural information and local detail information, supporting subsequent parameter learning or other technical processing. This improves the usability of multi-scale features and the convenience of engineering deployment. Attached Figure Description

[0032] Figure 1 This is a flowchart of the method of the present invention;

[0033] Figure 2 This is a system module diagram of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely for explaining the invention and are not intended to limit the scope of protection of the invention.

[0035] Example 1

[0036] Please see Figure 1 As shown, this embodiment provides a method for collaborative hierarchical processing of multi-scale remote sensing image features, including the following steps:

[0037] S1. Image acquisition and preprocessing;

[0038] Multi-source remote sensing images are integrated, and SIFT feature point matching and RANSAC are used to eliminate mismatches to obtain the coordinate transformation between images. An orthorectified reference is established using RPC combined with DEM, and relative radiometric normalization is performed using histogram matching. The intersection of the effective area masks of each image is calculated as the overlapping area. This step outputs aligned images and overlapping area identifiers.

[0039] Specifically, an image record is created for each input image. This record includes at least the image source identifier, acquisition time, band set, pixel size, number of rows and columns, geographic transformation parameters, coordinate reference system, and effective area mask. One image is selected as the target coordinate reference, and the coordinate transformation parameters for the remaining images are obtained through control point pairs or mutual information. After transformation, nearest neighbor resampling is applied to the effective area mask, and bilinear or cubic convolution resampling is applied to the radiance values. Overlapping areas are determined by the intersection of the effective area masks of each aligned image, generating an overlap area record with overlap area identifiers, boundary pixel sets, and geographic extent. If the number of effective control points is less than the minimum required by the transformation model, or if the reprojection error exceeds a preset threshold, the transformation model order is reduced, or the image is marked as unsuitable for fusion to prevent erroneous registration results from entering subsequent steps.

[0040] S2, Multi-scale feature extraction;

[0041] For each aligned image output, an image pyramid is constructed from coarse to fine, and features are extracted at each layer. Each layer's feature record includes the image source identifier, scale level, feature tensor, and effective region mask.

[0042] A feature pyramid is constructed for each image source. Let the scale level be... Then the first Layer feature records include image source identifiers. Scale level The scaling factor of this layer relative to the reference mesh, and the feature tensor. , corresponding effective area mask In addition to geographic transformation parameters, when constructing the next coarse-scale layer, the previous layer's image or features are low-pass filtered and then downsampled proportionally, with the mask being synchronously scaled using a nearest-neighbor approach. For feature locations falling within invalid regions, their mask values ​​are set to zero, ensuring that subsequent joint representation and complementary weighting do not read invalid pixels. Thus, features from different image sources within the same scale layer can correspond to geographic locations, while clear scale indices are maintained between different scale layers.

[0043] S3. Align the generated multi-scale features to a unified cross-scale joint feature representation;

[0044] Feature records from each image source are organized hierarchically by scale, preserving both the image source dimension and the scale dimension. For adjacent layers with different spatial resolutions, upsampling or downsampling is used to unify them to the target grid, and then they are stitched together by channel or weighted element-wise to form a joint representation; the effective area mask undergoes the same coordinate transformation synchronously.

[0045] Integrated expression record can be represented as ,in and The coordinates of the target grid location. This is the feature channel index. During processing, the target mesh at the current scale level is first selected, and features from adjacent scales are transformed to this mesh. When the number of channels is inconsistent, a unified channel dimension is achieved through linear projection or channel selection, and then the data is organized into a joint tensor along the scale dimension. The corresponding effective region mask is expressed as follows: Its source dimension and scale dimension are consistent with the joint tensor. During the alignment process, the original feature records are not overwritten, but the source feature identifier, target grid identifier, and transformation method are recorded so that each component in the integrated representation can be traced back to which image source and scale level it comes from.

[0046] Linear projection is implemented using 1×1 1×1 convolutional or fully connected layers to project features with different numbers of channels to a unified dimension; channel selection involves directly selecting the first N channels or filtering according to fixed rules. The weight matrix of linear projection consists of pre-determined and fixed weights stored in the system.

[0047] The weight matrix can be generated and fixed directly using a random initialization method, such as Xavier uniform initialization; or it can be obtained and fixed through offline training, such as self-supervised or supervised learning based on a multi-source remote sensing image sample set. This invention does not limit the specific values ​​of the weight matrix, as long as they are determined before use and remain unchanged.

[0048] S4. Apply cross-scale consistency constraints to the obtained integrated expression;

[0049] For features at adjacent scale levels of the same geographic location, the low-resolution layer features are first transformed to a high-resolution layer mesh. Then, the distance between the corresponding target mesh location features is calculated, and this distance is used as a consistency constraint term. The distance is Euclidean distance, cosine distance, or a weighted combination of both. After constraint correction, a collaborative representation record is formed, which retains the image source identifier, scale level, and effective area mask.

[0050] Taking Euclidean distance as an example, the scale level starts from the coarsest layer. Arranged down to the finest layer 1. Image source identification. ,according to = The order, with adjacent fine layers Correct the current layer and the set of target mesh locations where both masks are effective. Above calculation:

[0051]

[0052] In the formula, Image source identifier At the current scale level With adjacent fine-scale levels Consistency error between them; Take in sequence Up to 2; The set of target mesh locations for which effective area masks at two scale levels simultaneously take effective values; This represents the number of positions in the set. To scale hierarchy Feature upsampling transformation to scale level Upsampling and resampling operations on the grid; Index of all feature channels The feature vector formed at the target grid location; It represents the sum of squares of the differences between the components.

[0053] The feature vectors are first normalized to 0 to 1 channel by channel. The normalization adopts the channel-by-channel min-max normalization, that is, for each feature channel index, the maximum and minimum values ​​of all valid target grid positions (i.e., positions where the mask value of the effective area is true) under the current image source identifier at the current scale level are counted. The minimum value of the channel is subtracted from each feature value and then divided by the difference between the maximum and minimum values ​​of the channel, so that the feature values ​​of all valid positions of the channel are linearly mapped to the interval [0,1]. Only the positions within the effective area are included in the statistics and mapping, and invalid pixels are not included in the normalization calculation.

[0054] The upsampling and resampling operation uses bilinear interpolation upsampling; when the number of channels is inconsistent, the number of channels is first unified by linear projection. Dividing by the number of positions in the effective position set is used to eliminate the influence of the number of positions; when the effective position set is empty, the scale pair is skipped and marked as having no effective overlapping positions.

[0055] In this embodiment, the dimensionless error threshold is set to a preset value, preferably 0.25. (Regarding image source identification...) Current scale level ( From the coarsest layer up to the second layer 2) and adjacent fine-scale layers Calculate the consistency error on the set of valid locations. Features after correction at the current layer It shall be uniformly determined in the following manner:

[0056] when ( When the error threshold is reached, the integrated representation feature of the current layer is taken as the corrected feature of the current layer, i.e. = ;

[0057] when When the time is right, calculate using the following formula:

[0058] )

[0059] in, and These are the integrated representation feature vectors for the current layer and the adjacent fine layers, respectively. This indicates the scale hierarchy. Feature upsampling transformation to scale level Grid resampling operation, This indicates the corresponding downsampling recovery operation; min indicates taking the smaller value between the value in parentheses and 0.5 to limit the correction weight to no more than 0.5.

[0060] The above process is performed layer by layer from the coarsest layer to the second layer. In each round, only the features of the current layer are updated, without covering the fine layer features used as a reference. The finest layer retains its integrated representation features.

[0061] In this embodiment, the error threshold is the threshold that triggers correction; it is set to 0.25. The correction weight is greater than 0 and does not exceed 0.5. The downsampling and resampling operation is a resampling operation that averages the area according to the inter-layer width-to-height ratio and restores the temporary correction result to the original grid of the current layer. Taking the smaller value means taking the smaller of the two numbers in parentheses. Division means the ratio of the error exceeding the threshold. Multiplication means multiplying the weight by the feature vector component by component, and addition means adding the weighted vector component by component. The corrected features of the current layer have the same grid size and number of channels as the integrated feature representation of the current layer, so the size relationship of each layer from the coarsest to the finest remains unchanged. A collaborative representation record is recorded, which saves the image source identifier, scale level, target grid position, feature channel index, and correction value to form a collaborative representation record.

[0062] S5. Complementary and natural transition of overlapping areas;

[0063] Based on the identified overlapping region, spatially gradient complementary weighting is applied to the output collaborative representation record. Normalized position parameters are established along the overlapping region from image source A to image source B, with the weight of image source A decreasing from 1 to 0 and the weight of image source B increasing from 0 to 1, and the sum of the two weights at the same target grid location constrained to 1; a smooth, non-linear weighting curve can also be used. This complementary weighting is applied at each scale level, forming a fusion feature pyramid that preserves the scale dimension.

[0064] For the target grid location within the overlapping region Calculate the shortest Euclidean distances to the effective area boundaries of image source A and image source B, respectively. and The unit is the target grid pixel, and the shortest Euclidean distance is the distance from the target grid position within the overlapping area. The shortest straight-line distance to the effective area boundaries of image source A and image source B, in pixels of the target grid, is calculated pixel-by-pixel on the raster data by performing a distance transformation algorithm on the effective area masks of each image source. The local transition direction is only used to describe the spatial trend of weight gradients; the actual weight calculation is entirely based on the shortest Euclidean distance. The local transition direction is the shortest continuous direction between the two boundaries of the overlapping area. Linear weights are:

[0065]

[0066]

[0067] That is, the weight of image source A is equal to the distance to image source B divided by the sum of their distances, and the weight of image source B is equal to the distance to image source A divided by the sum of their distances. When the target grid position falls on the boundary of A, the distance to image source A is zero; when it falls on the boundary of B, the distance to image source B is 0. When the two distances are equal, both weights are 0.5. When the denominator is 0, the single-source weight is set according to the image source to which the nearest valid pixel belongs; if the distances on both sides are the same, both weights are set to 0.5. The fusion features at the scale level are:

[0068]

[0069] The fused feature vector at the target grid location at the current scale level is equal to the weight of image source A multiplied by the corrected feature vector of image source A, plus the weight of image source B multiplied by the corrected feature vector of image source B. Here, image source A and image source B are two image source identifiers; the two corrected feature vectors are the feature vectors in the collaborative representation record of the two image source identifiers after correction in step S4; the two weights are the dimensionless weights of the two image source identifiers at the target grid location, ranging from 0 to 1, and their sum is 1; multiplication represents multiplying the weights by the feature vectors component by component, and addition represents adding the two weighted vectors component by component.

[0070] If the effective area mask of an image source identifier at the target grid location is 0, then the weight of the other effective image source identifier is normalized to 1; if both are invalid, the output location is marked as invalid. Features from the collaborative representation record of the single image source identifier covering the target grid location are directly read outside the overlapping area. A fusion feature record is generated at each scale level, recording the scale level, target grid, fusion feature tensor, effective area mask, and weight map, collectively forming the fusion feature pyramid.

[0071] S6. Grading and output from coarse to fine;

[0072] The resulting fusion feature pyramid is processed hierarchically from coarse to fine scale. First, the coarsest scale level is read, and the fusion features of the coarsest layer are used to generate the processing state of that layer. Then, this processing state is upsampled to the next scale level and used together with the fusion features of that layer for refinement to obtain the processing state of that layer. This process is repeated, upsampling the processing state obtained from each layer and using it together with the fusion features of the next layer for refinement, until the finest scale level is reached.

[0073] Each processing round records the input scale level, the previous layer's state identifier, the current layer's fusion feature identifier, and the output state identifier. If a layer's fusion feature is missing, it is replaced by upsampling the most recent effective coarse layer's processing state, and that layer is marked as a downgraded layer. After the finest layer processing is completed, the final result is output, which includes a coordinate reference system, an effective region mask, and the source scale set to drive subsequent parameter learning or other processing techniques.

[0074] The refinement operation is performed by connecting a linear rectified function after convolution. In this embodiment, a 3x3 convolution with a stride of one and zero padding at the edges is used. The convolution parameters are pre-determined and fixed weights. These convolution parameters can be generated and fixed directly using random initialization methods (such as Xavier uniform initialization or Kaiming normal initialization), or they can be obtained and fixed after obtaining network parameters through offline training methods (such as supervised learning or self-supervised learning based on multi-source remote sensing image sample sets). This invention does not limit the specific values ​​of the convolution parameters, as long as they are determined before use and remain unchanged.

[0075] Example 2

[0076] Please see Figure 2 As shown, this embodiment provides a multi-scale remote sensing image feature collaborative hierarchical processing system, including the following units:

[0077] The preprocessing unit performs registration, scale alignment, and relative radiometric normalization on multi-source remote sensing images. It determines the intersection of the effective area masks of each image as the overlapping region and writes the aligned image and overlapping region identifier into the data storage unit. Specifically, the preprocessing unit establishes an image record for each input image. The image record includes at least the image source identifier, acquisition time, band set, pixel size, number of rows and columns, geographic transformation parameters, coordinate reference system, and effective area mask. One image is selected as the target coordinate reference, and the coordinate transformation parameters for the remaining images are obtained through control point pairs or mutual information. After transformation, the effective area mask is resampled using nearest neighbor resampling, and the radiometric values ​​are resampled using bilinear or cubic convolution. The overlapping region is determined by the intersection of the effective area masks of each aligned image, and an overlapping region record with overlapping region identifier, boundary pixel set, and geographic extent is generated. If the number of effective control points is less than the minimum required by the transformation model, or the reprojection error exceeds a preset threshold, the transformation model order is reduced, or the image is marked as unsuitable for fusion.

[0078] The multi-scale feature extraction unit reads the aligned images, constructs a coarse-to-fine image pyramid for each aligned image, extracts features at each layer, and writes the feature records of each layer into the data storage unit. Each layer's feature record includes the image source identifier, scale level, feature tensor, and effective region mask. The multi-scale feature extraction unit constructs a feature pyramid for each image source separately, without merging different image sources in advance at this step. When constructing the next coarse-scale level, the previous layer's image or features are low-pass filtered and downsampled proportionally, with the mask being synchronously scaled using a nearest neighbor method. For feature locations falling within invalid regions, their mask values ​​are set to zero, ensuring that subsequent integrated representation and complementary weighting do not read invalid pixels.

[0079] The integrated representation unit is used to read multi-scale features, transform features at different scale levels to the target grid, and form a cross-scale joint representation that preserves both the image source dimension and scale dimension, i.e., the integrated representation, which is then written into the data storage unit. Specifically, for adjacent levels with different spatial resolutions, upsampling or downsampling is used to unify them to the target grid, and then the joint representation is formed by channel stitching or element-wise weighting; the effective area mask performs the same coordinate transformation synchronously. When the number of channels is inconsistent, the channel dimension is unified through linear projection or channel selection; linear projection is implemented using 1×1 convolution or fully connected layers to project features with different numbers of channels to a unified dimension; channel selection is to directly select the first N channels or filter according to fixed rules; the weight matrix of linear projection is a pre-determined fixed weight stored in the data storage unit. During the alignment process, the original feature record is not overwritten, but the source feature identifier, target grid identifier, and transformation method are recorded so that each component in the integrated representation can be traced back to which image source identifier and scale level it comes from.

[0080] The collaborative representation unit is used to read the integrated representation, apply cross-scale consistency constraints to the integrated representation, compare the features of adjacent scales at the corresponding target grid location and correct them according to the magnitude of the differences, forming a collaborative representation record, which is then written to the data storage unit. Specifically, for features at adjacent scale levels of the same geographical location, the low-resolution layer features are first transformed to the high-resolution layer grid, and then the distance between the features at the corresponding target grid location is calculated, and this distance is used as a consistency constraint term; the distance adopts Euclidean distance, cosine distance, or a weighted combination of the two; after constraint correction, a collaborative representation record is formed, which retains the image source identifier, scale level, and effective area mask.

[0081] The overlapping region complementary unit is used to read the collaborative representation record. Based on the overlapping region, spatially gradual complementary weighting is performed on the collaborative representation features of different image sources within the overlapping region to generate a fusion feature pyramid that retains the scale dimension, and then written to the data storage unit. Specifically, normalized position parameters are established along the overlapping region from image source A to image source B, with the weight of image source A decreasing from 1 to 0 and the weight of image source B increasing from 0 to 1, and the sum of the two weights at the same target grid position constrained to 1; a smooth nonlinear weight curve can also be used. The complementary weighting is performed at each scale level, and a fusion feature record is generated at each scale level, recording the scale level, target grid, fusion feature tensor, effective area mask, and weight map, which together constitute the fusion feature pyramid.

[0082] The hierarchical processing unit reads the fusion feature pyramid and processes it hierarchically from coarse to fine scale. It first reads the coarsest scale to generate a processing state, then upsamples this state to the next scale and uses it together with the fusion features of that layer for refinement. This process is repeated until the finest scale, finally outputting the refined collaborative features and fusion result. In each round of processing, the hierarchical processing unit records the input scale level, the previous layer's state identifier, the current layer's fusion feature identifier, and the output state identifier. If a fusion feature of a certain layer is missing, the most recent valid coarse layer processing state is used for upsampling and the downgraded level is marked. After the finest layer processing is completed, the final result, its coordinate reference system, the effective region mask, and the source scale set are output. The refinement operation is a convolution followed by a linear rectified function. In this embodiment, a 3x3 convolution with a stride of one and zero padding at the edges is used. The convolution parameters are pre-determined and fixed weights stored in the data storage unit.

[0083] The data storage unit persistently stores images, multi-scale features, integrated representations, collaborative representation records, fused feature pyramids, and intermediate results, allowing each unit in the processing chain to read and write back sequentially according to the information flow. Aligned images and multi-scale features are organized in a raster format with geographic coordinates and scale hierarchy identifiers, enabling each unit to access the same object based on geographic location and scale hierarchy. Each unit transmits data through record identifiers: the preprocessing unit outputs image record identifiers and overlapping area record identifiers; the multi-scale feature extraction unit outputs feature record identifiers corresponding to the image source and scale; the integrated representation unit outputs target grid identifiers and channel counts; the collaborative representation unit outputs correction feature record identifiers; the overlapping area complementarity unit outputs fused feature record identifiers for each scale; and the hierarchical processing unit outputs status record identifiers. The data storage unit rejects writing records lacking image source identifiers, scale hierarchy, coordinate reference systems, or effective area masks.

[0084] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for collaborative hierarchical processing of multi-scale remote sensing image features, characterized in that, include: The multi-source remote sensing images were registered, scaled, and radiometrically normalized, and overlapping areas were identified. Multi-scale features of each image source are extracted separately, so that each feature record retains the image source identifier, scale level, feature tensor and effective area mask; Features at different scale levels are transformed to the target grid to form an integrated expression that preserves the image source dimension and scale dimension. On the basis of the integrated expression, consistency constraints are applied to the adjacent scale features at the corresponding target grid position to form a collaborative representation record. Complementary weighting is applied to the collaborative representation records of different image sources within the overlapping area to form a fusion feature pyramid; the fusion feature pyramid is then processed step by step in order of scale hierarchy from coarse to fine and the results are output.

2. The method for collaborative hierarchical processing of multi-scale remote sensing image features according to claim 1, characterized in that, Registration of multi-source remote sensing images includes: obtaining coordinate transformations between images through feature point matching or mutual information, and using resampling to achieve scale alignment; if the registration quality does not meet the preset conditions, the order of the transformation model is reduced or the image is marked as not suitable for fusion.

3. The method for collaborative hierarchical processing of multi-scale remote sensing image features according to claim 1, characterized in that, Transforming features at different scale levels to the target grid includes: upsampling or downsampling adjacent levels with different spatial resolutions to unify them to the target grid, and forming a joint representation by concatenating channels or weighting elements one by one; when the number of channels is inconsistent, unifying the channel dimension is achieved through linear projection or channel selection.

4. The method for collaborative hierarchical processing of multi-scale remote sensing image features according to claim 3, characterized in that, Applying consistency constraints to adjacent scale features of the corresponding target grid location on top of the integrated representation includes: calculating the distance between adjacent scale layer features of the same target grid location as the consistency error; correcting or not correcting the current layer features based on the comparison result of the consistency error and the preset threshold; and processing layer by layer from the coarsest layer to the second layer in a loop, while keeping the finest layer unchanged.

5. The method for collaborative hierarchical processing of multi-scale remote sensing image features according to claim 1, characterized in that, In the overlapping area, the collaborative representation records of different image sources are complementary and weighted to form a fusion feature pyramid. The fusion feature pyramid is processed step by step in order of scale hierarchy from coarse to fine, including: the weights of the two image sources in the overlapping area vary complementaryly along the spatial position and the sum of the weights at the same position is 1. The weights are determined based on the distance from the position to the effective area boundary of each image source. The complementary weighting is performed at each scale level. Then, the coarsest layer fusion feature is used to generate a coarse layer processing state. The previous layer processing state is upsampled and refined together with the current layer fusion feature. This process is carried out step by step until the finest scale level.

6. A collaborative hierarchical processing system based on multi-scale remote sensing image features, characterized in that, include: The preprocessing unit is used to register, scale-align, and radiometrically normalize multi-source remote sensing images, and to determine overlapping areas. The multi-scale feature extraction unit is used to extract multi-scale features from each image source, so that each feature record retains the image source identifier, scale level, feature tensor and effective area mask; An integrated representation unit is used to transform features at different scale levels to the target grid, forming an integrated representation that preserves both the image source dimension and the scale dimension. The collaborative representation unit is used to impose consistency constraints on the adjacent scale features of the corresponding target grid position on top of the integrated representation, forming a collaborative representation record; The overlapping area complementary unit is used to perform complementary weighting on the collaborative representation records of different image sources within the overlapping area to form a fusion feature pyramid. The hierarchical processing unit is used to process the fused feature pyramid step by step in order of scale from coarse to fine and output the results. And a data storage unit, used to persistently store intermediate results in the processing chain, for each unit to read and write back in the order of information flow.

7. The multi-scale remote sensing image feature collaborative hierarchical processing system according to claim 6, characterized in that, The preprocessing unit is configured to: obtain the coordinate transformation between images through feature point matching or mutual information, and achieve scale alignment by resampling; if the registration quality does not meet the preset conditions, the order of the transformation model is reduced or the image is marked as not eligible for fusion.

8. A multi-scale remote sensing image feature collaborative hierarchical processing system according to claim 6, characterized in that, The integrated representation unit is configured to: upsample or downsample adjacent levels with different spatial resolutions to the target grid, and form a joint representation by splicing by channel or weighting element by element; when the number of channels is inconsistent, the channel dimension is unified by linear projection or channel selection.

9. A multi-scale remote sensing image feature collaborative hierarchical processing system according to claim 6, characterized in that, The collaborative representation unit is configured to: calculate the distance between adjacent scale layer features at the same target grid location as the consistency error; correct or not correct the current layer features based on the comparison result of the consistency error and the preset threshold; process layer by layer from the coarsest layer to the second layer in a loop, while keeping the finest layer unchanged.

10. A multi-scale remote sensing image feature collaborative hierarchical processing system according to claim 6, characterized in that, The overlapping area complementary unit is configured such that the weights of the two image sources vary complementaryly along the spatial position and the sum of the weights at the same position is 1. The weights are determined based on the distance from the position to the effective area boundary of each image source, and the complementary weighting is performed at each scale level. The hierarchical processing unit is configured to generate a coarse-level processing state using the coarsest layer fusion features, then upsample the previous layer processing state and refine it together with the current layer fusion features, progressing layer by layer until the finest scale level.

Images