A Multi-Source Remote Sensing Image Classification Method Based on Behavioral Feature Recognition
By using a classification availability mask and block-level orthogonal Procrustes alignment, the problems of cross-sensor differences and data gaps in multi-source remote sensing image classification are solved, achieving higher accuracy and more stable image classification.
Patent Information
- Application Number
- CN202610366488.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-24
- Publication Date
- 2026-11-17
AI Technical Summary
Existing multi-source remote sensing image classification methods suffer from problems such as inconsistent feature spaces, drift of the discrimination boundary after fusion, and category confusion when there are significant differences across sensors, missing data, and noise. They also lack usability constraints at the level of classification units and feature blocks, which leads to low-quality or missing data participating in fusion and amplifying errors.
By employing classification availability masks and block-level mask constraints, block-level orthogonal transformation matrices are obtained through block-level orthogonal Procrustes alignment. Intrinsic points are selected based on alignment residuals and alignment embeddings are iteratively updated to construct a common reference behavior space to achieve unified alignment of multiple sources, thereby reducing the impact of cross-sensor differences and missing data.
It improves the accuracy and stability of multi-source remote sensing image classification, suppresses the interference of low-quality information such as cloud shadow occlusion, speckle noise, and missing holes on fusion classification, and enhances the reliability of alignment relationships and classification accuracy under cross-sensor difference conditions.
Smart Images

Figure CN123119085A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent processing technology for remote sensing images, and in particular to a multi-source remote sensing image classification method based on behavioral feature recognition. Background Technology
[0002] With the development of Earth observation technology, remote sensing images are widely used in scenarios such as urban fine interpretation, ecological monitoring, and disaster assessment. Existing multi-source remote sensing image classification methods typically use pixels, superpixels, or objects as classification units, extracting spectral, texture, structural, and temporal features from various data sources, and then using feature stitching, weighted fusion, or attention mechanisms to achieve classification inference. These methods can achieve certain results when the data quality is high and the differences between sensors are small, but they still have significant shortcomings when there are significant differences across sensors, data gaps, and noise.
[0003] On the one hand, multi-source remote sensing data have differences in resolution, radiometric characteristics, and statistical distribution. Directly stitching features or simply weighting them can easily lead to inconsistencies in the feature space, making it difficult to align the feature representations of the same ground feature under different data sources. This results in drift of the discrimination boundary and confusion of categories after fusion. On the other hand, optical data is often affected by clouds, shadows, and the atmosphere, radar data is affected by speckle noise and geometric distortion, elevation data may contain holes or outliers, and multi-temporal data may have observation gaps and temporal inconsistencies. Existing methods mostly process data with overall confidence or uniform thresholds, lacking availability constraints and missing data control mechanisms at the level of classification units and feature blocks. This leads to low-quality or missing data participating in fusion and amplifying errors.
[0004] Therefore, how to provide a multi-source remote sensing image classification method based on behavioral feature recognition is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] One objective of this invention is to propose a multi-source remote sensing image classification method based on behavioral feature recognition. This invention achieves multi-source behavioral feature block-level alignment through classification availability mask and block-level mask constraints, obtains block-level orthogonal transformation matrix by using block-level orthogonal Procrustes alignment, filters interior points based on alignment residuals and iteratively updates alignment embeddings, and constructs a common reference behavioral space to achieve unified multi-source alignment, reducing the impact of cross-sensor differences and missing data, and improving classification accuracy and stability.
[0006] A multi-source remote sensing image classification method based on behavioral feature recognition according to an embodiment of the present invention includes the following steps: Remote sensing data from different sensor sources for the same target area are acquired and preprocessed to obtain a multi-source remote sensing dataset. The set of classification units is determined from a multi-source remote sensing dataset, and quality and missing labels are generated for each classification unit, which are then converted into a classification availability mask. Using a set of classification units and a classification availability mask, a behavioral feature vector is constructed at each classification unit and divided into a set of behavioral feature blocks; Pair behavioral feature vectors from different data sources to form a cross-source pairing set, and generate a block-level mask based on the classification availability mask; A reference behavior space is selected, and block-level orthogonal Procrustes alignment is performed on each non-reference data source under block-level mask constraints to obtain a block-level orthogonal transformation matrix. The behavior feature vectors of the non-reference data sources are then mapped to the reference behavior space to obtain the alignment embedding. Calculate the alignment residual of the alignment embedding and filter the set of interior points. Iterate and update the block-level orthogonal transformation matrix on the set of interior points and recalculate the alignment embedding until the preset interior point ratio condition and alignment residual condition are met, and obtain the final alignment embedding. A common reference behavior space is constructed by the set of inliers and the final aligned embedding. The behavior feature vectors of each data source are aligned to the common reference behavior space to obtain a unified aligned embedding, which is then used for classification calculation to output the image classification result.
[0007] Optionally, obtaining the multi-source remote sensing dataset specifically includes: Acquire remote sensing data from different sensor sources for the same target area under the same coordinate reference, generate data source description information for each type of remote sensing data, and perform consistent cropping on the coverage area based on the data source description information to form the original multi-source remote sensing data set. Preprocessing is performed on remote sensing data from each sensor source in the original multi-source remote sensing dataset. The preprocessing includes radiometric calibration, atmospheric correction, geometric correction, orthorectification, registration, and resampling, and outputs uniform grid data. The unified grid data is organized according to the sensor source identifier and written to the storage medium to form a multi-source remote sensing dataset.
[0008] Optionally, obtaining the set of classification units and the classification availability mask specifically includes: Read the multi-source remote sensing dataset, determine the classification unit type, perform grid division on the multi-source remote sensing dataset according to the classification unit type, generate classification units, and assign a unique identifier to each classification unit to form a classification unit set; Generate quality and missing labels for each class unit in the class unit set and for each data source in the multi-source remote sensing dataset; A classification availability mask is established, which includes mask elements. Each mask element corresponds one-to-one with a classification unit in the classification unit set and a data source in the data source set. When the quality flag is one and the missing flag is one, the mask element is assigned a value of one. When the quality flag is zero or the missing flag is zero, the mask element is assigned a value of zero. After completing the assignment of mask elements, the classification availability mask is output.
[0009] Optionally, obtaining the set of behavioral feature blocks specifically includes: Iterate through each classification unit and each data source and read the mask value. When the mask value is set, establish a feature buffer and extract the corresponding data of the classification unit. For classification units with a mask value of one and data sources, feature calculations are performed and the data is written to the feature cache. The feature results in the feature cache are appended to form a behavior feature vector, and the append index is recorded to form a segmentation boundary table. The behavior feature vector is then segmented according to the segmentation boundary table to form a set of behavior feature blocks. Optionally, obtaining the block-level mask and the cross-source pairing set specifically includes: Using the identifier of the classification unit in the classification unit set as the pairing key, data source pairs are formed in the data source set according to the order of the data source identifier. For each data source pair, the behavioral feature vector, behavioral feature block set, and classification availability mask on both sides of the data source pair are obtained according to the pairing key, forming a cross-source pairing set. For each cross-source pairing set, check the classification availability mask of the data source on both sides. If the classification availability mask is zero, retain the cross-source pairing. If the classification availability mask is zero, delete the cross-source pairing. After deletion, the cross-source pairing set is obtained. Read the behavior feature block set one by one for the cross-source pairing set, generate block-level masks in order of block number. When the data source can locate the behavior feature block corresponding to the block number in both sides of the behavior feature block set, set the block-level mask to one. When there is a behavior feature block corresponding to the block number that cannot be located, set the block-level mask to zero. Complete the block number generation and output the block-level mask.
[0010] Optionally, the block-level orthogonal transformation matrix and the alignment embedding are obtained by specifically including: Select the data source with the smallest data source identifier from the data source set as the reference data source, and set the vector space where the behavioral feature vector of the reference data source is located as the reference behavioral space. Set the data sources other than the reference data source in the data source set as non-reference data sources. Under block-level mask constraints, cross-source pairs corresponding to reference data sources and non-reference data sources are selected from the cross-source pairing set. Block numbers with block-level masks set to one are screened out in order of block number. Reference components and non-reference components are extracted from the behavioral feature vectors of the reference data source and the behavioral feature vectors of the non-reference data source respectively according to the screened block numbers, and stacked in rows to form reference block matrices and non-reference block matrices. Centering is performed on the reference block matrix and the non-reference block matrix respectively. The column mean vector is calculated, and the column mean vector of the reference block matrix is subtracted element by element from the corresponding column of the reference block matrix. The column mean vector of the non-reference block matrix is subtracted element by element from the corresponding column of the non-reference block matrix to obtain the centered reference block matrix and the centered non-reference block matrix respectively. Block-level orthogonal Procrustes alignment is performed on the centralized non-reference block matrix and the centralized reference block matrix. The product matrix of the transpose of the centralized non-reference block matrix and the centralized reference block matrix is calculated, and singular value decomposition is performed on the product matrix to obtain the left orthogonal matrix, the singular value diagonal matrix, and the right orthogonal matrix. The product of the transpose of the left orthogonal matrix and the right orthogonal matrix is calculated to obtain the initial block-level orthogonal transformation matrix. The row-by-row alignment residuals are calculated using the initial block-level orthogonal transformation matrix and sorted in ascending order. At the same time, a truncated interior point re-estimation mechanism is performed to truncate the sorted results according to a preset ratio to obtain the interior point row set. The rows of the centralized non-reference block matrix and the centralized reference block matrix are extracted from the interior point row set to obtain the interior point non-reference block matrix and the interior point reference block matrix. The product matrix calculation, singular value decomposition, and matrix product calculation are repeated for the interior point non-reference matrix and the interior point reference block matrix to obtain the block-level orthogonal transformation matrix. Based on block-level mask constraints, non-reference components are extracted from the behavioral feature vectors of non-reference data sources. The non-reference components are then right-multiplied by a block-level orthogonal transformation matrix to obtain aligned components. The aligned components are then written back according to block numbers to form aligned embeddings.
[0011] Optionally, obtaining the final aligned embedding specifically includes: Obtain the cross-source pairing set, block-level mask, alignment embedding, and block-level orthogonal transformation matrix. Set the preset inlier ratio condition as the minimum ratio threshold of the number of inliers to the number of cross-source pairing sets, and set the preset alignment residual condition as the maximum threshold of the alignment residual statistic. For each cross-source pairing in the cross-source pairing set, the alignment embedding vector of the reference data source and the alignment embedding vector of the non-reference data source are extracted from the alignment embedding. Vector difference is performed to obtain the difference vector, and the alignment residual is calculated based on the difference vector to form an alignment residual sequence. The alignment residual sequence is sorted in ascending order to obtain a sorted sequence. The number of inliers is calculated according to the preset inlier ratio condition and the number of cross-source pairing sets. The cross-source pairing index corresponding to the number of inliers in the sorted sequence is selected to form an inlier set. Extract the behavioral feature vectors of the reference data source and the non-reference data source from the cross-source pairing set according to the inlier set, and retain the block-level mask constraints. Update the block-level orthogonal transformation matrix, and use the updated block-level orthogonal transformation matrix to perform mapping and write-back on the behavioral feature vectors of the non-reference data source to generate the updated aligned embedding. Recalculate the alignment residuals on the inlier set and summarize them to obtain the average alignment residual of the inlier set. Calculate the ratio of the number of inlier sets to the number of cross-source pairing sets to obtain the inlier ratio. When the inlier ratio meets the preset inlier ratio condition and the average alignment residual of the inlier set meets the preset alignment residual condition, output the final alignment embedding. Otherwise, replace the alignment embedding with the updated alignment embedding and continue to perform alignment residual calculation, inlier filtering, block-level orthogonal transformation matrix update, and alignment embedding recalculation until the final alignment embedding is output. Optionally, the output of the image classification result specifically includes: Establish a common reference vector set, locate the corresponding reference side alignment vector in the inlier set one by one, and write it into the common reference vector set; The common reference vector set is accumulated dimension by dimension, and the common reference vector is obtained by averaging dimension by dimension according to the number of vectors. The vector space where the common reference vector is located is set as the common reference behavior space. For each classification unit in the classification unit set, the behavior feature vector and corresponding block-level mask are read from the data source in the data source set. The behavior feature vector is processed in the order of block number. When the block-level mask is set to one, the block component is truncated and multiplied by the block-level orthogonal transformation matrix to obtain the aligned block component. When the block-level mask is set to zero, the original value of the block component is preserved. The aligned block component and the preserved block component are written back according to the block number to generate a common reference behavior space alignment vector. By selecting the common reference behavior space alignment vector of the same classification unit under various data sources, performing dimension-wise cumulative summation, participation count statistics and dimension-wise averaging, a unified alignment embedding is obtained. The class score is calculated for the uniformly aligned embedding, the class corresponding to the highest score is selected as the class label of the classification unit, and the class labels are output according to the set of classification units to form the image classification result.
[0012] The beneficial effects of this invention are: This invention constrains the availability information of classification units under various data sources through a classification availability mask, and generates a block-level mask based on cross-source pairing, so that the behavioral feature vector completes availability screening before entering the alignment stage, reducing the interference of low-quality information such as cloud shadow occlusion, speckle noise, and missing data holes on fusion classification, thereby improving the stability of multi-source remote sensing image classification.
[0013] This invention performs block-level orthogonal Procrustes alignment under block-level mask constraints, obtains a block-level orthogonal transformation matrix, and maps the behavioral feature vectors of non-reference data sources to the reference behavioral space. Combined with alignment residual calculation and interior point set screening, it achieves iterative updates and recalculation of alignment embedding, suppresses the influence of abnormal pairings and outlier samples on transformation estimation, and improves the reliability of alignment relationships and classification accuracy under cross-sensor difference conditions.
[0014] This invention constructs a common reference behavior space by using an interior point set and a final aligned embedding, and aligns the behavior feature vectors of each data source to the common reference behavior space to obtain a unified aligned embedding. Then, it performs classification calculations to output image classification results, making multi-source information comparable and convergent within the same reference system, reducing the result fluctuations caused by changes in the combination of different data sources, and further improving the accuracy and consistency of multi-source remote sensing image classification. Attached Figure Description
[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a multi-source remote sensing image classification method based on behavioral feature recognition proposed in this invention; Figure 2 This is a schematic diagram of block-level orthogonal Procrustes alignment under block-level mask constraints for a multi-source remote sensing image classification method based on behavioral feature recognition proposed in this invention. Detailed Implementation
[0016] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0017] refer to Figures 1-2 A multi-source remote sensing image classification method based on behavioral feature recognition includes the following steps: Remote sensing data from different sensor sources for the same target area are acquired and preprocessed to obtain a multi-source remote sensing dataset. The set of classification units is determined from a multi-source remote sensing dataset, and quality and missing labels are generated for each classification unit, which are then converted into a classification availability mask. Using a set of classification units and a classification availability mask, a behavioral feature vector is constructed at each classification unit and divided into a set of behavioral feature blocks; Pair behavioral feature vectors from different data sources to form a cross-source pairing set, and generate a block-level mask based on the classification availability mask; A reference behavior space is selected, and block-level orthogonal Procrustes alignment is performed on each non-reference data source under block-level mask constraints to obtain a block-level orthogonal transformation matrix. The behavior feature vectors of the non-reference data sources are then mapped to the reference behavior space to obtain the alignment embedding. Calculate the alignment residual of the alignment embedding and filter the set of interior points. Iterate and update the block-level orthogonal transformation matrix on the set of interior points and recalculate the alignment embedding until the preset interior point ratio condition and alignment residual condition are met, and obtain the final alignment embedding. A common reference behavior space is constructed by the set of inliers and the final aligned embedding. The behavior feature vectors of each data source are aligned to the common reference behavior space to obtain a unified aligned embedding, which is then used for classification calculation to output the image classification result.
[0018] In this embodiment, obtaining the multi-source remote sensing dataset specifically includes: Acquire remote sensing data from different sensor sources for the same target area under the same coordinate reference. Generate data source description information for each type of remote sensing data. The data source description information includes sensor type, imaging time, spatial resolution, band setting, projection coordinate system, and coverage area. Perform uniform clipping on the coverage area based on the data source description information to form the original multi-source remote sensing data set. Preprocessing is performed on remote sensing data from each sensor source in the original multi-source remote sensing dataset. The preprocessing includes radiometric calibration, atmospheric correction, geometric correction, orthorectification, registration, and resampling, and outputs uniform grid data. The unified grid data is organized according to the sensor source identifier and written to the storage medium to form a multi-source remote sensing dataset.
[0019] In this embodiment, obtaining the classification unit set and the classification availability mask specifically includes: Read the multi-source remote sensing dataset, determine the classification unit type, which includes pixel unit, superpixel unit, and ground object unit, and perform grid division on the multi-source remote sensing dataset according to the classification unit type to generate classification units. Assign a unique identifier to each classification unit to form a classification unit set. For each classification unit in the classification unit set and each data source in the multi-source remote sensing dataset, a quality label and a missing label are generated. The quality label has a value of 1 to indicate that the preset quality conditions are met and a value of 0 to indicate that the preset quality conditions are not met. The missing label has a value of 1 to indicate that the missing data does not exist and a value of 0 to indicate that the missing data exists. Create a classification availability mask. The classification availability mask contains mask elements. Each mask element corresponds one-to-one with a classification unit in the classification unit set and a data source in the data source set. When the quality flag is 1 and the missing flag is 1, the mask element is assigned the value 1. When the quality flag is 0 or the missing flag is 0, the mask element is assigned the value 0. After completing the assignment of mask elements, output the classification availability mask.
[0020] In this embodiment, obtaining the behavioral feature block set specifically includes: Iterate through each classification unit and each data source and read the mask value. When the mask value is set, establish a feature buffer and extract the corresponding data of the classification unit. For classification units and data sources with a mask value of one, feature calculations are performed and written to the feature cache. Feature calculations include band statistics calculation, exponential calculation, radar statistics and texture calculation, elevation statistics and topographic calculation, temporal difference and trend fitting. The feature results in the feature cache are appended to form a behavior feature vector, and the append index is recorded to form a segmentation boundary table. The behavior feature vector is then segmented according to the segmentation boundary table to form a set of behavior feature blocks. In this embodiment, obtaining the block-level mask and the cross-source pairing set specifically includes: Using the identifier of the classification unit in the classification unit set as the pairing key, data source pairs are formed in the data source set according to the order of the data source identifier. For each data source pair, the behavioral feature vector, behavioral feature block set, and classification availability mask on both sides of the data source pair are obtained according to the pairing key, forming a cross-source pairing set. For each cross-source pairing set, check the classification availability mask of the data source on both sides. If the classification availability mask is zero, retain the cross-source pairing. If the classification availability mask is zero, delete the cross-source pairing. After deletion, the cross-source pairing set is obtained. Read the behavior feature block set one by one for the cross-source pairing set, generate block-level masks in order of block number. When the data source can locate the behavior feature block corresponding to the block number in both sides of the behavior feature block set, set the block-level mask to one. When there is a behavior feature block corresponding to the block number that cannot be located, set the block-level mask to zero. Complete the block number generation and output the block-level mask.
[0021] In this embodiment, the acquisition of the block-level orthogonal transformation matrix and the alignment embedding specifically includes: Select the data source with the smallest data source identifier from the data source set as the reference data source, and set the vector space where the behavioral feature vector of the reference data source is located as the reference behavioral space. Set the data sources other than the reference data source in the data source set as non-reference data sources. Under block-level mask constraints, cross-source pairs corresponding to reference data sources and non-reference data sources are selected from the cross-source pairing set. Block numbers with block-level masks set to 1 are filtered out in block number order. Reference components and non-reference components are extracted from the behavioral feature vectors of the reference data source and the non-reference data source respectively according to the filtered block numbers, and stacked in rows to form reference block matrices and non-reference block matrices. Block-level mask constraints mean that the behavioral feature blocks corresponding to block numbers with block-level masks set to 1 participate in block-level orthogonal Procrustes alignment and alignment mapping, while the behavioral feature blocks corresponding to block numbers with block-level masks set to 0 do not participate in block-level orthogonal Procrustes alignment and alignment mapping. Centering is performed on the reference block matrix and the non-reference block matrix respectively. The column mean vector is calculated, and the column mean vector of the reference block matrix is subtracted element by element from the corresponding column of the reference block matrix. The column mean vector of the non-reference block matrix is subtracted element by element from the corresponding column of the non-reference block matrix to obtain the centered reference block matrix and the centered non-reference block matrix respectively. Block-level orthogonal Procrustes alignment is performed on the centralized non-reference block matrix and the centralized reference block matrix. The product matrix of the transpose of the centralized non-reference block matrix and the centralized reference block matrix is calculated, and singular value decomposition is performed on the product matrix to obtain the left orthogonal matrix, the singular value diagonal matrix, and the right orthogonal matrix. The product of the transpose of the left orthogonal matrix and the right orthogonal matrix is calculated to obtain the initial block-level orthogonal transformation matrix. The row-by-row alignment residuals are calculated using the initial block-level orthogonal transformation matrix and sorted in ascending order. At the same time, a truncated interior point re-estimation mechanism is performed to truncate the sorted results according to a preset ratio to obtain the interior point row set. The rows of the centralized non-reference block matrix and the centralized reference block matrix are extracted from the interior point row set to obtain the interior point non-reference block matrix and the interior point reference block matrix. The product matrix calculation, singular value decomposition, and matrix product calculation are repeated for the interior point non-reference matrix and the interior point reference block matrix to obtain the block-level orthogonal transformation matrix. Based on block-level mask constraints, non-reference components are extracted from the behavioral feature vectors of non-reference data sources. The non-reference components are then right-multiplied by a block-level orthogonal transformation matrix to obtain aligned components. The aligned components are then written back according to block numbers to form aligned embeddings.
[0022] This invention performs orthogonal Procrustes alignment on behavioral feature blocks under block-level mask constraints and introduces truncated interior point re-estimation to obtain a block-level orthogonal transformation matrix. This ensures that cross-source behavioral features maintain geometric consistency in the reference behavioral space and suppresses interference from anomalous blocks and noisy samples. In turn, it stably maps the behavioral feature vectors of non-reference data sources to aligned embeddings, thereby improving the reliability of cross-sensor alignment, the consistency of alignment embeddings, and the accuracy and stability of subsequent classification.
[0023] In this embodiment, the final alignment and embedding are obtained specifically including: Obtain the cross-source pairing set, block-level mask, alignment embedding, and block-level orthogonal transformation matrix. Set the preset inlier ratio condition as the minimum ratio threshold of the number of inliers to the number of cross-source pairing sets, and set the preset alignment residual condition as the maximum threshold of the alignment residual statistic. For each cross-source pairing in the cross-source pairing set, the alignment embedding vector of the reference data source and the alignment embedding vector of the non-reference data source are extracted from the alignment embedding. Vector difference is performed to obtain the difference vector, and the alignment residual is calculated based on the difference vector to form an alignment residual sequence. The alignment residual sequence is sorted in ascending order to obtain a sorted sequence. The number of inliers is calculated according to the preset inlier ratio condition and the number of cross-source pairing sets. The cross-source pairing index corresponding to the number of inliers in the sorted sequence is selected to form an inlier set. Extract the behavioral feature vectors of the reference data source and the non-reference data source from the cross-source pairing set according to the inlier set, and retain the block-level mask constraints. Update the block-level orthogonal transformation matrix. Use the updated block-level orthogonal transformation matrix to perform mapping write-back on the behavioral feature vectors of the non-reference data source to generate the updated aligned embedding. The update of the block-level orthogonal transformation matrix refers to reconstructing the block matrix under the inlier set and block-level mask constraints and re-finding the orthogonal alignment transformation matrix. Recalculate the alignment residuals on the inlier set and summarize them to obtain the average alignment residual of the inlier set. Calculate the ratio of the number of inlier sets to the number of cross-source pairing sets to obtain the inlier ratio. When the inlier ratio meets the preset inlier ratio condition and the average alignment residual of the inlier set meets the preset alignment residual condition, output the final alignment embedding. Otherwise, replace the alignment embedding with the updated alignment embedding and continue to perform alignment residual calculation, inlier filtering, block-level orthogonal transformation matrix update, and alignment embedding recalculation until the final alignment embedding is output. This invention employs an alignment residual-driven interior point screening and iterative update mechanism to robustly re-estimate the block-level orthogonal transformation matrix and continuously recalculate the alignment embedding under block-level mask constraints. This significantly reduces the risk of transformation estimation being dominated by anomalous pairings and noisy samples, and improves the consistency and transferability of cross-source behavioral representations. As a result, it stably obtains the final aligned embedding under conditions of cloud shadow occlusion, speckle noise, missing measurement holes, and temporal differences, thereby improving classification accuracy and stability.
[0024] In this embodiment, the output of the image classification result specifically includes: Establish a common reference vector set, locate the corresponding reference side alignment vector in the inlier set one by one, and write it into the common reference vector set; The common reference vector set is accumulated dimension by dimension, and the common reference vector is obtained by averaging dimension by dimension according to the number of vectors. The vector space where the common reference vector is located is set as the common reference behavior space. For each classification unit in the classification unit set, the behavior feature vector and corresponding block-level mask are read from the data source in the data source set. The behavior feature vector is processed in the order of block number. When the block-level mask is set to one, the block component is truncated and multiplied by the block-level orthogonal transformation matrix to obtain the aligned block component. When the block-level mask is set to zero, the original value of the block component is preserved. The aligned block component and the preserved block component are written back according to the block number to generate a common reference behavior space alignment vector. By selecting the common reference behavior space alignment vector of the same classification unit under various data sources, performing dimension-wise summation, participation count statistics, and dimension-wise averaging, a unified alignment embedding is obtained. Dimension-wise summation refers to summing the common reference behavior space alignment vector in each dimension to obtain the dimension and vector. Participation count statistics refers to counting the number of alignment vectors participating in the summation to obtain the quantity value. Dimension-wise averaging refers to dividing each dimension of the dimension and vector by the quantity value to obtain the unified alignment embedding. The class score is calculated for the uniformly aligned embedding, and the class corresponding to the highest score is selected as the class label of the classification unit. The class labels are output according to the set of classification units to form the image classification result. The image classification result refers to the multi-source remote sensing image classification map formed by writing the class labels obtained by the uniformly aligned embedding to the classification unit position and mapping them back to the spatial grid of the target area, using the set of classification units as the index.
[0025] This invention constructs a common reference behavior space by using an interior point set and aligns multi-source behavior features uniformly under block-level mask constraints. It employs block-level orthogonal transformation to achieve consistency across sensor feature spaces and fuses them within the same reference system to generate a unified aligned embedding. The consistent embedding is then used to complete category discrimination and output a multi-source remote sensing image classification map. This reduces the impact of cloud shadow occlusion, speckle noise, missing measurement holes, and temporal differences on the results, thereby improving the classification accuracy, stability, and cross-scene robustness of multi-source remote sensing images.
[0026] Example 1: To verify the feasibility of this invention in practice, it was applied to a multi-source remote sensing image classification task in a typical complex land cover area. The target area simultaneously includes typical land cover types such as densely built-up areas, hardened road areas, bare construction areas, farmland, woodland, shrubland, water bodies, and wetlands. The surface textures are significantly different, and there are phenomena such as fragmented plot boundaries, mixed pixels of land cover, and similar class spectra. Optical observations show radiation anomalies caused by cloud shadows and thin cloud transmission. Radar observations show speckle noise and local geometric distortion. Holes and abnormal elevation values are visible in the elevation raster. These problems lead to inconsistencies in cross-source features, fusion failures, and fluctuations in classification results when using conventional feature stitching and conventional fusion classification. This example focuses on verifying the improvement effect of this invention on the above problems through mechanisms such as classification availability mask, block-level orthogonal Procrustes alignment constrained by block-level mask, iterative update of alignment residual interior points, and unified alignment of common reference behavior space.
[0027] When applying this invention in this scenario, firstly, a set of classification units is determined based on a multi-source remote sensing dataset. For each classification unit, quality and missing labels are generated under each data source and converted into a classification availability mask. For optical data, the quality label is set to zero in cloud shadow areas and zeroed according to a threshold strategy in thin cloud areas. The missing label is set to zero in areas with no effective radiance. For radar data, the quality label is set to zero in areas with low signal-to-noise ratio and zeroed in areas with significant geometric distortion. For elevation raster, the missing label is set to zero in hole areas and the quality label is set to zero in areas with abnormal elevation. After completion, a classification availability mask is obtained and used for subsequent feature construction and pairing selection. Subsequently, a behavioral feature vector is constructed at each classification unit and divided into a set of behavioral feature blocks based on spectral behavior, scattering behavior, structural behavior, and change behavior. Spectral behavior comes from optical band statistics and exponential results, scattering behavior comes from radar amplitude statistics and texture results, structural behavior comes from elevation statistics and slope roughness results, and change behavior comes from multi-temporal sequence difference and trend fitting results. The behavioral feature vectors are written into and recorded in a segment boundary table in a unified order. After completion, behavioral feature vectors and a set of behavioral feature blocks are obtained.
[0028] Next, behavioral feature vectors from different data sources are paired to form a cross-source pairing set. A block-level mask is then generated based on the classification availability mask. This block-level mask controls the behavioral feature blocks participating in alignment at the block level. Cloud shadows cause spectral behavioral blocks to be zeroed out, voids cause structural behavioral blocks to be zeroed out, and radar distortion causes scattering behavioral blocks to be zeroed out. Subsequently, a reference behavioral space is selected, and block-level orthogonal Procrustes alignment is performed on non-reference data sources to obtain the block-level orthogonal transformation matrix. Block-level mask constraints are used to filter the block indices participating in alignment, and a reference block matrix and a non-reference block matrix are constructed. After centering the matrix, singular value decomposition is performed to obtain the initial block-level orthogonal transformation matrix. Then, a truncation interior point re-estimation mechanism is used using row-wise alignment residual sorting to obtain a more stable block-level orthogonal transformation matrix. Finally, the behavioral feature vectors from the non-reference data sources are mapped to the reference behavioral space to obtain the alignment embedding.
[0029] The alignment residuals of the alignment embedding are further calculated, and the set of inliers is filtered. The block-level orthogonal transformation matrix is iteratively updated on the set of inliers, and the alignment embedding is recalculated until the preset inlier ratio condition and alignment residual condition are met to obtain the final alignment embedding. Finally, a common reference behavior space is constructed from the set of inliers and the final alignment embedding. The behavior feature vectors of each data source are aligned to the common reference behavior space and converged into a unified alignment embedding. The unified alignment embedding is used to calculate the category label of the classification unit through category score and mapped back to the target region spatial grid to form the image classification result.
[0030] To demonstrate the beneficial effects, this embodiment sets up a comparison method and evaluates the results on the same data partition. The evaluation metrics used are overall accuracy, Kappa coefficient, macro-average F1, and key category F1. Robustness under cloud occlusion and missing data conditions is also evaluated. Comparison method one is multi-source feature direct concatenation followed by random forest classification; comparison method two is multi-source feature direct concatenation followed by support vector machine classification; comparison method three is attention-fusion multi-branch convolutional network; comparison method four is a global Procrustes alignment method without block-level mask constraints; comparison method five is an ablation version of this invention that removes alignment residual interior point iteration updates; and comparison method six is an ablation version of this invention that removes the common reference behavior space. Specific comparison data are shown in Table 1. Table 1. Comparison of different multi-source remote sensing image classification methods
[0031] As shown in Table 1, the complete solution of this invention achieves the best performance in overall accuracy OA, Kappa, and macro-average F1, reaching 91.80%, 0.89, and 0.91, respectively, which is a significant improvement compared to 83.20%, 0.79, and 0.82 for multi-source splicing and random forest. In terms of key category indicators, the F1 scores of the complete solution of this invention for roads, wetlands, and bare land are 0.87, 0.81, and 0.80, respectively, all of which are higher than the 0.83, 0.75, and 0.76 of attention fusion convolutional networks and the 0.84, 0.76, and 0.77 of overall Procrustes alignment without block-level masking, indicating that alignment and mask constraints have a benefit for easily confused categories. In robustness tests with approximately 50% effective optical coverage, the complete solution of this invention still maintains a robustness OA of 88.70% and a robustness macro F1 of 0.89, which is better than the 79.20% and 0.77 of multi-source splicing and random forest, and also better than the 84.60% and 0.84 of attention fusion convolutional networks. This indicates that the present invention has more stable classification output under conditions of missing values and quality inconsistency. Ablation comparison shows that after removing the interior point iterative update, the OA decreases to 89.50%, and after removing the common reference behavior space, the OA decreases to 90.10%, further confirming the contribution of interior point iterative update and common reference behavior space to performance improvement.
[0032] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A multi-source remote sensing image classification method based on behavioral feature recognition, characterized in that, Includes the following steps: Remote sensing data from different sensor sources for the same target area are acquired and preprocessed to obtain a multi-source remote sensing dataset. The set of classification units is determined from a multi-source remote sensing dataset, and quality and missing labels are generated for each classification unit, which are then converted into a classification availability mask. Using a set of classification units and a classification availability mask, a behavioral feature vector is constructed at each classification unit and divided into a set of behavioral feature blocks; Pair behavioral feature vectors from different data sources to form a cross-source pairing set, and generate a block-level mask based on the classification availability mask; A reference behavior space is selected, and block-level orthogonal Procrustes alignment is performed on each non-reference data source under block-level mask constraints to obtain a block-level orthogonal transformation matrix. The behavior feature vectors of the non-reference data sources are then mapped to the reference behavior space to obtain the alignment embedding. Calculate the alignment residual of the alignment embedding and filter the set of interior points. Iterate and update the block-level orthogonal transformation matrix on the set of interior points and recalculate the alignment embedding until the preset interior point ratio condition and alignment residual condition are met, and obtain the final alignment embedding. A common reference behavior space is constructed by the set of inliers and the final aligned embedding. The behavior feature vectors of each data source are aligned to the common reference behavior space to obtain a unified aligned embedding, which is then used for classification calculation to output the image classification result.
2. The multi-source remote sensing image classification method based on behavioral feature recognition according to claim 1, characterized in that, The acquisition of the multi-source remote sensing dataset specifically includes: Acquire remote sensing data from different sensor sources for the same target area under the same coordinate reference, generate data source description information for each type of remote sensing data, and perform consistent cropping on the coverage area based on the data source description information to form the original multi-source remote sensing data set. Preprocessing is performed on remote sensing data from each sensor source in the original multi-source remote sensing dataset. The preprocessing includes radiometric calibration, atmospheric correction, geometric correction, orthorectification, registration, and resampling, and outputs uniform grid data. The unified grid data is organized according to the sensor source identifier and written to the storage medium to form a multi-source remote sensing dataset.
3. The multi-source remote sensing image classification method based on behavioral feature recognition according to claim 1, characterized in that, The specific steps involved in obtaining the set of classification units and the classification availability mask are as follows: Read the multi-source remote sensing dataset, determine the classification unit type, perform grid division on the multi-source remote sensing dataset according to the classification unit type, generate classification units, and assign a unique identifier to each classification unit to form a classification unit set; Generate quality and missing labels for each class unit in the class unit set and for each data source in the multi-source remote sensing dataset; A classification availability mask is established, which includes mask elements. Each mask element corresponds one-to-one with a classification unit in the classification unit set and a data source in the data source set. When the quality flag is one and the missing flag is one, the mask element is assigned a value of one. When the quality flag is zero or the missing flag is zero, the mask element is assigned a value of zero. After completing the assignment of mask elements, the classification availability mask is output.
4. The multi-source remote sensing image classification method based on behavioral feature recognition according to claim 1, characterized in that, The specific steps to obtain the set of behavioral feature blocks include: Iterate through each classification unit and each data source and read the mask value. When the mask value is set, establish a feature buffer and extract the corresponding data of the classification unit. For classification units with a mask value of one and data sources, feature calculations are performed and the data is written to the feature cache. The feature results from the feature cache are appended to form a behavior feature vector, and the append index is recorded to form a segmentation boundary table. The behavior feature vector is then segmented according to the segmentation boundary table to form a set of behavior feature blocks.
5. The multi-source remote sensing image classification method based on behavioral feature recognition according to claim 1, characterized in that, The specific steps involved in obtaining the block-level mask and the cross-source pairing set are as follows: Using the identifier of the classification unit in the classification unit set as the pairing key, data source pairs are formed in the data source set according to the order of the data source identifier. For each data source pair, the behavioral feature vector, behavioral feature block set, and classification availability mask on both sides of the data source pair are obtained according to the pairing key, forming a cross-source pairing set. For each cross-source pairing set, check the classification availability mask of the data source on both sides. If the classification availability mask is zero, retain the cross-source pairing. If the classification availability mask is zero, delete the cross-source pairing. After deletion, the cross-source pairing set is obtained. Read the behavior feature block set one by one for the cross-source pairing set, generate block-level masks in order of block number. When the data source can locate the behavior feature block corresponding to the block number in both sides of the behavior feature block set, set the block-level mask to one. When there is a behavior feature block corresponding to the block number that cannot be located, set the block-level mask to zero. Complete the block number generation and output the block-level mask.
6. The multi-source remote sensing image classification method based on behavioral feature recognition according to claim 1, characterized in that, The specific steps for obtaining the block-level orthogonal transformation matrix and the alignment embedding include: Select the data source with the smallest data source identifier from the data source set as the reference data source, and set the vector space where the behavioral feature vector of the reference data source is located as the reference behavioral space. Set the data sources other than the reference data source in the data source set as non-reference data sources. Under block-level mask constraints, cross-source pairs corresponding to reference data sources and non-reference data sources are selected from the cross-source pairing set. Block numbers with block-level masks set to one are screened out in order of block number. Reference components and non-reference components are extracted from the behavioral feature vectors of the reference data source and the behavioral feature vectors of the non-reference data source respectively according to the screened block numbers, and stacked in rows to form reference block matrices and non-reference block matrices. Centering is performed on the reference block matrix and the non-reference block matrix respectively. The column mean vector is calculated, and the column mean vector of the reference block matrix is subtracted element by element from the corresponding column of the reference block matrix. The column mean vector of the non-reference block matrix is subtracted element by element from the corresponding column of the non-reference block matrix to obtain the centered reference block matrix and the centered non-reference block matrix respectively. Block-level orthogonal Procrustes alignment is performed on the centralized non-reference block matrix and the centralized reference block matrix. The product matrix of the transpose of the centralized non-reference block matrix and the centralized reference block matrix is calculated, and singular value decomposition is performed on the product matrix to obtain the left orthogonal matrix, the singular value diagonal matrix, and the right orthogonal matrix. The product of the transpose of the left orthogonal matrix and the right orthogonal matrix is calculated to obtain the initial block-level orthogonal transformation matrix. The row-by-row alignment residuals are calculated using the initial block-level orthogonal transformation matrix and sorted in ascending order. At the same time, a truncated interior point re-estimation mechanism is performed to truncate the sorted results according to a preset ratio to obtain the interior point row set. The rows of the centralized non-reference block matrix and the centralized reference block matrix are extracted from the interior point row set to obtain the interior point non-reference block matrix and the interior point reference block matrix. The product matrix calculation, singular value decomposition, and matrix product calculation are repeated for the interior point non-reference matrix and the interior point reference block matrix to obtain the block-level orthogonal transformation matrix. Based on block-level mask constraints, non-reference components are extracted from the behavioral feature vectors of non-reference data sources. The non-reference components are then right-multiplied by a block-level orthogonal transformation matrix to obtain aligned components. The aligned components are then written back according to block numbers to form aligned embeddings.
7. The multi-source remote sensing image classification method based on behavioral feature recognition according to claim 1, characterized in that, The final alignment embedding is obtained specifically by: Obtain the cross-source pairing set, block-level mask, alignment embedding, and block-level orthogonal transformation matrix. Set the preset inlier ratio condition as the minimum ratio threshold of the number of inliers to the number of cross-source pairing sets, and set the preset alignment residual condition as the maximum threshold of the alignment residual statistic. For each cross-source pairing in the cross-source pairing set, the alignment embedding vector of the reference data source and the alignment embedding vector of the non-reference data source are extracted from the alignment embedding. Vector difference is performed to obtain the difference vector, and the alignment residual is calculated based on the difference vector to form an alignment residual sequence. The alignment residual sequence is sorted in ascending order to obtain a sorted sequence. The number of inliers is calculated according to the preset inlier ratio condition and the number of cross-source pairing sets. The cross-source pairing index corresponding to the number of inliers in the sorted sequence is selected to form an inlier set. Extract the behavioral feature vectors of the reference data source and the non-reference data source from the cross-source pairing set according to the inlier set, and retain the block-level mask constraints. Update the block-level orthogonal transformation matrix, and use the updated block-level orthogonal transformation matrix to perform mapping and write-back on the behavioral feature vectors of the non-reference data source to generate the updated aligned embedding. The alignment residuals are recalculated on the inlier set and the average alignment residual of the inlier set is obtained. The ratio of the number of inlier sets to the number of cross-source pairing sets is calculated to obtain the inlier ratio. When the inlier ratio meets the preset inlier ratio condition and the average alignment residual of the inlier set meets the preset alignment residual condition, the final alignment embedding is output. Otherwise, the alignment embedding is replaced with the updated alignment embedding and the alignment residual calculation, inlier filtering, block-level orthogonal transformation matrix update, and alignment embedding recalculation are continued until the final alignment embedding is output.
8. The multi-source remote sensing image classification method based on behavioral feature recognition according to claim 1, characterized in that, The output of the image classification results specifically includes: Establish a common reference vector set, locate the corresponding reference side alignment vector in the inlier set one by one, and write it into the common reference vector set; The common reference vector set is accumulated dimension by dimension, and the common reference vector is obtained by averaging dimension by dimension according to the number of vectors. The vector space where the common reference vector is located is set as the common reference behavior space. For each classification unit in the classification unit set, the behavior feature vector and corresponding block-level mask are read from the data source in the data source set. The behavior feature vector is processed in the order of block number. When the block-level mask is set to one, the block component is truncated and multiplied by the block-level orthogonal transformation matrix to obtain the aligned block component. When the block-level mask is set to zero, the original value of the block component is preserved. The aligned block component and the preserved block component are written back according to the block number to generate a common reference behavior space alignment vector. By selecting the common reference behavior space alignment vector of the same classification unit under various data sources, performing dimension-wise cumulative summation, participation count statistics and dimension-wise averaging, a unified alignment embedding is obtained. The class score is calculated for the uniformly aligned embedding, the class corresponding to the highest score is selected as the class label of the classification unit, and the class labels are output according to the set of classification units to form the image classification result.
