A dual-stream adaptive fusion satellite component segmentation method based on RGB-D images

Abstract

This application belongs to the field of image processing technology. It provides a dual-stream adaptive fusion satellite component segmentation method based on RGB-D images. The embodiments of this disclosure address the problems of low segmentation accuracy and poor robustness of satellite components under extreme space lighting, occlusion, and small target conditions. A dual-stream convolutional neural network is constructed to extract texture features and geometric structures separately; a cross-modal adaptive weighting module is proposed to dynamically evaluate the contribution of RGB and depth features to the segmentation task and achieve weighted fusion, enhancing the model's robustness to lighting changes and occlusion; a semantic feature enhancement module is introduced, combining global context and local details to improve the recognition ability of small-scale components and complex structures; a multi-scale decoding structure and multi-scale supervised loss are used to gradually restore resolution and output high-precision component segmentation results. This method achieves accurate segmentation of satellite components under complex lighting conditions while maintaining lightweight design, making it suitable for environments with limited spaceborne resources.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a dual-stream adaptive fusion satellite component segmentation method based on RGB-D images. Background Technology

[0002] With the rapid development of on-orbit servicing missions, the autonomous perception and precise manipulation of non-cooperative targets such as failed satellites by space robots has become a crucial research direction in the aerospace field. In practical missions, whether it's on-orbit capture of a faulty satellite, refueling, or repair and replacement of critical components, robots are required to accurately identify the satellite itself and its key components, such as docking rings, solar panel supports, and antennas, and to achieve high-precision image segmentation of these components. It can be said that the accuracy of component-level segmentation directly determines the success or failure of subsequent pose estimation and robotic arm manipulation.

[0003] However, the space environment presents numerous practical challenges to visual perception. On the one hand, the lighting in space varies drastically. Direct sunlight can cause strong reflections or overexposure on the target surface, while insufficient light in shadow areas can result in an overall dark appearance and significant loss of texture information. On the other hand, the complex structure of satellites, with mutual occlusion between the solar panels and the satellite body, coupled with diverse surface materials (such as the high reflectivity of the insulation layer and the specular reflection of the solar panels), often makes traditional RGB image-based segmentation methods perform poorly under these extreme conditions. When texture is missing or there are sudden changes in lighting, these methods are prone to missed or false detections, especially for small, slender components (such as antennas and supports), which are difficult to segment accurately, thus affecting the stable execution of subsequent missions.

[0004] In recent years, RGB-D images have demonstrated stronger expressive power in complex environments due to their simultaneous inclusion of texture and geometric information. However, existing RGB-D segmentation methods mostly employ simple concatenation or element-wise addition fusion strategies, failing to fully consider the quality differences of different modal features and their contribution to the segmentation task. Furthermore, they still have shortcomings in semantic context modeling and small object recognition. In addition, limited spaceborne computing resources necessitate models that combine high accuracy with lightweight design.

[0005] Therefore, there is an urgent need for a lightweight RGB-D segmentation method that can achieve precise segmentation of satellite components under extreme space conditions and meet the requirements of real-time onboard processing.

[0006] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0007] The purpose of this disclosure is to provide a dual-stream adaptive fusion satellite component segmentation method based on RGB-D images, thereby overcoming at least to some extent one or more problems caused by the limitations and defects of related technologies.

[0008] According to embodiments of this disclosure, a dual-stream adaptive fusion satellite component segmentation method based on RGB-D images is provided, comprising: Step S1: Acquire RGB image and depth image, and preprocess the RGB image to obtain a preprocessed RGB image; Step S2: Construct the CFAWNet model; wherein the CFAWNet model includes a two-stream convolutional neural network, a CAC module, an SFE module, a GIS module, and a decoder; Step S3: The dual-stream convolutional neural network extracts features from the preprocessed RGB image and depth image respectively to obtain RGB features and depth features at different scales. Step S4: The CAC module dynamically evaluates the modal contribution and performs weighted fusion on the RGB features and corresponding depth features at each scale to obtain fused features at different scales. Step S5: The GIS module fuses RGB features at different scales and depth features at different scales to obtain RGB global information and depth global information. Step S6: The SFE module performs semantic feature enhancement on the fused features at different scales, RGB global information, and deep global information to obtain enhanced features at different scales; Step S7: The decoder progressively restores the resolution of the enhanced features at different scales, and outputs the final predicted image by combining the multi-scale supervised loss.

[0009] Furthermore, the two-stream convolutional neural network includes an RGB branch and a depth branch, both of which use MobileNetV3-large as the backbone network; among them, MobileNetV3-large has 12 featured layers, divided into 5 stages: The first stage is the initial convolutional layer, which uses 3×3 convolution, BatchNorm2d normalization, and Hard-Swish activation. The remaining stages extract features step by step through inverted residual blocks.

[0010] Furthermore, step S3 specifically includes: The RGB branch extracts features from the preprocessed RGB image, outputting RGB feature maps of size 16×H / 2×W / 2. RGB feature map with dimensions of 24×H / 4×W / 4 RGB feature map with dimensions of 40×H / 8×W / 8 RGB feature map with dimensions of 96×H / 16×W / 16 and an RGB feature map with dimensions of 96×H / 32×W / 32 ; The depth branch extracts features from the preprocessed depth image, outputting depth feature maps of size 16×H / 2×W / 2. A depth feature map with dimensions of 24×H / 4×W / 4 A depth feature map with dimensions of 40×H / 8×W / 8 A depth feature map with dimensions of 96×H / 16×W / 16 and a depth feature map with dimensions of 96×H / 32×W / 32 .

[0011] Furthermore, step S4 specifically includes: For each RGB feature map and depth feature map Perform one average pooling operation to obtain the RGB pooling features. and deep pooling features Then, channel attention is used to apply RGB pooling features. and deep pooling features Enhancement is performed to obtain the first layer of RGB features. and the first layer of depth features ; The first layer of RGB features and the first layer of depth features The data is concatenated along the channel dimension, and cross-modal information is extracted separately through 1×1 convolution. Then, the second layer of RGB features is obtained by applying the ReLU activation function and BN normalization. Second layer depth features ; For the second layer of RGB features Second layer depth features Spatial attention enhancement is performed to obtain the third layer of RGB features. and third layer depth features ; Based on the third layer RGB features and third layer depth features Global feature extraction is performed to obtain the fourth layer of RGB features. and fourth layer depth features ; Based on the fourth layer RGB features and fourth layer depth features The RGB segmentation quality score was calculated. and depth segmentation quality score ; Quality score based on RGB segmentation and depth segmentation quality score Calculate the RGB weighting coefficients and depth weighting coefficient ; Using RGB weighting coefficients and depth weighting coefficient For the fourth layer RGB features and fourth layer depth features Weighted fusion is performed to obtain initial fusion features at different scales. ; Based on initial fusion features After applying 3×3 convolution, normalization, and activation functions, residual connections are then performed to obtain fused features at different scales. .

[0012] Furthermore, step S5 specifically includes: By using 3×3 convolution and upsampling, RGB features and depth features at different scales are aligned, allowing high-level semantic information to be passed to lower levels, thereby generating RGB global information and depth global information containing global context information.

[0013] Furthermore, step S6 specifically includes: The fusion features at each scale are respectively RGB global information and depth global information are combined through depthwise separable convolution and cross-fusion to form concatenated features. These features are then further combined with depthwise separable convolution, 1×1 convolution, and residual connections to output enhanced features at different scales. Enhanced features Enhanced features Enhanced features and enhanced features .

[0014] Furthermore, the decoder includes: The AFA1 module is paired with transposed convolutional block 1; the AFA2 module with transposed convolutional block 2; the AFA3 module with transposed convolutional block 3; the AFA4 module with transposed convolutional block 4; and the AFA5 module with transposed convolutional block 5. Enhance features The AFA5 module and the transposed convolution block 5 are input sequentially to perform resolution restoration, and the output image is then processed. ; Enhance features After inputting the AFA4 module to unify the feature channel dimensions, it is compared with the image. The input is upsampled in transposed convolution block 4, and the resolution is restored by skip connections to output the image. ; Enhance features After inputting the AFA3 module to unify the feature channel dimensions, it is compared with the image. The input is upsampled in transposed convolution block 3, and resolution is restored by skip connections to output the image. ; Enhance features After inputting the AFA2 module to unify the feature channel dimensions, it is compared with the image. The input is upsampled in transposed convolution block 2, and the resolution is restored by skip connections to output the image. ; Enhance features After inputting the AFA1 module to unify the feature channel dimensions, it is compared with the image. The input is upsampled in transposed convolution block 1, and the resolution is restored by skip connections to output the image. , and serve as the final predicted image.

[0015] Furthermore, the multi-scale supervision loss is: .

[0016] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: In the embodiments of this disclosure, the above-described method achieves several advantages. First, the CAC module dynamically evaluates the quality contribution of RGB features and depth features based on different environmental scenarios and task requirements, thereby achieving complementary advantages and efficient fusion of the two modalities. This significantly improves the overall robustness and environmental adaptability of the CFAWNet model under extreme lighting conditions and complex environments with severe occlusion. Second, the SFE module effectively fuses global contextual semantic information and local fine detail features in the image, enabling more accurate and detailed recognition and analysis of small-scale components and complex geometric structures in the image, thus enhancing the model's semantic understanding and segmentation accuracy in complex scenes. Third, in terms of model architecture design, an optimized lightweight MobileNetV3 network is used as the feature extraction backbone, combined with efficient computational structures such as depthwise separable convolution. While ensuring high segmentation accuracy and performance, this significantly reduces the number of parameters and overall computational complexity, allowing the model to better adapt to resource-constrained embedded environments such as spaceborne platforms, meeting their practical application requirements for real-time image processing and low-power operation. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0018] Figure 1 The diagram illustrates the steps of a dual-stream adaptive fusion satellite component segmentation method based on RGB-D images in an exemplary embodiment of this disclosure. Figure 2 This diagram illustrates the overall framework of the CFAWNet model in an exemplary embodiment of this disclosure. Figure 3 This diagram illustrates the structure of the MobileNetV3 backbone network in an exemplary embodiment of this disclosure. Figure 4 This diagram illustrates the cross-modal adaptive weighting module structure in an exemplary embodiment of this disclosure. Figure 5 This diagram illustrates the structure of the global information statistics module in an exemplary embodiment of this disclosure. Figure 6 This diagram illustrates the structure of the semantic feature enhancement module in an exemplary embodiment of this disclosure. Figure 7 This diagram illustrates the decoder structure in an exemplary embodiment of this disclosure. Figure 8 This diagram shows a comparison of the segmentation effects of the method of this application and other methods on simulated images in an exemplary embodiment of this disclosure. Figure 9 The diagram illustrates the segmentation effect of the method of this application on a real ground image in an exemplary embodiment of this disclosure. Detailed Implementation

[0019] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0020] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0021] This example implementation provides a dual-stream adaptive fusion satellite component segmentation method based on RGB-D images. (Reference) Figure 1 As shown, the dual-stream adaptive fusion satellite component segmentation method based on RGB-D images may include: Step S1: Acquire RGB image and depth image, and preprocess the RGB image to obtain a preprocessed RGB image; Step S2: Construct the CFAWNet model; wherein the CFAWNet model includes a two-stream convolutional neural network, a CAC module, an SFE module, a GIS module, and a decoder; Step S3: The dual-stream convolutional neural network extracts features from the preprocessed RGB image and depth image respectively to obtain RGB features and depth features at different scales. Step S4: The CAC module dynamically evaluates the modal contribution and performs weighted fusion on the RGB features and corresponding depth features at each scale to obtain fused features at different scales. Step S5: The GIS module fuses RGB features at different scales and depth features at different scales to obtain RGB global information and depth global information. Step S6: The SFE module performs semantic feature enhancement on the fused features at different scales, RGB global information, and deep global information to obtain enhanced features at different scales; Step S7: The decoder progressively restores the resolution of the enhanced features at different scales, and outputs the final predicted image by combining the multi-scale supervised loss.

[0022] Through the above methods, on the one hand, the CAC module can dynamically evaluate the quality contribution of RGB features and depth features according to different environmental scenarios and task requirements, thereby achieving complementary advantages and efficient fusion of the two modalities. This significantly improves the overall robustness and environmental adaptability of the CFAWNet model under extreme lighting conditions and complex environments with severe occlusion. On the other hand, the SFE module effectively fuses global contextual semantic information and local fine detail features in the image, achieving more accurate and detailed recognition and analysis of small-scale parts and complex geometric structures in the image, thus enhancing the model's semantic understanding and segmentation accuracy in complex scenes. Furthermore, in terms of model architecture design, an optimized lightweight MobileNetV3 network is used as the feature extraction backbone, combined with efficient computational structures such as depthwise separable convolution. While ensuring high segmentation accuracy and performance, this significantly reduces the number of parameters and overall computational complexity, enabling the model to better adapt to resource-constrained embedded environments such as spaceborne platforms, meeting their practical application requirements for real-time image processing and low-power operation.

[0023] Below, we will refer to Figures 1 to 9 The steps of the dual-stream adaptive fusion satellite component segmentation method based on RGB-D images described in this example embodiment will be explained in more detail.

[0024] In step S1, an RGB image and a depth image are acquired, and the RGB image is preprocessed to obtain a preprocessed RGB image.

[0025] Specifically, for the training set required for subsequent missions, RGB-D cameras (such as the Intel Realsense D435i) were used to simultaneously acquire RGB and depth images in a space-based on-orbit service ground simulation laboratory scenario. The acquisition process covered diverse scenarios with different lighting conditions (strong light, weak light, side light), different degrees of occlusion, and different targets.

[0026] Labelme software was used to perform pixel-level annotations on RGB images, including key components such as the satellite body, solar panels, solar panel supports, antennas, docking rings, and nozzles. JSON-formatted annotation files were generated to construct a spatial object segmentation dataset. During the preprocessing stage, image alignment, normalization, and noise filtering were performed on the RGB and depth images to ensure consistent data distribution across the two modalities.

[0027] In step S2, a Complementary Feature Adaptive Weighting based Network (CFAWNet) is constructed. The CFAWNet model includes a two-stream convolutional neural network, a CAC (Cross-modal Adaptive-weighted Combination Module), an SFE (Semantic Feature Enhancement Module), a GIS (Global Information Statistics Module), and a decoder. Figure 2 The diagram shown is the overall framework of the CFAWNet model.

[0028] In step S3, the dual-stream convolutional neural network extracts features from the preprocessed RGB image and depth image respectively to obtain RGB features at different scales and depth features at different scales.

[0029] Specifically, this involves dual-stream feature extraction. For example... Figure 3As shown, a two-stream convolutional neural network is used as the encoder, with two branches taking RGB images and depth images as inputs, respectively. Each branch uses MobileNetV3-large as the backbone network, which extracts multi-scale features while maintaining lightweight design through depthwise separable convolutions and inverted residual structures.

[0030] The MobileNetV3 large feature layer consists of 12 layers, which can be divided into 5 stages: The first stage is the initial convolutional layer, which outputs a 16×H / 2×W / 2 feature map through 3×3 convolution, BatchNorm2d normalization, and Hard-Swish activation; subsequent stages extract deeper features step by step through inverted residual blocks, finally outputting feature maps at five scales, denoted as follows: ~ (RGB branch) and ~ (Depth branch), the feature map sizes are 16×H / 2×W / 2, 24×H / 4×W / 4, 40×H / 8×W / 8, 96×H / 16×W / 16, and 96×H / 32×W / 32.

[0031] In step S4, the CAC module dynamically evaluates the modal contribution and performs weighted fusion on the RGB features and corresponding depth features at each scale to obtain fused features at different scales.

[0032] Specifically, it involves cross-modal adaptive weighted fusion. For each scale of RGB features... and depth features ,enter Figure 4 The cross-modal adaptive weighting module (CAC module) shown below has the following specific process: (1) Enhanced channel attention: for and Perform average pooling once each to obtain and Then, channel attention enhancement is used to obtain enhanced features. and .

[0033] (2) Cross-modal interaction: combining features of two modalities and The data is concatenated along the channel dimension, and cross-modal information is extracted separately through 1×1 convolution. Then, it is obtained by applying the ReLU activation function and BN normalization. and .

[0034] (3) Spatial attention enhancement: Spatial attention enhancement is applied to the features obtained after cross-modal interaction to obtain features. and .

[0035] (4) Segmentation quality assessment: It is necessary to evaluate the contribution of the two modal features to the segmentation task. Based on the third layer features, global features are first extracted to obtain... and As shown in formulas (1) and (2). The segmentation quality score is then calculated using learnable weight coefficients. and As shown in formulas (3) and (4).

[0036] (1) (2) (3) (4) In the formula, and These are the weight values ​​initialized in the model. These two values ​​will be automatically learned and updated during training through backpropagation and gradient descent. Features with higher quality scores are more beneficial for segmentation tasks and will receive higher weights in subsequent fusion. This is an implicit learning process that does not require manual labeling of quality scores.

[0037] (5) Adaptive weighted fusion: The fusion feature is obtained by weighting and fusing the segmentation quality scores obtained from the segmentation quality assessment. The specific mathematical expression is as follows: (5) In the formula, and These are the weighting coefficients calculated from the segmentation quality scores of the RGB image and the depth image, respectively. , , The specific calculation method is as follows: (6) (7) (8) (9) In the formula, The global features of the two modalities in the fourth layer are concatenated; The global features are concatenated with the segmentation quality score; The final weight coefficients are obtained by calculating and normalizing the learned weight matrix.

[0038] (6) Feature refinement: After using 3×3 convolution, normalization and activation function, residual connection is then performed to finally achieve adaptive fusion of RGB image and depth image.

[0039] In steps S5 and S6, the GIS module fuses RGB features and depth features at different scales to obtain RGB global information and depth global information, respectively; the SFE module performs semantic feature enhancement on the fused features, RGB global information and depth global information at different scales to obtain enhanced features at different scales.

[0040] Specifically, it enhances semantic features. This module aggregates global information from RGB features and deep features. , and the fusion features in step three First, the global information statistics module (i.e., the GIS module) is used to summarize contextual information, and then semantic feature enhancement is achieved through cross-fusion. The global information statistics module includes, for example... Figure 5 As shown, through 3×3 convolution and upsampling, features at different scales are aligned, allowing high-level semantic information to be passed to lower layers, forming contextual information containing the whole. The semantic enhancement module (SFE module) is as follows: Figure 6 As shown, the global information of the two modalities is combined. and fusion features As input, features are concatenated through depthwise separable convolution and cross-fusion, then further processed by depthwise separable convolution, 1×1 convolution, and residual concatenation to output semantically enhanced features. This process enhances the ability of features to jointly express both the overall structure and local details. In step S7, the decoder progressively restores the resolution of the enhanced features at different scales, and outputs the final predicted image by combining the multi-scale supervised loss.

[0041] Specifically, multi-scale decoding and output are employed. A top-down decoding structure is used to progressively restore the feature map resolution. The decoder (decoder structure as follows) Figure 7 (As shown) It includes an Adaptive Feature Aggregation (AFA) module and a transposed convolutional block. During training, a multi-scale supervised loss is used, with the loss function being: (10) In the formula, arrive Enhanced features arrive The output obtained after decoding through AFA and transposed convolutional blocks.

[0042] In a specific embodiment, experimental verification and performance analysis were conducted. The application was validated on a self-built SpaceRS-Dataset simulation and real dataset. Training employed the Adam optimizer with an initial learning rate of 0.002, a cosine annealing strategy, a batch size of 4, and 150 training epochs. Evaluation metrics included mean intersection-over-union (mIoU) and mean pixel accuracy (MPA).

[0043] Ablation experiments show that the cross-modal adaptive weighting module improves mIoU by 0.9%, the semantic feature enhancement module improves mIoU by 1.3%, and the combined use of both improves mIoU by 2.7%, reaching 84.1%, while MPA reaches 87.9%. Compared with mainstream methods such as PSPNet, DeepLabv3+, ACNet, and AsymFormer, this application achieves the best results in both mIoU and MPA, especially showing significant advantages in small target categories such as solar panel supports and antennas.

[0044] Figure 8 The comparison of segmentation results on simulated images is shown. The method of this application can still accurately segment the main body of the satellite and its components, especially small components such as the solar panel support, even under complex backgrounds and shadow occlusion. Figure 9 The segmentation results on ground-based measured images are demonstrated, further validating the generalization ability and engineering applicability of the method.

[0045] This method addresses the problems of poor segmentation performance and poor small target recognition in existing satellite component segmentation under complex lighting and occlusion conditions. It improves upon existing segmentation algorithms based on dual-stream convolutional neural networks by using cross-modal adaptive weighting and semantic feature enhancement to achieve high-precision segmentation of satellite components under complex lighting conditions while maintaining a lightweight model.

[0046] It should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise" in the above description indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this disclosure and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this disclosure.

[0047] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0048] In the embodiments of this disclosure, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.

[0049] In embodiments of this disclosure, unless otherwise expressly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature being directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0050] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0051] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A dual-stream adaptive fusion satellite component segmentation method based on RGB-D images, characterized in that, include: Step S1: Acquire RGB image and depth image, and preprocess the RGB image to obtain a preprocessed RGB image; Step S2: Construct the CFAWNet model; wherein the CFAWNet model includes a two-stream convolutional neural network, a CAC module, an SFE module, a GIS module, and a decoder; Step S3: The dual-stream convolutional neural network extracts features from the preprocessed RGB image and depth image respectively to obtain RGB features and depth features at different scales. Step S4: The CAC module dynamically evaluates the modal contribution and performs weighted fusion on the RGB features and corresponding depth features at each scale to obtain fused features at different scales. Step S5: The GIS module fuses RGB features at different scales and depth features at different scales to obtain RGB global information and depth global information. Step S6: The SFE module performs semantic feature enhancement on the fused features at different scales, RGB global information, and deep global information to obtain enhanced features at different scales; Step S7: The decoder progressively restores the resolution of the enhanced features at different scales, and outputs the final predicted image by combining the multi-scale supervised loss.

2. The method of claim 1, wherein the method further comprises: The two-stream convolutional neural network includes an RGB branch and a depth branch, both of which use MobileNetV3-large as the backbone network; among them, MobileNetV3-large has 12 featured layers, divided into 5 stages: The first stage is the initial convolutional layer, which uses 3×3 convolution, BatchNorm2d normalization, and Hard-Swish activation. The remaining stages extract features step by step through inverted residual blocks.

3. The method of claim 2, wherein the method further comprises: Step S3 specifically includes: The RGB branch performs feature extraction on the preprocessed RGB image, and respectively outputs RGB feature maps with sizes of 16xH / 2xW / 2 , 24xH / 4xW / 4 , 40xH / 8xW / 8 , 96xH / 16xW / 16 , and 96xH / 32xW / 32 . The depth branch extracts features from the preprocessed depth image, outputting depth feature maps of size 16×H / 2×W / 2. A depth feature map with dimensions of 24×H / 4×W / 4 A depth feature map with dimensions of 40×H / 8×W / 8 A depth feature map with dimensions of 96×H / 16×W / 16 and a depth feature map with dimensions of 96×H / 32×W / 32 .

4. The dual-stream adaptive fusion satellite component segmentation method based on RGB-D images according to claim 3, characterized in that, Step S4 specifically includes: For each RGB feature map and depth feature map Perform one average pooling operation to obtain the RGB pooling features. and deep pooling features Then, channel attention is used to apply RGB pooling features. and deep pooling features Enhancement is performed to obtain the first layer of RGB features. and the first layer of depth features ; The first layer of RGB features and the first layer of depth features The data is concatenated along the channel dimension, and cross-modal information is extracted separately through 1×1 convolution. Then, the second layer of RGB features is obtained by applying the ReLU activation function and BN normalization. Second layer depth features ; For the second layer of RGB features Second layer depth features Spatial attention enhancement is performed to obtain the third layer of RGB features. and third layer depth features ; Based on the third layer RGB features and third layer depth features Global feature extraction is performed to obtain the fourth layer of RGB features. and fourth layer depth features ; Based on the fourth layer RGB features and fourth layer depth features The RGB segmentation quality score was calculated. and depth segmentation quality score ; Quality score based on RGB segmentation and depth segmentation quality score Calculate the RGB weighting coefficients and depth weighting coefficient ; Using RGB weighting coefficients and depth weighting coefficient For the fourth layer RGB features and fourth layer depth features Weighted fusion is performed to obtain initial fusion features at different scales. ; Based on initial fusion features After applying 3×3 convolution, normalization, and activation functions, residual connections are then performed to obtain fused features at different scales. .

5. The dual-stream adaptive fusion satellite component segmentation method based on RGB-D images according to claim 4, characterized in that, Step S5 specifically includes: By using 3×3 convolution and upsampling, RGB features and depth features at different scales are aligned, allowing high-level semantic information to be passed to lower levels, thereby generating RGB global information and depth global information containing global context information.

6. The dual-stream adaptive fusion satellite component segmentation method based on RGB-D images according to claim 5, characterized in that, Step S6 specifically includes: The fusion features at each scale are respectively RGB global information and depth global information are combined through depthwise separable convolution and cross-fusion to form concatenated features. These features are then further combined with depthwise separable convolution, 1×1 convolution, and residual connections to output enhanced features at different scales. Enhanced features Enhanced features Enhanced features and enhanced features .

7. The dual-stream adaptive fusion satellite component segmentation method based on RGB-D images according to claim 6, characterized in that the decoder... include: The AFA1 module is paired with transposed convolutional block 1; the AFA2 module with transposed convolutional block 2; the AFA3 module with transposed convolutional block 3; the AFA4 module with transposed convolutional block 4; and the AFA5 module with transposed convolutional block 5. Enhance features The AFA5 module and the transposed convolution block 5 are input sequentially to perform resolution restoration, and the output image is then processed. ; Enhance features After inputting the AFA4 module to unify the feature channel dimensions, it is compared with the image. The input is upsampled in transposed convolution block 4, and the resolution is restored by skip connections to output the image. ; Enhance features After inputting the AFA3 module to unify the feature channel dimensions, it is compared with the image. The input is upsampled in transposed convolution block 3, and resolution is restored by skip connections to output the image. ; Enhance features After inputting the AFA2 module to unify the feature channel dimensions, it is compared with the image. The input is upsampled in transposed convolution block 2, and the resolution is restored by skip connections to output the image. ; Enhance features After inputting the AFA1 module to unify the feature channel dimensions, it is compared with the image. The input is upsampled in transposed convolution block 1, and the resolution is restored by skip connections to output the image. , and serve as the final predicted image.

8. The dual-stream adaptive fusion satellite component segmentation method based on RGB-D images according to claim 7, characterized in that, The multi-scale supervision loss is: 。

Images