Spacecraft appearance damage detection method and system based on multi-modal data fusion

Abstract

The application relates to the technical field of on-orbit health monitoring of spacecraft, and particularly discloses a spacecraft appearance damage detection method and system based on multi-modal data fusion, which comprises the following steps: acquiring current multi-modal data of an outer surface damage area of a target spacecraft and performing data preprocessing and cross-modal alignment to obtain target multi-modal data; performing feature extraction on the target multi-modal data to obtain damage features of each mode, and performing damage detection on the damage features of each mode to obtain damage detection results of each mode; and performing cross-modal feature fusion on the damage detection results of each mode and dynamic weights to obtain appearance damage detection results of the target spacecraft. Through multi-modal fusion and innovative process design, the precision of spacecraft appearance damage detection is improved, and the method is especially suitable for on-orbit service tasks in a complex orbit environment, and provides an efficient solution for real-time evaluation and autonomous decision-making of the health state of a spacecraft.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of on-orbit health monitoring of spacecraft, and particularly relates to a spacecraft appearance damage detection method and system based on multi-modal data fusion. BACKGROUND

[0002] With the rapid development of on-orbit service tasks (such as spacecraft maintenance, fuel refueling and life extension operations), co-orbit service spacecraft need to perform high-precision health monitoring on the outer surface and key components (such as solar panels, thrusters, thermal protection layers, etc.) of the target spacecraft. The target spacecraft is long-term exposed to complex space environment (such as micro-meteoroid impact, extreme temperature cycle, atomic oxygen erosion and space radiation, etc.), and its external structure is prone to damage such as micro-cracks, coating peeling and deformation, which directly affects the safety and life of the task. Traditional ground observation methods are limited by orbital distance, dynamic viewing angle and insufficient resolution, making it difficult to capture millimeter-level damage in real time; and existing on-orbit monitoring technologies mostly rely on a single sensor (such as an optical camera or a laser radar), which has problems such as monitoring blind area, high false detection rate and insufficient sensitivity to weak defects.

[0003] In addition, the co-orbit service task has very high requirements for detection timeliness, and needs to complete damage identification and risk assessment within a few minutes, while the traditional method relies on manual review or ground command interaction, which is difficult to meet the demand for autonomous decision-making. Therefore, there is an urgent need for a technical solution that can fuse multi-modal data, dynamically adapt to complex orbital environment, and achieve high-precision damage detection, in order to ensure the safety and efficiency of on-orbit service tasks. SUMMARY

[0004] To solve the above technical problems, the present application provides a spacecraft appearance damage detection method and system based on multi-modal data fusion.

[0005] In a first aspect, the present application provides a spacecraft appearance damage detection method based on multi-modal data fusion, and the technical scheme of the method is as follows:

[0006] Obtain the current multi-modal data of the damage area of the outer surface of the target spacecraft and perform data preprocessing and cross-modal alignment to obtain target multi-modal data;

[0007] Perform feature extraction on the target multi-modal data to obtain damage features of each modality, and perform damage detection on the damage features of each modality to obtain damage detection results of each modality;

[0008] Perform cross-modal feature fusion according to the damage detection results of each modality and dynamic weights to obtain the appearance damage detection results of the target spacecraft.

[0009] The spacecraft appearance damage detection method based on multi-modal data fusion has the following beneficial effects:

[0010] The method of the application improves the accuracy of spacecraft appearance damage detection through multi-modal fusion and innovative process design, especially suitable for on-orbit service tasks in complex orbital environments, providing an efficient solution for real-time evaluation of spacecraft health status and autonomous decision-making.

[0011] On the basis of the above scheme, the spacecraft appearance damage detection method based on multi-modal data fusion of the application can be further improved as follows.

[0012] In an optional manner, it further comprises:

[0013] The current multi-modal data of the target spacecraft's outer surface damage area is collected by a small on-orbit service spacecraft equipped with multi-modal sensors.

[0014] In an optional manner, the multi-modal sensor comprises a camera and a laser radar; wherein the camera is used to collect two-dimensional image data, and the laser radar is used to collect three-dimensional point cloud data.

[0015] In an optional manner, the current multi-modal data includes current two-dimensional image data and current three-dimensional point cloud data; and the target multi-modal data includes target two-dimensional image data and target three-dimensional point cloud data.

[0016] The step of data preprocessing and cross-modal alignment of the current multi-modal data of the target spacecraft's outer surface damage area to obtain the target multi-modal data comprises:

[0017] The current two-dimensional image data is subjected to image distortion correction to obtain preprocessed two-dimensional image data, and the current three-dimensional point cloud data is subjected to denoising filtering to obtain preprocessed three-dimensional point cloud data.

[0018] The mapping relationship between pixels and point clouds is constructed by using the extrinsic matrix between the camera and the laser radar, and the preprocessed two-dimensional image data and the preprocessed three-dimensional point cloud data are subjected to cross-modal alignment according to the mapping relationship to obtain the aligned target two-dimensional image data and target three-dimensional point cloud data.

[0019] In an optional manner, the step of feature extraction of the target multi-modal data to obtain damage features of each modality, and damage detection of the damage features of each modality to obtain damage detection results of each modality comprises:

[0020] The target two-dimensional image data is subjected to texture and color feature extraction to obtain two-dimensional damage features, and the target three-dimensional point cloud data is subjected to curvature and depth feature extraction to obtain three-dimensional damage features.

[0021] inputting the two-dimensional damage feature into an improved U-Net network to obtain a two-dimensional damage detection result containing a surface damage type and a corresponding confidence; wherein the improved U-Net network is that an attention gate module is embedded between an encoder and a decoder of an original U-Net network;

[0022] inputting the three-dimensional damage feature into a three-dimensional point cloud model to obtain a three-dimensional damage detection result containing a structure damage type and a corresponding confidence.

[0023] In an optional mode, the method further comprises:

[0024] adjusting the initial weight of each modality based on the current environment information to generate a dynamic weight of each modality.

[0025] In an optional mode, the method further comprises:

[0026] based on the two-dimensional damage feature and the three-dimensional damage feature, and combining a damage degree quantification formula, obtaining and determining a risk level of the target spacecraft according to an appearance damage index of the target spacecraft.

[0027] In a second aspect, the application provides a spacecraft appearance damage detection system based on multi-modal data fusion, and the technical scheme of the system is as follows:

[0028] comprising a first processing module, a second processing module and a damage detection module;

[0029] The first processing module is configured to obtain current multi-modal data of a target spacecraft's outer surface damage area and perform data preprocessing and cross-modal alignment to obtain target multi-modal data.

[0030] The second processing module is configured to extract features from the target multi-modal data to obtain damage features of each modality, and perform damage detection on the damage features of each modality to obtain damage detection results of each modality.

[0031] The damage detection module is configured to perform cross-modal feature fusion on the damage detection results of each modality and the dynamic weight to obtain an appearance damage detection result of the target spacecraft.

[0032] The spacecraft appearance damage detection system based on multi-modal data fusion has the following beneficial effects:

[0033] The system of the application improves the accuracy of spacecraft appearance damage detection through multi-modal fusion and innovative process design, and is especially suitable for on-orbit service tasks in complex orbital environments, providing an efficient solution for real-time evaluation and autonomous decision-making of spacecraft health status.

[0034] In a third aspect, a technical solution of an electronic device of the present application is as follows:

[0035] The electronic device comprises a memory, a processor and a program stored in the memory and running on the processor, and the processor implements the steps of the spacecraft appearance damage detection method based on multi-modal data fusion of the present application when running the program.

[0036] In a fourth aspect, a technical solution of a computer readable storage medium provided by the present application is as follows:

[0037] The computer readable storage medium stores instructions, and when the computer readable storage medium reads the instructions, the computer readable storage medium executes the steps of the spacecraft appearance damage detection method based on multi-modal data fusion of the present application.

[0038] The above description is only a summary of the technical solutions of the present application, in order to more clearly understand the technical means of the present application, the content of the specification can be implemented, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings are only used to show the embodiments, and are not considered as limiting the present application. Moreover, the same reference signs are used to represent the same parts throughout the drawings. In the drawings:

[0040] Figure 1 The flowchart is a flowchart of an embodiment of the spacecraft appearance damage detection method based on multi-modal data fusion of the present application;

[0041] Figure 2 The flowchart is a flowchart of wide-angle lens distortion correction;

[0042] Figure 3 The structural diagram is a structural diagram of an embodiment of the spacecraft appearance damage detection system based on multi-modal data fusion of the present application;

[0043] Figure 4 The structural diagram is a structural diagram of an embodiment of the electronic device of the present application. DETAILED DESCRIPTION

[0044] Exemplary embodiments of the present application will be described in detail with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein.

[0045] Figure 1A flowchart of an embodiment of a spacecraft appearance damage detection method based on multi-modal data fusion provided by the present application is shown, which can be executed by terminal equipment or server and the like electronic equipment. Among them, the terminal equipment can be any fixed or mobile terminal such as user equipment (User Equipment, UE), mobile equipment, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device, wearable device, etc. The server can be a separate server or a server cluster composed of multiple servers. Any electronic equipment can realize the spacecraft appearance damage detection method based on multi-modal data fusion by calling the computer readable instructions stored in the memory through the processor. As shown in Figure 1 includes the following steps:

[0046] S1, obtaining the current multi-modal data of the target spacecraft's outer surface damage area and performing data preprocessing and cross-modal alignment to obtain target multi-modal data.

[0047] Among them, the target spacecraft is the spacecraft that needs to be detected for appearance damage in this embodiment. The outer surface damage area can be a certain area on the spacecraft cabin shell or solar panel, which is not limited here. The current multi-modal data includes but is not limited to: current two-dimensional image data and current three-dimensional point cloud data. The target multi-modal data is the multi-modal data after data preprocessing and cross-modal alignment, which specifically includes: target two-dimensional image data and target three-dimensional point cloud data.

[0048] Before S1, it also includes:

[0049] The current multi-modal data of the target spacecraft's outer surface damage area is collected by a small co-orbit service spacecraft equipped with multi-modal sensors.

[0050] Among them, the multi-modal sensor includes: a camera and a laser radar. The camera is used to collect two-dimensional image data to obtain spacecraft surface images under multiple angles and multiple lighting conditions; the laser radar is used to collect three-dimensional point cloud data, which contains depth, surface roughness and material reflectivity information. The small co-orbit service spacecraft is a micro spacecraft that runs in orbit, has modular design and co-orbit flight capability, and is mainly used to perform on-orbit service tasks such as orbit adjustment, attitude control, component replacement, etc. The specific resolution of the camera (black and white camera and color camera) carried by the small co-orbit service spacecraft can be adjusted according to actual conditions, which is not limited here. It should be noted that a navigation sensor can also be provided in the multi-modal sensor to provide the relative pose, velocity and timestamp between the small co-orbit service spacecraft and the target spacecraft.

[0051] In S1, the current multi-modal data of the outer surface damage area of the target spacecraft is preprocessed and aligned across modalities to obtain target multi-modal data, including:

[0052] The current two-dimensional image data is corrected for image distortion to obtain preprocessed two-dimensional image data, and the current three-dimensional point cloud data is filtered for noise to obtain preprocessed three-dimensional point cloud data.

[0053] The current two-dimensional image data is corrected for image distortion using camera calibration parameters. The preprocessed two-dimensional image data is two-dimensional image data that has been corrected for image distortion. Deep space background noise points in the current three-dimensional point cloud data are removed by statistical filtering (Statistical Outlier Removal), and after downsampling using a voxel grid, the three-dimensional point cloud data (multi-view point cloud) is aligned using the ICP algorithm to obtain preprocessed three-dimensional point cloud data.

[0054] It should be noted that the core of wide-angle lens distortion correction is to obtain distortion parameters through camera calibration and to perform inverse mapping and interpolation repair on the image based on the Brown-Conrady model. Key technologies include high-precision corner detection, multi-view calibration data acquisition, and edge region optimization. Wide-angle lenses are prone to radial distortion (Radial Distortion) and tangential distortion (Tangential Distortion) due to their wide viewing angle:

[0055] 1) Radial distortion can be described by a polynomial model:

[0056] x corrected =x·(1+k1r 2 +k2r 4 +k3r 6 )

[0057] y corrected =y·(1+k1r 2 +k2r 4 +k3r 6 )

[0058] where r 2 =x 2 +y 2 , k1, k2, k3 are radial distortion coefficients.

[0059] 2) Tangential distortion, lens and sensor plane installation not parallel, resulting in image stretching or tilting:

[0060] x corrected =x+[2p1xy+p2(r 2 +2x 2 )]

[0061] y corrected =y+[p1(r 2 +2y 2 )+2p2xy]

[0062] where p1 and p2 are tangential distortion coefficients.

[0063] As shown in the following equation, wide-angle lens distortion correction needs to obtain distortion parameters through camera calibration and apply reverse transformation to restore the image. Figure 2

[0064] 1) Camera calibration. Use a checkerboard calibration board, whose regular corner distribution is convenient for automatic detection. Take at least 10-20 calibration board images at different positions, angles, and distances to cover the entire field of view. Use OpenCV's cv2.findChessboardCorners() to detect calibration board corner coordinates. Call cv2.calibrateCamera() to calculate the camera intrinsic matrix K (focal length, principal point) and distortion coefficients (radial distortion coefficients k1, k2, k3, tangential distortion coefficients p1, p2).

[0065] 2) Distortion correction model. Use the Brown-Conrady model: a classic model that combines radial and tangential distortion, and the distortion correction formula is:

[0066] x corrected =x·(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0067] y corrected =y·(1+k1r 2 +k2r 4 +k3r 6 )+p1(r 2 +2y 2 )+2p2xy

[0068] where x represents the original horizontal coordinate of a pixel point in two-dimensional image data, y represents the original vertical coordinate of a pixel point in two-dimensional image data, x corrected represents the corrected horizontal coordinate of a pixel point in two-dimensional image data, y corrected represents the corrected vertical coordinate of a pixel point in two-dimensional image data, r 2 =x 2 +y 2 ​; k1 represents a first radial distortion coefficient, k2 represents a second radial distortion coefficient, k3 represents a third radial distortion coefficient, p1 represents a first tangential distortion coefficient, and p2 represents a second tangential distortion coefficient.

[0069] 3) Image correction implementation, reverse mapping is performed. A distortion mapping table is generated, and the coordinates of each pixel of the original image in the ideal non-distorted image are calculated based on the distortion coefficients obtained by calibration. The corrected image is filled smoothly using Bilinear Interpolation to avoid pixel loss.

[0070] The mapping relationship between pixels and point clouds is constructed using the extrinsic matrix between the camera and the lidar, and the preprocessed two-dimensional image data and the preprocessed three-dimensional point cloud data are cross-modal aligned according to the mapping relationship, to obtain the aligned target two-dimensional image data and the target three-dimensional point cloud data.

[0071] The mapping relationship is the mapping relationship between two-dimensional pixels and three-dimensional point clouds, and is used to realize accurate matching of spatial positions to obtain the cross-modal aligned target two-dimensional image data and the target three-dimensional point cloud data.

[0072] S2, feature extraction is performed on the target multi-modal data to obtain damage features of each modality, and damage detection is performed on the damage features of each modality to obtain damage detection results of each modality.

[0073] Specifically, S2 includes:

[0074] Texture and color feature extraction is performed on the target two-dimensional image data to obtain two-dimensional damage features, and curvature and depth feature extraction is performed on the target three-dimensional point cloud data to obtain three-dimensional damage features.

[0075] The two-dimensional damage features refer to the two-dimensional feature maps obtained after texture and color feature extraction. The three-dimensional damage features refer to the three-dimensional point cloud data obtained after curvature and depth feature extraction.

[0076] The process of texture and color feature extraction includes: ①Retinex algorithm optimization: for the target two-dimensional image data, the image is decomposed by multi-scale Retinex (MSR) to separate the illumination component and the reflection component, and further suppress the interference of uneven illumination on color difference detection. The Gaussian kernel scale is 15, 80, 200, and the weighting coefficient is 0.3, 0.5, 0.2, to enhance the contrast of the reflection component in the color difference area. ②Local texture enhancement: based on the image enhanced by CLAHE, the multi-directional texture features are extracted using Gabor filter set (4 directions x 3 scales) to highlight the periodic structure of scratches and micro-cracks. The process of curvature and depth feature extraction includes: ①Local geometric feature calculation: for the target three-dimensional point cloud data, the normal vector of each point is calculated based on k-nearest neighbors (k=30), and the principal curvatures (λ1, λ2) are obtained by covariance matrix decomposition. Through local geometric feature calculation (k-nearest neighbor normal vector, covariance matrix decomposition), the surface micro-geometric deformation (such as pits and protrusions) is reflected. The Gaussian curvature (K) and the average curvature (H) are calculated by the following formula: K = λ1·λ2; When K > 0.015 mm 2 or H > 0.015 mm -1 is marked as a potential pit. ②Depth anomaly detection: taking each point in the target three-dimensional point cloud data as the center, the average depth of the point cloud in the 5mm radius neighborhood is calculated respectively, and compared with the theoretical depth after aligning with the ideal CAD model. By comparing the actual depth with the theoretical depth of the ideal CAD model, macro-structure deviations (such as overall depression or protrusion) are detected. The deviation formula is: ΔD = |D 实际 -D 理论 |. When ΔD > 0.1 mm (aluminum alloy) or ΔD > 0.05 mm (composite material) is marked as an anomaly. For example, it is sensitive to local micro-deformation (such as micro-meteorite impact pit), even if the depth change is small (ΔD does not exceed the threshold value), the curvature anomaly (such as K > 0.015 mm 2 ) can still mark potential damage. For detecting large-area or deep-structure damage (such as cabin deformation), even if the curvature is not significantly abnormal, the depth deviation (such as ΔD > 0.1 mm) can still trigger an alarm. Curvature (K / H) and ΔD detect damage from two dimensions of micro-geometric deformation and macro-structure deviation respectively, cross-verify through spatial consistency verification, and improve the detection accuracy and robustness.

[0077] The two-dimensional damage features are input into the improved U-Net network to obtain two-dimensional damage detection results containing surface damage types and corresponding confidence.

[0078] The improved U-Net network is: embedding an attention gate module between the encoder and the decoder of the original U-Net network. The input of the improved U-Net network is a two-dimensional damage feature, and the output is a pixel-level damage mask. The attention gate module focuses on the low-contrast damage area. The pixel-level damage mask is finally output by a classifier to include a two-dimensional damage detection result containing a surface damage type and a corresponding confidence. The surface damage type includes: oxidation discoloration (color difference), scratch, and coating peeling. The confidence is the credibility of the surface damage type output by the Softmax probability, and the value range is 0-1.

[0079] The three-dimensional damage feature is input into a three-dimensional point cloud model to obtain a three-dimensional damage detection result containing a structure damage type and a corresponding confidence.

[0080] The input of the three-dimensional point cloud model (PointNet++) is a three-dimensional damage feature, and the output is a three-dimensional damage detection result containing a structure damage type and a corresponding confidence. The three-dimensional damage feature is specifically a registered point cloud (ICP error <0.5mm) containing depth and curvature features. The structure damage type includes: pit, deformation, and curvature anomaly, and the confidence is the credibility of the structure damage type output by the Softmax probability, and the value range is 0-1.

[0081] It should be noted that in the training process of the improved U-Net network, multiple labeled images (containing low-contrast damage and background interference samples) are used for iterative training. In the training process of the three-dimensional point cloud model, multiple point cloud samples (covering different depth and geometric deformation scenes) are used for iterative training.

[0082] S3, according to the damage detection result of each modality and the dynamic weight, cross-modal feature fusion is performed to obtain the appearance damage detection result of the target spacecraft.

[0083] The two-dimensional damage detection result is: P 2D =[p 划痕 ,p 氧化 ,p 涂层 ], and the three-dimensional damage detection result is: P 3D =[q 凹坑 ,q 形变 ,q 曲率 ]. The appearance damage detection result is: w 2D ·P 2D +w 3D ·P 3D =argmax[Score 划痕 ,Score 氧化 ,Score 涂层 ,Score 凹坑 ,Score 形变,Score 曲率 ]。w 2D represents a two-dimensional dynamic weight corresponding to the two-dimensional damage detection result, w 3D represents a three-dimensional dynamic weight corresponding to the three-dimensional damage detection result.

[0084] In S3, further comprising:

[0085] Based on the current environment information, the initial weight of each modality is adjusted to generate the dynamic weight of each modality.

[0086] wherein the two-dimensional initial weight is by default: w 2D = 0.7, and the three-dimensional initial weight w 3D = 0.3. The current environment information includes: light intensity and noise intensity. For example, in a strong light scene, w 2D = 0.4, w 3D = 0.6; in a high noise scene, w 2D = 0.8, w 3D = 0.2.

[0087] It should be noted that in the present embodiment, a spatial consistency verification method is adopted, specifically by cross-modality projection verification, to exclude artifacts and adjust confidence, thereby improving detection accuracy. Specifically:

[0088] 1) Projection mapping: two-dimensional damage area to three-dimensional point cloud. The damage area detected in the two-dimensional damage feature is accurately mapped to the point cloud coordinate system of the three-dimensional damage feature, establishing a cross-modality spatial correlation.

[0089] ① Load the camera-laser radar calibration extrinsic matrix (rotation matrix R and translation vector T) and camera intrinsic parameters (focal length f x , f y , principal point c x , c x ).

[0090] ② Pixel coordinates to three-dimensional coordinates conversion: for each damage pixel point (u, v) in the two-dimensional image, the ray direction in three-dimensional space is calculated through inverse perspective transformation:

[0091]

[0092] Convert the two-dimensional pixel coordinates (u, v) and the corresponding three-dimensional point cloud coordinates (x, y, z) to the global coordinate system through the extrinsic matrix [R|T]:

[0093] ③ Region projection and matching: use KD-Tree to accelerate three-dimensional point cloud search to find all point clouds within the projection range of the two-dimensional damage area. If the projection area is not covered by point clouds (such as occlusion), mark it as "to be manually reviewed".

[0094] 2) Geometric consistency check: verify the geometric characteristics of the two-dimensional damage area in three-dimensional space to meet the expected value, and exclude artifacts; The specific way includes depth anomaly detection and local geometric feature calculation.

[0095] 3) Confidence dynamic correction: according to the results of geometric consistency check, dynamically adjust the confidence of damage probability. Specifically:

[0096] ① Consistency passes: if the depth, curvature and reflectivity of the two-dimensional damage area in three-dimensional space meet the expected value. For example, surface damage (such as oxidation, scratches) corresponds to a 20% increase in confidence (from 0.7 to 0.84); For example, structural damage (such as pits) corresponds to a 15% increase in confidence (from 0.6 to 0.69).

[0097] ② Consistency conflict: if the three-dimensional data and the two-dimensional detection results are contradictory. For example, the surface damage area has depth anomaly (ΔD>0.1mm) corresponding to 50% reduction in confidence (from 0.8 to 0.4), marked as "suspected structural damage needs to be reviewed"; For example, the two-dimensional image of the structural damage area has no texture anomaly: the confidence is zero, and it is determined as three-dimensional noise.

[0098] ③ Artificial review flag triggering: when the confidence is still lower than the threshold (such as <0.3) after correction, an artificial review request is automatically generated, and the conflict reason (such as "depth anomaly" or "reflectivity inconsistency") is marked.

[0099] In an optional way, it also includes:

[0100] Based on the two-dimensional damage characteristics and the three-dimensional damage characteristics, and combined with the damage degree quantification formula, the appearance damage index of the target spacecraft is obtained and determined according to the risk level of the target spacecraft.

[0101] Wherein, the damage degree quantification formula is: MDDI = 0.4 × D depth +0.3 × S area +0.2C color +0.1 × R risk ; MDDI represents the appearance damage index, D depth represents the damage depth, S area represents the damage area, C color represents the color difference, and R riskrepresents the position risk. The weight of each index in the damage degree quantification formula is obtained by the analytic hierarchy process (AHP) and data-driven correction. The principle of AHP is that 10 aerospace structure experts are invited to score the relative importance of the four types of damage indexes, and the initial weight is obtained through consistency test (CR < 0.1). The principle of data-driven correction is that based on 300 groups of damage samples in ground simulation experiments, the weight coefficients are optimized by multiple linear regression to maximize the correlation between MDDI and the measured residual strength (R 2 = 0.92).

[0102] wherein: ① damage depth D depth is the core index of structural integrity, which directly affects the material residual strength and fatigue life. Experimental data show that when the depth of the micro meteorite impact crater exceeds 10% of the material thickness, the failure risk increases significantly (referring to the NASA-STD-3001 standard). The weight of damage depth D depth accounts for the highest proportion (40%) reflecting its dominant influence on safety. Damage depth D depth is specifically the ratio of ΔD to the material threshold, that is: D0 is the material threshold; if ΔD = 0.12 mm (the material threshold D0 of aluminum alloy is 0.1 mm), then D depth = 1.2. ② damage area S area represents the damage diffusion range, and large-area damage may cause the stress concentration area to expand. According to the finite element simulation, the weakening effect of area on local stiffness is about 0.75 times that of depth, so the weight of damage area S area is set to 30%. Specifically, based on the geometric data of the projected area in the three-dimensional point cloud, the actual physical area (unit: mm 2 ) of the damage area is calculated. The actual area is compared with the preset safety threshold to output the normalized area parameter (for example: actual area / safety threshold) for comprehensive risk assessment. Wherein, S 实际损伤面积 = pixel area x unit pixel actual area; the unit pixel actual area is calculated through camera calibration parameters such as focal length and shooting distance. If the safety threshold = 10 mm 2 , the actual area = 3 mm 2 , then S area = 0.3. ③ color difference C color is used to evaluate the degree of atomic oxygen erosion, and its correlation with the material oxide layer thickness is calibrated by spectral analysis (experimental R 2 = 0.87). The weight of color difference C color is 20% to reflect the secondary important influence of material property degradation on functional life. The RGB image after Retinex processing is converted to LAB color space to separate brightness (L) and chroma (a, b). The color difference formula is: (4) Position risk R risk Based on the list of spacecraft key components (such as propellant pipelines, docking mechanisms), the position weight is quantified by Failure Mode and Effects Analysis (FMEA). High-risk areas (such as sealing structures) are assigned a weight of 1.0, and non-critical areas are assigned a weight of 0.1-0.5, with a weighted average of 10%. Specifically, a list of spacecraft key components is established, including high-risk areas such as propellant pipelines, docking mechanisms, and sealing structures. The position risk is quantified by expert evaluation and historical failure data. Criticality classification: high-risk areas (such as sealing structures): R risk = 1.0; medium-risk areas (such as solar panel connections): R risk = 0.5; low-risk areas (non-load-bearing shell): R risk = 0.1.

[0103] wherein the risk classification is determined according to the size of the appearance damage index. Specifically: ① MDDI < 0.4 is low risk, record and monitor regularly. ② 0.4 ≤ MDDI < 0.7 is medium risk, generate a maintenance work order, handle within 1 month. ③ MDDI ≥ 0.7 is high risk, trigger an emergency alert, immediately disable the relevant components.

[0104] To better illustrate the technical solutions of the present embodiment, the following two examples are used for illustration:

[0105] Example 1: Solar panel surface oxidation discoloration and micro-meteorite crater detection. Solar panels are exposed to the Low Earth Orbit (LEO) environment for a long time, and are easily impacted by micro-meteors (forming craters) and atomic oxygen erosion (causing surface oxidation discoloration). Traditional detection methods cannot distinguish between these damage types, and have a high false detection rate.

[0106] 1) Obtain the current multi-modal data of the solar panel surface of the target spacecraft and perform data preprocessing and cross-modal alignment to obtain the target multi-modal data.

[0107] ① The spacecraft flies around the solar panel and uses a multi-spectral camera (RGB + near-infrared) to take high-resolution images at a 60° angle of side light, highlighting scratch and color difference textures. In the image, suspected oxidation areas (C color = 4.2) and scratches (length 2mm) are detected.

[0108] ② Laser radar scans the panel surface to generate a dense point cloud (accuracy ±1mm) that captures local depth and curvature information. The point cloud shows that a certain area has a depth deviation ΔD = 0.8mm (exceeding the safety threshold of 0.5mm) and an abnormal curvature (Gaussian curvature K = 0.025mm -2 ).

[0109] ③Adopt Retinex algorithm to eliminate uneven illumination, and CLAHE to enhance the contrast of ΔE in the oxidized area by 30%. Improve ICP algorithm to register multi-view point cloud, and the registration error is less than 0.5mm. Remove background noise points (such as deep space stray points).

[0110] ④Based on the calibration parameters, accurately map the two-dimensional oxidized area to the three-dimensional point cloud coordinate system to ensure spatial position consistency.

[0111] 2) Feature extraction is performed on the target multi-modal data to obtain damage features of each modality, and damage detection is performed on the damage features of each modality to obtain damage detection results of each modality.

[0112] ① Two-dimensional feature analysis: the U-Net model outputs an oxidation discoloration probability of 0.85 and a scratch probability of 0.10.

[0113] ② Three-dimensional feature analysis: the curvature abnormal area is marked as a pit (probability 0.70), and the depth deviation area is marked as a structural damage (probability 0.65).

[0114] 3) Cross-modal feature fusion is performed according to the damage detection results of each modality and dynamic weights to obtain the appearance damage detection results of the target spacecraft.

[0115] ① The current environment is strong light (direct sunlight), the three-dimensional weight is increased to 0.6, and the two-dimensional weight is reduced to 0.4.

[0116] ② The weighted result is: Score 氧化 : 0.4x0.85+0.6x0=0.34; Score 凹坑 : 0.4x0+0.6x0.70=0.42; Preliminary judgment: pit (Score=0.42).

[0117] ③ The pit corresponds to a two-dimensional area without significant texture abnormalities, but the three-dimensional point cloud shows a depth ΔD=0.8mm and a curvature anomaly. Therefore, by reducing the three-dimensional point cloud confidence by 50% without two-dimensional texture abnormalities, it is determined to be a three-dimensional noise. Finally, it is determined to exclude the pit and confirm the oxidation discoloration (because C color =4.2 exceeds the threshold).

[0118] 4) Based on the two-dimensional damage features and three-dimensional damage features, and combined with the damage degree quantification formula, the appearance damage index of the target spacecraft is obtained, and the risk level of the target spacecraft is determined.

[0119] The risk level is determined to be high risk (MDDI≥0.7), triggering an emergency alarm; it is recommended to immediately disable the sail module and prioritize on-orbit maintenance. It should be noted that the technical effect comparison of the above example and the prior art is shown in Table 1.

[0120] Table 1:

[0121]

[0122] Example Two: Micro-crack detection of spacecraft cabin shell. Micro-cracks (<0.1mm) are generated on the cabin shell due to long-term thermal cycling, and traditional optical detection is easily disturbed by metal reflection and missed detection.

[0123] 1) Obtain the current multi-modal data of the solar panel surface of the target spacecraft and perform data preprocessing and cross-modal alignment to obtain target multi-modal data.

[0124] ①The aircraft is adjusted to a side light 30° angle, and a high-resolution black and white camera (20 million pixels) is used to shoot the cabin surface to enhance the edge contrast of micro-cracks. A suspected crack area (length 0.08mm, confidence 75%) is detected in the image. Laser radar scans the cabin surface to generate a dense point cloud (accuracy ±0.05mm) to capture local curvature changes. The point cloud shows that the average curvature of the crack area is H=0.12mm -1 (super threshold 0.1mm -1 ).

[0125] ②CLAHE algorithm is used to enhance the crack contrast in low light areas, and the signal-to-noise ratio is improved by 25%; NDT (normal distribution transformation) is used to register multi-view point clouds, and a large range of surface registration is adapted (error <0.3mm).

[0126] ③Map the two-dimensional crack area to the three-dimensional point cloud to ensure accurate spatial position matching.

[0127] 2) Feature extraction is performed on the target multi-modal data to obtain damage features for each modality, and damage detection is performed on the damage features of each modality to obtain damage detection results for each modality.

[0128] The improved U-Net model outputs a micro-crack probability of 0.75 and an oxidation discoloration probability of 0.05. The curvature anomaly area is marked as structural damage (probability 0.85), and the depth deviation ΔD=0.03mm (normal range).

[0129] 3) Cross-modal feature fusion is performed according to the damage detection results of each modality and dynamic weights to obtain the appearance damage detection results of the target spacecraft.

[0130] ①The current environment is low light, the two-dimensional weight is increased to 0.8, and the three-dimensional weight is reduced to 0.2.

[0131] ②Weighted results: Score 微裂纹 : 0.8x0.75+0.2x0=0.600.8x0.75+0.2x0=0.60; Score 结构损伤: 0.8x0 + 0.2x0.85 = 0.17. Preliminary judgment: Micro crack (Score = 0.60).

[0132] ③Three-dimensional point cloud corresponding area curvature continuous anomaly (H = 0.12mm -1 ), and the depth has no significant deviation. Since the two-dimensional and three-dimensional features are consistent, the two-dimensional confidence is increased by 20%.

[0133] 4) Based on the two-dimensional damage feature and the three-dimensional damage feature, and combined with the damage degree quantification formula, the appearance damage index of the target spacecraft is obtained, and the risk level of the target spacecraft is determined.

[0134] The risk level is determined as low risk (MDDI < 0.4), and a regular monitoring suggestion is generated. It is suggested to recheck after 3 months, and immediate repair is not required. It should be noted that the technical effect comparison of the above example and the prior art is shown in Table 2.

[0135] Table 2:

[0136]

[0137] The technical scheme of the embodiment improves the accuracy of spacecraft appearance damage detection without relying on specific experimental data, and provides a better solution for spacecraft on-orbit health monitoring.

[0138] Figure 3 An embodiment of a spacecraft appearance damage detection system 200 provided by the application is shown in the structural schematic diagram. As shown in Figure 3 The system 200 includes a first processing module 210, a second processing module 220 and a damage detection module 230.

[0139] The first processing module 210 is configured to obtain current multi-modal data of the target spacecraft's outer surface damage area and perform data preprocessing and cross-modal alignment to obtain target multi-modal data.

[0140] The second processing module 220 is configured to extract features from the target multi-modal data to obtain damage features of each modality, and perform damage detection on the damage features of each modality to obtain damage detection results of each modality.

[0141] The damage detection module 230 is configured to perform cross-modal feature fusion according to the damage detection results of each modality and dynamic weights to obtain the appearance damage detection results of the target spacecraft.

[0142] In an optional mode, further comprising: an acquisition module; the acquisition module is configured to:

[0143] The current multi-modal data of the damaged area of the outer surface of the target spacecraft is acquired by using a small common rail service spacecraft carrying multi-modal sensors.

[0144] In an optional mode, the multi-modal sensor comprises: a camera and a laser radar; wherein the camera is configured to acquire two-dimensional image data, and the laser radar is configured to acquire three-dimensional point cloud data.

[0145] In an optional mode, the current multi-modal data comprises: current two-dimensional image data and current three-dimensional point cloud data; the target multi-modal data comprises: target two-dimensional image data and target three-dimensional point cloud data; and the first processing module 210 is specifically configured to:

[0146] perform image distortion correction on the current two-dimensional image data to obtain preprocessed two-dimensional image data, and perform denoising filtering processing on the current three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data;

[0147] construct a mapping relationship between pixels and point clouds by using an extrinsic matrix between the camera and the laser radar, and perform cross-modal alignment on the preprocessed two-dimensional image data and the preprocessed three-dimensional point cloud data according to the mapping relationship to obtain the target two-dimensional image data and the target three-dimensional point cloud data after alignment.

[0148] In an optional mode, the second processing module 220 is specifically configured to:

[0149] extract texture and color features from the target two-dimensional image data to obtain two-dimensional damage features, and extract curvature and depth features from the target three-dimensional point cloud data to obtain three-dimensional damage features;

[0150] input the two-dimensional damage features into an improved U-Net network to obtain two-dimensional damage detection results containing surface damage types and corresponding confidence levels; wherein the improved U-Net network is a attention gate module embedded between an encoder and a decoder of an original U-Net network.

[0151] input the three-dimensional damage features into a three-dimensional point cloud model to obtain three-dimensional damage detection results containing structural damage types and corresponding confidence levels.

[0152] In an optional mode, further comprising: a generation module; the generation module is configured to:

[0153] adjust the initial weight of each modality based on current environmental information to generate dynamic weights of each modality.

[0154] In an alternative mode, further comprising: a decision module; the decision module is used for:

[0155] Based on the two-dimensional damage features and the three-dimensional damage features, and in combination with a damage degree quantification formula, an appearance damage index of the target spacecraft is obtained, and a risk level of the target spacecraft is determined according to the appearance damage index.

[0156] It should be noted that the beneficial effects of the spacecraft appearance damage detection system 200 based on multi-modal data fusion provided by the above embodiments are the same as those of the spacecraft appearance damage detection method based on multi-modal data fusion, which will not be repeated here. In addition, when the system provided by the above embodiments implements its functions, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the system is divided into different functional modules according to actual conditions to complete all or part of the above described functions. In addition, the system and method embodiments provided by the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0157] Among them, the spacecraft appearance damage detection system based on multi-modal data fusion of the present application can be a computer program (including program code) running in a computer device, for example, the spacecraft appearance damage detection system based on multi-modal data fusion of the present application is an application software, which can be used to execute the corresponding steps in the spacecraft appearance damage detection method based on multi-modal data fusion of the present application.

[0158] In some embodiments, the spacecraft appearance damage detection system based on multi-modal data fusion of the present application can be realized in a combination of software and hardware, for example, the spacecraft appearance damage detection system based on multi-modal data fusion of the present application can be a hardware decoding processor form of processor, which is programmed to execute the spacecraft appearance damage detection method based on multi-modal data fusion of the present application, for example, the hardware decoding processor form of processor can adopt one or more application specific integrated circuits (ASIC, Application Specific Integrated Circuit), DSP, programmable logic device (PLD, Programmable Logic Device), complex programmable logic device (CPLD, Complex Programmable Logic Device), field programmable gate array (FPGA, Field-Programmable Gate Array) or other electronic components.

[0159] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the names of the modules do not limit the modules themselves.

[0160] An electronic device according to an embodiment of the present application can include a processor and a memory. The memory can store a computer program. The processor can execute the computer program to perform the method of detecting damage to the appearance of a spacecraft based on multi-modal data fusion according to any of the embodiments of the present application.

[0161] In an optional embodiment, an electronic device is provided, as shown in Figure 4 Figure 4 The electronic device 4000 shown in the embodiment includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 can also include a transceiver 4004, which can be used for data interaction, such as data transmission and / or data reception, between the electronic device and other electronic devices. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not limit the embodiments of the present application.

[0162] The processor 4001 can be a CPU (Central Processing Unit, central processing unit), a general-purpose processor, a DSP (Digital Signal Processor, digital signal processor), an ASIC (Application Specific Integrated Circuit, application specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the present disclosure. The processor 4001 can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc.

[0163] ​The bus 4002 can include a path that transmits information between the above-described components. The bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 4002 can be divided into an address bus, a data bus, a control bus, or the like. For convenience of representation, Figure 4 The bus 4002 is represented by only one thick line, but it does not mean that there is only one bus or one type of bus.

[0164] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0165] The memory 4003 is used to store application code (computer program) for executing the scheme of the present application, and is controlled by the processor 4001 to execute. The processor 4001 is used to execute the application code stored in the memory 4003 to realize the content shown in the foregoing method embodiments.

[0166] Among them, the electronic device can also be a terminal device, and the terminal device can be any terminal device that can install an application and access a webpage through the application, including at least one of a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart television, and a smart vehicle device.

[0167] It should be noted that, Figure 4 The electronic device shown is only an example and should not limit the functions and use range of the embodiments of the present application.

[0168] The computer readable storage medium of the embodiment of the present application, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement any of the above space vehicle appearance damage detection methods based on multi-modal data fusion.

[0169] Optionally, the computer readable storage medium can be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), a magnetic tape, a floppy disk and an optical data storage device, etc.

[0170] In the exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the electronic device execute the above space vehicle appearance damage detection method based on multi-modal data fusion.

[0171] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0172] It should be understood that the flow diagrams and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of various embodiments of the present application. In this regard, each block in the flow diagrams and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each of the blocks of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0173] The computer readable storage medium of embodiments of the present application can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, the computer readable storage medium can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0174] The computer readable storage medium described above can bear one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0175] The above description merely provides preferred embodiments of the present application and a principle of applied technology. It should be understood by those skilled in the art that the disclosed range of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or equivalent features without departing from the disclosed concept. For example, the above features can be replaced with the technical features disclosed in the present application (but not limited to) having similar functions to form technical solutions.

[0176] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and do not represent a specific order or sequential order. The order of use of similar objects can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described.

[0177] Those skilled in the art know that the present application can be implemented as a system, a method or a computer program product, so the present application can be specifically implemented as follows: it can be a complete hardware, a complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present application can also be implemented as a computer program product in one or more computer readable media, which contains computer readable program code.

[0178] Although the embodiments of the present application have been shown and described above, it should be understood that the above-described embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

Claims

1. A method for detecting surface damage on spacecraft based on multimodal data fusion, characterized in that, include: A small co-orbital servicing spacecraft equipped with multimodal sensors is used to collect current multimodal data of the damaged area on the outer surface of the target spacecraft; wherein, the multimodal sensors include: a camera and a lidar; the camera is used to collect two-dimensional image data, and the lidar is used to collect three-dimensional point cloud data; the current multimodal data includes: current two-dimensional image data and current three-dimensional point cloud data; Image distortion correction is performed on the current two-dimensional image data to obtain preprocessed two-dimensional image data, and noise reduction filtering is performed on the current three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data. Using the extrinsic parameter matrix between the camera and the lidar, a mapping relationship between pixels and point clouds is constructed. Based on the mapping relationship, cross-modal alignment is performed on the preprocessed two-dimensional image data and the preprocessed three-dimensional point cloud data to obtain aligned target two-dimensional image data and target three-dimensional point cloud data. Texture and color features are extracted from the target two-dimensional image data to obtain two-dimensional damage features, and curvature and depth features are extracted from the target three-dimensional point cloud data to obtain three-dimensional damage features. The two-dimensional damage features are input into an improved U-Net network to obtain a two-dimensional damage detection result that includes the surface damage type and the corresponding confidence level; wherein, the improved U-Net network is: an attention gating module is embedded between the encoder and decoder of the original U-Net network; The three-dimensional damage features are input into a three-dimensional point cloud model to obtain three-dimensional damage detection results that include structural damage types and corresponding confidence levels. Cross-modal feature fusion is performed based on the damage detection results of each modality and dynamic weights to obtain the external damage detection results of the target spacecraft; wherein, the two-dimensional damage detection results are: The three-dimensional damage detection results are as follows: The results of the appearance damage detection are as follows: ; This represents the two-dimensional dynamic weights corresponding to the two-dimensional damage detection results. This represents the three-dimensional dynamic weights corresponding to the three-dimensional damage detection results.

2. The spacecraft appearance damage detection method based on multimodal data fusion according to claim 1, characterized in that, Also includes: Based on the current environmental information, the initial weights of each modality are adjusted to generate dynamic weights for each modality.

3. The spacecraft appearance damage detection method based on multimodal data fusion according to claim 1, characterized in that, Also includes: Based on the two-dimensional damage features and the three-dimensional damage features, and combined with the damage degree quantification formula, the risk level of the target spacecraft is obtained and determined according to the appearance damage index of the target spacecraft.

4. A spacecraft exterior damage detection system based on multimodal data fusion, characterized in that, include: The system comprises an acquisition module, a first processing module, a second processing module, and a damage detection module. The acquisition module is used to: acquire current multimodal data of the damaged area on the outer surface of the target spacecraft using a small co-orbital servicing spacecraft equipped with multimodal sensors; wherein, the multimodal sensors include: a camera and a lidar; the camera is used to acquire two-dimensional image data, and the lidar is used to acquire three-dimensional point cloud data; the current multimodal data includes: current two-dimensional image data and current three-dimensional point cloud data; The first processing module is used to: perform image distortion correction on the current two-dimensional image data to obtain preprocessed two-dimensional image data; and perform noise reduction filtering on the current three-dimensional point cloud data to obtain preprocessed three-dimensional point cloud data; using the extrinsic parameter matrix between the camera and the lidar, construct a mapping relationship between pixels and point clouds; and according to the mapping relationship, perform cross-modal alignment between the preprocessed two-dimensional image data and the preprocessed three-dimensional point cloud data to obtain aligned target two-dimensional image data and target three-dimensional point cloud data. The second processing module is used to: extract texture and color features from the target two-dimensional image data to obtain two-dimensional damage features, and extract curvature and depth features from the target three-dimensional point cloud data to obtain three-dimensional damage features; input the two-dimensional damage features into an improved U-Net network to obtain two-dimensional damage detection results containing surface damage types and corresponding confidence levels; input the three-dimensional damage features into a three-dimensional point cloud model to obtain three-dimensional damage detection results containing structural damage types and corresponding confidence levels; wherein, the improved U-Net network is: an attention gating module is embedded between the encoder and decoder of the original U-Net network; The damage detection module is used to: perform cross-modal feature fusion based on the damage detection results of each modality and dynamic weights to obtain the appearance damage detection results of the target spacecraft; wherein, the two-dimensional damage detection results are: The three-dimensional damage detection results are as follows: The results of the appearance damage detection are as follows: ; This represents the two-dimensional dynamic weights corresponding to the two-dimensional damage detection results. This represents the three-dimensional dynamic weights corresponding to the three-dimensional damage detection results.

5. An electronic device, characterized in that, The electronic device includes a processor coupled to a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the spacecraft appearance damage detection method based on multimodal data fusion as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer-readable storage medium to implement the spacecraft appearance damage detection method based on multimodal data fusion as described in any one of claims 1 to 3.

Images