A monocular image sequence satellite attitude estimation method

Abstract

The application discloses a monocular image sequence satellite attitude estimation method, and belongs to the technical field of space target perception, and comprises the following steps: acquiring satellite sequence images shot by a space-based monocular camera as input images; completing segmentation of a solar panel and a load main body; obtaining a single-frame satellite attitude based on a segmentation result; screening feature points in the satellite sequence images based on a feature extraction deep learning network and a preliminary semantic segmentation result; guiding feature point matching based on semantic information of an extracted mask; iteratively optimizing the feature point matching by using a polar line constraint; obtaining an essential matrix between adjacent images; calculating satellite attitude changes in the satellite sequence images; and combining the single-frame satellite attitude to obtain a series of attitudes of the satellite in the satellite sequence images. According to the technical scheme, the attitude estimation task can be better completed without labeled data, and the generalization performance for a non-cooperative target satellite is improved.

Description

Technical Field

[0001] This invention belongs to the field of space target situational awareness, and specifically relates to a method for satellite attitude estimation of monocular image sequences. Background Technology

[0002] As more and more countries gain the ability to access and utilize space, possessing stronger space situational awareness capabilities has become an urgent need for major powers. In recent years, space activities of various countries have become increasingly frequent, the number of space targets in orbit has continued to increase, and people are paying more and more attention to the in-orbit status of space targets. Estimating the attitude of space targets has become an important computer vision problem.

[0003] Currently, accurate measurement of the attitude of space targets in orbit relies on high-quality, continuous, high-resolution imaging sensing using radar and optics. Monocular cameras are widely used in spacecraft due to their relative simplicity, small size, low power and weight requirements, and ease of integration. Therefore, obtaining the pose of observed objects from images acquired by space-based monocular cameras is a worthwhile research topic. However, long-distance observations are affected by extreme conditions such as lighting, resulting in blurred images and low resolution. Furthermore, for attitude estimation of non-cooperative space targets, there is a lack of readily available labels and known 3D models to train neural network algorithms.

[0004] The inventors of this application have discovered that satellite attitude estimation based on sequential images determines the satellite's attitude and attitude changes from continuous images. Compared to pose estimation based on a single frame image, it can determine satellite attitude changes by combining temporal information. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the aforementioned problems, this invention proposes a method for estimating satellite attitude from monocular image sequences. Using optical satellite image sequences captured by a spaceborne monocular camera during satellite convergence as input, the images are preprocessed using a pre-trained Segment Anything Model (SAM). This preprocessing eliminates the need for retraining the model with annotation information, enabling segmentation of the solar panels, payload, and background information. Based on the segmentation results, the geometric assumption that the solar panels are rectangular planes is fused, and the vanishing point principle is used to obtain the solar panel attitude, i.e., the satellite attitude. Based on the correlation between the image sequences, a deep learning algorithm is used as the backbone network to extract feature points. The semantic segmentation results are fused to classify and filter key points based on their criticality. Image matching is guided by semantic information, the fundamental matrix is ​​solved, and the Simpson error is calculated using epipolar geometric constraints. The least squares method is used to iteratively optimize key point selection. The matching points are further optimized based on the solar panel plane assumption, constraining the feature point positions. Finally, the satellite attitude changes between the image sequences are calculated.

[0007] (II) Technical Solution

[0008] This invention proposes a method for satellite attitude estimation of monocular image sequences, comprising the following steps:

[0009] S1: Obtain satellite sequence images captured by a space-based monocular camera as input images, without the need for annotation information;

[0010] S2, based on the pre-trained semantic segmentation large model, performs preliminary semantic segmentation on the acquired satellite sequence images, calculates the shape features of the mask obtained from the preliminary semantic segmentation, and performs clustering and classification to complete the segmentation of the solar panels and the main body of the payload;

[0011] S3. Based on the segmentation results, two sets of parallel lines are extracted using the solar panel. Rectangular constraints are added based on the two sets of parallel lines, and the solar panel normal vector is calculated to obtain the satellite attitude of a single frame.

[0012] S4. Based on the feature extraction deep learning network, the feature points in the satellite sequence images are screened by fusing the preliminary semantic segmentation results. Based on the semantic information of the mask extracted in step S2, the feature point matching is guided. The feature point matching is iteratively optimized by using epipolar constraints to obtain the essential matrix between adjacent images. The satellite attitude changes in the satellite sequence images are calculated. Combined with the satellite attitude of a single frame, a series of satellite attitudes in the satellite sequence images are obtained.

[0013] The above technical solutions bring the following beneficial effects:

[0014] This method integrates a pre-trained semantic segmentation model with the planar assumption of the solar panels to achieve attitude estimation of dual-solar-panel satellites based on sequential images acquired by a monocular camera. It can also perform attitude estimation tasks well even without labeled data, thus improving the generalization performance for non-cooperative target satellites.

[0015] Using visible light images, without the need for radar data to provide depth information, without the need for labeled information to train neural networks, and without the need for individual training for each satellite, it can generalize attitude estimation for satellites with symmetrical solar panels.

[0016] For the first time, the vanishing point principle and template matching method are used to achieve coarse satellite attitude estimation, and image matching based on a pre-trained semantic segmentation model and deep learning neural network is used to determine the relationship between adjacent images to achieve more refined attitude estimation.

[0017] By introducing a pre-trained semantic segmentation model, which eliminates the need for separate training, the model integrates the geometric prior information of the sailboard to complete the segmentation of the sailboard and the load body, significantly improving the feature matching success rate and reducing the adverse effects of poor image quality and lack of texture information on feature point matching. Attached Figure Description

[0018] Figure 1This is a schematic flowchart of a monocular image sequence satellite attitude estimation method according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of a reference coordinate system according to an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of this application. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0021] Figure 1 This is a schematic flowchart of a monocular image sequence satellite attitude estimation method according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0022] S1: Obtain satellite sequence images captured by a space-based monocular camera as input images, without the need for annotation information;

[0023] S2, based on the pre-trained semantic segmentation large model, performs preliminary semantic segmentation on the acquired satellite sequence images, calculates the shape features of the mask obtained from the preliminary semantic segmentation, and performs clustering and classification to complete the segmentation of the solar panels and the main body of the payload;

[0024] S3. Based on the segmentation results, two sets of parallel lines are extracted using the solar panel. Rectangular constraints are added based on the two sets of parallel lines, and the solar panel normal vector is calculated to obtain the satellite attitude of a single frame.

[0025] S4. Based on the feature extraction deep learning network, the feature points in the satellite sequence images are screened by fusing the preliminary semantic segmentation results. Based on the semantic information of the mask extracted in step S2, the feature point matching is guided. The feature point matching is iteratively optimized by using epipolar constraints to obtain the essential matrix between adjacent images. The satellite attitude changes in the satellite sequence images are calculated. Combined with the satellite attitude of a single frame, a series of satellite attitudes in the satellite sequence images are obtained.

[0026] The specific implementation method of step S1 in the above embodiment is as follows:

[0027] Input data is acquired without the need for annotation. The data consists of visible light images of satellite sequences captured by actual observations or simulations using a space-based monocular camera. In one embodiment, this step may include resizing the original image to 1024×1024 pixels for subsequent processing by a pre-trained semantic segmentation model.

[0028] For step S2, preliminary semantic segmentation is performed on the acquired satellite sequence images based on the pre-trained semantic segmentation large model. The shape features of the mask obtained from the preliminary semantic segmentation are then clustered and classified to complete the segmentation of the solar panels and the payload body, including:

[0029] The pre-trained semantic segmentation large model SAM (Segment Anything Model) performs semantic segmentation on satellite sequence images in an unsupervised manner. A detection output module for solar panels is added to the mask output head. Based on the rectangular and symmetrical properties of solar panels, solar panel detection aggregation is achieved to segment the solar panels, payload, and background.

[0030] In one embodiment, in terms of parameter settings, the initial semantic segmentation algorithm adopts a uniform sampling strategy, adjusting the image size to the same size of 1024×1024, and adaptively adjusting the size of the minimum segmented mask according to the images acquired at different distances.

[0031] In one implementation, a pre-trained semantic segmentation model (SAM, SegmentAnything Model) is used to perform preliminary semantic segmentation on satellite image sequences. The number of sampling points per edge (where an edge refers to the length and width of the input satellite image sequence) is adaptively adjusted based on the preliminary semantic segmentation results. In one implementation, this number of sampling points is initially set to 32. The preliminary semantic segmentation employs a uniform sampling semantic segmentation strategy. If fewer than 5 categories are obtained, the number of sampling points per edge is doubled until the preliminary segmentation can be performed satisfactorily, enabling the segmentation of each individual solar panel and the payload itself.

[0032] Regarding the segmentation of solar panels: In calculating the shape features of the mask, the pixel-level area of ​​each segmented mask is calculated, and the minimum quadrilateral is fitted to approximate the mask. Feature information such as aspect ratio, crossover ratio, and tilt rate are calculated. Based on this feature information, the masks are clustered and divided. The mask that satisfies the rectangle prior and has the largest number is the solar panel mask.

[0033] In the initial output of SAM, each sailboard is treated as a different category. Combining this with the principle of parallel projection (based on long-distance perspective projection), a clustering module is added after the mask output head. For each mask, a minimum quadrilateral is fitted, and its aspect ratio, intersection-to-union ratio, tilt rate, and center point position are calculated as feature information. This clustering of the masks results in an output feature map with a mask dimension of H. W 3, representing the sailboard, the load body, and the background, respectively, as the final output.

[0034] Specifically, based on the preliminary semantic segmentation, and taking advantage of the prior information that the solar panel is rectangular, masks that satisfy the parallelogram shape are searched in the preliminary semantic segmentation results. The area, center point position, symmetry, and shape of each mask are calculated. A threshold is selected based on the size of the satellite sequence image and the area occupied by the satellite in the image. The similarity between different masks is calculated based on the threshold. Masks that meet the following conditions are output as solar panels: similar area, center point positions on the same straight line, similar included angles between the two sides, and symmetry. The central part excluding the solar panel is output as the payload body. After step S2, the satellite's solar panel information and payload body information are obtained, laying the foundation for subsequent attitude estimation and feature point matching.

[0035] For step S3, based on the segmentation results, two sets of parallel lines are extracted using the solar panels. Rectangular constraints are added based on the two sets of parallel lines, and the solar panel normal vector is calculated to obtain the satellite attitude for a single frame. Specifically, this step includes obtaining the preliminary attitude based on the segmentation results, using the assumption of parallel lines existing on the solar panels, and using the principle of the vanishing point of parallel lines. It can be further divided into the following steps:

[0036] S31, extract the solar panel image based on the solar panel mask obtained from the preliminary semantic segmentation, extract the edge information in the solar panel image based on the prior information that the solar panel has a rectangular geometric property, and use edge detection to extract the edge information in the solar panel image and use Hough transform to detect the straight lines in the solar panel image;

[0037] In one embodiment, the minimum line segment length for Hough transform detection is adaptively set according to the size of the sail mask.

[0038] S32, based on the triangle area algorithm, the straight lines are classified into two sets of parallel lines. For each set of parallel lines, a vanishing point is solved. Regardless of whether the vanishing point is inside or outside the windsurfing image, the center of the windsurfing image is connected to the vanishing point to obtain line segments in two directions. Through these two line segments, the edge line segments of the windsurfing with similar slopes are found to obtain two sets of parallel lines.

[0039] S33, based on the fact that two sets of parallel lines in the windsurfing image have parallel and perpendicular relationships, and considering that the windsurfing image satisfies the perspective projection law, the normal vector of the windsurfing plane is solved by... By combining the rotational relationship between the solar panel's normal vector in the target coordinate system and the camera coordinate system normal vector, the solar panel attitude is obtained, which represents the initial attitude of a single frame of the satellite.

[0040] For detailed calculation procedures, please refer to... Figure 2 , Figure 2 In the coordinate system Represents the target coordinate system. Representing the camera coordinate system, first solve the plane. normal vector ,Will It can be represented in the following direction cosine form:

[0041] ,

[0042] in They are respectively The angle between the plane and the three coordinate axes, let the plane be... The equation is

[0043] ,

[0044] Where A, B, and C are the components of the plane normal vector in the x, y, and z axes, respectively.

[0045] We can obtain:

[0046] ,

[0047] In the formula, .

[0048] The following analysis These four pass through the origin. To solve for the plane (0,0,0), we need to use the plane... For example, efgh is the image plane, and the plane... Intersects with the image plane at That is, the eh line, let its equation on the image plane be:

[0049] ,

[0050] in, It is the x-coordinate on the image plane. It is the ordinate on the image plane. It is a plane Line of intersection with the image plane The direction angle, Intersecting lines Intercept on the image plane.

[0051] On the image plane, Two points: , The spatial coordinates of these two points are: , Combining this with the origin, we can obtain... The three-point form of the plane equation:

[0052] ,

[0053] In the formula, f is the focal length. Combining the above formula, we can obtain the following result:

[0054] ,

[0055] ,

[0056] ,

[0057] in, , , These are the coefficients in the relevant equations of the plane EFO. It is the focal length f and the orientation angle The product of the cosine values, It is the focal length f and the orientation angle The product of the sine values, It is the intercept. The opposite of the number is taken as , , Simplified representation.

[0058] Therefore, as long as the position of a straight line in the image plane is known on the image, and the camera's intrinsic parameters are also known, the camera coordinate system can be determined. Find the equation of the plane that passes through the origin corresponding to the line below.

[0059] With the three-point form of the equation of a plane, we can obtain the normal vector of the plane. The normal vector is as follows:

[0060] ,

[0061] The normal vectors corresponding to other planes can be obtained using the corresponding... To express.

[0062] It can also be known that the plane The four sides of the diagram are perpendicular to each other. and corresponding Still in plane For example, its relationship with a plane The intersection line is set as , by vector This means that, according to the cross product theorem, the cross product of two vectors results in a vector perpendicular to the original two vectors. It can be represented as:

[0063] ,

[0064] Easy to push, Similarly, other plane intersections also have corresponding relationships. Since the plane of the windsurfing board is rectangular, it satisfies... Similarly, the other correspondences are: , , , , , .

[0065] by For example, It can be solved as follows:

[0066] ,

[0067] in, , , These are the coefficients in the equation of the plane where the FGO plane intersects the image plane, similar to... , , Follow-up , , It is a plane The coefficients in the equation of the plane intersecting the image plane. , , It is a plane The coefficients in the equation of the plane that intersects the image plane.

[0068] make:

[0069] ,

[0070] ,

[0071] ,

[0072] Simplifying, we get:

[0073] ,

[0074] , , Since all parameters are given by the straight lines in the image, it can be seen that the above equation has an analytical solution.

[0075] And from , can be obtained ,by For example, simplification can be made during calculation.

[0076] ,

[0077] Since this problem studies long-distance imaging, it is known that Compared to other parameters, it is a very small value and can be ignored in the calculation, therefore It can be simplified to the following formula:

[0078] ,

[0079] ,

[0080] so It can be simplified to:

[0081] ,

[0082] And from We can obtain:

[0083] ,

[0084] United:

[0085] ,

[0086] ,

[0087] ,

[0088] Given two unknowns m and n and three equations, this is an overdetermined problem. The nonlinear least squares method is used to find the most suitable m and n, thus obtaining the solution. , to obtain a plane The normal vector is the representation of the sail in the camera coordinate system, while the sail's coordinates in its own target coordinate system are determined by different models, commonly (0,0,1), (0,1,0), and (1,0,0), depending on the specific model. This is achieved by solving the plane... The rotation matrix between the normal vector of the solar panel and the normal vector of the satellite solar panel is used to obtain the initial attitude of the solar panel.

[0089] Specifically, for step S4, feature points in the satellite sequence images are filtered based on the preliminary semantic segmentation results fused by the feature extraction deep learning network. Feature point matching is guided by semantic information, and epipolar constraints are used to iteratively optimize the feature point matching, obtaining the essential matrix between adjacent images. The satellite attitude changes in the satellite sequence images are calculated, and a series of satellite attitudes in the satellite sequence images are obtained by combining the satellite attitude of a single frame. Specifically, feature points in the satellite images are filtered based on the semantic segmentation results fused by the feature extraction deep learning network to obtain key points of different parts of the satellite, such as the corner points of the solar panels and the main feature points of the payload. Image matching is performed between the satellite sequence images, and Simpson error is used for iterative optimization to determine the attitude changes between the sequence images and thus determine the satellite attitude in the sequence images. The specific implementation steps include the following process:

[0090] S41, based on the deep learning algorithm pre-trained on the simulation dataset, feature points are extracted from satellite sequence images. Based on the descriptor information of the solar panel mask extracted from the satellite sequence images, and based on the position of the edge and corner of the solar panel semantic mask extracted in S2, the feature points closest to the corner of the solar panel mask are obtained. That is, the feature points at the edge and corner of the solar panel are used as the feature points with the highest confidence. Based on these feature points, a certain number (e.g., 8) of feature points in the payload body are adaptively extracted to make up 8 feature points, which are then matched with the features of subsequent images.

[0091] S42: Based on the matching relationship between feature points of any two adjacent images in the satellite sequence, and combined with camera intrinsic parameters K (including focal length, principal point, etc.), calculate the essential matrix between the two images. Obtain the epipolar line of the feature points extracted in step S41 in one image in the other image. Calculate the distance between the feature points and the epipolar line, i.e., the Simpson error. Design a loss function based on the Simpson error, minimize the loss function using the least squares method, iteratively optimize the matching point pairs between the two images, and select the matching point pair with the minimum loss function as the final matching point pair. Here, the matching relationship refers to the feature point position of a feature point in one image on the other image.

[0092] S43: Based on the matching point pairs obtained in S42, the fundamental matrix between two adjacent images is calculated using the eight-point algorithm. The satellite attitude change between adjacent images is calculated based on the fundamental matrix. Combined with the single-frame satellite attitude obtained from the single-frame image, a series of satellite attitudes in the satellite sequence image are calculated.

[0093] The above steps may also include: based on the calculated matching relationship between images and the essential matrix, fusing the mask of the load subject obtained from the preliminary semantic segmentation, calculating the feature point matching degree of the load subject, and adaptively determining whether the solar panel has flipped over when the front and back sides are the same, based on the feature point matching degree.

[0094] Specifically, in step S41, a pre-trained neural network is used to extract rich feature points, which are then fused with the solar panel segmentation mask information, i.e., the solar panel mask obtained from semantic segmentation in one implementation method. Specifically, this mask is a binary image used to increase the descriptor dimension of feature points. Feature points are extracted at the edge and corner positions as feature points with higher confidence. Based on the number of extracted feature points, feature points with high matching success rate in the payload body are adaptively extracted. In this example, a deep learning algorithm is used for feature point matching, and a graph neural network is used to further enhance the descriptor of feature points. More than 6 feature points are extracted from the satellite, of which the outermost 4 corner points are selected for the solar panel, and more than 2 key points with the highest confidence are selected for the payload body, for subsequent feature point matching and calculation of the fundamental matrix.

[0095] Specifically, step S42 also includes calculating the fundamental matrix F based on the matching information. The fundamental matrix F describes the epipolar constraint of corresponding points in the two images. In epipolar geometry, for a point p in one image, there is an epipolar line in the other image. Correspondingly, the matching point for this point Also there Above, the fundamental matrix F is such a projective mapping from a point to a line. The fundamental matrix F has the following form:

[0096] ,

[0097] - These are the unknown parameters to be solved, representing the geometric relationship between the two images.

[0098] If there are two matching points and Then the following correspondence exists:

[0099] ,

[0100] The camera intrinsic parameter matrix K is a known parameter determined by the camera, and takes the following form:

[0101] ,

[0102] in, and represents the camera's focal length in the horizontal and vertical directions, respectively, and s represents the non-orthogonality between the image axes. and It represents the projection point of the optical axis onto the image plane.

[0103] The intrinsic parameter matrix K has the following relationship with the fundamental matrix F and the essential matrix E:

[0104] ,

[0105] The fundamental matrix F can be obtained by matching feature points optimized by semantic segmentation results between two images. The distance from point p to the corresponding epipolar line is calculated in reverse using the fundamental matrix F. The distance is accumulated as a loss function by assigning weights based on the inner product of the feature vectors of the matching key points. Key points of the payload body are selected iteratively, and the fundamental matrix F is calculated in combination with the feature points of the solar panel. The selection of key points is optimized by the least squares method, and finally the fundamental matrix F and the essential matrix E are obtained. The rotation matrix R is calculated through the essential matrix to obtain the x, y, z axis attitude changes of the satellite between adjacent images. The attitude calculated by the single frame image is used as the initial attitude to obtain a series of satellite attitudes in the sequence of images.

[0106] Specifically, in step S43, based on the assumption that the sailboard is planar, all feature points on the plane need to satisfy the homography transformation of the plane, as shown in the following equation:

[0107] ,

[0108] In the case of long-distance imaging, it simplifies to the following formula:

[0109] ,

[0110] Where z represents the distance from the point to the camera. This represents the depth from point A to cameras A and B. The normal vector representing the plane. It is the camera's intrinsic parameter matrix. and These are the pixel coordinates of a point on the image. and These are the pixel coordinates of the same point in image A and image B. This represents the distance from the plane to the origin. These are the intrinsic parameter matrices for cameras A and B, respectively, assuming the two images were taken by the same camera. and They are all the same. It is the rotation matrix from camera A to camera B. It is the translation vector from camera A to camera B. It is the vertical distance from the image plane to the imaging plane of camera A. This is the normal vector of the imaging plane of camera A. The subscript 'a' represents the parameters related to camera A, and the subscript 'b' represents the parameters related to camera B. If both images are taken by the same camera, then the parameters of camera A are the same as those of camera B, and the subscript 'ba' represents the transformation relationship from camera A to camera B.

[0111] That is, corresponding points on the same plane must satisfy the above formula. Based on the above formula, the feature points on the windsurf plane are constrained during the iteration process, which can better improve the matching accuracy and success rate.

[0112] Specifically, for step S43, in the embodiment process, based on the calculated matching relationship between images and the essential matrix, the rotation matrix R is calculated through the essential matrix to obtain the x, y, z axis attitude changes of the satellite between adjacent images. The attitude calculated from a single frame image is used as the initial attitude to obtain a series of satellite attitudes in the sequence of images.

[0113] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0114] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes, and not for the purpose of explaining or limiting the subject matter of the invention.

Claims

1. A method for satellite attitude estimation using monocular image sequences, characterized in that, Includes the following steps: S1: Obtain satellite sequence images captured by a space-based monocular camera as input images, without the need for annotation information; S2, based on the pre-trained semantic segmentation large model, performs preliminary semantic segmentation on the acquired satellite sequence images, calculates the shape features of the mask obtained from the preliminary semantic segmentation, and performs clustering and classification to complete the segmentation of the solar panels and the main body of the payload; S3. Based on the segmentation results, two sets of parallel lines are extracted using the solar panel. Rectangular constraints are added based on the two sets of parallel lines, and the solar panel normal vector is calculated to obtain the satellite attitude of a single frame. S4. Based on the feature extraction deep learning network, the feature points in the satellite sequence images are screened by fusing the preliminary semantic segmentation results. Based on the semantic information of the mask extracted in step S2, the feature point matching is guided. The feature point matching is iteratively optimized by using epipolar constraints to obtain the essential matrix between adjacent images. The satellite attitude change in the satellite sequence images is calculated. Combined with the satellite attitude of a single frame, a series of satellite attitudes in the satellite sequence images are obtained. The process of segmenting solar panels includes: calculating the pixel-level area of ​​each segmented mask in terms of mask shape features; fitting a minimum quadrilateral to approximate the mask; calculating feature information including aspect ratio, cross-union ratio, and tilt rate; and clustering the masks based on the feature information. The mask that satisfies the rectangle prior and has the largest number of masks is the solar panel mask. In the preliminary semantic segmentation results, masks that satisfy the parallelogram shape are searched, and the area, center point position, symmetry, and shape of each mask are calculated. A threshold is selected based on the size of the satellite sequence image and the area of ​​the image occupied by the satellite. The similarity between different masks is calculated based on the threshold. Masks that meet the following conditions are output as solar panels: similar area, center point position on the same straight line, similar included angle between the two sides, and symmetry. The central part other than the solar panel is output as the main load body. S3 includes: S31, extract the solar panel image based on the solar panel mask obtained from the preliminary semantic segmentation, extract the edge information in the solar panel image based on the prior information that the solar panel has a rectangular geometric property, and use edge detection to extract the edge information in the solar panel image and use Hough transform to detect the straight lines in the solar panel image; S32, based on the triangle area algorithm, the straight lines are classified into two sets of parallel lines. For each set of parallel lines, a vanishing point is solved. The center of the windsurf image is connected to the vanishing point to obtain line segments in two directions. Through these two line segments, the edge line segments of the windsurf with similar slopes are found to obtain two sets of parallel lines. S33, based on the fact that two sets of parallel lines in the windsurfing image have parallel and perpendicular relationships, and considering that the windsurfing image satisfies the perspective projection law, the normal vector of the windsurfing plane is solved by... By combining the rotational relationship between the solar panel's normal vector in the target coordinate system and the camera coordinate system normal vector, the solar panel attitude is obtained, which represents the initial attitude of a single frame of the satellite.

2. The monocular image sequence satellite attitude estimation method according to claim 1, characterized in that, S1 includes: The original image size was adjusted to 1024×1024 so that it could be fed into a pre-trained semantic segmentation model for further processing.

3. The monocular image sequence satellite attitude estimation method according to claim 1, characterized in that, S2 include: The pre-trained semantic segmentation large model performs semantic segmentation on satellite sequence images under unsupervised conditions. A detection output module for solar panels is added to the mask output head. Based on the rectangular and symmetrical properties of solar panels, solar panel detection aggregation is achieved to segment the solar panels, payload body and background.

4. The monocular image sequence satellite attitude estimation method according to claim 3, characterized in that, Preliminary semantic segmentation of satellite sequence images is performed using a pre-trained semantic segmentation model. The number of points uniformly sampled for each edge is adaptively adjusted based on the preliminary semantic segmentation results. An edge refers to the length and width of the input satellite sequence image. The number of points is initially set to 32. The preliminary semantic segmentation adopts a uniform sampling semantic segmentation strategy. If the number of categories obtained is less than 5, the number of sampling points for each edge is doubled until each individual solar panel and payload body is segmented.

5. The monocular image sequence satellite attitude estimation method according to claim 1, characterized in that, S4 includes: S41, based on the deep learning algorithm pre-trained on the simulation dataset, feature points are extracted from satellite sequence images. Based on the descriptive information of the feature points of the solar panel mask extracted from the satellite sequence images, and based on the position of the edge and corner of the solar panel semantic mask extracted in S2, the feature points closest to the corner of the solar panel mask are obtained. That is, the feature points at the edge and corner of the solar panel are used as the feature points with the highest confidence. Based on these feature points, eight feature points in the payload body are adaptively extracted and matched with the features of subsequent images. S42, based on the matching relationship between feature points of any two adjacent images in the satellite sequence image, and combined with camera intrinsic parameters, calculate the essential matrix between the two images, obtain the epipolar line of the feature points extracted in step S41 in one image in the other image, calculate the distance between the feature points and the epipolar line, i.e. Simpson error, design a loss function based on Simpson error, use the least squares method to minimize the loss function, iteratively optimize the matching point pairs between the two images, and select the matching point pairs with the minimum loss function as the final matching point pairs; S43: Based on the matching point pairs obtained in S42, the fundamental matrix between two adjacent images is calculated using the eight-point algorithm. The satellite attitude change between adjacent images is calculated based on the fundamental matrix. Combined with the single-frame satellite attitude obtained from the single-frame image, a series of satellite attitudes in the satellite sequence images are calculated.

6. The monocular image sequence satellite attitude estimation method according to claim 5, characterized in that, In step S41, a pre-trained neural network is used to extract feature points, which are then fused with the solar panel segmentation mask information to increase the descriptor dimension of the feature points. Feature points at the edges and corners are extracted as feature points with higher confidence. Based on the number of extracted feature points, feature points with high matching success rate in the payload body are adaptively extracted. Deep learning algorithms are used to match feature points, and graph neural networks are used to further enhance the descriptors of feature points. More than 6 feature points are extracted from the satellite, with the outermost 4 corners selected for the solar panel and more than 2 key points with the highest confidence selected for the payload body. Subsequent feature point matching and calculation of the fundamental matrix are then performed.

7. The monocular image sequence satellite attitude estimation method according to claim 5, characterized in that, Step S42 further includes calculating the fundamental matrix F based on the matching information. The fundamental matrix F describes the epipolar constraint of corresponding points in the two images. In epipolar geometry, for a point p in one image, there is an epipolar line in the other image. Correspondingly, the matching point for this point Also there Above, the fundamental matrix F is such a projective mapping from a point to a line: ， - These are the unknown parameters to be solved, representing the geometric relationship between the two images; If there are two matching points and Then the following correspondence exists: ， The camera intrinsic parameter matrix K is a known parameter determined by the camera, and takes the following form: ， in, and represents the camera's focal length in the horizontal and vertical directions, respectively, and 's' represents the non-orthogonality between the image axes. and This represents the projection point of the optical axis onto the image plane; The intrinsic parameter matrix K has the following relationship with the fundamental matrix F and the essential matrix E: ， The fundamental matrix F is obtained by matching feature points between two images using the semantic segmentation results. The distance from point p to the corresponding epipolar line is calculated in reverse using the fundamental matrix F. The distance is accumulated as a loss function by assigning weights based on the inner product between the feature vectors of the matching key points. Key points of the main body of the load are selected iteratively. The fundamental matrix F is calculated by combining the feature points of the solar panel. The selection of key points is optimized by the least squares method. Finally, the fundamental matrix F and the essential matrix E are obtained. In step S43, based on the assumption that the sailboard is planar, all feature points on the plane need to satisfy the homography transformation of the plane, as shown in the following equation: ， In the case of long-distance imaging, it simplifies to the following formula: ， Where z represents the distance from the point to the camera. This represents the depth from point A to cameras A and B. The normal vector representing the plane. It is the camera's intrinsic parameter matrix. and These are the pixel coordinates of a point on the image. and These are the pixel coordinates of the same point in image A and image B. This represents the distance from the plane to the origin. These are the intrinsic parameter matrices for cameras A and B, respectively. It is the rotation matrix from camera A to camera B. It is the translation vector from camera A to camera B. It is the vertical distance from the image plane to the imaging plane of camera A. It is the normal vector of the imaging plane of camera A. The subscript a represents the parameters related to camera A, and b represents the parameters related to camera B. When both images are taken by the same camera, the parameters of camera A are the same as those of camera B. The subscript ba represents the transformation relationship from camera A to camera B. Based on the calculated matching relationship between images and the essential matrix, the rotation matrix R is calculated through the essential matrix to obtain the x, y, z axis attitude changes of the satellite between adjacent images. The attitude calculated from a single frame image is used as the initial attitude to obtain a series of satellite attitudes in the sequence of images.

Images