A method for measuring the position of a satellite in orbit based on parallelogram fitting
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- AEROSPACE DONGFANGHONG DEV LTD
- Filing Date
- 2022-12-02
- Publication Date
- 2026-06-23
Smart Images
Figure CN116012443B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of aerospace technology, and in particular to an on-orbit satellite attitude measurement method based on parallelogram fitting. Background Technology
[0002] With the increase in space activities of various countries, space missions are becoming more and more complex, and tasks such as searching, detecting, approaching, and escorting satellites are increasing. Visual detection is an important method in this process, so there is a need to develop high-efficiency and high-precision visual measurement algorithms. Summary of the Invention
[0003] To address the problems in the prior art, this invention provides an on-orbit satellite attitude measurement method based on parallelogram fitting.
[0004] This invention provides an on-orbit satellite pose measurement method based on parallelogram fitting, comprising the following steps:
[0005] Step A: Acquire images.
[0006] Step B: Design an image processing algorithm to extract the contours.
[0007] Step C: Combine contour points according to the set judgment conditions.
[0008] Step D: Target contour selection.
[0009] Step E: Use a parallelogram fitting algorithm to fit the target contour points to a parallelogram.
[0010] Step F: Extract corner points and calculate the pose of the target satellite.
[0011] As a further improvement of the present invention, step B further includes:
[0012] Step B1, image distortion removal.
[0013] Step B2: Set the ROI search area based on the region of the target satellite in the image.
[0014] Step B3: Convert the image into an HLS format suitable for machine processing.
[0015] Step B4: Perform median filtering on the image to reduce noise interference.
[0016] Step B5: Extract the target region based on the HLS values of each pixel in the image and convert the image into a binary image.
[0017] Step B6 uses morphological closing operations to reduce interference in the image, fill gaps in the target region, and reduce extraction errors.
[0018] Step B7: Extract the contour.
[0019] As a further improvement of the present invention, step C further includes:
[0020] Step C1: Fit each set of contour points using the minimum envelope rectangle to obtain the center point, side length, and rotation angle around the center point of the fitted rectangle.
[0021] Step C2: Starting from the first contour, sequentially determine the relationship between the distance between two contours and the side length of the contour fitting rectangle. If the difference is greater than the threshold, merge the contour point set into the previous contour point set and delete the next contour point set.
[0022] Step C3, return to step C1, repeat steps C2 and C3 until no more contours can be merged.
[0023] Step C4: Filter the contour point sets based on the number of points in each contour point set, and delete contours with fewer contour points.
[0024] As a further improvement of the present invention, in step D, extracting the contour corresponding to the solar panel as the target includes:
[0025] Step D1: Select the contour with the largest number of contour points as the contour of the target solar panel.
[0026] Step D2: Distinguish the outlines of the windsurfing boards according to their positional relationship on the image plane. Among the center point coordinates (u, v), the one with the smallest u is the leftmost windsurfing board, the one with the middle u is the middle windsurfing board, and the one with the largest u is the rightmost windsurfing board.
[0027] As a further improvement of the present invention, step E further includes:
[0028] Step E1: Obtain the set of contour points to be fitted, and calculate the center point of the contour, i.e., the average value of the coordinates of each point (u, v).
[0029] Step E2: Use the least squares method to find the optimal fitted line ax + by + c = 0, which passes through the center point of the contour point set.
[0030] Step E3: Determine the set of points on the two edge lines of the parallelogram that is approximately parallel to the optimal line.
[0031] Step E4: Use the least squares method to fit the best-fitting straight lines of the upper and lower contour point sets respectively. The two fitted lines must be parallel. After fitting the two lines with the two contour point sets respectively, solve the deflection angle of the lines. Calculate the weighted average of the deflection angle of the lines based on the number of elements in the two contour point sets, and then determine a, b, c1, and c2 of the two lines.
[0032] Step E5: Determine the point set of the other two parallel lines of the fitted parallelogram.
[0033] Step E6: Fit two more parallel lines based on the point set obtained in step E5.
[0034] Step E7: Based on the initially determined point set of the four sides of the fitted parallelogram and the initially confirmed linear equations of the four sides, the optimal linear equations of the four sides of the parallelogram are gradually approximated through iteration.
[0035] As a further improvement of the present invention, step E3 further includes:
[0036] Step E30: Based on the positional relationship between the contour points and the straight line, divide the contour points into two parts: points above the straight line (including those on the straight line) and points below the straight line.
[0037] Step E31: Calculate the distance from each contour point to the line, and find the maximum distance from the point above the line and the point below the line to the line.
[0038] In step E32, for points above and below the line, assign them to different point sets based on the difference between the distance of each point to the line and the maximum distance obtained in step E31.
[0039] Step E33: For points above and below, if the number of elements in the farthest point set is the largest, or the number of elements in the farthest point set is greater than the threshold, then the farthest point set is the set containing the contour edge points; otherwise, the point set with the largest number of elements in the remaining point set is the set containing the contour edge points.
[0040] As a further improvement of the present invention, step E5 further includes:
[0041] Step E50: Find the points in the point set from step E4 whose distance from the two fitted lines is less than the threshold and which are on the edge of the point set, and use them as the starting search points for the other two sides of the parallelogram to be fitted.
[0042] Step E51: Solve for the straight line passing through the two pairs of initial search points obtained in step E50. Use these two straight lines as the two initial sides of the parallelogram to be fitted. Search for points whose distance from the straight line is less than a threshold as the point set of the two sides of the fitted parallelogram. Fit the straight line formed by the two point sets, using the same method as step E4.
[0043] As a further improvement of the present invention, step E7 further includes:
[0044] Step E70: For the set of points used to fit the four sides of the parallelogram, determine the distance of each point from the fitted line. If the distance is less than the threshold, keep the point; otherwise, delete the point.
[0045] Step E71: Fit the two pairs of parallel edge lines of the parallelogram using the point set processed in step E70, in the same way as step E4.
[0046] Step E72: Determine whether the number of points deleted in the previous two iterations is less than the threshold. If yes, end the process; otherwise, return to step E70. The distance threshold used in step E70 will decrease as the number of iterations increases.
[0047] As a further improvement of the present invention, step F further includes:
[0048] Step F1: Solve for the four corner points formed by the intersection of the four fitted lines.
[0049] Step F2: Determine the satellite's pose in the camera coordinate system.
[0050] As a further improvement of the present invention, in step F2, if the three-dimensional dimensions of the solar panel are known, the P12P algorithm can be used for a monocular camera. If the fitting error of a certain corner point is large, the corner point can be excluded and the remaining corner points can be used for fitting. For a binocular camera or a multi-camera, the satellite pose can be calculated using the P12P algorithm for each monocular camera image and then the average value can be taken. If the three-dimensional dimensions of the solar panel are unknown, the position of each corner point can be calculated using a binocular camera or a multi-camera to obtain the satellite pose.
[0051] The beneficial effects of this invention are: 1. The on-orbit satellite pose measurement method of this invention has many applicable scenarios and high fitting accuracy; 2. The on-orbit satellite pose measurement method of this invention gradually approaches the best fitting line through iteration, which can eliminate the interference of points with large errors, and is applicable to both closed contour points and discrete point sets. Attached Figure Description
[0052] Figure 1 This is a schematic diagram of the satellite model of the present invention;
[0053] Figure 2 This is a flowchart of the on-orbit satellite attitude measurement method of the present invention;
[0054] Figure 3 This is a flowchart of the parallelogram fitting process for contour points in this invention;
[0055] Figure 4 This is a diagram showing the effect of fitting the parallelogram shape of the satellite solar panel of this invention. Detailed Implementation
[0056] Solar panels have a large surface area and are easily identifiable once deployed; therefore, this invention uses solar panels as the visual inspection object. The surface of a solar panel is generally composed of multiple sets of rectangular arrays. Individual rectangles are too small, and the spacing between them is small, making it difficult to ensure complete differentiation in an image. Therefore, the entire solar panel is used as the visual inspection object. The edges of the small rectangles within the panel and the gaps between them can cause significant interference to image recognition. The on-orbit satellite pose measurement method based on parallelogram fitting proposed in this invention can eliminate interference within the contour, achieving solar panel detection through parallelogram fitting of contour points.
[0057] The purpose of this invention is to detect the pose of solar panels, which are typically found on satellites, during missions such as satellite rendezvous and satellite approach. This invention utilizes visual detection to determine the pose of the solar panels. Addressing the issue of textured surfaces on solar panels significantly impacting edge extraction, the invention obtains contours from the image through image processing. These contours are then processed and combined using methods such as minimum rectangle envelope. Detection conditions are set based on the solar panel's pose in Cartesian space and its position in the image, extracting the contours corresponding to three solar panels. Finally, the invention employs a proposed parallelogram fitting algorithm to solve for the four corner points of the contours. This invention has wide applicability and high fitting accuracy.
[0058] This invention proposes a novel parallelogram fitting algorithm for fitting rectangular objects in Cartesian space onto an image plane. The algorithm uses the least squares method to find the optimal line for fitting all contour points, initially identifying the point sets for fitting two parallel lines, and further identifying the point sets for fitting two other parallel lines. Based on this, points offset from the fitting line are iteratively removed to solve for the four sides of the fitted parallelogram. This algorithm iteratively approximates the optimal fitting line, effectively eliminating interference from points with large errors, and is applicable to both closed contour points and discrete point sets.
[0059] The object of measurement in the technical solution of this invention is a satellite. For example... Figure 1 As shown, the satellite has three solar panels on its surface, and these three solar panels are used as the testing objects of this invention.
[0060] like Figure 2 As shown, this invention discloses an on-orbit satellite pose measurement method based on parallelogram fitting, comprising the following steps:
[0061] Step A: Image Acquisition; use a monocular or multi-view camera to capture images of the satellite.
[0062] Step B: Design an image processing algorithm to extract the contours.
[0063] Step C: Combine contour points according to the set judgment conditions.
[0064] Step D: Target contour selection.
[0065] Step E: Use a parallelogram fitting algorithm to fit the target contour points to a parallelogram.
[0066] Step F: Extract corner points and calculate the pose of the target satellite.
[0067] Step B includes:
[0068] Step B1, image distortion removal.
[0069] Step B2: Based on the region of the target satellite in the image, set the ROI (Local Area of Interest) search region to improve computational efficiency.
[0070] Step B3 converts the image into an HLS format that is more suitable for machine judgment.
[0071] Step B4: Perform median filtering on the image to reduce noise interference.
[0072] Step B5: Extract the target region based on the HLS values of each pixel in the image and convert the image into a binary image.
[0073] Step B6 uses morphological closing operations to reduce interference in the image, fill gaps in the target region, and reduce extraction errors.
[0074] Step B7: Extract the contour.
[0075] Step C further includes:
[0076] Step C1: Fit each set of contour points using the minimum envelope rectangle to obtain the center point, side length, and rotation angle around the center point of the fitted rectangle.
[0077] Step C2: Starting from the first contour, sequentially determine the relationship between the distance between two contours and the side length of the contour fitting rectangle. If the difference is greater than the threshold, merge the contour point set into the previous contour point set and delete the next contour point set.
[0078] Step C3, return to step C1, repeat steps C2 and C3 until no more contours can be merged.
[0079] Step C4: Filter the contour point sets based on the number of points in each contour point set, and delete contours with fewer contour points.
[0080] In step D, the target is the outline corresponding to the solar panel, which specifically includes:
[0081] Step D1: Select the three contours with the largest number of contour points to obtain the contours of the three target solar panels.
[0082] Step D2: Distinguish the outlines of the three windsurfing boards according to their positional relationship on the image plane. Among the center point coordinates (u, v), the one with the smallest u is the leftmost windsurfing board, the one with the middle u is the middle windsurfing board, and the one with the largest u is the rightmost windsurfing board.
[0083] like Figure 3 As shown, step E further includes:
[0084] Step E1: Obtain the set of contour points to be fitted, and calculate the center point of the contour, i.e., the average value of the coordinates of each point (u, v).
[0085] Step E2: Use the least squares method to find the optimal fitted line ax + by + c = 0, which passes through the center point of the contour point set.
[0086] Step E3: Determine the set of points on the two edge lines of the parallelogram that is approximately parallel to the optimal line.
[0087] Step E4: Use the least squares method to fit the best-fitting straight lines of the upper and lower contour point sets respectively. The two fitted lines must be parallel lines because the two opposite sides of the fitted parallelogram should be parallel. After fitting the two lines with the two contour point sets respectively, solve the deflection angle of the lines. Calculate the weighted average of the line deflection angles based on the number of elements in the two contour point sets, and then determine a, b, c1, and c2 of the two lines.
[0088] Step E5: Determine the point set of the other two parallel lines of the fitted parallelogram.
[0089] Step E6: Fit two more parallel lines based on the point set obtained in step E5.
[0090] Step E7: Based on the initially determined point set of the four sides of the fitted parallelogram and the initially confirmed linear equations of the four sides, the optimal linear equations of the four sides of the parallelogram are gradually approximated through iteration.
[0091] Step E3 further includes:
[0092] Step E30: Based on the positional relationship between the contour points and the straight line, divide the contour points into two parts: points above the straight line (including those on the straight line) and points below the straight line.
[0093] Step E31: Calculate the distance from each contour point to the line, and find the maximum distance from the point above the line and the point below the line to the line.
[0094] In step E32, for points above and below the line, assign them to different point sets based on the difference between the distance of each point to the line and the maximum distance obtained in step E31.
[0095] Step E33: For points above and below, if the farthest point set has the most elements, or the number of elements in the farthest point set is greater than a threshold, then the farthest point set is the set containing the contour edge points; otherwise, the point set with the most elements in the remaining point set is the set containing the contour edge points. This is because, generally, contour points should be distributed in a relatively concentrated area, and contour points should generally be located at the relatively edge of the entire contour point set.
[0096] Step E5 also includes:
[0097] Step E50: Find the points in the point set from step E4 whose distance from the two fitted lines is less than the threshold and which are on the edge of the point set, and use them as the starting search points for the other two sides of the parallelogram to be fitted.
[0098] Step E51: Solve for the straight line passing through the two pairs of initial search points obtained in step E50. Use these two straight lines as the two initial sides of the parallelogram to be fitted. Search for points whose distance from the straight line is less than a threshold as the point set of the two sides of the fitted parallelogram. Fit the straight line formed by the two point sets, using the same method as step E4.
[0099] Step E7 also includes:
[0100] Step E70: For the set of points used to fit the four sides of the parallelogram, determine the distance of each point from the fitted line. If the distance is less than the threshold, keep the point; otherwise, delete the point.
[0101] Step E71: Fit the two pairs of parallel edge lines of the parallelogram using the point set processed in step E70, in the same way as step E4.
[0102] Step E72: Determine whether the number of points deleted in the previous two iterations is less than the threshold. If yes, end the process; otherwise, return to step E70. Repeat steps E70 and E71 to continue the determination until the number of points deleted from the four point sets after the previous two processing iterations is less than the threshold. The distance threshold used in step E70 will decrease as the number of iterations increases.
[0103] Step F further includes:
[0104] Step F1: Solve for the four corner points formed by the intersection of the four fitted straight lines, which are the four corner points of the fitted parallelogram. There are a total of 12 corner points for the three contours.
[0105] Step F2: Determine the satellite's pose in the camera coordinate system.
[0106] In step F2, if the three-dimensional dimensions of the satellite solar panel are known, the P12P algorithm can be used for a monocular camera. If the fitting error of a certain corner point is large, that corner point can be excluded, and the remaining corner points can be used for fitting. For a binocular or multi-view camera, the satellite pose can be calculated using the P12P algorithm for each monocular camera image and then the average value can be taken. If the three-dimensional dimensions of the satellite solar panel are unknown, the position of each corner point can be calculated using a binocular or multi-view camera to obtain the satellite pose.
[0107] like Figure 4 As shown, this is the fitting effect of the three solar panels of the satellite. The connecting lines are the four sides of the parallelogram, and the intersection points of each side and the round dots are the corner points of the fitted parallelogram. It can be seen from the fitted image that the algorithm has a high fitting accuracy, and the entire algorithm has a smaller computational load compared with algorithms such as line extraction.
[0108] Compared with existing technologies, the present invention has the following advantages:
[0109] 1. This invention provides a method for detecting and measuring satellite pose. Using solar panels, a feature commonly found on satellites, as the detection target, and considering the numerous textures on solar panels that significantly impact edge extraction, the method obtains contours from the image through image processing. These contours are then processed and combined using methods such as minimum rectangle envelope. Detection conditions are set based on the solar panel's pose in Cartesian space and its position in the image, extracting contours corresponding to three solar panels. Finally, the parallelogram fitting algorithm proposed in this invention is used to solve for the four corner points of the contours. This on-orbit satellite pose measurement method has wide applicability and high fitting accuracy.
[0110] 2. This invention provides a novel parallelogram fitting algorithm. Cartesian space contains many objects with rectangular surfaces; theoretically, their shape in the image plane is a parallelogram. This algorithm uses the least squares method to find the best line for fitting all contour points. This initially identifies the point set used to fit two parallel lines, and further identifies the point set used to fit two other parallel lines. Based on this, points offset from the fitting line are removed through iteration, and the four sides of the fitted parallelogram are solved. This algorithm gradually approximates the best fitting line through iteration, and can eliminate interference from points with large errors. It is applicable to both closed contour points and discrete point sets.
[0111] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.
Claims
1. An on-orbit satellite attitude measurement method based on parallelogram fitting, characterized in that, Includes the following steps: Step A: Acquire images; Step B: Design an image processing algorithm to extract the contours; Step C: Combine contour points according to the set judgment conditions; Step D, target contour selection; Step E: Use a parallelogram fitting algorithm to fit parallelograms to the target contour points; Step F: Extract corner points and calculate the pose of the target satellite; Step E also includes: Step E1: Obtain the set of contour points to be fitted, and calculate the center point of the contour, i.e., the average value of the (u, v) coordinates of each point. Step E2: Use the least squares method to find the best fitted line ax + by + c = 0, which passes through the center point of the contour point set. Step E3: Determine the set of points on the two edge lines of the parallelogram that are approximately parallel to the fitted optimal line; Step E4: Use the least squares method to fit the best-fitting straight lines of the upper and lower contour point sets respectively. The two fitted lines must be parallel. After fitting the two lines with the two contour point sets respectively, solve the deflection angle of the lines. Calculate the weighted average of the line deflection angles based on the number of elements in the two contour point sets, and then determine the a, b, c1, and c2 of the two lines. Step E5: Determine the point set of the other two parallel lines of the fitted parallelogram; Step E6: Fit two more parallel lines based on the point set obtained in step E5. Step E7: Based on the initially determined point set of the four sides of the fitted parallelogram and the initially confirmed linear equations of the four sides, the optimal linear equations of the four sides of the parallelogram are gradually approximated through iteration. Step E5 also includes: Step E50: Find the points in the point set from step E4 that are less than the threshold distance from the two fitted lines and are on the edge of the point set, and use them as the starting search points for the other two sides of the parallelogram to be fitted. Step E51: Solve for the straight line passing through the two pairs of initial search points obtained in step E50. Use these two straight lines as the two initial sides of the parallelogram to be fitted. Search for points whose distance from the straight line is less than a threshold as the point set of the two sides of the fitted parallelogram. Fit the straight line formed by the two point sets, using the same method as step E4.
2. The on-orbit satellite attitude measurement method according to claim 1, characterized in that, Step B further includes: Step B1, image distortion removal; Step B2: Set the ROI search area based on the region of the target satellite in the image; Step B3: Convert the image into an HLS format suitable for machine processing; Step B4: Perform median filtering on the image to reduce noise interference; Step B5: Extract the target region based on the HLS values of each pixel in the image and convert the image into a binary image; Step B6: Use morphological closing operations to reduce interference in the image, fill gaps in the target region, and reduce extraction error; Step B7: Extract the contour.
3. The on-orbit satellite attitude measurement method according to claim 1, characterized in that, Step C further includes: Step C1: Fit each contour point set using the minimum envelope rectangle to obtain the center point, side length, and rotation angle around the center point of the fitted rectangle. Step C2: Starting from the first contour, determine the relationship between the distance between two contours and the side length of the contour fitting rectangle. If the difference is greater than the threshold, merge the contour point set into the previous contour point set and delete the next contour point set. Step C3, return to step C1, and repeat steps C2 and C3 until no more contours can be merged; Step C4: Filter the contour point sets based on the number of points in each contour point set, and delete contours with fewer contour points.
4. The on-orbit satellite attitude measurement method according to claim 1, characterized in that, In step D, the target is the outline corresponding to the solar panel, which specifically includes: Step D1: Select the contour with the largest number of contour points as the contour of the target solar panel. Step D2: Distinguish the outlines of the windsurfing boards according to their positional relationship on the image plane. Among the center point coordinates (u, v), the one with the smallest u is the leftmost windsurfing board, the one with the middle u is the middle windsurfing board, and the one with the largest u is the rightmost windsurfing board.
5. The on-orbit satellite attitude measurement method according to claim 1, characterized in that, Step E3 further includes: Step E30: Based on the positional relationship between the contour points and the straight line, divide the contour points into two parts: points above the straight line and points below the straight line. The points above the straight line include the points on the straight line. Step E31: Calculate the distance from each contour point to the line, and find the maximum distance from the point above the line and the point below the line to the line respectively; Step E32: For points above and below the line, assign them to different point sets based on the difference between the distance of each point to the line and the maximum distance obtained in step E31. Step E33: For points above and below, if the number of elements in the farthest point set is the largest, or the number of elements in the farthest point set is greater than the threshold, then the farthest point set is the set containing the contour edge points; otherwise, the point set with the largest number of elements in the remaining point set is the set containing the contour edge points.
6. The on-orbit satellite attitude measurement method according to claim 1, characterized in that, Step E7 also includes: Step E70: For the set of points used to fit the four sides of the parallelogram, determine the distance of each point from the fitted line. If the distance is less than the threshold, keep the point; otherwise, delete the point. Step E71: Fit the two pairs of parallel edge lines of the parallelogram using the point set processed in step E70, in the same way as step E4. Step E72: Determine whether the number of points deleted in the previous two iterations is less than the threshold. If yes, end the process; otherwise, return to step E70. The distance threshold used in step E70 will decrease as the number of iterations increases.
7. The on-orbit satellite attitude measurement method according to claim 1, characterized in that, Step F further includes: Step F1: Solve for the four corner points formed by the intersection of the four fitted lines; Step F2: Determine the satellite's pose in the camera coordinate system.
8. The on-orbit satellite attitude measurement method according to claim 7, characterized in that, In step F2, if the three-dimensional dimensions of the solar panel are known, the P12P algorithm can be used for a monocular camera. If the fitting error of a certain corner point is large, that corner point can be excluded, and the remaining corner points can be used for fitting. For a binocular or multi-view camera, the satellite pose can be calculated using the P12P algorithm for each monocular camera image and then the average value can be taken. If the three-dimensional dimensions of the solar panel are unknown, the position of each corner point can be calculated using a binocular or multi-view camera to obtain the satellite pose.