A method for satellite docking quick finding

Abstract

The application discloses a method for satellite docking quick search, comprising the following steps: (1) quickly rotating a holder to search for a target satellite, so that the target satellite is in a field of view range; (2) detecting the target satellite by adopting a target detection algorithm based on regression; and (3) tracking a specific target on each frame of image by adopting a target tracking algorithm based on deep learning and outputting position information. The relative distance between the satellites is calculated according to the position information of the target satellite on the image. The application realizes the quick search for the target satellite by the target detection and tracking algorithm of binocular vision and the control of the holder. The method reduces the energy consumption by the holder mode and can quickly search for the target satellite. The position relationship between the two docking satellites is established through the coordinate system conversion, so that the position information of the target is determined.

Description

Technical Field

[0001] This invention relates to a method for target finding, and more particularly to a method for rapid target finding for satellite docking. Background Technology

[0002] In recent years, with the rapid development of computer technology, real-time target detection and tracking technology has become a research hotspot. Target detection, also known as target extraction, combines target segmentation and recognition. Accurate target detection has a significant impact on subsequent tracking systems. Target tracking is based on the initial state of the target and uses target tracking algorithms to track the target's trajectory in space. Compared to other technologies, visual tracking has the advantage of better recognition rate for the types of tracked objects, and visual sensors are less expensive. Therefore, visual target detection and tracking algorithms are used to detect and track target objects, and a gimbal is controlled to rotate the camera.

[0003] Since the target satellite's location needs to be determined before docking, a binocular camera is placed on a gimbal to detect and track the target. By controlling the gimbal, the target is kept within the field of view. At the same time, the binocular camera can acquire the target's depth information, which helps with subsequent processing. Summary of the Invention

[0004] The purpose of this invention is to provide a method for quickly locating target satellites before satellite docking.

[0005] To achieve the above objectives, this invention provides a method for rapidly locating target satellites by constructing a gimbal-mounted binocular camera on a satellite and employing target detection and tracking algorithms, comprising:

[0006] A method for rapid satellite docking location includes the following steps:

[0007] 1) Install a gimbal binocular camera on the reference satellite, quickly rotate the gimbal binocular camera to acquire images, detect the target satellite in the images, and input the target bounding box of the first frame image when the target satellite is detected.

[0008] 2) Generate candidate boxes in the second frame image, extract features from the objects in the candidate boxes, obtain confidence scores, and determine the candidate box with the highest confidence score as the target candidate box;

[0009] 3) Obtain the target bounding box of each frame image through steps 1) and 2), output the position information of the target satellite in the image, and keep the target satellite in the center of the field of view by rotating the binocular camera of the gimbal.

[0010] 4) Using the left eye of the binocular camera on the gimbal as a reference, calculate the distance from the target satellite to the reference satellite using the binocular camera parallax formula.

[0011] Preferably, the implementation process of step 1) is as follows:

[0012] Step 1.1) Obtain multiple images containing the target satellite and divide them into training set and validation set. Input the training set into the target detection model to train the target detection model and verify the performance of the trained target detection model through the validation set.

[0013] Step 1.2) Quickly rotate the gimbal's binocular camera and input the images acquired by the gimbal's binocular camera into the trained target detection model. When a target satellite is detected, input the target bounding box of the first frame image.

[0014] Preferably, the implementation process of step 2) is as follows: a candidate box is generated in the second frame image, the candidate region is determined by sliding window sampling, and appearance modeling is performed. Then, the confidence score of the object in the candidate box is calculated based on the appearance modeling, and the highest confidence score is determined as the target candidate box.

[0015] Preferably, the implementation process of step 3) is as follows: the target bounding box of each frame image is obtained through steps 1) and 2), and the position information of the target satellite in the image is output with the center pixel of the target bounding box as the position of the target satellite, and the gimbal binocular camera is controlled to rotate so that the target satellite is at the center of the field of view.

[0016] Preferably, step 4) is implemented as follows: taking the left eye camera of the binocular camera as the reference, the target satellite coordinates in the world coordinate system are transformed to the camera coordinate system, then to the image coordinate system, and finally to the pixel coordinate system. The binocular visual parallax formula is then used. Calculate the distance Z from the target satellite to the reference satellite; where T is the parallax of the center point of the target frame matched on the images of the left and right eyes of the gimbal's binocular camera, f is the focal length of the left and right eyes, and d is the distance between the optical centers of the left and right eyes.

[0017] Preferably, in step 4), before performing coordinate transformation, the intrinsic and extrinsic parameters of the gimbal binocular camera are determined using a rapid defocus calibration method. This specifically includes the following steps:

[0018] Step A) Obtain the precise sub-pixel coordinates of feature points in the camera's out-of-focus state by using a phase-shift encoded circular pattern;

[0019] Step B) Calculate the initial parameters of the monocular camera based on the precise sub-pixel coordinates of the feature points obtained in step (A), construct the monocular objective function with the minimum reprojection error of the monocular camera based on the initial parameters, and calculate the internal parameters of the monocular camera based on the objective function.

[0020] Step C) After obtaining relatively accurate internal parameters for each camera, the reprojection error function of the stereo camera is optimized to obtain accurate internal parameters and external parameters of the stereo camera.

[0021] Preferably, the implementation process of step A) is as follows:

[0022] Obtain the phase shift encoded circular pattern and acquire the corresponding image. The phase shift value of the phase shift encoded circular pattern is 2 / 3π, and the light intensity distribution function is:

[0023]

[0024] In the formula, I1(x,y) is the pixel grayscale value of the phase-coded circular pattern, I'(x,y) is the average background grayscale of the corresponding image, with a value of 0.5; I'(x,y) is the modulated grayscale of the corresponding image, with a value of 0.5; Φ(x,y) is the principal phase value, defined as:

[0025]

[0026] In the formula, T is the period of the phase-shift coding circular pattern, and r(x,y) is the Euclidean distance from a point (x,y) on the phase-shift coding circular pattern to the center (x0,y0) of the phase-shift coding circular pattern, expressed as:

[0027]

[0028] The phase principal values ​​of the acquired images are calculated, and the phase principal values ​​of the pixels in the image are:

[0029]

[0030] In the formula, I1, I2, and I3 are the gray values ​​of the corresponding pixels of the three phase-shift coded circular patterns. The calculated principal phase values ​​are used to extract the contour and fit an ellipse to obtain the precise sub-pixel coordinates of the feature points.

[0031] Preferably, step C) is implemented as follows: the reprojection error function of the binocular camera is:

[0032]

[0033] Where m i For each image, represents a 2D point observed in the image; g is the projection equation; A is the intrinsic parameter matrix, whose initial values ​​use parameters provided by the camera hardware manufacturer; K is the distortion parameter; R and t are the extrinsic parameters between each camera and the target, whose initial values ​​are obtained through an n-point perspective algorithm; M i represents the three-dimensional spatial coordinates of the feature point; err represents the reprojection error.

[0034] Beneficial effects:

[0035] This invention achieves rapid target satellite acquisition through a binocular vision target detection and tracking algorithm and gimbal control. This method, using a gimbal, reduces energy consumption while enabling rapid target satellite location. By transforming the coordinate system, the positional relationship between two docking satellites can be established, thereby determining the target's location information. Attached Figure Description

[0036] Figure 1 This is a flowchart of a method for rapid satellite docking search according to the present invention. Detailed Implementation

[0037] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] This invention provides a method for rapid satellite docking search, comprising: S1: rapidly rotating a gimbal to detect a target satellite in a video stream; when a target satellite is detected, inputting the target bounding box of the first frame. S2: generating candidate bounding boxes in the second frame, extracting features from objects within the candidate bounding boxes, obtaining confidence scores, and determining the candidate bounding box with the highest score as the target bounding box. S3: obtaining the target bounding box for each frame through S1 and S2, and outputting the target's position information in the image; controlling the gimbal to ensure the target satellite remains centered in the field of view. S4: calculating the target's distance using the parallax principle of binocular cameras, with the left eye camera as a reference.

[0039] The specific process described above for rapid satellite docking can be set as follows:

[0040] Firstly, object detection algorithms are categorized into traditional object detection algorithms, candidate region-based object detection algorithms, regression-based object detection algorithms, and reinforcement learning-based object detection algorithms. A comprehensive comparison of the advantages and disadvantages of each algorithm is shown in Table 1 below:

[0041]

[0042]

[0043] Table 1 Summary of features of target detection algorithms

[0044] This method is used for rapid satellite docking, which requires fast target detection and high accuracy. Regression-based target detection algorithms can be selected, such as the YOLO (You Only Look Once) series of algorithms and SSD (Single Shot multibox Detector).

[0045] S1: In the YOLOv4 algorithm, the input image is first resized to 416×416. Then, the CSPDarknet-53 feature extraction network divides the input image into S×S cells.

[0046] Then, three different sizes of feature maps were obtained, namely 13×13, 26×26, and 52×52, which were used for large, medium, and small targets, respectively. At the same time, the regression prediction in each cell had three anchor boxes predicting three bounding boxes, and the one with the highest confidence was selected as the final detection result.

[0047] S2: Regarding tracking algorithms, discriminative tracking algorithms can be selected. Discriminative tracking algorithms can be further divided into sparse representation-based tracking algorithms, correlation filtering-based tracking algorithms, and deep learning-based tracking algorithms. Since deep learning-based tracking algorithms are insensitive to deformation, blurring, and partial occlusion and can achieve fast and accurate tracking, they are used to track target satellites.

[0048] S3: The target satellite is quickly detected and tracked through S1 and S2. The target satellite's frame is displayed in the video stream. The position of the target satellite is determined by the pixel at the center of the frame, and the position information of the target satellite is output. By controlling the gimbal to rotate the binocular camera, the target satellite is kept within the field of view, thereby achieving fast and accurate tracking of the target satellite. During this process, the satellite's attitude remains unaffected.

[0049] S4: Using the left-eye camera as a reference, the target satellite's position coordinates need to be transformed into pixel coordinates. Assume the target satellite's point in the world coordinate system corresponding to the center pixel of the target bounding box is (X... w ,Y W Z W Transform the world coordinate system into the camera coordinate system using rotation and translation matrices:

[0050]

[0051] Write it in secondary coordinate form:

[0052]

[0053] Points in the camera coordinate system are transformed to the image coordinate system using a similarity transformation:

[0054]

[0055] Finally, the conversion formula from image coordinates to pixel coordinates is used:

[0056]

[0057] Transform the center point coordinates of the target satellite in the world coordinate system to the pixel coordinate system of the target bounding box. Solve the coordinate transformation formulas simultaneously:

[0058]

[0059] Using the binocular vision parallax formula The depth Z can be calculated. Here, T is the disparity of the center points of the target bounding boxes matched on the left and right camera images, f is the focal length of the camera, and d is the distance between the optical centers of the left and right cameras, i.e., the distance between them.

[0060] To address the challenges and inefficiencies in calibrating cameras for large-scale vision systems, a rapid defocus calibration method based on phase shift encoding circles is proposed.

[0061] First, feature point extraction under defocus conditions is achieved through phase shift coding circles; then, a target optimization function for the monocular camera is constructed using the target spatial position information, and the camera's internal parameters are solved using the nonlinear least squares method; finally, a target optimization function for the binocular system is established to calculate the camera system's external parameters.

[0062] This paper uses a three-step phase-shifting method to generate a phase-shifted coded circular pattern with a phase shift value of 2 / 3π. The light intensity distribution function can be expressed as follows:

[0063]

[0064] In the formula, I'(x,y) is the average gray level of the image background, with a value of 0.5; I'(x,y) is the modulated gray level of the image, with a value of 0.5; Φ(x,y) is the principal phase value, described as...

[0065]

[0066] In the formula, T is the period of the phase-shift coding circle, i.e., the number of pixels that the phase principal value spans from 0 to 2π. Its value is determined by the screen resolution and the total number of rows and columns of the planned phase-shift coding circle. r(x,y) is the Euclidean distance from a point (x,y) on the phase-shift coding circle to its center (x0,y0), which can be expressed as:

[0067]

[0068] Image processing is performed on the phase circle pattern to obtain the precise location of its feature points. The phase principal value of the pixels in the image acquired by the camera has changed since the generation process; therefore, it is first necessary to calculate the phase principal value of the acquired image. The phase principal value of the pixels in the phase-shift encoded circle image is...

[0069]

[0070] In the formula, Φ(x,y) is the principal phase value to be determined at pixel (x,y), and I1, I2, and I3 are the gray values ​​of the corresponding pixels in the three phase shift patterns. The principal phase values ​​obtained by phase calculation range from 0 to 2π. Due to the properties of the arctangent function, the principal phase values ​​will be discontinuous at 2π. Therefore, the calculated principal phase values ​​can be directly used to extract the contour and fit an ellipse to obtain the precise sub-pixel coordinates of the feature points.

[0071] The system parameter calibration process involves solving for the camera's intrinsic and extrinsic parameter matrices. The camera's intrinsic parameter matrix is ​​only related to the camera's own internal parameters, while the extrinsic parameter matrix of a stereo system is generally a transformation from the right camera to the left camera. For each camera in the stereo system, its intrinsic parameters are calculated separately, and an objective function is established to minimize the reprojection error of the monocular camera. After obtaining relatively accurate intrinsic parameter results for each camera, the reprojection error function of the stereo camera is then established and optimized to obtain precise intrinsic and extrinsic parameters. The reprojection error function is constructed as follows:

[0072]

[0073] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for satellite docking quick finding, characterized in that, The method comprises the following steps: 1) installing a gimbal binocular camera on a reference satellite, rapidly rotating the gimbal binocular camera to obtain pictures, detecting a target satellite in the pictures, and inputting a target frame of a first picture when the target satellite is detected; 2) generating a candidate frame in a second picture, performing feature extraction on an object in the candidate frame, obtaining a confidence score, and determining the candidate frame with the highest confidence score as a target candidate frame; 3) obtaining the target frame of each picture through steps 1) and 2), outputting position information of the target satellite in the picture, and rotating the gimbal binocular camera to keep the target satellite at the center of the field of view; 4) taking the left camera of the gimbal binocular camera as a reference, and calculating the distance from the target satellite to the reference satellite by using a binocular camera parallax formula; The implementation process of step 4) is as follows: taking the left eye camera of the pan-tilt binocular camera as a reference, converting from the target satellite coordinates in the world coordinate system to the camera coordinate system, to the image coordinate system, and finally to the pixel coordinate system by coordinate transformation, and using the binocular vision parallax formula The distance Z of the target satellite to the reference satellite is calculated; wherein T is the parallax of the matching target frame center point on the left and right eye camera images of the pan-tilt binocular camera, f is the focal length of the left and right eye cameras, and d is the distance between the optical centers of the left and right eye cameras. Before the coordinate conversion in step 4), the internal and external parameters of the gimbal binocular camera are obtained by using a gimbal binocular camera defocus rapid calibration method, and the method comprises the following steps: Step A) obtaining accurate sub-pixel coordinates of feature points in a defocus state of the camera by using a phase shift encoding circle pattern; Step B) calculating initial parameters of the monocular camera according to the accurate sub-pixel coordinates of the feature points obtained in step (A), constructing a monocular target function with minimum monocular re-projection error according to the initial parameters, and calculating internal parameters of the monocular camera according to the target function; Step C) after obtaining relatively accurate internal parameters of each camera, the re-projection error function of the binocular camera is optimized to obtain accurate internal parameters and external parameters of the binocular camera; The implementation process of step C) is that the re-projection error function of the binocular camera is: wherein m i is the image two-dimensional point observed by the image; g is a projection equation; A is an intrinsic matrix, whose initial value uses the parameters provided by the camera hardware manufacturer; K is a distortion parameter; R, t are the external parameters between each camera and the target, whose initial value is obtained by an n-point perspective algorithm; M i is the spatial three-dimensional coordinates of the feature points; and err is the re-projection error.

2. A method for satellite docking quick finding as claimed in claim 1 wherein, The implementation process of step 1) is: Step 1.1) obtaining multiple pictures containing the target satellite, and dividing the pictures into a training set and a verification set, inputting the training set into a target detection model to obtain a trained target detection model, and verifying the performance of the trained target detection model by using the verification set; Step 1.2) rapidly rotating the gimbal binocular camera, inputting the pictures obtained by the gimbal binocular camera into the trained target detection model, and inputting a target frame of a first picture when the target satellite is detected.

3. A method for satellite docking quick finding as claimed in claim 2, wherein, The implementation process of step 2) is that a candidate frame is generated in a second picture, the candidate region is determined by using a sliding window sampling, appearance modeling is performed, the confidence score of the object in the candidate frame is calculated according to the appearance modeling, and the candidate frame with the highest confidence score is determined as a target candidate frame.

4. A method for satellite docking quick finding as claimed in claim 3 wherein, The implementation process of step 3) is that the target frame of each picture is obtained through steps 1) and 2), the position of the target satellite in the picture is taken as the center pixel point of the target frame, the position information of the target satellite is outputted, and the gimbal binocular camera is controlled to rotate to keep the target satellite at the center of the field of view.

5. The pan-tilt binocular camera defocus fast calibration method of claim 4, wherein, The implementation process of step A) is: A phase shift encoding circle pattern is obtained, and an image corresponding to the phase shift encoding circle pattern is collected, the phase shift value of the phase shift encoding circle pattern is 2 / 3pi, and the light intensity distribution function is: In the formula, I1(x, y) is the pixel gray value of the phase encoding circle pattern, I ′ (x, y) is the background average gray value of the corresponding image, and is 0.5; I ” (x, y) is the modulation gray value of the corresponding image, and is 0.5; and Φ(x, y) is the phase principal value, and is defined as: In the formula, T is the period of the phase shift encoding circle pattern, r(x, y) is the Euclidean distance from a point (x, y) on the phase shift encoding circle pattern to the center (x0, y0) of the phase shift encoding circle pattern, and is expressed as: The phase principal value of the collected image is calculated, and the phase principal value of the pixel point on the image is: In the formula, I1, I2 and I3 are respectively the gray values of the pixels corresponding to the three phase shift encoding circle patterns, the calculated phase principal value is subjected to contour extraction and ellipse fitting, so as to obtain the accurate sub-pixel coordinates of the feature points.

Images