A target identification method, docking system and docking method in a space environment

Abstract

This invention discloses a target recognition method, docking system, and docking method in a space environment. By adjusting the output intensity of the sunlight simulator's light source and the position and angle of the simulated work equipment, different types of simulated target images are obtained, forming a target photo library. The simulated target images are processed into small target images and then placed into a small target image library. This small target image library is merged with the target photo library augmented by GAN data, and divided into training and testing libraries proportionally. An SA-YOLOX network is constructed. The SA-YOLOX network is trained using the training library to obtain an SA-YOLOX target detection model. The augmented target photo library is tested using the SA-YOLOX target detection model and the testing library to obtain test results. Based on the test results and model metrics, an evaluation result is output. This invention effectively overcomes the performance index problem of the YOLO series algorithms in solving the special situation of high-contrast image target recognition in complex space environments.

Description

Technical Field

[0001] This invention relates to a target recognition method, docking system, and docking method in a space environment, belonging to the field of computer vision target detection technology. Background Technology

[0002] The detection and docking method and system in a space environment is a method that uses target recognition technology based on the SA-YOLOX algorithm under space illumination to identify specific targets on a target object, thereby obtaining the target object's position and related information for tracking. This method and system has been widely applied in areas such as automatic docking and tracking of spacecraft, reconnaissance, surveillance and tracking of mobile robots, and automated inspection of industrial machine tools.

[0003] Traditional machine vision-related spacecraft using image processing typically employ DSPs as their core processors, resulting in poor accuracy and performance in target recognition. Deep learning-based target detection technology, currently one of the most popular techniques in computer vision, primarily utilizes a feature extraction and classification approach. Feature extraction is performed using CNNs, while classification is achieved using SVMs or similar neural networks. Early deep learning-based target detection techniques used sliding windows and exhaustive search for regions, leading to excessively high computational costs. Later, region-candidate-based target detection methods emerged, achieving better accuracy but exhibiting poor real-time performance. Subsequently, researchers proposed regression-based deep learning target detection methods, such as the YOLO series and SSD algorithms. However, current algorithms are no longer sufficient to meet the performance requirements in scenarios involving target recognition in high-contrast images within complex spatial environments. Summary of the Invention

[0004] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a target recognition method, docking system and docking method in a space environment. The system of this invention adopts a target recognition method based on the SA-YOLOX algorithm in a space illumination environment, which effectively overcomes the index problem of YOLO series algorithms in solving the special situation of high contrast image target recognition in complex space environments.

[0005] The technical solution of this invention is:

[0006] A target identification method in a space environment includes:

[0007] By adjusting the output intensity of the sunlight simulator light source and the position and angle of the simulated work equipment, different types of simulated target images are obtained, forming a target photo library;

[0008] Determine whether the simulated target image is a small target; if so, process the simulated target image into a small target image library and then put it into the small target image library; if not, put the simulated target image into the target image library.

[0009] GAN data augmentation was performed on the target image library;

[0010] The small target image library is merged with the target photo library after GAN data augmentation, and divided into training library and test library according to the ratio.

[0011] Construct an SA-YOLOX network;

[0012] The SA-YOLOX network is trained using the aforementioned training library to obtain the SA-YOLOX object detection model;

[0013] The SA-YOLOX target detection model and the test library were used to test the expanded target image library, and the test results were obtained.

[0014] Based on the test results and model metrics, output the performance evaluation results.

[0015] In the above identification method, the step of performing GAN data augmentation on the target image library specifically involves:

[0016] Generative adversarial networks (GANs) are generated, including generators and discriminators.

[0017] The generator produces synthetic data from a given random noise vector;

[0018] The discriminator evaluates the synthesized data and outputs a probability value;

[0019] When the probability value output by the discriminator is A, it indicates that the discriminator cannot distinguish the output data of the generator, and the output data of the generator is used to expand the target photo library; otherwise, the generator and the discriminator are trained until the probability value output by the discriminator is A; A is a positive number less than 1.

[0020] In the above identification method, the objective function of the GAN generative adversarial network is:

[0021]

[0022] Where D(x) represents the probability of the real sample, G(z) represents the probability of the synthetic sample, and P data (x) represents the true sample distribution, p z (z) represents the distribution of the synthetic data.

[0023] In the above identification method, the SA-YOLOX network includes a backbone network, a neck section, and a head section, wherein:

[0024] The backbone network uses the CSPDarknet structure to slice and convolve the input image to form a convolutional image, which is then output to the Neck part.

[0025] In the Neck section, the path aggregation network PANet is used to upsample and downsample the convolutional image, and the extracted features are fused again to obtain image features, which are then output to the Head section.

[0026] The Head section employs a decoupled head structure to perform prediction, classification, and regression operations on the image features, resulting in the SA-YOLOX loss function.

[0027] In the above recognition method, the Head part includes a convolutional Conv module (1), a classification branch (2), a convolutional module (3), a regression branch (4), the center point coordinates of the predicted target box (5), the prediction confidence (6), and a spatial balance loss corresponding branch (7). After the data enters the convolutional Conv module (1), it is divided into a classification branch (2) and a regression branch (4). The convolutional result output by the classification branch (2) is processed by the convolutional module (3) to output the category loss Cls. The convolutional result output by the regression branch (4) is processed by the center point coordinates of the predicted target box (5) to output the bounding box loss Reg. The convolutional result output by the regression branch (4) is processed by the prediction confidence (6) to output the intersection-union ratio (IoU) of the center points of the ground truth box and the candidate box. The convolutional result output by the regression branch (4) is processed by the spatial balance loss corresponding branch (7) to output the spatial balance loss Space. The SA-YOLOX loss function is obtained from the category loss Cls, the bounding box loss Reg, the intersection-union ratio (IoU), and the spatial balance loss Space.

[0028] In the above identification method, the SA-YOLOX loss function is specifically as follows:

[0029] c = L cls +λL reg +μL space

[0030]

[0031] L reg =-log(IoU(B gt B pred ))

[0032]

[0033] Where c is the total loss function of SA-YOLOX, L cls For class loss, L reg For bounding box loss, L space For spatial equilibrium loss, (B gt B pred ) represents the center point of the ground truth bounding box and the candidate bounding box, ρ is the Euclidean distance; c is the slant distance of the smallest rectangle covering the ground truth bounding box and the candidate bounding box; IoU(B gt B pred ) represents the intersection-union ratio (IoU) of the center points of the ground truth bounding box and the candidate bounding box; L cls Binary cross-entropy is used to calculate the difference between the class predicted by the model and the true class, (t1,...,t). i ,...,t n (p1,...,p) represents the one-hot vector of the ground truth bounding box in each square of the image. i ,...,p n ) represents the score vector for category prediction.

[0034] In the above identification method, the model metrics, including precision, recall, and intersection-over-union (IoU), are calculated using the following formulas:

[0035]

[0036]

[0037]

[0038] Where Precision is the accuracy, TP is the number of correctly detected positive examples, FP is the number of negative examples incorrectly identified as positive examples; Recall is the recall, FN is the number of positive examples incorrectly identified as negative examples; IoU is the intersection-over-union ratio, ya is the predicted value, and zb is the true value.

[0039] In the above identification method, the small target photo processing of the simulated target image specifically includes:

[0040] A feature network is extracted from small target images to determine whether a target exists within the detection box.

[0041] The Region Proposal Network (RPN) is used to generate horizontal bounding boxes, classify foreground and background, and obtain foreground targets.

[0042] The foreground target is subjected to tilt nonmaximum suppression processing to obtain the small target image category.

[0043] This invention discloses a detection and docking system in a space environment, employing a target recognition method in a space environment. The system includes: a simulated operation device, a simulated operation device control system, a machine vision system, a sunlight simulator, a simulated workpiece, and a remote monitoring platform and evaluation system. The machine vision system is installed at the end of the simulated operation device to capture images of targets on the simulated workpiece. The simulated operation device control system is installed within the simulated operation device and uses the target recognition method in a space environment to identify targets based on the images captured by the machine vision system. The identification results are then sent to the remote monitoring platform and evaluation system. The remote monitoring platform and evaluation system analyzes the detection results and controls the movement of the simulated operation device based on these results. The sunlight simulator simulates sunlight of varying intensities illuminating the target.

[0044] This invention discloses a target detection and docking method in a space environment, employing a detection and docking system in a space environment, comprising:

[0045] Adjust the sunlight simulator to the specified light intensity;

[0046] The machine vision system installed on the space simulation work equipment is used to take on-site photos of the target on the simulated work equipment, and the photo stream is stored in the machine vision system.

[0047] The SA-YOLOX target detection model in the machine vision system is used to perform real-time recognition of the photo stream captured on site, and the recognition results of the photo stream are saved; the recognition results of the photo stream are sent to the remote monitoring platform and evaluation system;

[0048] The remote monitoring platform and evaluation system analyze the recognition results of the photo stream, obtain the position and angle of the simulated operating equipment through coordinate transformation, and control the movement of the simulated operating equipment.

[0049] The advantages of this invention over the prior art are as follows:

[0050] (1) This invention uses GAN to augment the dataset: real data is difficult to collect or create, but can be augmented by GAN, which can quickly and effectively construct a large amount of data of spatially complex environment types, making it convenient to quickly carry out model training and optimization.

[0051] (2) This invention provides a loss function scheme that is effective for spatial images, which can effectively deal with problems such as reflection, weak texture, and occlusion, and improve the robustness of the model.

[0052] (3) This invention provides a new network training strategy that can achieve rapid network convergence and accelerate training speed.

[0053] (4) This invention provides a novel automated evaluation scheme that can quickly evaluate test results and accelerate model validity verification.

[0054] (5) This invention detects small targets, ensuring the comprehensiveness and effectiveness of the detection results. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the system of the present invention;

[0056] Figure 2 This is a schematic diagram of the system target detection model training process of the present invention;

[0057] Figure 3 This is a system flowchart of the present invention;

[0058] Figure 4 This is the backbone network of the SA-YOLOX of this invention;

[0059] Figure 5 This is the Neck structure of the SA-YOLOX of this invention;

[0060] Figure 6 This invention relates to the SA-YOLOX decoupling head network structure;

[0061] Figure 7 This invention relates to the GAN (Generative Adversarial Network).

[0062] Figure 8 This is the small target image library processing flow of the present invention. Detailed Implementation

[0063] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0064] like Figure 2 As shown, this embodiment provides a target identification method in a space environment, including:

[0065] Step 1: By adjusting the output intensity of the sunlight simulator light source and the position and angle of the simulated work equipment, different types of simulated target images are obtained to form a target photo library;

[0066] Step 2: Determine whether the simulated target image is a small target; if so, process the simulated target image into a small target image library and then put it into the small target image library; if not, put the simulated target image into the target image library.

[0067] Step 3: Perform GAN data augmentation on the target image library;

[0068] Step 4: Merge the small target image library with the target photo library after GAN data augmentation, and divide them into training and testing libraries according to the ratio;

[0069] Step 5: Construct the SA-YOLOX network;

[0070] Step 6: Train the SA-YOLOX network using the training library to obtain the SA-YOLOX object detection model;

[0071] Step 7: Using the SA-YOLOX target detection model and test library, test the amplified target image library and obtain the test results;

[0072] Step 8: Output the effect evaluation results based on the test results and model indicators.

[0073] Preferably, in step 3, the target image library is augmented with GAN data, specifically as follows:

[0074] Generative adversarial networks (GANs) are generated, including generators and discriminators.

[0075] The generator produces synthetic data from a given random noise vector;

[0076] The discriminator evaluates the synthesized data and outputs probability values;

[0077] When the probability value output by the discriminator is A, it indicates that the discriminator cannot distinguish the output data of the generator, and the output data of the generator is used to expand the target photo library; otherwise, the generator and the discriminator are trained until the probability value output by the discriminator is A; A is a positive number less than 1.

[0078] Preferably, the objective function of the GAN (Generative Adversarial Network) is:

[0079]

[0080] Where D(x) represents the probability of the real sample, G(z) represents the probability of the synthetic sample, and P data (x) represents the true sample distribution, p z (z) represents the distribution of the synthetic data.

[0081] Preferably, the SA-YOLOX network includes a backbone network, a neck section, and a head section, wherein:

[0082] The backbone network uses the CSPDarknet structure to slice and convolve the input image to form a convolutional image, which is then output to the Neck part.

[0083] In the Neck section, the path aggregation network PANet is used to upsample and downsample the convolutional image, and the extracted features are fused again to obtain image features, which are then output to the Head section.

[0084] The Head part employs a decoupled head structure to perform prediction, classification, and regression operations on image features, resulting in the SA-YOLOX loss function.

[0085] Preferably, the Head part includes a convolutional Conv module (1), a classification branch (2), a convolutional module (3), a regression branch (4), the center point coordinates of the predicted target box (5), the prediction confidence (6), and a branch corresponding to the spatial balance loss (7). After the data enters the convolutional Conv module (1), it is divided into a classification branch (2) and a regression branch (4). The convolutional result output by the classification branch (2) is processed by the convolutional module (3) to output the category loss Cls. The convolutional result output by the regression branch (4) is processed by the center point coordinates of the predicted target box (5) to output the bounding box loss Reg. The convolutional result output by the regression branch (4) is processed by the prediction confidence (6) to output the intersection-union ratio (IoU) of the center points of the ground truth box and the candidate box. The convolutional result output by the regression branch (4) is processed by the branch corresponding to the spatial balance loss (7) to output the spatial balance loss Space. The SA-YOLOX loss function is obtained from the category loss Cls, the bounding box loss Reg, the intersection-union ratio (IoU), and the spatial balance loss Space.

[0086] Preferably, the SA-YOLOX loss function is as follows:

[0087] c = L cls +λL reg +μL space

[0088]

[0089] L reg =-log(IoU(B gt B pred ))

[0090]

[0091] Where c is the total loss function of SA-YOLOX, L cls For class loss, L reg For bounding box loss, L space For spatial equilibrium loss, (B gt B pred ) represents the center point of the ground truth bounding box and the candidate bounding box, ρ is the Euclidean distance; c is the slant distance of the smallest rectangle covering the ground truth bounding box and the candidate bounding box; IoU(B gt B pred ) represents the intersection-union ratio (IoU) of the center points of the ground truth bounding box and the candidate bounding box; L cls Binary cross-entropy is used to calculate the difference between the class predicted by the model and the true class, (t1,...,t). i ,...,tn (p1,...,p) represents the one-hot vector of the ground truth bounding box in each square of the image. i ,...,p n ) represents the score vector for category prediction.

[0092] Preferably, in step 8, the model metrics, including precision, recall, and intersection-over-union (IoU), are calculated using the following formula:

[0093]

[0094]

[0095]

[0096] Where Precision is the accuracy, TP is the number of correctly detected positive examples, FP is the number of negative examples incorrectly identified as positive examples; Recall is the recall, FN is the number of positive examples incorrectly identified as negative examples; IoU is the intersection-over-union ratio, ya is the predicted value, and zb is the true value.

[0097] Preferably, in step 2, the simulated target image undergoes small target photo processing, specifically as follows:

[0098] A feature network is extracted from small target images to determine whether a target exists within the detection box.

[0099] The Region Proposal Network (RPN) is used to generate horizontal bounding boxes, classify foreground and background, and obtain foreground targets.

[0100] By performing tilt nonmaximum suppression on the foreground target, the small target image category is obtained.

[0101] like Figure 1 As shown, this embodiment provides a detection and docking system in a space environment, employing a target recognition method in a space environment. The system includes: a simulated operation device, a simulated operation device control system, a machine vision system, a sunlight simulator, a simulated workpiece, and a remote monitoring platform and evaluation system. The machine vision system is installed at the end of the simulated operation device to capture images of the target on the simulated workpiece. The simulated operation device control system is installed within the simulated operation device and uses the target recognition method in a space environment to identify the target based on the images captured by the machine vision system. The identification results are then sent to the remote monitoring platform and evaluation system. The remote monitoring platform and evaluation system analyzes the detection results and controls the movement of the simulated operation device based on these results. The sunlight simulator simulates sunlight of different intensities illuminating the target.

[0102] This embodiment provides a target detection and docking method in a space environment, employing a detection and docking system in a space environment, including:

[0103] Adjust the sunlight simulator to the specified light intensity;

[0104] The machine vision system installed on the space simulation work equipment is used to take on-site photos of the target on the simulated work equipment, and the photo stream is stored in the machine vision system.

[0105] The SA-YOLOX target detection model in the machine vision system is used to perform real-time recognition of the photo stream captured on site, and the recognition results of the photo stream are saved; the recognition results of the photo stream are sent to the remote monitoring platform and evaluation system;

[0106] The remote monitoring platform and evaluation system analyze the recognition results of the photo stream, obtain the position and angle of the simulated operating equipment through coordinate transformation, and control the movement of the simulated operating equipment.

[0107] Example

[0108] This embodiment provides a target recognition system for spatial environments, employing a target recognition method based on the SA-YOLOX algorithm under spatial lighting conditions. This effectively overcomes the performance limitations of YOLO series algorithms in addressing the specific scenarios of high-contrast image target recognition in complex spatial environments. The system fully utilizes GANs for dataset augmentation, overcoming the difficulty of collecting or creating real-world data, and quickly and effectively constructing large batches of data from complex spatial environments, facilitating rapid model training and optimization. For high-contrast image recognition scenarios, the SA-YOLOX loss function effectively addresses issues such as reflection, weak texture, and occlusion, thereby improving the model's robustness.

[0109] like Figure 8 As shown, for cases where a large portion of images are high-resolution small target images, a proposed R2CNN (Rotational Region CNN) algorithm network processing layer method is used for images exceeding the standard input size of the target image. Three different length and width sizes are applied to the ROI pooling stage for the detection box size, and two additional sizes are designed for the detection of horizontal and vertical targets. The ROI features extracted from the small target images are fused and used as input for subsequent prediction branches.

[0110] 1) Extract the feature network from the small target image to determine whether there is a target within the detection box;

[0111] 2) Use RPN (Region Proposal Net) to generate axis-aligned boxes, and perform classification and simplification, i.e., coarse detection, classifying only the foreground and background;

[0112] 3) The core of the algorithm, namely the prediction of the inclined box, is obtained by performing inclined NMS non-maximum suppression. Features of different sizes are merged to predict specific target scores, horizontal boxes, and associated inclined minimum region boxes, i.e., fine detection, to distinguish the specific category corresponding to small target images.

[0113] Meanwhile, by designing software to automatically test the accuracy of model recognition, replacing manual methods, and combining innovative aspects of the algorithm, an automated evaluation scheme can be realized to quickly evaluate test results and accelerate the model's effectiveness verification process.

[0114] This embodiment provides a target identification method in a space environment, including:

[0115] Step 1: By adjusting the output intensity of the sunlight simulator light source and the position and angle of the simulated work equipment, different types of simulated target images are obtained to form a target photo library;

[0116] Step 2: Determine whether the simulated target image is a small target; if so, process the simulated target image into a small target image library and then put it into the small target image library; if not, put the simulated target image into the target image library.

[0117] Step 3: Perform GAN data augmentation on the target image library;

[0118] Step 4: Merge the small target image library with the target photo library after GAN data augmentation, and divide them into training and testing libraries according to the ratio;

[0119] Step 5: Construct the SA-YOLOX network;

[0120] Step 6: Train the SA-YOLOX network using the training library to obtain the SA-YOLOX object detection model;

[0121] Step 7: Using the SA-YOLOX target detection model and test library, test the amplified target image library and obtain the test results;

[0122] Step 8: Output the effect evaluation results based on the test results and model indicators.

[0123] Specifically, a Generative Adversarial Network (GAN) is used to augment the target image database. The GAN mainly consists of a generator and a discriminator. Figure 7As shown, specifically:

[0124] Generative Adversarial Networks (GANs) are unsupervised generative models that implicitly learn the underlying distribution. Within the GAN framework, the learning process is a minimax game between two networks, such as... Figure 7 As shown, one is a generator, responsible for generating synthetic data for a given random noise vector; the other is a discriminator, responsible for distinguishing between real data and the generator's synthetic data.

[0125] The generator learns and adjusts itself to deceive the discriminator, while the discriminator learns and adjusts itself passively to identify whether the generator's synthesized data is real or synthetic.

[0126] Ultimately, GANs (Generative Adversarial Networks) will reach an equilibrium where the generator cannot fool the discriminator, and the discriminator cannot distinguish between genuine and fake generator results. In other words, a Nash equilibrium is reached when the generator and discriminator work together to produce the most realistic simulation data possible, while the discriminator cannot recognize the synthetic data generated by the generator.

[0127] Once a new equilibrium is reached, the discriminator outputs a value representing the probability that a sample comes from a real sample: 1 represents that the sample comes from a real sample, 0 represents that the data comes from a sample synthesized by the generator, and 0.5 means that it cannot be distinguished.

[0128] As the network continues to train, the generator becomes more and more powerful, generating samples that are identical to real samples, making it impossible for the discriminator to distinguish them, and thus outputting a probability of 0.5.

[0129] The generator and discriminator structures are flexible and generally composed of a combination of neural networks, such as CNN, RNN, ORU, LSTM, etc. The objective function of a GAN (Generative Adversarial Network) is:

[0130]

[0131] Where D(x) represents the probability of the real sample, G(z) represents the probability of the synthetic sample, and P data (x) represents the true sample distribution, p z (z) represents the distribution of the synthetic data.

[0132] In the game between the generator and discriminator in a Generative Adversarial Network (GAN), the training objective is to minimize the overall loss of both the generator and the discriminator. Due to the instability of the generator and discriminator, the GAN is trained separately, that is, the generator is optimized once, and the discriminator is optimized multiple times.

[0133] This embodiment uses a Generative Adversarial Network (GAN) to augment the source target image database, obtaining a large number of generated samples for the source target image database, thereby greatly increasing the number of target recognition image databases.

[0134] The SA-YOLOX object detection model includes a backbone network, a neck part, and a head part, among which:

[0135] (1) Backbone network, such as Figure 4 As shown, it uses the CSPDarknet architecture and is divided into three parts: the Focus module, the CSP module, and the SPP module, as follows:

[0136] a. The Focus module performs a slice operation on the input image before it enters the Backbone network, such as... Figure 4 As shown in (I): Spatial information is concentrated into channel information. Although this increases the computational load compared to ordinary downsampling, it preserves more complete image downsampling information for subsequent target image feature extraction.

[0137] The b.CSP module splits the original input feature map into two branches, performing CBS convolution operations on each branch to halve the number of channels. Then, one branch performs a Bottleneck*N operation, stacking the residual block Res units. Finally, the two branches are concatted. This reduces computational cost while maintaining accuracy, alleviating the previous problem of requiring extensive inference computation.

[0138] c. The SPP module is a max-pooling module with a kernel size of k×k (k=1,5,9,13) that connects the outputs of convolutions. Figure 4 As shown in (III), using multiple pooling windows (Maxpool) effectively increases the receptive field of the backbone network features, improves the scale invariance of the image, and makes the network more likely to converge.

[0139] (2) Neck, such as Figure 5 As shown, Neck uses PANet (Path Aggregation Network) for upsampling and downsampling, and then fuses the extracted features again;

[0140] Head, such as Figure 6As shown, the Head classification and regression network adopts a decoupled head structure to implement prediction classification and regression operations separately. After the data enters the SA-YOLOX decoupled head, it first enters the convolutional Conv module (box 1). After output, it is divided into two branches: one is the classification branch Cls. (boxes 2 and 3), and the other is the regression branch (box 4). The regression branch can be divided into three branches: one is the center point coordinate of the predicted target box shown in box 5; the second is the prediction confidence shown in box 6; and the last is the branch corresponding to the spatial balance loss for spatial high contrast target image detection added in this patent.

[0141] This embodiment provides a space environment detection and docking system, including:

[0142] 1. Simulation Equipment: Simulation equipment is a type of mechanical equipment commonly used in space missions. Its direction and angle can be remotely controlled via a remote monitoring platform and evaluation system. Based on the mission being performed in space, it is mainly divided into two types: intravehicular simulation equipment and extravehicular simulation equipment.

[0143] In-cabin simulation equipment is used in manned spaceflight to assist or replace astronauts in tasks such as assembly, experimentation, daily care, and technical verification.

[0144] Extravehicular activity (EVA) simulation equipment is used for on-orbit assembly and maintenance of spacecraft, supporting extravehicular activities of astronauts, completing on-orbit missions such as scientific and technological experiments, and installing space vehicles such as space shuttles, satellites, and space stations.

[0145] The simulation equipment in this embodiment mainly involves visual perception, task planning, and end effectors.

[0146] The robot simulation operation equipment system uses a mechanical vision system installed at the end of the simulation operation equipment to photograph the target. Under the monitoring of the remote operation and evaluation platform, the simulated target on the simulation operation equipment within the field of view of the mechanical vision system is automatically identified.

[0147] Furthermore, it calculates the high-precision relative pose correspondence between the simulated work equipment and the simulated work equipment in real time and continuously, which is used for servo control of the simulated work equipment in machine vision on the remote monitoring platform and evaluation system or for real-time judgment of astronaut operations.

[0148] 2. Machine Vision System: A machine vision system, installed at the end of a simulated work equipment, is a device that uses optical devices and non-contact sensors to capture images of simulated target patterns on the equipment. With the aid of a remote monitoring platform and evaluation system, relevant information is extracted and analyzed from the simulated target images to obtain the true coordinates of the corresponding spatial points on the simulated equipment in the world coordinate system and the projections of these points onto the image. Through transformations such as PnP, the pose of the machine vision system is obtained. This pose information is then output to the remote monitoring and evaluation system for further processing, enabling target detection and control of the simulated work equipment. Machine vision systems offer good real-time performance and high positioning accuracy, effectively increasing the flexibility and automation of simulated work equipment systems.

[0149] 3. Sunlight Simulator: The purpose of the sunlight simulator is to simulate sunlight, allowing the system to obtain a suitable solar light source unaffected by external lighting conditions. This enables the capture of simulated target images under complex spatial lighting environments, such as varying light intensity, angle, and scale. A sufficiently large image library is created based on these target images captured under different conditions. This library is then augmented using GAN data to obtain a sufficient number of target images for training and testing.

[0150] 4. Simulated Docked Equipment: Simulated docked equipment includes spacecraft such as space shuttles, satellites, and space stations. It refers to the object being detected and docked within the detection and docking methods and systems used in the space environment. Simulated docked equipment includes simulated targets and functional work areas. The functions of each module are described below:

[0151] (1) Simulated Target: The simulated target on the simulated workpiece is a specific marker used by the machine vision system within the simulated workpiece to track the workpiece. The machine vision system in the simulated workpiece identifies the simulated target and, through operations such as PnP transformation (i.e., by using the correspondence between the n pixels of the simulated target and actual spatial points), combined with known intrinsic parameters of the machine vision system, solves for the extrinsic parameters of the machine vision system (including rotation matrix and translation vector), thereby determining the pose of the simulated target within the machine vision system. After processing by the functional work area of ​​the simulated workpiece, the result is returned to the remote monitoring platform and evaluation system.

[0152] (2) Functional Operation Area: The functional operation area on the simulated operational equipment consists of computing modules and systems corresponding to the different types of simulated operational equipment, such as space shuttles, satellites, and space stations. The simulated operational equipment, in conjunction with specific space missions, completes tasks such as machine vision perception, space mission planning, end effector operation, space teleoperation, and ground experimental verification within the functional operation area of ​​the simulated operational equipment.

[0153] 5. Remote monitoring platform and evaluation system: such as Figure 3 As shown, the remote monitoring platform and evaluation system can remotely monitor the target identification process during detection and docking within the entire space environment and provide automated evaluation of the target identification effectiveness. The remote monitoring platform and evaluation system can be deployed with virtual reality or augmented reality systems for better real-time monitoring of the current status of all equipment in the aerospace system. Ground technicians can use the remote monitoring platform and evaluation system for virtual remote operation, interactive viewing and rapid evaluation of mission completion, and immersive real-time observation of the actual mission operation site, thus more effectively assisting astronauts in completing their aerospace missions. Data related to the simulated equipment obtained by the simulation equipment can also be returned to the remote monitoring platform and evaluation system in real time, helping ground personnel to observe and analyze the execution of aerospace missions in real time.

[0154] This embodiment provides a target detection and docking method in a space environment, including:

[0155] 1) Adjusting the output intensity of the sunlight simulator light source: To simulate complex lighting environments in space, a sunlight simulator is used in conjunction with the current ambient light source. Throughout the system process, the simulated target is illuminated from different angles by light sources of varying intensities output by the sunlight simulator, and a sufficient number of simulated target images are captured by a machine vision system.

[0156] 2) Adjust the position and angle of the simulation equipment: In conjunction with step 1), change the position and angle of the machine vision system, as well as the intensity and direction of the light, to change the type and number of simulated target images captured.

[0157] 3) The machine vision system takes a certain number of target photos to form a target photo library: By combining steps 1) and 2), a sufficient number of target photos under complex environments such as different lighting, different angles and different scales are obtained.

[0158] 4) GAN data augmentation of target photo library: Since it is difficult to collect or create real simulated target photo data, GAN can be used to augment the data in actual experiments. This can quickly and effectively build a large amount of data on complex spatial environments, which facilitates rapid model training and optimization.

[0159] 5) Test the expanded target image library with the SA-YOLOX model that meets the requirements: In order to better meet the needs of high-contrast image processing in spatial environment, the proposed SA-YOLOX model mainly includes two strategies: one is the SA-YOLOX loss function, and the second strategy is the image segmentation and deduplication method around small target detection.

[0160] a) SA-YOLOX Loss Function: The SA-YOLOX algorithm requires label assignment when calculating the loss function; this is a key concept of SA-YOLOX. The loss function calculation is used for label assignment. The bounding box (bbox) loss and class loss are calculated below:

[0161] √Bounding box loss:

[0162] L reg =-log(IoU(B gt B pred ))

[0163] √Category loss:

[0164]

[0165] The original loss function formula for SA-YOLOX was:

[0166]

[0167] Where λ is the balance coefficient.

[0168] In this embodiment, since it involves high-contrast spatial image optimization and recognition in complex spatial environments, a spatial balance loss needs to be introduced:

[0169]

[0170] The center points of the ground truth bounding box and the candidate bounding box are represented by (B). gt B pred The Euclidean distance is ρ, where c is the slant distance of the smallest rectangle covering the true bounding box and the candidate bounding box.

[0171] Therefore, the loss function of SA-YOLOX, which is more suitable for spatial high-contrast image recognition scenarios, is:

[0172] c = L cls +λL reg +μL space

[0173] Where λ and μ are balance coefficients.

[0174] c is the SA-YOLOX loss function used for label assignment, L clsFor class loss, L reg For the loss of the bounding box (bbox), L space To address the spatial balance loss in high-contrast target image recognition in a spatial environment.

[0175] The SA-YOLOX network constructs convolutional layers using the SA-YOLOX decoupling head; forward inference is performed between network layers; Cls, Reg, and IoU are decoupled and merged using the SA-YOLOX decoupling head, and the coordinates of each grid on the feature map are calculated; then, feature map network coordinate points are created, and the bounding boxes (bboxes) of the neural network's forward inference are projected onto the size of the input image; various losses during training are calculated. This is used to evaluate the performance of image recognition.

[0176] b) Image segmentation and deduplication method for small target detection: This is an R2CNN algorithm network processing layer method for small target images, used as a supplementary detection method for a portion of small target images outside of the SA-YOLOX image detection algorithm training. The R2CNN algorithm applies three different length and width sizes to the ROI pooling stage, and designs two additional sizes for horizontal and vertical target detection. The ROI features extracted from the small target images are fused and used as input for subsequent prediction branches.

[0177] 6) Develop software to automatically record and output the accuracy, recall, and intersection-over-union (IoU) of the current model test to the terminal: By analyzing the confusion matrix output by the model and the terminal results, a program can be developed to automatically calculate accuracy, recall, and IoU, and display the output on the terminal to provide feedback to the system administrator. Specifically:

[0178]

[0179]

[0180] 7) Build an automated evaluation system for SA-YOLOX under complex lighting conditions in space: Develop automated evaluation software that records and outputs the difference between the accuracy, recall, and intersection-union ratio (IoU) of the current model test and the corresponding index requirements of the space project, and quickly provides evaluation and verification results of the solution test effect.

[0181] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above description. Therefore, the scope of protection of the present invention should be defined by the appended claims.

[0182] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A target identification method in a space environment, characterized in that, include: By adjusting the output intensity of the sunlight simulator light source and the position and angle of the simulated work equipment, different types of simulated target images are obtained, forming a target photo library; Determine whether the simulated target image is a small target; if so, process the simulated target image into a small target image library and then put it into the small target image library; if not, put the simulated target image into the target image library. GAN data augmentation was performed on the target image library; The small target image library is merged with the target photo library after GAN data augmentation, and divided into training library and test library according to the ratio. Construct an SA-YOLOX network; The SA-YOLOX network is trained using the aforementioned training library to obtain the SA-YOLOX object detection model; The SA-YOLOX target detection model and the test library were used to test the expanded target image library, and the test results were obtained. Based on the test results and model indicators, output the effect evaluation results; The SA-YOLOX network includes a backbone network, a neck section, and a head section, wherein: The backbone network uses the CSPDarknet structure to slice and convolve the input image to form a convolutional image, which is then output to the Neck part. In the Neck section, the path aggregation network PANet is used to upsample and downsample the convolutional image, and the extracted features are fused again to obtain image features, which are then output to the Head section. The Head section employs a decoupled head structure to perform prediction, classification, and regression operations on the image features, resulting in the SA-YOLOX loss function. The Head section includes a Convolutional Conv module (1), a classification branch (2), a convolutional module (3), a regression branch (4), the center point coordinates of the predicted target box (5), the prediction confidence (6), and a branch corresponding to the spatial balance loss (7). After the data enters the Convolutional Conv module (1), it is divided into a classification branch (2) and a regression branch (4). The convolutional result output by the classification branch (2) is processed by the convolutional module (3) to output the category loss Cls. The convolutional result output by the regression branch (4) is processed by the center point coordinates of the predicted target box (5) to output the bounding box loss Reg. The convolutional result output by the regression branch (4) is processed by the prediction confidence (6) to output the intersection-union ratio (IoU) of the center points of the ground truth box and the candidate box. The convolutional result output by the regression branch (4) is processed by the branch corresponding to the spatial balance loss (7) to output the spatial balance loss Space. The SA-YOLOX loss function is obtained from the category loss Cls, the bounding box loss Reg, the intersection-union ratio (IoU), and the spatial balance loss Space. The SA-YOLOX loss function is as follows: Where c is the total loss function of SA-YOLOX. For category loss, For bounding box loss, For spatial equilibrium loss, (B gt B pred ) represents the center point of the ground truth bounding box and the candidate bounding box, ρ is the Euclidean distance; c is the slant distance of the smallest rectangle covering the ground truth bounding box and the candidate bounding box; IoU(B gt B pred ) represents the intersection-union ratio of the center points of the ground truth bounding box and the candidate bounding box; Binary cross-entropy is used to calculate the difference between the class predicted by the model and the true class, (t1,...,t...). i ,...,t n (p1,...,p) represents the one-hot vector of the ground truth bounding box in each square of the image. i ,...,p n ) represents the score vector for category prediction.

2. The target identification method in a space environment according to claim 1, characterized in that: The target image library is augmented using GAN data, specifically as follows: Generative adversarial networks (GANs) are generated, including generators and discriminators. The generator produces synthetic data from a given random noise vector; The discriminator evaluates the synthesized data and outputs a probability value; When the probability value output by the discriminator is A, it indicates that the discriminator cannot distinguish the output data of the generator, and the output data of the generator is used to expand the target photo library; otherwise, the generator and the discriminator are trained until the probability value output by the discriminator is A; A is a positive number less than 1.

3. The target identification method in a space environment according to claim 2, characterized in that: The objective function of the GAN (Generative Adversarial Network) is: in, Represents the probability of a real sample. Represents a synthetic sample. Represents the true sample distribution. This represents the distribution of synthetic data.

4. The target identification method in a space environment according to claim 1, characterized in that: The model metrics, including precision, recall, and intersection-over-union (IoU), are calculated using the following formulas: Where Precision is the accuracy, TP is the number of correctly detected positive examples, FP is the number of negative examples incorrectly identified as positive examples; Recall is the recall, FN is the number of positive examples incorrectly identified as negative examples; IoU is the intersection-over-union ratio, ya is the predicted value, and zb is the true value.

5. The target identification method in a space environment according to claim 1, characterized in that: The simulated target image is processed into a small target image, specifically as follows: A feature network is extracted from small target images to determine whether a target exists within the detection box. The Region Proposal Network (RPN) is used to generate horizontal bounding boxes, classify foreground and background, and obtain foreground targets. The foreground target is subjected to tilt nonmaximum suppression processing to obtain the small target image category.

6. A detection and docking system for a space environment, characterized in that, The target recognition method in a space environment as described in claim 1 includes: a simulated operation equipment, a simulated operation equipment control system, a machine vision system, a sunlight simulator, a simulated workpiece, and a remote monitoring platform and evaluation system; wherein, the machine vision system is installed at the end of the simulated operation equipment to capture images of the target on the simulated workpiece; the simulated operation equipment control system is installed in the simulated operation equipment and uses the target recognition method in a space environment to recognize the target based on the images captured by the machine vision system, and sends the recognition results to the remote monitoring platform and evaluation system; the remote monitoring platform and evaluation system analyzes the detection results and controls the movement of the simulated operation equipment based on the detection results; the sunlight simulator simulates sunlight of different intensities illuminating the target.

7. A target detection and docking method in a space environment, characterized in that, The space environment detection and docking system as described in claim 6 includes: Adjust the sunlight simulator to the specified light intensity; The machine vision system installed on the space simulation work equipment is used to take on-site photos of the target on the simulated work equipment, and the photo stream is stored in the machine vision system. The SA-YOLOX target detection model in the machine vision system is used to perform real-time recognition of the photo stream captured on site, and the recognition results of the photo stream are saved; the recognition results of the photo stream are sent to the remote monitoring platform and evaluation system; The remote monitoring platform and evaluation system analyze the recognition results of the photo stream, obtain the position and angle of the simulated operating equipment through coordinate transformation, and control the movement of the simulated operating equipment.

Images