A visual detection method for approaching non-cooperative dynamic targets in space

Abstract

The present invention discloses a method for visually detecting approaching non-cooperative dynamic targets in space, which relates to the technical field of optical visual sensor detection. The method includes: establishing a software and hardware framework for a non-cooperative dynamic target approaching visual detection system in space, using a deep residual contraction network to achieve target detection in environments with strong noise interference; establishing a coaxial three-dimensional visual measurement model to achieve rapid measurement of the target's three-dimensional point cloud and motion parameters; and perceiving and predicting the target's motion through reinforcement learning training of the target's motion state, thereby providing support for intelligent perception and maneuvering decisions of the flight vehicle. The present invention is suitable for high-precision, intelligent detection of dynamic targets in space during their approach, and is of great significance for ensuring detection and perception in areas such as close-range capture of spacecraft, near-Earth asteroid warning, and aerial refueling docking.

Description

Technical Field

[0001] The present invention relates to the technical field of optical vision sensor detection, and in particular to a method for visual detection of approaching non-cooperative dynamic targets in space. Background Art

[0002] Visual detection and intelligent perception technology for non-cooperative dynamic targets in space, with its non-contact, wide-range, long-distance, and high-precision capabilities, is widely used in areas such as aerial refueling and docking, space monitoring and early warning, and asteroid impact experiments. Unlike cooperative targets, detection and perception of non-cooperative targets requires capturing, identifying, and acquiring three-dimensional point cloud information to accurately detect their motion parameters and other information. This provides data support for target detection and perception, assisting the aircraft in making operational decisions. For example, during aerial refueling, the receiving aircraft must control its flight state to monitor the motion parameters of the drogue released by the tanker in real time. From far to near, the drogue must be accurately identified, three-dimensionally measured, and its motion state predicted, enabling timely adjustments to the receiving aircraft's flight posture for precise docking. Typical application scenarios for approach detection of non-cooperative dynamic targets in space include Starlink formation flying, asteroid defense, air combat, and unmanned aerial vehicles.

[0003] Space measurement environments, often affected by high maneuverability, strong vibration, variable light conditions, and complex backgrounds, can severely hinder target tracking, leading to reduced system accuracy and even failure. For example, both atmospheric and space environments present complex backgrounds, light interference, high target maneuverability, and the limited power consumption of the flight vehicle. These factors make it difficult to accurately detect targets, simultaneously acquire 2D texture and 3D information, and perform intelligent target perception, limiting early warning and effective avoidance.

[0004] The process of non-cooperative dynamic target detection and perception in space (dynamic target approach visual detection process) is a process from far to near and from coarse to fine. Figure 1 The horizontal axis distance in the figure is divided into several different stages, as shown in the following example: Figure 1 As shown, from far to near, the process includes searching, locking, tracking, guiding, measuring, guiding, docking, capturing, or landing. Initially, the requirements for measurement accuracy are relatively low, but the requirements for detection accuracy are relatively high. As the approach distance shortens, the corresponding measurement accuracy becomes increasingly higher. When the aircraft reaches a close range, high-precision detection and intelligent perception are required to provide data support for subsequent decision-making and actions.

[0005] Existing visual detection and perception technologies for approaching non-cooperative dynamic targets in space primarily encompass the following aspects: Image acquisition: High-resolution cameras, multi-camera systems, or other imaging devices are used to capture images or videos of non-cooperative dynamic targets in space. However, image quality can be unstable due to factors such as lighting, weather, and target pose. Target detection and recognition: Targets are detected and identified through processing and analysis of captured images or videos. However, detection and recognition accuracy is limited by factors such as image quality, target pose, and occlusion. Pose estimation: Target poses, such as angle and velocity, are estimated by analyzing target images or videos. However, pose estimation accuracy is easily affected by factors such as image quality, target pose, and sensor accuracy. Path planning: Paths are planned based on target position and pose information to achieve safe and accurate approaches. However, path planning accuracy and reliability are limited by factors such as target pose and environmental factors. Perception and obstacle avoidance: Sensors and other auxiliary devices are used to detect obstacles in the surrounding environment and avoid collisions using obstacle avoidance algorithms. Automatically sensing and avoiding obstacles is susceptible to factors such as sensor accuracy and obstacle avoidance algorithms, limiting its accuracy and reliability and resulting in a low level of intelligence. While it is possible to improve the accuracy and reliability of detection and perception by integrating data from multiple sensors, this increases system complexity and costs.

[0006] Intelligent perception of space targets includes key technologies such as pose measurement, 3D reconstruction, and part identification. Currently, non-cooperative rendezvous and docking primarily uses microwave or lidar for ranging, with visual cameras assisting with angle measurement. At close range, imaging lidar and visual cameras are used to resolve relative pose. Sensors used for space target detection primarily include microwave radar, lidar, Kinect, binocular cameras, time-of-flight cameras, multi-camera cameras, light field cameras, and event cameras, each with its own unique characteristics and applications in different scenarios.

[0007] With the rapid development of artificial intelligence technology, deep learning target detection algorithms have been extensively studied. However, factors such as spatial environment image noise, light interference, and complex backgrounds have limited the accuracy and stability of conventional detection. At the same time, in stereoscopic vision-based three-dimensional reconstruction, it is necessary to accurately match the target image features. However, based on binocular vision, the matching accuracy is easily affected by factors such as perspective projection and occlusion. At the same time, to ensure measurement accuracy, the baseline distance needs to be increased, and the installation space is limited for slender flying vehicles. Reinforcement learning is used for autonomous flight control and aerial confrontation of drones, and the optimal control instructions of drones are calculated in real time to complete corresponding tactical actions. Therefore, the relevant methods cannot be directly used for accurate target detection in complex spatial interference environments. They are small in size, low in power consumption, and do not require prior information about the target to achieve intelligent perception of dynamic targets.

[0008] Based on the above analysis, a single camera can achieve 3D reconstruction of space targets from far to near at different object distances or focal lengths. While this system offers advantages such as small size and low power consumption, it cannot achieve real-time, continuous 3D imaging and detection of dynamic targets in space. For non-cooperative dynamic targets in space, which present complex environments, low imaging quality, and large target scale variations, simple single-camera systems still cannot meet detection requirements. 3D reconstruction must be robust against image noise. Binocular stereo vision requires a certain baseline distance, making it impractical to directly install and operate within limited flight environments. For example, long-baseline binocular stereo sensors cannot be directly installed and operated within the confines of refueling aircraft, missiles, and small satellite platforms due to their limited space. Non-cooperative targets in space are unknown, and their motion characteristics are characterized by temporal uncertainty. Therefore, fully utilizing historical target motion data for modeling and analysis, and predicting subsequent motion, is crucial for intelligent perception and decision-making. Summary of the Invention

[0009] To address the aforementioned technical issues, the present invention provides a method for visually detecting approaching non-cooperative dynamic targets in space. This method features a compact structure, high intelligence, strong versatility, and simple system configuration. It is capable of detecting and intelligently sensing non-cooperative dynamic targets in space under the conditions of a confined space flight vehicle. The present invention utilizes two sets of single cameras to construct a coaxial stereoscopic vision measurement system, which ensures both measurement accuracy and a compact system structure, enabling the detection and sensing of approaching non-cooperative dynamic targets in space under complex environments.

[0010] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0011] A method for visually detecting a non-cooperative dynamic target approaching in space comprises the following steps:

[0012] Step 11. Build a visual detection device for approaching non-cooperative dynamic targets in space. The device is a coaxial three-dimensional visual imaging device that collects information about space targets. A beam splitter and a reflector are used to transmit homologous optical information of space targets to the main camera for imaging, and then to the auxiliary camera for imaging through a reflector. The main and auxiliary cameras are coaxial structures to ensure that the collected images are homologous and synchronized, thereby achieving visual detection of space targets. A coaxial binocular stereo vision measurement mode is used, supplemented by target pose solution and motion pattern prediction, to achieve three-dimensional detection and intelligent perception of approaching non-cooperative space targets.

[0013] Step 12: Detect targets in a strong noise interference environment.

[0014] Step 13: Construct a coaxial 3D vision measurement model to simulate the eagle-eye visual perception model. Use a spectroscope combined with a reflector to distribute the homologous light field. After the spatial light field information enters the bionic coaxial vision 3D imaging and measurement module, it is divided into two groups by the spectroscope. One group passes through the spectroscope directly into the main camera for perspective projection imaging, and the other group is reflected by the spectroscope to the reflector installed on the side, and then reaches the auxiliary camera through the reflector. The auxiliary camera and the main camera are equivalent to a coaxial vision module, which synchronously images the spatial homologous light field information.

[0015] Step 14: After densely matching the two sets of image feature points, dense reconstruction of the spatial target is achieved, the target depth information is obtained, and then converted into a three-dimensional point cloud in the camera coordinate system;

[0016] Step 15: After analyzing the three-dimensional point cloud of the space target, the target coordinate system is established, and the pose parameters of the space target relative to the camera coordinate system are obtained, providing data support for the calculation of target motion parameters and intelligent perception;

[0017] Step 16: Based on the current image and state information of the space target and its motion information, the future state information of the target is calculated, a flight vehicle decision framework is established, and the maneuver instructions of the flight vehicle are solved. The flight vehicle perceives the target's motion state from the environment and predicts the future motion state of the space target. Features are extracted from the sensor data and used to predict the future motion state of the space target. Reinforcement learning and long-short-term memory networks are used to process the flight parameters of the space target to generate the optimal strategy model of the flight vehicle.

[0018] Step 17: Based on the generated optimal strategy model, the flight vehicle implements maneuvering decisions for visual detection and intelligent perception of approaching non-cooperative targets in space.

[0019] The beneficial effects of the present invention compared with the prior art are:

[0020] The present invention designs an optical probe that combines a splitter / reflector and multiple groups of cameras to form a homologous coaxial visual light path. Combined with coaxial imaging and projection methods, it can realize visual detection of non-cooperative dynamic targets approaching in space under the conditions of using two cameras and a return light path; the present invention is based on the target detection method of improving the deep residual shrinkage network to a strong noise image, which realizes the fast and high-precision positioning of dynamic targets in a complex spatial environment with strong interference, and adopts embedded parallel acceleration technology to realize the parallelization of image processing algorithms to meet the real-time requirements of the measurement system; the measurement system involved in the present invention is compact and small, with high measurement accuracy, which can effectively solve the problem that the traditional measurement system has a complex structure and a large volume and is not suitable for installation on small carriers for dynamic targets in space. The difficulty of approach detection has important theoretical significance and practical value in promoting the introduction of visual detection in the closed-loop control of the space motion process, improving the navigation positioning accuracy and perception warning, etc.; the present invention segments the geometric features of the measured target and establishes the target local coordinate system by fitting, thereby obtaining the target's posture parameters. This method is applicable to the posture parameter measurement of rigid or flexible deformable objects and has certain versatility; the present invention realizes the perception and prediction of the motion state of dynamic targets in space by combining the intelligent perception model of target motion parameters, solves the problem of existing visual sensor measurement systems in small carrier installation and adaptation to limited environments, and is very suitable for navigation and real-time warning applications during the movement of small flying carriers in complex environments.

[0021] In summary, the present invention designs the software and hardware framework, structure, measurement mechanism and working mode of a miniaturized, high-precision space non-cooperative dynamic target approach visual detection and intelligent perception system, solves key scientific problems such as system software and hardware framework structure optimization design, space dynamic target approach visual detection, dynamic target three-dimensional reconstruction, and motion parameter timing analysis and intelligent perception, and constructs a measurement system verification prototype, providing an effective technical approach and detection means for space dynamic target approach visual detection and intelligent perception, which has important theoretical significance and practical value for promoting dynamic target approach detection, further improving application scenarios such as aerial autonomous refueling, drone landing / ship, asteroid detection and early warning, and high-precision navigation in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the visual detection process for approaching dynamic targets in space;

[0023] Figure 2 This is a flow chart of a method for visually detecting a non-cooperative dynamic target approach in space according to the present invention;

[0024] Figure 3 This is the principle diagram of coaxial visual three-dimensional imaging;

[0025] Figure 4Schematic diagram of determining the target coordinate system for measuring its pose based on the target's geometric features;

[0026] Figure 5 Schematic diagram of the target coordinate system. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0028] The basic idea of ​​the present invention is to design the hardware and software framework structure of the measurement system, and adopt a deep residual shrinkage network to realize target detection in a strong noise interference environment; establish a coaxial three-dimensional visual measurement model to realize the rapid measurement of the target's three-dimensional point cloud and motion parameters; perceive and predict the target's motion status through reinforcement learning training on the target's motion state, and provide support for the flight vehicle's intelligent perception and maneuver decision-making of the target; construct a prototype of a spatial dynamic target approach visual detection and intelligent perception system, and experimentally verify and evaluate its various performance indicators.

[0029] like Figure 1 , Figure 2 As shown, a method for visually detecting a non-cooperative dynamic target approaching in space according to the present invention comprises the following steps:

[0030] Step 11: Build a space non-cooperative dynamic target approach visual detection device. The space non-cooperative dynamic target approach visual detection device is a coaxial visual three-dimensional imaging device to realize the collection of space target information. The spectrometer and reflector are used to transmit the homologous optical information of the space target to the main camera for imaging, and then reflect it into the auxiliary camera for imaging through the reflector. The main camera and the auxiliary camera are coaxial structures to ensure that the collected images are homologous and synchronized, thereby realizing visual detection of space targets.

[0031] like Figure 3 The figure shows a coaxial vision three-dimensional imaging module, which includes a beam splitter, a reflector, a lens, a main camera, and an auxiliary camera. The light from the non-cooperative target in space passes through the beam splitter, one path passes through the beam splitter to enter the main camera, and the other path is reflected by the beam splitter to the reflector, and then reaches the auxiliary camera after passing the reflector, so that the main camera and the auxiliary camera can simultaneously image the space target.

[0032] The captured image data is transmitted to the receiving camera's intelligent perception module. The image data then enters a low-signal-to-noise ratio (SNR) target detection phase based on a deep residual contraction network to detect spatial targets. Simultaneously, the bionic coaxial visual 3D imaging and measurement module performs 3D reconstruction and pose measurement of spatial targets. Finally, intelligent target motion parameter perception (a module that processes data obtained from target detection by visual sensors, perceiving target motion states and characteristics, such as pose parameters) based on DQN (Deep Q-Network) and LSTM (Long Short-term Memory Networks) is employed to achieve perception and early warning of dynamic, non-cooperative targets in space. The detection information from the intelligent perception module is then fed into the flight vehicle to perform tasks such as visual guidance. The intelligent perception module controls the bionic coaxial visual 3D imaging and measurement module through feedback control, ensuring detection quality and accuracy.

[0033] Step 12: Target detection in a strong noise interference environment, including:

[0034] During the approach of a dynamic target in space, image quality can be poor due to environmental factors. Furthermore, the image contains many other objects, which can be interpreted as "noise." Conventional deep learning-based target detection algorithms cannot accurately detect the target. The Deep Residual Shrinkage Network, an improved version of the Deep Residual Network, integrates the Deep Residual Network, an attention mechanism, and a soft thresholding function. This mechanism can, to a certain extent, focus on and retain unimportant features, while using a soft thresholding function to set them to zero, thereby enhancing the deep neural network's ability to extract useful features from noisy signals.

[0035] like Figure 4 As shown in the figure, the deep residual shrinkage network implements soft thresholding under the deep attention mechanism. Through the side sub-network, a set of thresholds can be learned to perform soft thresholding on each feature channel. The deep residual shrinkage network includes multiple sets of deep residual modules. In the side sub-network, the absolute values ​​of all features of the input feature map (image features processed by the convolutional network) are first solved. Then, after global mean pooling and averaging, a set of features are obtained. In another path, the feature map after global mean pooling is input into a small fully connected network. This fully connected network uses the Sigmoid function as the last layer to normalize the output to between 0 and 1, obtaining a coefficient, which is recorded as The final threshold can be expressed as . Therefore, the threshold is a number between 0 and 1 × the average of the absolute value of the feature map. This method not only ensures that the threshold is positive, but also not too large. In this way, different samples have different thresholds, which enables the entire network to pay attention to the features related to the current task and improve the network's detection accuracy under noise interference conditions. Finally, a certain number of residual shrinkage network modules, as well as convolutional layers, batch normalization, activation functions, global mean pooling, and fully connected output layers are stacked to obtain a complete deep residual shrinkage network.

[0036] The deep residual shrinkage network can be used as a general feature learning method. The deep residual shrinkage network can use the attention mechanism to notice these "noises", and then use soft thresholding to set the features corresponding to these "noises" to zero, effectively improving the accuracy of image classification.

[0037] Figure 4 In this model, the input image undergoes convolution to extract features, passes through a multi-stage deep residual contraction network, and finally outputs the target category through a fully connected network. The deep residual module includes a shortcut connection and a side threshold calculation module. The threshold calculation module averages the image features, passes the fully connected branch through a sigmoid activation function, and then integrates them to generate a soft threshold. C and W represent the size of the image features.

[0038] Step 13: Construct a coaxial 3D vision measurement model, including:

[0039] The spatial dynamic target is in the process of approaching from far to near. In order to achieve the synchronous acquisition of its two-dimensional image and three-dimensional point cloud data, the installation carrier is in a moving state and the environment is affected by random uncontrollable factors such as strong interference and high noise. Therefore, the coaxial visual three-dimensional imaging device proposed in the present invention is required to be able to perform synchronous measurement of the two-dimensional image and three-dimensional point cloud of the spatial target, so as to realize the intelligent perception of the target motion parameters.

[0040] The present invention simulates the eagle-eye visual perception model and distributes the homologous light field by combining a spectroscope with a reflector. After the spatial light field information enters the bionic coaxial visual three-dimensional imaging and measurement module, it is divided into two groups by the spectroscope. One group passes through the spectroscope directly into the main camera for perspective projection imaging, and the other group is reflected by the spectroscope to the reflector installed on the side, and then reaches the auxiliary camera through the reflector. At this point, the auxiliary camera and the main camera are equivalent to a coaxial visual module, which can synchronously image the spatial homologous light field information.

[0041] In the pinhole imaging model, the bionic coaxial visual 3D imaging and measurement module is equivalent to the optical center moving along the optical axis, and the change in the distance between the optical center of the camera and the object is equivalent to the change in focal length. In the bionic coaxial visual 3D imaging and measurement module, the focal lengths of the main camera and the auxiliary camera are different. By imaging the same target with cameras of different focal lengths, the depth information of the object can be obtained. Figure 3 As shown in FIG, by configuring cameras with different focal lengths to form a coaxial stereo vision baseline distance, spatial stereo measurement is achieved, and the horizontal baseline distance is changed to the vertical baseline distance, which saves horizontal space occupation and is suitable for installation on slender flying carriers. The relevant calculation formula is shown in formula (1).

[0042] (1)

[0043] in, 、 They are the focal length of the main camera and the focal length of the auxiliary camera, and the angular distance difference between the two , 、 It is the radial radius of a pair of matching points in a coaxial vision system, which can be determined by matching the image feature points (the corresponding feature points of the spatial target in the main camera and the auxiliary camera). Figure 3 In, O c 、X c 、Y c , Z c are the origin, X-axis, Y-axis, and Z-axis of the main camera coordinate system. The virtual camera is equivalent to the spatial distribution of the auxiliary camera in the main camera coordinate system. The virtual camera here is just for illustration and does not actually exist.

[0044] Step 14: Reconstruct the target 3D point cloud, including:

[0045] Since the two sets of camera images in the bionic coaxial visual three-dimensional imaging and measurement module are coaxial and homologous, and the resulting target images differ by a certain scale, there are no problems such as perspective projection and occlusion caused by changes in viewing angle. Therefore, it is only necessary to densely match the feature points of the two sets of images and substitute them into the above formula (1) to achieve dense reconstruction of the spatial target, obtain the target depth information, and then convert it into a three-dimensional point cloud in the camera coordinate system.

[0046] In order to achieve dense matching of image feature points, this paper adopts a local image feature matching method. First, image feature detection, description and matching are established at a coarse granularity, and then dense matching at the sub-pixel level is refined at a fine granularity. By referring to Transformer, self-attention layer and mutual attention layer are used to obtain feature descriptors of the two images, so that dense matching can be generated in areas with less texture, which is suitable for complex scenes such as outdoor scenes with less texture and light interference. Among them, the total loss L includes coarse granularity loss and fine-grained loss , which can be expressed as:

[0047] (2)

[0048] set up is the main camera image feature point, is the auxiliary camera image feature point, is the radial epipolar constraint matrix, then:

[0049] (3)

[0050] By matching the image feature points of the two coaxial cameras and substituting them into formula (1), the three-dimensional dense point cloud of the spatial target can be obtained. At the same time, a one-to-one mapping between the two-dimensional image of the spatial target and the dense three-dimensional point cloud after dense matching of the image feature points is achieved, and the two-dimensional and three-dimensional information of the spatial target is obtained synchronously, effectively improving the target recognition and posture measurement accuracy.

[0051] Step 15: Measure target motion parameters, including:

[0052] By analyzing the center of the three-dimensional point cloud of the space target and establishing the target coordinate system, the pose parameters of the space target relative to the camera coordinate system can be obtained, providing data support for the calculation of target motion parameters and intelligent perception. Based on the three-dimensional dense point cloud of the space target obtained in step 14, the three-dimensional point cloud of the space target is segmented and fitted to determine the target coordinate system. Here, a space satellite is taken as an example. Figure 5 As shown. The three adjacent planes of the satellite are selected as the calculation objects, and the normal vectors of the three planes are orthogonal to each other. The Ransac algorithm is used to fit the left plane, and its equation is obtained as follows: , and determine the normal vector of the first plane as ( Figure 5 middle The equation of the second plane can be obtained by the same logic: , whose normal vector is ( Figure 5 middle The equation of fitting plane 3 is: , whose normal vector is ,in is the number of three-dimensional point clouds of the object being measured. Similarly, the normal vectors of the other two planes can be obtained 、 .

[0053] Taking into account the influence of point cloud error, the average value of the point set at the corner points of the three planes is taken as the coordinate origin of the satellite , establish the target coordinate system , 、 、 、 They are the coordinate origin, x-axis, y-axis, and z-axis respectively. By establishing the coordinate system of the target to be measured in space, the external parameter matrix of the target to be measured in space in the camera coordinate system is determined based on the three-dimensional point cloud of the space target. , considering the influence of 3D point cloud noise, it is necessary to adjust the external parameter matrix of the camera coordinate system Perform SVD (Singular Value Decomposition) orthogonalization processing. Further, we can get the Euler angles of the axis , around Euler angles of the axis , around Euler angles of the axis , the specific calculation formula is as follows:

[0054] (4)

[0055] in, Represents the external parameter matrix No. Row and List.

[0056] Step 16: Reinforce the learning target motion state, including:

[0057] Based on the current image and state information (position, attitude, orientation, velocity) of the space target, as well as its motion information (acceleration, pitch angle, and yaw angle), the future state of the target is calculated. A flight vehicle decision framework is established to determine the maneuver instructions for the flight vehicle. The flight vehicle needs to perceive the target's motion state from the environment and predict the future motion state of the measured space target. This is achieved by extracting features from sensor data and using these features to predict the future motion state of the space target.

[0058] A trajectory prediction model for moving targets is built using a reinforcement learning DQN network and an LSTM network with long-term memory. The target's motion parameters are used to construct a continuous state space for the flight vehicle. A reward function is designed based on the flight vehicle's mission requirements to implement a competitive strategy between the flight vehicle and the target. An LSTM network is combined with a fully connected network to construct a flight vehicle motion value network and a target network. The network is trained using historical flight data to fit the value function, enabling intelligent decision-making for the flight vehicle in any state. This approach uses reinforcement learning and a long-short-term memory network to process the flight parameters of the spatially measured target and generate an optimal strategy model for the flight vehicle.

[0059] make At this moment, the agent obtains state information from the environment and return information , and perform actions Acting on the environment, the state of the environment changes and moves to the next state information , and feedback information The goal of reinforcement learning is to find a strategy that maximizes the long-term accumulated rewards of the agent, namely:

[0060] (5)

[0061] in, represents the optimal strategy, Take the maximum value, is the value function, For strategy, is the discount factor used to calculate the cumulative return.

[0062] According to the motion characteristics of the flight vehicle and the non-cooperative target in the space during the approach process, based on the detection of the target motion parameters, the state space between the flight vehicle and the target in three-dimensional space is established. , and form the action space of the flight carrier, that is, the action instructions, as shown in Table 1, which are the action instructions that meet different mission requirements .

[0063] At the same time, the motion reward function of the flight vehicle is designed according to different mission requirements. It should be noted that different tasks here correspond to different flight mission requirements, such as docking, impact, capture, etc., which depend on the actual tasks of the space flight vehicle during the approach process of the non-cooperative target.

[0064] The network input layer is the state information of the agent at each moment , the network output is the value function of each action. To enhance the agent’s The medium perception ability makes the decision-making action have a certain continuity. LSTM is introduced as the neurons of the hidden layer, combined with the fully connected layer to realize the decision output of the intelligent agent.

[0065] Step 17: Sense and predict target motion, including:

[0066] After the training of the optimal strategy model generated in step 16 is complete, the flight vehicle can still make reasonable maneuvering decisions during the variable maneuvers during the approach to a non-cooperative space target, with a decision-making time of less than 20ms, meeting real-time requirements. This has important theoretical value and practical significance for the flight vehicle's maneuvering decisions based on visual detection and intelligent perception of the approach of a non-cooperative space target. Furthermore, the present invention has certain versatility and is suitable for assisting intelligent flight action decisions for slender flight vehicles in aviation or aerospace. Based on the above content, the space non-cooperative dynamic target approach visual detection device can analyze the relevant actions and commands generated after perceiving and predicting the space target, as shown in Table 1.

[0067] Table 1

[0068]

Claims

1. A method for visual detection of non-cooperative dynamic target approach in space, characterized in that: The steps include: Step 11. Build a visual detection device for approaching non-cooperative dynamic targets in space. The device is a coaxial three-dimensional visual imaging device that collects information about space targets. A beam splitter and a reflector are used to transmit homologous optical information of space targets to the main camera for imaging, and then to the auxiliary camera for imaging through a reflector. The main and auxiliary cameras are coaxial structures to ensure that the collected images are homologous and synchronized, thereby achieving visual detection of space targets. A coaxial binocular stereo vision measurement mode is used, supplemented by target pose solution and motion pattern prediction, to achieve three-dimensional detection and intelligent perception of approaching non-cooperative space targets. Step 12: Detect targets in a strong noise interference environment. Step 13: Construct a coaxial 3D vision measurement model to simulate the eagle-eye visual perception model. Use a spectroscope combined with a reflector to distribute the homologous light field. After the spatial light field information enters the bionic coaxial vision 3D imaging and measurement module, it is divided into two groups by the spectroscope. One group passes through the spectroscope directly into the main camera for perspective projection imaging, and the other group is reflected by the spectroscope to the reflector installed on the side, and then reaches the auxiliary camera through the reflector. The auxiliary camera and the main camera are equivalent to a coaxial vision module, which synchronously images the spatial homologous light field information. Step 14: After densely matching the two sets of image feature points, dense reconstruction of the spatial target is achieved, the target depth information is obtained, and then converted into a three-dimensional point cloud in the camera coordinate system; Step 15: After analyzing the three-dimensional point cloud of the space target, the target coordinate system is established, and the pose parameters of the space target relative to the camera coordinate system are obtained, providing data support for the calculation of target motion parameters and intelligent perception; Step 16: Based on the current image and state information of the space target and its motion information, the future state information of the target is calculated, a flight vehicle decision framework is established, and the maneuver instructions of the flight vehicle are solved. The flight vehicle perceives the target's motion state from the environment and predicts the future motion state of the space target. Features are extracted from the sensor data and used to predict the future motion state of the space target. Reinforcement learning and long-short-term memory networks are used to process the flight parameters of the space target to generate the optimal strategy model of the flight vehicle. Step 17: Based on the generated optimal strategy model, the flight vehicle implements maneuvering decisions for visual detection and intelligent perception of approaching non-cooperative targets in space.

2. The method for visual detection of non-cooperative dynamic target approach in space according to claim 1, characterized in that: The step 11 comprises: By transmitting the collected image data to the intelligent perception module of the receiving camera, the image data enters the low signal-to-noise ratio target detection link based on the deep residual shrinkage network to realize the detection of space targets. At the same time, the three-dimensional reconstruction and posture measurement of space targets are realized through the bionic coaxial visual three-dimensional imaging and measurement module. Finally, the intelligent perception of target motion parameters based on DQN and LSTM is adopted to realize the perception and early warning of dynamic non-cooperative targets in space. The detection information from the intelligent perception module is connected to the flight carrier to complete tasks such as visual guidance. The intelligent perception module realizes the control of the bionic coaxial visual three-dimensional imaging and measurement module through the feedback control link to ensure the detection quality and accuracy.

3. The method for visual detection of approaching non-cooperative dynamic targets in space according to claim 1, characterized in that: The step 12 includes: The deep residual shrinkage network is used to implement soft thresholding under the deep attention mechanism. Through the side sub-network, a set of thresholds are learned to perform soft thresholding on each feature channel. The deep residual shrinkage network includes multiple groups of deep residual modules. In the side sub-network, first, the absolute values ​​of all features of the input feature map are solved; then, after global mean pooling and averaging, a set of features are obtained. In the other path, the feature map after global mean pooling is input into a small fully connected network; the fully connected network uses the Sigmoid function as the last layer to normalize the output to between 0 and 1, obtaining a coefficient, which is recorded as ; The final threshold is expressed as .

4. The method for visual detection of non-cooperative dynamic target approach in space according to claim 1, characterized in that: The step 13 comprises: By configuring cameras with different focal lengths to form a coaxial stereo vision baseline distance, spatial stereo measurement is achieved, and the horizontal baseline distance is changed to the vertical baseline distance. The relevant calculation formula is shown in formula (1): （1） in, 、 They are the focal length of the main camera and the focal length of the auxiliary camera, and the angular distance difference between the two , 、 It is the radial radius of a pair of matching points in a coaxial vision system, which is determined by matching the corresponding feature points of the space target when imaged by the main camera and the auxiliary camera.

5. The method for visual detection of approaching non-cooperative dynamic targets in space according to claim 4, characterized in that: The step 14 comprises: Based on the local image feature matching method, we first establish image feature detection, description and matching at a coarse granularity, and then refine the dense matching at the sub-pixel level at a fine granularity. By referring to the Transformer, we use the self-attention layer and the mutual attention layer to obtain the feature descriptors of the two images. The total loss L includes the coarse granularity loss. and fine-grained loss , expressed as: （2） set up is the main camera image feature point, is the auxiliary camera image feature point, is the radial epipolar constraint matrix, then: （3） After matching the image feature points of the two coaxial cameras, the three-dimensional dense point cloud of the spatial target is obtained by substituting them into formula (1). At the same time, a one-to-one mapping between the two-dimensional image of the spatial target and the dense three-dimensional point cloud is achieved, and the two-dimensional and three-dimensional information of the spatial target is obtained synchronously.

6. The method for visual detection of approaching non-cooperative dynamic targets in space according to claim 1, characterized in that: The step 15 comprises: Take the average value of the point set at the corner points of the three planes as the coordinate origin of the satellite , establish the target coordinate system , 、 、 、 They are the coordinate origin, x-axis, y-axis, and z-axis respectively; by establishing the coordinate system of the target to be measured in space, based on the three-dimensional point cloud of the space target, the external parameter matrix of the space target to be measured in the camera coordinate system is determined , the external parameter matrix of the camera coordinate system Perform SVD (singular value decomposition) orthogonalization processing; respectively obtain the surrounding Euler angles of the axes , around Euler angles of the axes , around Euler angles of the axes , the specific calculation formula is as follows: （4） in, Represents the external parameter matrix No. Row and List.

7. The method for visual detection of approaching non-cooperative dynamic targets in space according to claim 1, characterized in that: The step 16 includes: using reinforcement learning DQN and LSTM network with long-term memory capacity to build a trajectory prediction model for the moving target, using the target motion parameters to construct a continuous state space of the flight carrier, designing a reward function according to the mission requirements of the flight carrier, and realizing the confrontation strategy between the flight carrier and the target; using LSTM and a fully connected network to build a flight carrier motion value network and a target network; using historical flight data to train the network, complete the value function fitting, and realize the intelligent decision-making of the flight carrier in any state; according to the motion characteristics of the flight carrier and the non-cooperative target in the space during the approach process, based on the detection of the target motion parameters, establish the state space between the flight carrier and the target in three-dimensional space. , and form the action space of the flying carrier, that is, the action instruction .

Images