Spatial non-cooperative dynamic target approaching visual detection method

Abstract

The invention discloses a spatial non-cooperative dynamic target approach visual detection method, and relates to the technical field of optical visual sensor detection, and the method comprises the steps: building a spatial non-cooperative dynamic target approach visual detection system software and hardware frame structure, and employing a deep residual shrinkage network to achieve target detection in a strong noise interference environment; establishing a coaxial three-dimensional vision measurement model to realize rapid measurement of target three-dimensional point cloud and motion parameters; the motion state of the target is subjected to reinforcement learning training, the motion condition of the target is sensed and predicted, and support is provided for a flight carrier to intelligently sense the target and make a maneuver decision. The method is suitable for high-precision and intelligent detection in the approaching process of a space dynamic target, and has important significance in guaranteeing detection and perception in the fields of spacecraft short-distance capture, near-earth asteroid early warning, air refueling docking and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of optical vision sensor detection, and in particular to a method for visual detection of a space non-cooperative dynamic target approach. Background Art

[0002] The visual detection and intelligent perception technology of non-cooperative dynamic targets in space is widely used in the fields of aerial refueling, rendezvous and docking, space monitoring and early warning, asteroid impact experiments, etc., with its non-contact, large-scale, long-distance and high-precision characteristics. Non-cooperative targets have no prior information compared to cooperative targets. To detect and perceive them, it is necessary to capture, identify and obtain their three-dimensional point cloud information in order to accurately detect their motion parameters and other information, and provide data support for target detection and perception for the flight carrier, and assist it in making action decisions. For example, during aerial refueling, the receiving aircraft needs to control its own flight state to perceive the state parameters of the drogue released by the tanker in real time in the air. From far to near, it is necessary to accurately identify the drogue, measure in three dimensions and predict its motion state, and adjust the flight posture of the receiving aircraft in time to achieve the final precise docking. Typical application scenarios of approach detection of non-cooperative dynamic targets in space include Starlink formation flying, asteroid near-earth defense, air combat and unmanned driving.

[0003] In the space measurement environment, under aviation / aerospace conditions, it is usually affected by factors such as high maneuverability, strong vibration, variable light, and complex background, which seriously lead to difficulties in target tracking, reduced system measurement accuracy, or even failure. For example, both the atmospheric environment and the space environment have complex backgrounds, light interference, strong maneuverability of the target, and limited power consumption of the flight carrier space volume, making it difficult to achieve accurate target detection, simultaneous acquisition of two-dimensional texture and three-dimensional information, and intelligent perception of the target, limiting its early warning and effective avoidance of the target.

[0004] The process of non-cooperative dynamic target detection and perception in space (dynamic target approach visual detection process) is a process from far to near and from coarse to fine. Figure 1 The horizontal axis distance in the figure is divided into several different stages, as shown in the figure below: Figure 1 As shown, from far to near, it includes searching, locking, tracking, guiding, measuring, guiding, docking, capturing or landing, etc. In the early stage, the requirements for measurement accuracy are relatively low, but the requirements for detection accuracy are relatively high. As the approach distance shortens, the corresponding measurement accuracy becomes higher and higher. When it reaches the close range, it needs to be detected with high precision and intelligent perception to provide data support for subsequent decisions and actions.

[0005] The existing visual detection and perception technologies for non-cooperative dynamic target approach in space mainly include the following aspects: in terms of image acquisition, images or videos of non-cooperative dynamic targets in space are acquired through high-resolution cameras, multi-camera systems or other imaging devices. However, the image quality is unstable due to factors such as lighting, weather, and target posture. In terms of target detection and recognition, the acquired images or videos are processed and analyzed to detect and identify the target. However, the accuracy of detection and recognition is limited by factors such as image quality, target posture, and occlusion. In terms of posture estimation, the posture of the target, such as angle and speed, is estimated by analyzing the target image or video. However, it is easily affected by factors such as image quality, target posture, and sensor accuracy, and the accuracy of posture estimation is limited. In terms of path planning, the motion path is planned according to the position, posture and other information of the target to achieve safe and accurate approach. However, it is easily affected by factors such as target posture and environmental factors, and the accuracy and reliability of path planning are limited. In terms of perception and obstacle avoidance, obstacles in the surrounding environment are perceived through sensors and other auxiliary equipment, and obstacle avoidance algorithms are used to avoid collisions. Automatically sense and avoid obstacles, but it is easily affected by factors such as sensor accuracy and obstacle avoidance algorithm. The accuracy and reliability of obstacle avoidance are limited, and the degree of intelligence is low. Although the accuracy and reliability of detection and perception can be improved by comprehensively utilizing data from multiple sensors, it will increase system complexity and cost.

[0006] Intelligent perception of space targets includes key technologies such as posture measurement, 3D reconstruction and part recognition. Currently, non-cooperative rendezvous and docking mainly uses microwave or lidar for ranging, visual cameras for assisted angle measurement, and imaging lidar and visual cameras for short distances to resolve relative posture. Sensors for space target detection mainly include microwave radar, lidar, Kinect, binocular camera, TOF camera, multi-camera, light field camera and event camera, which are applied to different scenarios due to their respective characteristics.

[0007] With the rapid development of artificial intelligence technology, deep learning target detection algorithms have been widely studied. However, factors such as spatial environment image noise, light interference, and complex backgrounds have limited the accuracy and stability of conventional detection. At the same time, in three-dimensional reconstruction based on stereo vision, it is necessary to accurately match the target image features, but binocular vision is easily affected by factors such as perspective projection and occlusion. At the same time, in order to ensure the measurement accuracy, the baseline distance needs to be increased, and the installation space is limited for slender flying carriers. Reinforcement learning is used for autonomous flight control and air confrontation of drones, and the optimal control instructions of drones are calculated in real time to complete the corresponding tactical actions. Therefore, the current relevant methods cannot be directly used for accurate target detection in complex interference environments in space, and they are small in size, low in power consumption, and do not require prior information on the target to achieve intelligent perception of dynamic targets.

[0008] Based on the above analysis, it can be seen that a single camera can realize three-dimensional reconstruction of space targets from far to near at different object distances or focal lengths. Although it has the advantages of small size and low power consumption, it cannot realize real-time and continuous three-dimensional imaging and detection of dynamic space targets. For non-cooperative dynamic space targets, there are complex environments, low imaging quality, and large target change scales. Simple single-camera systems still cannot meet detection requirements. It is necessary to resist image noise interference and achieve three-dimensional reconstruction. If binocular stereo vision is used, it needs to meet certain baseline distance requirements. It cannot be directly installed and used under limited flight carrier conditions. For example, due to the small space of refueling aircraft, missiles, and small satellite platforms, long-baseline binocular stereo vision sensors cannot be directly installed and used. Non-cooperative space targets are unknown targets, and their motion characteristics have the characteristics of time series uncertainty. How to make full use of the target motion history data for modeling and analysis and predict subsequent motion conditions has also become the key to intelligent perception and decision-making. Summary of the invention

[0009] In order to solve the above technical problems, the present invention provides a method for visually detecting the approach of non-cooperative dynamic targets in space, which has the characteristics of compact structure, high intelligence, strong versatility, simple system composition, etc., and can realize the detection and intelligent perception of non-cooperative dynamic targets in space under the condition of limited space flight carrier. The present invention adopts two sets of single cameras to construct a coaxial stereoscopic vision measurement system, which can ensure both the measurement accuracy and the compact system structure, and meet the requirements of approach detection and perception of non-cooperative dynamic targets in space under complex environments.

[0010] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0011] A space non-cooperative dynamic target approach visual detection method comprises the following steps:

[0012] Step 11, build a space non-cooperative dynamic target approach visual detection device. The space non-cooperative dynamic target approach visual detection device is a coaxial visual three-dimensional imaging device to realize the collection of space target information. The same source optical information of the space target is transmitted to the main camera for imaging through a beam splitter and a reflector, and then is reflected into the auxiliary camera for imaging through a reflector. The main camera and the auxiliary camera are coaxial structures to ensure that the collected images are homologous and synchronized, so as to realize the visual detection of the space target. The coaxial binocular stereo vision measurement mode is adopted, assisted by the target posture solution and motion mode prediction, so as to realize the three-dimensional detection and intelligent perception of the approach of the space non-cooperative target;

[0013] Step 12: Detect targets in a strong noise interference environment;

[0014] Step 13: Construct a coaxial 3D vision measurement model, simulate the eagle-eye vision perception model, and use a beam splitter combined with a reflector to distribute the homologous light field. After the spatial light field information enters the bionic coaxial vision 3D imaging and measurement module, it is divided into two groups by the beam splitter. One group directly enters the main camera through the beam splitter for perspective projection imaging, and the other group is reflected by the beam splitter to the reflector installed on the side, and then reaches the auxiliary camera through the reflector. The auxiliary camera and the main camera are equivalent to a coaxial vision module, which synchronously images the spatial homologous light field information;

[0015] Step 14: After dense matching of the two sets of image feature points, dense reconstruction of the spatial target is achieved, the target depth information is obtained, and then converted into a three-dimensional point cloud in the camera coordinate system;

[0016] Step 15: After analyzing the three-dimensional point cloud of the space target, the target coordinate system is established, and the posture parameters of the space target relative to the camera coordinate system are obtained, providing data support for the calculation of target motion parameters and intelligent perception;

[0017] Step 16: Calculate the future state information of the target based on the current image and state information and actuation information of the space target, establish a flight carrier decision framework, and solve the maneuvering instructions of the flight carrier; the flight carrier perceives the target motion state from the environment and predicts the future motion state of the space target; extract features from the sensor data and use these features to predict the future motion state of the space target; use reinforcement learning and long short-term memory network to process the flight parameters of the space target and generate the optimal strategy model of the flight carrier;

[0018] Step 17: Based on the generated optimal strategy model, the flight vehicle implements maneuvering decisions for visual detection and intelligent perception of approaching non-cooperative targets in space.

[0019] The beneficial effects of the present invention compared with the prior art are:

[0020] The present invention designs an optical probe that combines a splitter / reflector and multiple groups of cameras to form a homologous coaxial visual optical path. Combined with coaxial imaging and projection methods, the visual detection of non-cooperative dynamic targets approaching in space can be realized under the conditions of using two cameras and a return optical path. The present invention is based on an improved deep residual shrinkage network to detect targets in strong noise images, so as to realize fast and high-precision positioning of dynamic targets in complex spatial environments with strong interference, and adopts embedded parallel acceleration technology to realize parallelization of image processing algorithms to meet the real-time requirements of the measurement system. The measurement system involved in the present invention is compact and small, with high measurement accuracy, and can effectively solve the problem of traditional measurement systems with complex structures and large volumes that are not suitable for installation on small carriers for dynamic targets in space. The problem of approach detection has important theoretical significance and practical value in promoting the introduction of visual detection in the closed-loop control of the space motion process, improving the navigation positioning accuracy and perception warning. The present invention segments the geometric features of the measured target and establishes the target local coordinate system by fitting, thereby obtaining the target's posture parameters. This method is applicable to the measurement of posture parameters of rigid or flexible deformable objects and has certain versatility. The present invention combines the target motion parameter intelligent perception model to realize the perception and prediction of the motion state of dynamic targets in space, solves the problem of existing visual sensor measurement systems in small carrier installation and limited adaptability to the environment, and is very suitable for navigation and real-time warning applications in the motion of small flying carriers in complex environments.

[0021] In summary, the present invention designs the software and hardware framework, structure, measurement mechanism and working mode of a miniaturized, high-precision space non-cooperative dynamic target approach visual detection and intelligent perception system, solves key scientific problems such as system software and hardware framework structure optimization design, space dynamic target approach visual detection, dynamic target three-dimensional reconstruction, and motion parameter timing analysis and intelligent perception, and constructs a measurement system verification prototype to provide an effective technical approach and detection means for space dynamic target approach visual detection and intelligent perception, which has important theoretical significance and practical value for promoting dynamic target approach detection, further improving application scenarios such as aerial autonomous refueling in complex environments, drone landing / ship, and high-precision navigation in asteroid detection and early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is the principle diagram of the visual detection process of a space dynamic target approach;

[0023] Figure 2 A flow chart of a space non-cooperative dynamic target approach visual detection method of the present invention;

[0024] Figure 3 This is the principle diagram of coaxial visual three-dimensional imaging;

[0025] Figure 4A schematic diagram of determining the target coordinate system for measuring its position and pose according to the target geometric features;

[0026] Figure 5 Schematic diagram of the target coordinate system. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0028] The basic idea of ​​the invention is: to design the hardware and software framework structure of the measurement system, and to use a deep residual shrinkage network to realize target detection in a strong noise interference environment; to establish a coaxial three-dimensional visual measurement model to realize the rapid measurement of the target three-dimensional point cloud and motion parameters; to perceive and predict the target motion state through reinforcement learning training on the target motion state, and to provide support for the flight carrier's intelligent perception and maneuvering decision-making of the target; to construct a prototype of a spatial dynamic target approach visual detection and intelligent perception system, and to experimentally verify and evaluate its various performance indicators.

[0029] like Figure 1 , Figure 2 As shown, a space non-cooperative dynamic target approach visual detection method of the present invention comprises the following steps:

[0030] Step 11: Build a space non-cooperative dynamic target approach visual detection device. The space non-cooperative dynamic target approach visual detection device is a coaxial visual three-dimensional imaging device to realize the collection of space target information. A beam splitter and a reflector are used to transmit the homologous optical information of the space target to the main camera for imaging, and then reflected by the reflector into the auxiliary camera for imaging. The main camera and the auxiliary camera are coaxial structures to ensure that the collected images are homologous and synchronized, thereby realizing visual detection of space targets.

[0031] like Figure 3 The figure shows a coaxial vision three-dimensional imaging module, which includes a beam splitter, a reflector, a lens, a main camera and an auxiliary camera. The light from the non-cooperative space target passes through the beam splitter, one path passes through the beam splitter to enter the main camera, and the other path is reflected by the beam splitter to the reflector, and then reaches the auxiliary camera after passing the reflector, so that the main camera and the auxiliary camera can image the space target at the same time.

[0032] By transmitting the collected image data to the intelligent perception module of the receiving camera, the image data enters the low signal-to-noise ratio target detection link based on the deep residual shrinkage network to realize the detection of space targets. At the same time, the bionic coaxial visual three-dimensional imaging and measurement module realizes the three-dimensional reconstruction and posture measurement of space targets. Finally, the intelligent perception of target motion parameters based on DQN (Deep Q-Network) and LSTM (Long Short-term Memory Networks) (a module that processes the data obtained after detecting the target through the visual sensor, perceives the target motion state and characteristics, such as posture parameters, etc.) to realize the perception and early warning of dynamic non-cooperative targets in space, and connects the detection information from the intelligent perception module to the flight carrier to complete tasks such as visual guidance. Among them, the intelligent perception module realizes the control of the bionic coaxial visual three-dimensional imaging and measurement module through the feedback control link to ensure the detection quality and accuracy.

[0033] Step 12: Detect targets in a strong noise interference environment, including:

[0034] When a dynamic target approaches, the image quality will be poor due to the influence of external environmental factors. At the same time, it contains many other objects, which can be understood as "noise". Conventional target detection algorithms based on deep learning cannot accurately detect the target. As an improved version of the deep residual network, the deep residual shrinkage network is an integration of the deep residual network, the attention mechanism and the soft threshold function. It can notice unimportant features through the attention mechanism to a certain extent and retain them, and set them to zero through the soft threshold function, thereby enhancing the ability of the deep neural network to extract useful features from noisy signals.

[0035] like Figure 4 As shown in the figure, the deep residual shrinkage network implements soft thresholding under the deep attention mechanism. Through the side sub-network, a set of thresholds can be learned to perform soft thresholding on each feature channel. The deep residual shrinkage network includes multiple sets of deep residual modules. In the side sub-network, first, the absolute values ​​of all features of the input feature map (image features processed by the convolutional network) are solved. Then, after global mean pooling and averaging, a set of features is obtained. In the other path, the feature map after global mean pooling is input into a small fully connected network. This fully connected network uses the Sigmoid function as the last layer to normalize the output to between 0 and 1 and obtain a coefficient, denoted as The final threshold can be expressed as . Therefore, the threshold is a number between 0 and 1 × the average of the absolute value of the feature map. This method not only ensures that the threshold is positive, but also not too large. In this way, different samples have different thresholds, so that the entire network can pay attention to the features related to the current task and improve the detection accuracy of the network under noise interference conditions. Finally, a certain number of residual shrinkage network modules, convolutional layers, batch normalization, activation functions, global mean pooling, and fully connected output layers are stacked to obtain a complete deep residual shrinkage network.

[0036] The deep residual shrinkage network can be used as a general feature learning method. The deep residual shrinkage network can use the attention mechanism to notice these "noises", and then use soft thresholding to set the features corresponding to these "noises" to zero, effectively improving the accuracy of image classification.

[0037] Figure 4 In the example, the input image is extracted through convolution, passes through a multi-level deep residual shrinkage network, and then fully connected to output the target category. The deep residual module includes a quick connection and a side threshold calculation module. The threshold calculation module averages the image features, and the fully connected branch passes through the Sigmoid activation function to integrate and generate a soft threshold. C and W represent the size of the image features.

[0038] Step 13: Construct a coaxial 3D visual measurement model, including:

[0039] The spatial dynamic target is in the process of approaching from far to near, and it is necessary to realize the synchronous acquisition of its two-dimensional image and three-dimensional point cloud data. At the same time, the installation carrier is in a moving state, and the environment is affected by random uncontrollable factors such as strong interference and high noise. Therefore, the coaxial visual three-dimensional imaging device proposed in the present invention is required to be able to perform synchronous measurement of the two-dimensional image and three-dimensional point cloud of the spatial target, so as to realize the intelligent perception of the target motion parameters.

[0040] The present invention simulates the eagle-eye visual perception model, and distributes the homologous light field by combining a beam splitter with a reflector. After the spatial light field information enters the bionic coaxial visual three-dimensional imaging and measurement module, it is divided into two groups by the beam splitter. One group directly enters the main camera through the beam splitter for perspective projection imaging, and the other group is reflected by the beam splitter to the reflector installed on the side, and then reaches the auxiliary camera through the reflector. At this point, the auxiliary camera and the main camera are equivalent to a coaxial visual module, which can synchronously image the spatial homologous light field information.

[0041] In the pinhole imaging model, the bionic coaxial visual 3D imaging and measurement module is equivalent to the movement of the optical center along the optical axis, and the change in the distance between the optical center of the camera and the object is equivalent to the change in focal length. In the bionic coaxial visual 3D imaging and measurement module, the focal lengths of the main camera and the auxiliary camera are different. The depth information of the object can be obtained by imaging the same target with cameras of different focal lengths, such as Figure 3 As shown, by configuring cameras with different focal lengths to form a coaxial stereo vision baseline distance, spatial stereo measurement is achieved, and the horizontal baseline distance is changed to the vertical baseline distance, which saves the horizontal space occupation and is suitable for installation on slender flying carriers. The relevant calculation formula is shown in formula (1).

[0042] (1)

[0043] in, , They are the focal length of the main camera and the focal length of the auxiliary camera, and the angular distance difference between the two is , , It is the radial radius of a pair of matching points in a coaxial vision system, which can be determined by matching the image feature points (the corresponding feature points of the space target in the main camera and the auxiliary camera). Figure 3 In, O c , X c , Y c , Z c They are the origin, X-axis, Y-axis, and Z-axis of the main camera coordinate system. The virtual camera is equivalent to the spatial distribution of the auxiliary camera in the main camera coordinate system. The virtual camera here serves as a reference and does not actually exist.

[0044] Step 14: Reconstruct the target 3D point cloud, including:

[0045] Since the two camera groups in the bionic coaxial visual 3D imaging and measurement module are coaxial and homologous, and the resulting target images differ by a certain scale, there are no problems such as perspective projection and occlusion caused by changes in viewing angle. Therefore, it is only necessary to densely match the feature points of the two groups of images and substitute them into the above formula (1) to achieve dense reconstruction of the spatial target, obtain the target depth information, and then convert it into a 3D point cloud in the camera coordinate system.

[0046] In order to achieve dense matching of image feature points, the present invention adopts a matching method based on local image features. First, image feature detection, description and matching are established at a coarse granularity, and then dense matching at the sub-pixel level is refined at a fine granularity. By referring to Transformer, self-attention layer and mutual attention layer are used to obtain feature descriptors of the two images, so that dense matching can be generated in areas with less texture, which is suitable for complex scenes such as outdoor areas with less texture and light interference. Among them, the total loss L includes coarse-grained loss and fine-grained loss , which can be expressed as:

[0047] (2)

[0048] set up is the main camera image feature point, is the auxiliary camera image feature point, is the radial polar constraint matrix, then it satisfies:

[0049] (3)

[0050] By matching the image feature points of the two coaxial cameras and substituting them into equation (1), the three-dimensional dense point cloud of the spatial target can be obtained. At the same time, a one-to-one mapping between the two-dimensional image of the spatial target and the dense three-dimensional point cloud after dense matching of the image feature points is achieved, and the two-dimensional and three-dimensional information of the spatial target is obtained synchronously, effectively improving the accuracy of target recognition and posture measurement.

[0051] Step 15: Measure target motion parameters, including:

[0052] By analyzing the center of the three-dimensional point cloud of the space target and establishing the target coordinate system, the position parameters of the space target relative to the camera coordinate system can be obtained, providing data support for the calculation of target motion parameters and intelligent perception. Based on the three-dimensional dense point cloud of the space target obtained in step 14, the three-dimensional point cloud of the space target is segmented and fitted to determine the target coordinate system. Here, a space satellite is taken as an example. Figure 5 As shown. The three adjacent planes of the satellite are selected as the calculation objects, and the normal vectors of the three planes are orthogonal to each other. The Ransac algorithm is used to fit the left plane, and its equation is obtained as follows: , and determine the normal vector of the first plane as ( Figure 5 middle Similarly, the equation of the second plane is , whose normal vector is ( Figure 5 middle The equation of the fitting plane 3 is: , whose normal vector is ,in is the number of three-dimensional point clouds of the object being measured. Similarly, the normal vectors of the other two planes can be obtained , .

[0053] Taking into account the influence of point cloud error, the average value of the point set at the corner points of the three planes is taken as the coordinate origin of the satellite , establish the target coordinate system , , , , They are the coordinate origin, x-axis, y-axis, and z-axis respectively. By establishing the coordinate system of the target to be measured in space, the external parameter matrix of the target to be measured in space in the camera coordinate system is determined based on the three-dimensional point cloud of the acquired space target. Considering the influence of 3D point cloud noise, the external parameter matrix of the camera coordinate system needs to be Perform SVD (singular value decomposition) orthogonalization. Further, we can get the Euler angles of the axis , around Euler angles of the axis , around Euler angles of the axis , the specific calculation formula is as follows:

[0054] (4)

[0055] in, Represents the external parameter matrix No. Row and List.

[0056] Step 16: Reinforce the learning target motion state, including:

[0057] According to the current image and state information (position, attitude, orientation, speed) and actuation information (acceleration, pitch angle, yaw angle) of the space target, the future state information of the target is calculated, the flight carrier decision framework is established, and the maneuvering instructions of the flight carrier are solved. The flight carrier needs to perceive the target motion state from the environment and predict the future motion state of the space target. Features are extracted from the sensor data and used to predict the future motion state of the space target.

[0058] The DQN reinforcement learning and the LSTM network with long-term memory are used to build a trajectory prediction model for moving targets. The target motion parameters are used to construct the continuous state space of the flight carrier. The reward function is designed according to the mission requirements of the flight carrier to realize the confrontation strategy between the flight carrier and the target. The LSTM and fully connected network are combined to build the flight carrier motion value network and target network. The network is trained using historical flight data to complete the value function fitting and realize the intelligent decision-making of the flight carrier in any state. That is, the flight parameters of the measured target in space are processed by reinforcement learning and long short-term memory network to generate the optimal strategy model of the flight carrier.

[0059] make At this moment, the agent obtains state information from the environment and return information , and perform actions Acting on the environment, the state of the environment changes and transfers to the next state information , and feedback information The goal of reinforcement learning is to find a strategy that maximizes the long-term accumulated rewards of the agent, namely:

[0060] (5)

[0061] in, represents the optimal strategy, Take the maximum value, is the value function, For strategy, is the discount factor used to calculate the cumulative return.

[0062] According to the motion characteristics of the flight carrier and the non-cooperative target in the space during the approach process, based on the detection of the target motion parameters, the state space between the flight carrier and the target in three-dimensional space is established. , and form the action space of the flight carrier, that is, the action instructions, as shown in Table 1, which are the action instructions that meet different mission requirements .

[0063] At the same time, the motion reward function of the flight vehicle is designed according to different mission requirements. It should be pointed out that the different tasks here correspond to different flight mission requirements, such as docking, collision, capture, etc., which depend on the actual tasks of the space flight vehicle in the approach process to the non-cooperative target.

[0064] The network input layer is the state information of the agent at each moment , the network output is the value function of each action. To enhance the agent from state information The perception ability makes the decision-making action have a certain continuity. LSTM is introduced as the neuron of the hidden layer, combined with the fully connected layer to realize the decision output of the intelligent agent.

[0065] Step 17: Sense and predict target motion, including:

[0066] After the training of the optimal strategy model generated in step 16 is completed, the flight carrier can still make reasonable maneuvering decisions during the variable maneuvers in the process of approaching a non-cooperative target in space, and the decision time is less than 20ms, which can meet the real-time requirements. It has important theoretical value and practical significance for the maneuvering decision of the flight carrier for visual detection and intelligent perception of the approach of non-cooperative targets in space. At the same time, the present invention has certain versatility and is suitable for auxiliary intelligent flight action decision-making of slender flight carriers in aviation or aerospace. Based on the above content, the space non-cooperative dynamic target approach visual detection device can analyze the relevant actions and instructions generated after perceiving the space target and making motion predictions as shown in Table 1.

[0067] Table 1

Claims

1. A space non-cooperative dynamic target approach visual detection method, characterized in that: The steps include: Step 11, build a space non-cooperative dynamic target approach visual detection device. The space non-cooperative dynamic target approach visual detection device is a coaxial visual three-dimensional imaging device to realize the collection of space target information. The same source optical information of the space target is transmitted to the main camera for imaging through a beam splitter and a reflector, and then is reflected into the auxiliary camera for imaging through a reflector. The main camera and the auxiliary camera are coaxial structures to ensure that the collected images are homologous and synchronized, so as to realize the visual detection of the space target. The coaxial binocular stereo vision measurement mode is adopted, assisted by the target posture solution and motion mode prediction, so as to realize the three-dimensional detection and intelligent perception of the approach of the space non-cooperative target; Step 12: Detect targets in a strong noise interference environment; Step 13: Construct a coaxial 3D vision measurement model, simulate the eagle-eye vision perception model, and use a beam splitter combined with a reflector to distribute the homologous light field. After the spatial light field information enters the bionic coaxial vision 3D imaging and measurement module, it is divided into two groups by the beam splitter. One group directly enters the main camera through the beam splitter for perspective projection imaging, and the other group is reflected by the beam splitter to the reflector installed on the side, and then reaches the auxiliary camera through the reflector. The auxiliary camera and the main camera are equivalent to a coaxial vision module, which synchronously images the spatial homologous light field information; Step 14: After dense matching of the two sets of image feature points, dense reconstruction of the spatial target is achieved, the target depth information is obtained, and then converted into a three-dimensional point cloud in the camera coordinate system; Step 15: After analyzing the three-dimensional point cloud of the space target, the target coordinate system is established, and the posture parameters of the space target relative to the camera coordinate system are obtained, providing data support for the calculation of target motion parameters and intelligent perception; Step 16: Calculate the future state information of the target based on the current image and state information and actuation information of the space target, establish a flight carrier decision framework, and solve the maneuvering instructions of the flight carrier; the flight carrier perceives the target motion state from the environment and predicts the future motion state of the space target; extract features from the sensor data and use these features to predict the future motion state of the space target; use reinforcement learning and long short-term memory network to process the flight parameters of the space target and generate the optimal strategy model of the flight carrier; Step 17: Based on the generated optimal strategy model, the flight vehicle implements maneuvering decisions for visual detection and intelligent perception of approaching non-cooperative targets in space.

2. The space non-cooperative dynamic target approach visual detection method according to claim 1 is characterized in that: The step 11 comprises: By transmitting the collected image data to the intelligent perception module of the receiving camera, the image data enters the low signal-to-noise ratio target detection link based on the deep residual contraction network to realize the detection of space targets. At the same time, the bionic coaxial visual three-dimensional imaging and measurement module realizes the three-dimensional reconstruction and posture measurement of space targets. Finally, the intelligent perception of target motion parameters based on DQN and LSTM is used to realize the perception and early warning of dynamic non-cooperative targets in space, and the detection information from the intelligent perception module is connected to the flight carrier to complete visual guidance and other tasks. Among them, the intelligent perception module realizes the control of the bionic coaxial visual three-dimensional imaging and measurement module through the feedback control link to ensure the detection quality and accuracy.

3. The space non-cooperative dynamic target approach visual detection method according to claim 1, characterized in that: The step 12 comprises: The deep residual shrinkage network is used to implement soft thresholding under the deep attention mechanism. Through the side sub-network, a set of thresholds are learned to perform soft thresholding on each feature channel. The deep residual shrinkage network includes multiple sets of deep residual modules. In the side sub-network, first, the absolute values ​​of all features of the input feature map are solved; then, after global mean pooling and averaging, a set of features is obtained. ; In the other path, the feature map after global mean pooling is input into a small fully connected network; the fully connected network uses the Sigmoid function as the last layer to normalize the output to between 0 and 1 and obtain a coefficient, which is recorded as ; The final threshold is expressed as .

4. The space non-cooperative dynamic target approach visual detection method according to claim 1, characterized in that: The step 13 comprises: By configuring cameras with different focal lengths to form a coaxial stereo vision baseline distance, spatial stereo measurement is achieved, and the horizontal baseline distance is changed to the vertical baseline distance. The relevant calculation formula is shown in formula (1): （1） in, , They are the focal length of the main camera and the focal length of the auxiliary camera, and the angular distance difference between the two is , , It is the radial radius of a pair of matching points in a coaxial vision system, which is determined by matching the corresponding feature points of the space target when imaged by the main camera and the auxiliary camera.

5. The space non-cooperative dynamic target approach visual detection method according to claim 4 is characterized in that: The step 14 comprises: Based on the local image feature matching method, we first establish image feature detection, description and matching at a coarse granularity, and then refine the dense matching at the sub-pixel level at a fine granularity. By referring to Transformer, we use the self-attention layer and the mutual attention layer to obtain the feature descriptors of the two images. The total loss L includes the coarse-grained loss and fine-grained loss , expressed as: （2） set up is the main camera image feature point, is the auxiliary camera image feature point, is the radial polar constraint matrix, then it satisfies: （3） After matching the image feature points of the two coaxial cameras, the three-dimensional dense point cloud of the space target is obtained by substituting them into formula (1). At the same time, the one-to-one mapping between the two-dimensional image of the space target and the dense three-dimensional point cloud is realized, and the two-dimensional information and three-dimensional information of the space target are obtained synchronously.

6. The space non-cooperative dynamic target approach visual detection method according to claim 1, characterized in that: The step 15 comprises: Take the average value of the point set at the corner points of the three planes as the coordinate origin of the satellite , establish the target coordinate system , , , , They are the coordinate origin, x-axis, y-axis, and z-axis respectively; by establishing the coordinate system of the target to be measured in space, based on the three-dimensional point cloud of the acquired space target, the external parameter matrix of the space target to be measured in the camera coordinate system is determined , the external parameter matrix of the camera coordinate system Perform SVD (singular value decomposition) orthogonalization processing; respectively obtain the surrounding Euler angles of the axis , around Euler angles of the axis , around Euler angles of the axis , the specific calculation formula is as follows: （4） in, Represents the external parameter matrix No. Row and List.

7. The space non-cooperative dynamic target approach visual detection method according to claim 1, characterized in that: The step 16 includes: using the reinforcement learning DQN and the LSTM network with long-term memory capability to build a trajectory prediction model for the moving target, using the target motion parameters to construct the continuous state space of the flight carrier, designing the reward function according to the mission requirements of the flight carrier, and realizing the confrontation strategy between the flight carrier and the target; using LSTM and a fully connected network to build a flight carrier motion value network and a target network; using historical flight data to train the network, completing the value function fitting, and realizing the intelligent decision-making of the flight carrier in any state; according to the motion characteristics of the flight carrier and the space non-cooperative target during the approach process, based on the detection of the target motion parameters, establishing the state space between the flight carrier and the target in the three-dimensional space , and form the action space of the flying vehicle, that is, the action instruction .

Images