Deep space exploration sky-ground collaborative teleoperation system and deep space exploration sky-ground collaborative teleoperation method

Abstract

The invention discloses a deep space exploration space-ground collaborative teleoperation system and method. The system comprises an interface and information management module, a terrain reconstruction and positioning module, a navigation planning module and a data management and virtual simulation verification system. On the basis that the starting point and the target point are determined, one or more factors such as the shortest time, the shortest mileage and the highest safety are used as optional conditions, the optimal passing path in the navigation planning range of the detector is determined, and a control strategy for movement of the detector is generated.

Description

Technical Field

[0001] The present invention relates to the field of aerospace technology, and more specifically to a deep space exploration ground-ground collaborative remote control system and method. Background Art

[0002] Deep space exploration is an extremely important field in human space activities. It is the first step for humans to understand the Earth, the solar system and the universe, and then to explore, survey and settle other celestial bodies in the solar system. With the continuous deepening of deep space exploration, space teleoperation is facing an increasingly complex environment. For example, for the Mars probe, communication conditions are severely limited, and there will be phenomena such as delays in uploading commands, obstructions in stellar space and low data transmission rates; for the lunar probe, when exploring the South Pole of the Moon, it will face challenges such as extremely low solar altitude angles and large shadow areas on the lunar surface.

[0003] At present, the remote operation of deep space probes is mainly carried out in a low-latency, non-time-varying imaging environment, while space remote operation has a complex working environment different from that of the earth, and is subject to time-varying conditions such as energy and communication and harsh extraterrestrial environments. In the prior art, a Chinese invention patent with publication number CN109794918A (an interactive space remote operation system) proposes a master-slave interactive remote operation system to improve the control performance of the robot and improve the stability of the space remote operation system to a certain extent. However, this technology lacks comprehensive consideration of the overall structure of the remote operation and does not consider complex environmental factors.

[0004] Based on the above background, considering the current complex mission environment of deep space exploration, it is crucial to study a space remote operation method and system that takes into account complex time-varying constraints and can perform real-time ground simulation and real-time interaction with the ground, in order to efficiently complete deep space exploration missions. Summary of the invention

[0005] In view of this, the present invention provides a deep space exploration ground-ground collaborative teleoperation system and method, which is used for rapid response and safe supervision and management of human-computer interaction in ground teleoperation of the detector, so as to realize efficient operation of the detector.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] The present invention first discloses a deep space exploration ground-ground collaborative teleoperation system, comprising:

[0008] Interface and information management module: responsible for receiving the telemetry source code of the detector, and passing the decoded telemetry information and image data to the terrain reconstruction and positioning module; at the same time, responsible for converting the verified navigation planning results into on-board remote control instructions and injecting them into the detector;

[0009] Terrain reconstruction and positioning module: responsible for reconstructing the three-dimensional scene of the surface of the detected planet using the decoded telemetry information and image data, and determining the position of the detector on the surface of the detected planet;

[0010] Navigation planning module: Based on optional constraints, path planning is performed on the reconstructed three-dimensional scene of the probe planet surface to obtain the navigation planning result for the probe to reach the target point;

[0011] Data management and virtual simulation verification system: responsible for managing the decoded telemetry information, and using the digital twin simulation platform to simulate and verify the navigation planning results.

[0012] Furthermore, the terrain reconstruction and positioning module performs reconstruction of the three-dimensional scene of the detected planet surface, specifically comprising the following steps:

[0013] By using the left and right parallax pairs obtained after matching the images of the on-board binocular camera of the detector, a three-dimensional point cloud of the detected planet's surface in the coordinate system of the detector's reference camera is obtained through forward intersection. Then, the three-dimensional point cloud coordinates of the planet's surface are converted to the coordinate system of the detector itself through the exterior orientation elements of the detector's reference camera, thus obtaining the terrain reconstruction result completed by the reconstructed point cloud.

[0014] In a specific embodiment, the reference coordinate system is a left camera coordinate system and the left camera coordinate system may be used.

[0015] Furthermore, a multi-scale cost aggregation matching algorithm is used in the on-device binocular camera image matching process to complete the disparity map generation.

[0016] Furthermore, in the terrain reconstruction and positioning module, determining the position of the probe on the surface of the detection planet specifically includes the following steps:

[0017] Coordinate extraction: Extract the coordinates of the features on the detector's ring image corresponding to the features on the actual digital orthophoto map DOM of the star catalog, and obtain the coordinate values ​​of the features in the local set coordinate system;

[0018] Detector positioning: Match the features of the detector's ring image with the digital orthophoto map DOM of the detected planet's star catalog, and use the detector image features to match the features of the digital orthophoto map DOM of the star catalog to obtain the topological relationship of the actual objects, and obtain the position of the detector based on the topological relationship; specifically, the digital orthophoto map DOM of the detected planet's star catalog has the global coordinates of each pixel point, and the global coordinates of the feature points are marked on the orthophoto map DOM, and then the global coordinates of the center point are calculated inversely through the relative position relationship between the feature points and the center point, which is the position coordinates of the detector.

[0019] Furthermore, in the terrain reconstruction and positioning module, determining the position of the probe on the surface of the detection planet specifically includes the following steps:

[0020] The three-dimensional coordinates of the objects around the rough position of the detector are obtained by detecting the digital elevation model DEM and the digital orthophoto map DOM of the planet's surface. The multi-image space resection is performed according to the circular images taken by the navigation camera of the detector to solve the position and attitude of the detector in the DOM projection coordinate system, and the position of the detector is corrected by the following error equation:

[0021] V=AX-L

[0022] in,

[0023]

[0024] V represents the image point coordinates (x i ,y i ) is a correction matrix; A represents the coefficient matrix, that is, the correction matrix of the unknown φ j ,ω j , κ j The partial derivative of ; X represents the position and posture of the photography center, that is, the unknown number to be solved; L represents the matrix of the difference between the observed coordinates of the image point and the calculated value; j represents the jth image; i represents the i-th feature point on the jth image; each image requires at least 3 feature points, and the unknown number can be solved by the least squares method φ j ,ω j , κ j , and then get the accurate detector position.

[0025] Furthermore, in one embodiment, the navigation planning module performs path planning on the reconstructed three-dimensional scene of the detected planet surface based on the optional constraint conditions, specifically comprising the following steps:

[0026] Determine the specific value of the cost coefficient in the heuristic algorithm according to the optional constraint condition, and determine the cost function of the path planning according to the specific value of the cost coefficient; the specific value of the cost coefficient is obtained by the following steps:

[0027] When the direction of the next node and the current node in the path planning search are within the solar occlusion angle range, the cost coefficient w is taken as +∞, indicating that the cost is infinite; otherwise, the cost coefficient w is taken as 1, indicating that there is no additional cost;

[0028] When the direction of the next node and the current node in the path planning search are within the communication obstruction angle range, the cost coefficient w is taken as +∞, indicating that the cost is infinite; otherwise, the cost coefficient w is taken as 1, indicating that there is no additional cost;

[0029] The optimal path within the scope of the detector navigation planning is determined based on the estimated cost value from the starting node through the current node to the target node.

[0030] Further, in another embodiment, the navigation planning module performs path planning on the reconstructed three-dimensional scene of the detected planet surface based on the optional constraint conditions, and the specific process includes:

[0031] First, the path cost from each node to the target point is calculated by linear interpolation. The path cost includes the path length and the cumulative value of the elevation change of the points passed by the path. During the linear interpolation process, the heading angle is constrained according to the illumination angle and communication angle constraints.

[0032] Secondly, the path consumption value of each node is used to extract the path with the smallest path consumption value from the starting point to the target point as the optimal passing path within the navigation planning range of the detector.

[0033] The present invention also discloses a deep space exploration ground-ground collaborative remote operation method, comprising the following steps:

[0034] (1) Receive the telemetry source code from the detector and pass the decoded telemetry information and image data to the terrain reconstruction and positioning module;

[0035] (2) reconstructing a three-dimensional scene of the surface of the detected planet using the decoded telemetry information and image data, and determining the position of the probe on the surface of the detected planet;

[0036] (3) Perform path planning on the reconstructed three-dimensional scene of the probe planet surface based on optional constraints to obtain the navigation planning result of the probe reaching the target point;

[0037] (4) Use the digital twin simulation platform to simulate and verify the navigation planning results;

[0038] (5) The verified navigation planning results are transmitted to the detector.

[0039] Furthermore, in one embodiment, path planning is performed on the reconstructed three-dimensional scene of the detected planet surface based on the optional constraint conditions, specifically including the following steps:

[0040] Determine the specific value of the cost coefficient in the heuristic algorithm according to the optional constraint condition, and determine the cost function of the path planning according to the specific value of the cost coefficient; the specific value of the cost coefficient is obtained by the following steps:

[0041] When the direction of the next node and the current node in the path planning search are within the solar occlusion angle range, the cost coefficient w is taken as +∞, indicating that the cost is infinite; otherwise, the cost coefficient w is taken as 1, indicating that there is no additional cost;

[0042] When the direction of the next node and the current node in the path planning search are within the communication obstruction angle range, the cost coefficient w is taken as +∞, indicating that the cost is infinite; otherwise, the cost coefficient w is taken as 1, indicating that there is no additional cost;

[0043] The optimal path within the scope of the detector navigation planning is determined based on the estimated cost value from the starting node through the current node to the target node.

[0044] Furthermore, the cost function of the path planning specifically includes the following expression:

[0045] f(n)=w*g(n)+h(n)

[0046] Among them, f(n) is the estimated cost from the starting node through the current node to the target node;

[0047] g(n) is the dissipation function, which represents the actual cost from the starting node to the current node;

[0048] h(n) is the heuristic function, which represents the estimated cost from the current node to the target node;

[0049] w is the cost coefficient.

[0050] It can be seen from the above technical solutions that, compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention provides a deep space exploration multi-device joint teleoperation system that is adaptable to complex mission environments, solves the problems of efficient mission route planning, behavior action planning, mobile path planning and safety supervision management of detectors under time-varying complex constraints, and improves the computational efficiency of teleoperation, which is suitable for the on-orbit mission implementation of deep space detectors.

[0052] Since deep space exploration teleoperation is affected by time-varying factors such as lighting, communication, and shadows under high latency conditions, effective planning is carried out by comprehensively considering the task constraints. By integrating various functional interfaces, efficient data transmission is achieved and planning efficiency is improved. This method solves the problem that conventional teleoperation is limited in efficiency and cannot meet special conditions such as low-light areas. At the same time, by establishing a data-driven twin simulation platform, the interactive simulation verification and analysis of the instrument and the ground is carried out, and the safety supervision and management of the detector is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0054] Figure 1 An architecture diagram of a deep space exploration ground-ground collaborative teleoperation system provided for an embodiment of the invention.

[0055] Figure 2 A schematic diagram of the star catalog constraint coupling relationship provided in an embodiment of the present invention.

[0056] Figure 3 A schematic diagram of a star chart terrain reconstruction process provided by an embodiment of the present invention.

[0057] Figure 4 A schematic diagram of the solar azimuth angle and its rate of change for ephemeris analysis provided in an embodiment of the present invention.

[0058] Figure 5 A schematic diagram of the internal structure of the digital twin simulation platform provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] The present invention provides a deep space exploration ground-ground collaborative teleoperation system, comprising:

[0061] Interface and information management module: responsible for receiving the telemetry source code of the detector, and passing the decoded telemetry information and image data to the terrain reconstruction and positioning module; at the same time, responsible for converting the verified navigation planning results into on-board remote control instructions and injecting them into the detector;

[0062] Terrain reconstruction and positioning module: responsible for reconstructing the three-dimensional scene of the surface of the detected planet using the decoded telemetry information and image data, and determining the position of the detector on the surface of the detected planet;

[0063] Navigation planning module: Based on optional constraints, path planning is performed on the reconstructed three-dimensional scene of the probe planet surface to obtain the navigation planning result for the probe to reach the target point;

[0064] Data management and virtual simulation verification system: responsible for managing the decoded telemetry information and using the digital twin simulation platform to simulate and verify the navigation planning results

[0065] In a specific embodiment, the overall framework structure of the deep space exploration ground-to-earth collaborative teleoperation system is as follows: Figure 1As shown, the deep space exploration ground-to-earth collaborative teleoperation system inputs the telemetry and data transmission source codes transmitted from the on-orbit detector; and outputs the original frame of the detector's mobile remote control command to guide the detector to move on the surface of the target planet.

[0066] The specific deep space exploration ground-to-earth collaborative teleoperation system includes the following aspects:

[0067] 1) Interface and information management is responsible for data exchange with the on-orbit detector. The receiver receives telemetry and digital transmission data, which are decoded for teleoperation tasks. It is also responsible for converting planning results into telecontrol instructions for on-orbit operation.

[0068] 2) Terrain reconstruction and positioning module and navigation planning module are basic functions to support teleoperation tasks. According to the telemetry information, image or laser data on the device, planning results or positioning results are given.

[0069] 3) The data management and virtual simulation verification system is the data management core of the teleoperation system. It is responsible for the data management of the entire system and the simulation verification of planning results. At the same time, it can display various terrain products and planning movement results on the platform in real time.

[0070] In addition to the above functional modules, the deep space exploration ground-to-earth collaborative teleoperation system also includes:

[0071] 4) The image database can display, maintain and update all historical remote sensing images, orbiter images, DEM analysis images (lighting, slope, etc.) and on-board perception images, and can be quickly retrieved according to the needs of the remote operation working group

[0072] Through the above system, the teleoperation working group only needs to provide detection targets or planning targets. The teleoperation planning system can provide planning results and display simulation verification results in a short time, and generate on-device instructions that can be injected.

[0073] The specific data processing process of some functional modules is described in detail below.

[0074] The interface and information management module mainly implements the following functions:

[0075] 1) The interface and information management module receives the telemetry and data transmission code sent by the measurement and control system, and forwards it to the detector teleoperation system according to the established communication protocol;

[0076] 2) The interface and information management module receives the overall remote control instructions of the detector remote operation and forwards them to the measurement and control system according to the specified communication protocol.

[0077] 3) On the established ground hardware and software platform, data interaction with the measurement and control system and ground application system is realized through TCP\UDP communication protocol, including telemetry data, digital transmission data, image data, terrain products and planning results; at the same time, data

[0078] 4) Store, process, distribute, display and manage telemetry data, digital transmission data and image data on the overall teleoperation hardware platform.

[0079] Terrain reconstruction and positioning module, the input data of this module may include: the original image or three-dimensional point cloud data file with the detector posture, mechanism and sensor parameters provided by the detector remote operation system; and the final output is the terrain product file and positioning results.

[0080] First, as a pre-launch preparation, the internal and external parameters of the detector and the position of the sensor in the detector coordinate system can be obtained by calibrating the detector's sensing system;

[0081] Secondly, according to Figure 3 The process shown in the figure obtains the star catalog point cloud data, fuses the sequence images of sensors such as optical cameras and 3D laser radars and multi-site images, and generates a typical star catalog 3D scene; its main processes include:

[0082] Using the parallax pairs obtained in the matching process, we obtain the 3D point cloud in the left camera system through forward intersection (triangulation), and then transform the point cloud coordinates into the detector body coordinate system through the left camera exterior orientation elements, thus obtaining the terrain reconstruction result completed by the reconstructed point cloud.

[0083] In order to meet the characteristics of sparse texture in extreme star environment, a multi-scale cost aggregation matching algorithm is used to generate the disparity map, making full use of the image context information in the matching cost volume.

[0084] Then, the absolute positioning can be performed using the star catalog remote sensing images and the global perception images after the probe lands. The specific process includes:

[0085] Coordinate extraction: The coordinates of the characteristic objects on the image obtained by the detector's circular shooting correspond to the objects on the actual DOM, and the coordinate values ​​of the objects with obvious characteristics in the local north-east coordinate system are extracted.

[0086] The characteristics of star map objects can be learned through the convolutional neural network (CNN) method to achieve automatic classification of different ground object characteristics in star map images.

[0087] Detect the features of objects on the planet's surface (referred to as star surface) and extract the features of star surface objects through deep learning methods.

[0088] The detector's ring image is matched with the star table DOM. The detector image features obtained by feature extraction are matched with the star table DOM to obtain the topological relationship of the actual object, where the center point is the position of the detector.

[0089] The precise positioning of the detector is achieved by obtaining the three-dimensional coordinates of the objects around the rough position of the detector, such as craters, sand dunes, mountain ridges and shadows, through the digital elevation model DEM and digital orthophoto map DOM of the planet's surface. The multi-image space resection is performed based on the navigation camera's ring image to solve the position and attitude of the detector in the DOM projection coordinate system. The error equation is as follows:

[0090] V=AX-L

[0091] in,

[0092]

[0093] V represents the image point coordinates (x i ,y i ) is a correction matrix; A represents the coefficient matrix, that is, the correction matrix of the unknown φ j ,ω j , κ j The partial derivative of ; X represents the position and posture of the photography center, that is, the unknown number to be solved; L represents the matrix of the difference between the observed coordinates of the image point and the calculated value; j represents the jth image; i represents the i-th feature point on the jth image. Each image requires at least 3 feature points, and the unknown number can be solved by the least squares method φ j ,ω j , κ j , and then get the accurate detector position.

[0094] The navigation planning module inputs the following data:

[0095] 1) Point cloud data and DEM images generated by the terrain construction module;

[0096] 2) Time-varying constraints such as illumination and communication;

[0097] 3) Plan the target point.

[0098] The final output is the path planning result. Specifically, the three-dimensional terrain recovery is described by generating a raster map through point cloud data and DEM data. An improved filtering algorithm is used to reduce noise and remove external points on the DEM map. In addition, the terrain data generated by multiple imaging can be fused and spliced ​​to complete the scene reconstruction.

[0099] Furthermore, based on the detector's position information, star surface topography, obstacles and other information, combined with the detector's own position, movement and GNC performance, a safe passing threshold is set, the surrounding environment is analyzed, and a passability map of the detector's surrounding environment is obtained.

[0100] In a specific embodiment, the optional constraints such as illumination, communication and shadow on the moving path of the detector are analyzed by combining the ephemeris and terrain information. A schematic diagram of the constrained coupling relationship of the star table is shown as follows: Figure 3 As shown in the figure, based on the orbital ephemeris of the probe's orbit around the sun, the relative position relationship over time is calculated through the probe's position information, and the azimuth and altitude of the sun in the local north-east coordinate system at the time of planning are calculated as the input of the path planning constraint. The angle between the sun's azimuth and the probe's yaw angle is not allowed to be within [θ sun1 ,θ sun2 ] range, the specific value of this range is determined by the angle at which the detector camera avoids direct sunlight (for example, for a certain model of detector, the range is [-30,30]).

[0101] According to the orbit of the relay satellite or orbiter around the probe planet measured in orbit, the relative position relationship over time is calculated through the position information of the probe, and the azimuth of the communication target (relay satellite or orbiter) in the local north-east coordinate system at that time is calculated according to the planning time, which is used as the input of the path planning constraint condition. The angle between the azimuth of the relay satellite and the yaw angle of the probe is not allowed to be within [θ relay1 ,θ relay2 ] range, the specific value of this range is determined according to the shielding angle of the detector's solar panels, masts and other structures to the directional antenna (for example, for a certain model of detector, the range is [150,210]).

[0102] Based on the DEM generated by the terrain construction module, the shadow analysis calculation is performed according to the local solar altitude angle and azimuth as input, and the generated shadow map is binarized according to the presence or absence of lighting conditions as the input of path planning. The light area is the passable area (white pixels after binarization), and the dark area is the impassable area (black pixels after binarization).

[0103] By processing the local environment perception information, the search map of the navigation planning is obtained. On the basis of the determined starting point and target point, one or more factors such as the shortest time, the shortest mileage, and the highest safety are used as optional conditions to determine the optimal path within the scope of the detector navigation planning and generate the control strategy for the movement of the detector. The specific process includes:

[0104] First, the path cost from each node to the target point is calculated by linear interpolation. The path cost includes options such as path length and the cumulative value of elevation change of the points passed on the path. During the linear interpolation process, the heading angle is constrained according to the illumination angle and communication angle constraints. That is, the linear interpolation is performed in the non-constrained direction and does not enter the non-constrained direction. The farther the node is from the target point, the greater the path cost value calculated by the node.

[0105] Secondly, the path consumption value of each node is used to extract the path with the minimum path consumption value from the starting point to the target point.

[0106] Relevant adjustable parameters, such as obstacle surmounting capability, are added during the path planning process, and the navigation planning results are matched with the onboard mobility strategy.

[0107] The digital twin data display platform can input remote control, telemetry and data transmission original frames, as well as planning results provided by navigation planning and terrain products provided by terrain construction; output visualization results such as teleoperation and telemetry data, and ephemeris analysis, mechanism planning, and mobile planning results. Among them, a schematic diagram of the ephemeris analysis of the solar azimuth angle and its rate of change is shown below: Figure 4 shown.

[0108] The digital twin data display platform in the present invention can establish accurate detector models and terrain models, and realize the interaction and display between models. Figure 5 As shown in the figure, the digital twin data display platform can perform three-dimensional modeling of the probe movement area based on the planet surface DEM, DOM and other data, and conduct data docking and interaction with the probe control system and equipment monitoring system. According to the real-time monitoring data of the virtual monitoring system, the accuracy and reliability of the kinematic model, mechanism motion model, gait model and path planning model in the virtual simulation system and virtual planning system are verified, and the model is iteratively optimized and dynamically corrected. Specifically, it includes the following aspects:

[0109] 1) Display and verify the terrain products and planning results of the remote operation system, and combine the visualization tools such as the remote operation large screen, workstation, and control terminal to form a complete detector visualization system, including:

[0110] Ability to review and display verification of planning strategies such as path planning results.

[0111] It can load reconstructed terrain files and combine them with pre-bound ephemeris to complete the three-dimensional model of telemetry-driven and command-driven detectors, simulate and render the terrain environment, lighting, shadows, running ruts and moving wheels, and realize dynamic display of different perspectives to show the positional relationship between the detector and the star catalog.

[0112] 2) It can dynamically display the data of key parts related to the detector (temperature, current, voltage, etc.) and display the relevant data of the equipment on each corresponding device.

[0113] The internal operation process of the system of the present invention is as follows:

[0114] 1) Interface and information management: As the interactive interface between the probe mission support system and the outside, the receiver forwards the probe telemetry and data transmission source code; it is also responsible for sending remote control command information to control the on-orbit probe;

[0115] 2) Interface and information management transmits image information, laser data information, and engineering telemetry to the lunar topography construction module;

[0116] 3) Interface and information management transmits landing image data to the positioning function module;

[0117] 4) Interface and information management transmits telemetry information such as detector status to data management and virtual simulation modules;

[0118] 5) Interface and information management receives the planning results of navigation path planning and the results of mission planning and organization planning;

[0119] 6) Interface and information management forwards the telemetry and data transmission data source code subscribed by the software to the fast teleoperation software, and receives the image software and path planning results decoded by the fast teleoperation software;

[0120] 7) The data management and virtual simulation system receives the planning results of navigation path planning, mission and organization planning, and verifies the results;

[0121] 8) The image database receives and stores the terrain products reconstructed by the lunar terrain construction module;

[0122] 9) The image database forwards the image products in the database according to the subscription status of modules such as positioning, navigation path planning, mission and organization planning.

[0123] The present invention designs a deep space exploration multi-device joint remote operation system architecture for application in complex mission environments. The planning process comprehensively considers factors such as lighting, communication, and shadows. The ground remote operation working group only needs to provide planning goals and detection goals. Automated operation can achieve rapid response of human-computer interaction in ground remote operation of the detector and realize efficient operation of the on-orbit spacecraft.

[0124] The present invention designs a digital twin simulation platform structure driven by on-orbit data, which can verify the probe planning strategy in real time, realize the visualization of telemetry and data transmission, mobile and mechanism planning results, terrain products, etc. Establishing an image database can support mission planning under low-light conditions such as the lunar polar regions.

[0125] The present invention aims to realize the construction of a system for deep space teleoperation under complex mission conditions.

[0126] The ground-ground collaborative remote operation of deep space exploration uses telemetry and digital transmission data to obtain the working status and environmental information of the detector on the ground, establish a digital twin platform, complete the establishment of the terrain of the environment around the detector, analyze the working status of the detector, generate and verify the planning strategy, and support the normal operation of the detector in orbit.

[0127] Based on the above system, the present invention also discloses a deep space exploration ground-to-earth collaborative remote operation method, comprising the following steps:

[0128] (1) Receive the telemetry source code from the detector and pass the decoded telemetry information and image data to the terrain reconstruction and positioning module;

[0129] (2) reconstructing a three-dimensional scene of the surface of the detected planet using the decoded telemetry information and image data, and determining the position of the probe on the surface of the detected planet;

[0130] (3) Perform path planning on the reconstructed three-dimensional scene of the probe planet surface based on optional constraints to obtain the navigation planning result of the probe reaching the target point;

[0131] (4) Use the digital twin simulation platform to simulate and verify the navigation planning results;

[0132] (5) The verified navigation planning results are transmitted to the detector.

[0133] Furthermore, in one embodiment, path planning is performed on the reconstructed three-dimensional scene of the detected planet surface based on the optional constraint conditions, specifically including the following steps:

[0134] Determine the specific value of the cost coefficient in the heuristic algorithm according to the optional constraint condition, and determine the cost function of the path planning according to the specific value of the cost coefficient; the specific value of the cost coefficient is obtained by the following steps:

[0135] When the direction of the next node and the current node in the path planning search are within the solar occlusion angle range, the cost coefficient w is taken as +∞, indicating that the cost is infinite; otherwise, the cost coefficient w is taken as 1, indicating that there is no additional cost;

[0136] When the direction of the next node and the current node in the path planning search are within the communication obstruction angle range, the cost coefficient w is taken as +∞, indicating that the cost is infinite; otherwise, the cost coefficient w is taken as 1, indicating that there is no additional cost;

[0137] The optimal path within the scope of the detector navigation planning is determined based on the estimated cost value from the starting node through the current node to the target node.

[0138] The cost function of the path planning in the above implementation manner specifically includes the following expression:

[0139] f(n)=w*g(n)+h(n)

[0140] Among them, f(n) is the estimated cost from the starting node through the current node to the target node;

[0141] g(n) is the dissipation function, which represents the actual cost from the starting node to the current node;

[0142] h(n) is the heuristic function, which represents the estimated cost from the current node to the target node;

[0143] w is the cost coefficient.

[0144] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0145] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A deep space exploration ground-to-earth collaborative teleoperation system, characterized in that: include: Interface and information management module: responsible for receiving the telemetry source code of the detector, and passing the decoded telemetry information and image data to the terrain reconstruction and positioning module; at the same time, responsible for converting the verified navigation planning results into on-board remote control instructions and injecting them into the detector; Terrain reconstruction and positioning module: responsible for reconstructing the three-dimensional scene of the surface of the detected planet using the decoded telemetry information and image data, and determining the position of the detector on the surface of the detected planet; Navigation planning module: Based on optional constraints, path planning is performed on the reconstructed three-dimensional scene of the probe planet surface to obtain the navigation planning result for the probe to reach the target point; Data management and virtual simulation verification system: responsible for managing the decoded telemetry information, and using the digital twin simulation platform to simulate and verify the navigation planning results.

2. A deep space exploration ground-to-earth collaborative teleoperation system according to claim 1, characterized in that: The terrain reconstruction and positioning module reconstructs the three-dimensional scene of the detected planet surface, specifically including the following steps: By using the left and right parallax pairs obtained after matching the images of the on-board binocular camera of the detector, a three-dimensional point cloud of the detected planet's surface in the coordinate system of the detector's reference camera is obtained through forward intersection. Then, the three-dimensional point cloud coordinates of the planet's surface are converted to the coordinate system of the detector itself through the exterior orientation elements of the detector's reference camera, thus obtaining the terrain reconstruction result completed by the reconstructed point cloud.

3. A deep space exploration ground-to-earth collaborative teleoperation system according to claim 2, characterized in that: In the process of on-device binocular camera image matching, a multi-scale cost aggregation matching algorithm is used to generate the disparity map.

4. The deep space exploration ground-to-earth collaborative teleoperation system according to claim 1, characterized in that: In the terrain reconstruction and positioning module, determining the position of the probe on the surface of the detection planet specifically includes the following steps: Coordinate extraction: Extract the coordinates of the features on the detector's ring image corresponding to the features on the actual digital orthophoto map DOM of the star catalog, and obtain the coordinate values ​​of the features in the local set coordinate system; Detector positioning: Match the features of the detector's ring image with the digital orthophoto map DOM of the detected planet's star catalog, and use the detector image features to match the features of the digital orthophoto map DOM of the star catalog to obtain the topological relationship of the actual objects, and obtain the position of the detector based on the topological relationship; specifically, the digital orthophoto map DOM of the detected planet's star catalog has the global coordinates of each pixel point, and the global coordinates of the feature points are marked on the orthophoto map DOM, and then the global coordinates of the center point are calculated inversely through the relative position relationship between the feature points and the center point, which is the position coordinates of the detector.

5. The deep space exploration ground-to-earth collaborative teleoperation system according to claim 1, characterized in that: In the terrain reconstruction and positioning module, determining the position of the probe on the surface of the detection planet specifically includes the following steps: The three-dimensional coordinates of the objects around the rough position of the detector are obtained by detecting the digital elevation model DEM and the digital orthophoto map DOM of the planet's surface. The multi-image space resection is performed according to the circular images taken by the navigation camera of the detector to solve the position and attitude of the detector in the DOM projection coordinate system, and the position of the detector is corrected by the following error equation: V=AX-L in, V represents the image point coordinates (x i ,y i ) is a correction matrix; A represents the coefficient matrix, that is, the correction matrix of the unknown number φ j ,ω j , κ j The partial derivative of ; X represents the position and posture of the photography center, that is, the unknown number to be solved; L represents the matrix of the difference between the observed coordinates of the image point and the calculated value; j represents the jth image; i represents the i-th feature point on the jth image; each image requires at least 3 feature points, and the unknown number can be solved by the least squares method φ j ,ω j , κ j , and then get the accurate detector position.

6. The deep space exploration ground-to-earth collaborative teleoperation system according to claim 1, characterized in that: The navigation planning module performs path planning in the reconstructed three-dimensional scene of the detected planet surface based on the optional constraints, specifically including the following steps: Determine the specific value of the cost coefficient in the heuristic algorithm according to the optional constraint condition, and determine the cost function of the path planning according to the specific value of the cost coefficient; the specific value of the cost coefficient is obtained by the following steps: When the direction of the next node and the current node in the path planning search are within the solar occlusion angle range, the cost coefficient w is taken as +∞, indicating that the cost is infinite; otherwise, the cost coefficient w is taken as 1, indicating that there is no additional cost; When the direction of the next node and the current node in the path planning search are within the communication obstruction angle range, the cost coefficient w is taken as +∞, indicating that the cost is infinite; otherwise, the cost coefficient w is taken as 1, indicating that there is no additional cost; The optimal path within the scope of the detector navigation planning is determined based on the estimated cost value from the starting node through the current node to the target node.

7. The deep space exploration ground-to-earth collaborative teleoperation system according to claim 1, characterized in that: The navigation planning module performs path planning in the reconstructed three-dimensional scene of the detected planet surface based on the optional constraints, and the specific process includes: First, the path cost from each node to the target point is calculated by linear interpolation. The path cost includes the path length and the cumulative value of the elevation change of the points passed by the path. During the linear interpolation process, the heading angle is constrained according to the illumination angle and communication angle constraints. Secondly, the path consumption value of each node is used to extract the path with the smallest path consumption value from the starting point to the target point as the optimal passing path within the navigation planning range of the detector.

8. A method for deep space exploration ground-to-earth coordinated remote operation, characterized in that: The following steps are involved: (1) Receive the telemetry source code from the detector and pass the decoded telemetry information and image data to the terrain reconstruction and positioning module; (2) reconstructing a three-dimensional scene of the surface of the detected planet using the decoded telemetry information and image data, and determining the position of the probe on the surface of the detected planet; (3) Perform path planning on the reconstructed three-dimensional scene of the probe planet surface based on optional constraints to obtain the navigation planning result of the probe reaching the target point; (4) Use the digital twin simulation platform to simulate and verify the navigation planning results; (5) The verified navigation planning results are transmitted to the detector.

9. The method for deep space exploration ground-to-earth coordinated teleoperation according to claim 8, characterized in that: Path planning is performed based on the optional constraints in the reconstructed three-dimensional scene of the detected planet surface, specifically including the following steps: Determine the specific value of the cost coefficient in the heuristic algorithm according to the optional constraint condition, and determine the cost function of the path planning according to the specific value of the cost coefficient; the specific value of the cost coefficient is obtained by the following steps: When the direction of the next node and the current node in the path planning search are within the solar occlusion angle range, the cost coefficient w is taken as +∞, indicating that the cost is infinite; otherwise, the cost coefficient w is taken as 1, indicating that there is no additional cost; When the direction of the next node and the current node in the path planning search are within the communication obstruction angle range, the cost coefficient w is taken as +∞, indicating that the cost is infinite; otherwise, the cost coefficient w is taken as 1, indicating that there is no additional cost; The optimal path within the scope of the detector navigation planning is determined based on the estimated cost value from the starting node through the current node to the target node.

10. The method for deep space exploration ground-to-earth coordinated teleoperation according to claim 9, characterized in that: The cost function of the path planning specifically includes the following expression: f(n)=w*g(n)+h(n) Among them, f(n) is the estimated cost from the starting node through the current node to the target node; g(n) is the dissipation function, which represents the actual cost from the starting node to the current node; h(n) is the heuristic function, which represents the estimated cost from the current node to the target node; w is the cost coefficient.

Images