Full-process High-real-time Digital Simulation System for On-orbit Manipulation of Dynamic Targets

Abstract

The present application provides a full-process high-real-time digital simulation system for on-orbit manipulation of dynamic targets, which relates to the technical field of on-orbit manipulation of dynamic targets. The digital simulation system includes: a mission planning and management module for receiving manipulation missions and generating path planning schemes; a visual servo module for visually measuring dynamic targets and outputting path planning schemes; a dynamics and control module for generating motion state data of a manipulation mechanism, a dynamic target, and / or a floating base according to the path planning schemes and / or manipulation missions sent by the mission planning and management module and / or the visual servo module; and a mission display module for presenting a digital simulation scene of the dynamic target according to the motion state data and sending a camera image of the digital simulation scene to the visual servo module for visual measurement. This system can effectively and highly real-timely reflect the dynamic characteristics of the system, providing technical support for the simulation analysis of on-orbit manipulation missions of dynamic targets.

Description

Technical Field

[0001] This application relates to the technical field of on-orbit operation and control of dynamic targets. Specifically, it relates to a full-process high-real-time digital simulation system for on-orbit operation and control of dynamic targets. Background Art

[0002] The on-orbit operation and control of dynamic targets play a key role in tasks such as space station assembly and construction, and spacecraft docking and repair. It is an important technology to ensure space safety and promote space cooperation. With the rapid development of space technology, the design of spacecraft systems has become increasingly complex. Given the long cycle and high cost of space experiments, ground experiments can only test typical working conditions and cannot elaborate on the on-orbit state in detail. Digital simulation of on-orbit operation tasks has become an important link to verify task indicators and technical feasibility. In on-orbit operation tasks, complex space operations such as dynamic target capture are hot research topics internationally. Facing such challenging tasks, the importance of digital simulation technology becomes even more prominent.

[0003] On-orbit digital simulation technology has application prospects such as pre-task simulation analysis, in-task status monitoring, and on-orbit fault analysis. Over the years, certain progress has been made in its modeling and development. For example, AGI Company in the United States has developed the commercial analysis software STK (satellite tool kit), and NASA has led the development of the GMAT (general mission analysis tool), a general space mission simulation tool. These software support the visual scene simulation of space missions and are suitable for simulating general mission scenarios such as spacecraft design, launch, and orbital motion. However, they cannot provide support for the process simulation and real-time calculation of complex space operations. If we want to simulate the complex space operation process, traditional kinematic planning and dynamic simulation usually rely on software platforms such as Matlab and Adams. These software are good at calculations but can only present the calculation results in the form of curves or simple animations, making it difficult to truly display the physical scene. Digital simulation scenarios can achieve a highly realistic real-time rendering effect using development tools such as Unity3D and Unreal Engine. However, these tools are generally used to produce functional display animations, and these tools themselves do not have the modeling calculation and algorithm debugging capabilities for complex dynamics and control.

[0004] To conduct systematic on-orbit digital simulation analysis for specific complex space operations, in recent years, researchers have developed a virtual visual simulation platform for on-orbit operation, rendezvous and docking, and other mission scenarios based on the FMI (functional mock-up interface) standard. However, the current virtual visual simulation platform mainly provides simulation results of kinematic parameters such as position and velocity, lacking the result support that can reflect dynamic characteristics. And in the on-orbit operation task of dynamic targets, there are simulation difficulties such as arm-base coupling caused by floating bases, multi-arm coupling caused by cooperative operations, and contact coupling caused by grasping dynamic targets. The dynamic and control methods integrated in the simulation platform still face challenges. In addition, the current on-orbit digital simulation technology usually transmits data to the simulation scenario in a single-threaded form, lacking the interactive feedback of the simulation scenario represented by visual signals to the data, and failing to simulate the full-process on-orbit operation process including visual servoing. Summary of the Invention

[0005] To solve at least one of the above problems, the present application proposes a full-process high-real-time digital simulation system for on-orbit operation of dynamic targets.

[0006] According to the first aspect of the present application, at least one embodiment of the present application provides a full-process high-real-time digital simulation system for on-orbit operation of dynamic targets, including: a task planning and management module, configured to receive the operation task of the dynamic target and generate a path planning scheme according to the requirements of the operation task; a visual servo module, connected to the task planning and management module, configured to receive the first control instruction sent by the task planning and management module to perform visual measurement on the dynamic target and output a path planning scheme; a dynamics and control module, respectively connected to the task planning and management module and the visual servo module, configured to receive the second control instruction sent by the task planning and management module and generate motion state data of the manipulation mechanism, the dynamic target, and / or the floating base according to the path planning scheme and / or the operation task sent by the task planning and management module, and / or the path planning scheme sent by the visual servo module, wherein the manipulation mechanism is used to manipulate the dynamic target, and the manipulation mechanism is installed on the floating base; a task display module, respectively connected to the task planning and management module, the visual servo module, and the dynamics and control module, configured to receive the third control instruction sent by the task planning and management module to present a digital simulation scene of the dynamic target according to the motion state data generated by the dynamics and control module and send the camera image of the digital simulation scene to the visual servo module for the visual servo module to use for visual measurement.

[0007] For example, in some embodiments of the present application, the task planning and management module includes: a Cartesian space path planning unit, configured to generate a path planning scheme with a limited speed when the manipulation task includes a speed constraint condition; a joint space path planning unit, configured to generate a path planning scheme for each joint variable when the manipulation task includes a given joint angle constraint condition; an obstacle mode path planning unit, configured to generate a path planning scheme for obstacle avoidance when the manipulation task includes an obstacle mode constraint condition; a joint failure path planning unit, configured to generate a path planning scheme for joint failure when the manipulation task includes a joint failure constraint condition; a task analysis, configuration and scheduling unit, configured to receive the manipulation task of the dynamic target, and according to the content of the manipulation task, allocate the manipulation task to the Cartesian space path planning unit, the joint space path planning unit, the obstacle mode path planning unit, and / or the joint failure path planning unit, and receive the generated path planning scheme from the Cartesian space path planning unit, the joint space path planning unit, the obstacle mode path planning unit, and / or the joint failure path planning unit to which the manipulation task is allocated.

[0008] For example, in some embodiments of the present application, the task analysis, configuration and scheduling unit is connected to the visual servo module, and is configured to send the first control instruction and the category of the dynamic target, and receive the feedback information of the visual servo module; the task analysis, configuration and scheduling unit is connected to the dynamics and control module, and is configured to send the second control instruction, the path planning scheme, and / or the manipulation task, and receive the feedback information of the dynamics and control module; the task analysis, configuration and scheduling unit is connected to the task display module, and is configured to send the third control instruction, and receive the feedback information of the task display module.

[0009] For example, in some embodiments of the present application, the visual servo module includes: a visual measurement unit, configured to determine the relative pose of the dynamic target in the digital simulation scene in the coordinate system of the manipulation mechanism, that is, the expected end pose of the manipulation mechanism; a motion path planning unit, configured to obtain the actual end pose of the manipulation mechanism from the dynamics and control module, calculate the difference between the expected end pose and the actual end pose to obtain the end pose error, and calculate the end velocity according to the end pose error, and obtain a path planning scheme according to the end velocity.

[0010] For example, in some embodiments of the present application, when the dynamic target is a cooperative target, the visual measurement unit is configured to extract features from the target image of the cooperative target in the digital simulation scene to obtain the relative pose of the dynamic target in the coordinate system of the manipulation mechanism.

[0011] For example, in some embodiments of the present application, when the dynamic target is a non-cooperative target, the vision measurement unit is configured to: acquire pixel points of the non-cooperative target in the digital simulation scene, collect point cloud data of the pixel points, and perform filtering processing; for the first-frame point cloud data in the point cloud data, perform rough registration using the random sample consensus algorithm and perform fine registration using the generalized iterative closest point algorithm to obtain the first-frame pose data; for the non-first-frame point cloud data in the point cloud data, perform fine registration using the non-rigid iterative closest point algorithm to obtain the non-first-frame pose data; determine the relative pose of the dynamic target in the coordinate system of the manipulation mechanism according to the first-frame pose data and the non-first-frame pose data.

[0012] For example, in some embodiments of the present application, the dynamics and control module includes: a control implementation strategy unit, connected to the task planning and management module and the vision servo module, for receiving a path planning scheme and / or a manipulation task, and generating a control scheme; a dynamics modeling method unit, respectively connected to the control implementation strategy unit and the task display module, for generating motion state data of the manipulation mechanism, the dynamic target, and / or the floating base according to the control scheme, and outputting the data to the task display module.

[0013] For example, in some embodiments of the present application, the control implementation strategy unit includes: a tracking control sub-unit, for generating a tracking control scheme according to the path planning scheme; a contact control sub-unit, for generating a contact control scheme according to the manipulation task; a de-spin control sub-unit, for generating a de-spin control scheme according to the manipulation task.

[0014] For example, in some embodiments of the present application, the dynamics modeling method unit is configured to: perform modeling using a multi-body dynamics algorithm, for generating motion state data of the manipulation mechanism, the dynamic target, and / or the floating base according to the tracking control scheme, the contact control scheme, and / or the de-spin control scheme; wherein, the multi-body dynamics algorithm includes a contact dynamics algorithm and a de-spin dynamics algorithm.

[0015] For example, in some embodiments of the present application, the task display module includes: a display unit for presenting a digital simulation scene of the dynamic target according to the motion state data; a scene interaction unit for receiving an external input instruction and adjusting the presentation perspective of the digital simulation scene; a vision camera for acquiring a camera image of the digital simulation scene; a network communication unit for connecting to the dynamics and control module and the vision servo module, receiving the motion state data and sending the camera image in the digital simulation scene to the vision servo module; a safety detection unit for sending an alarm message when the motion state data exceeds a set value; and a task interruption unit for interrupting the presentation and / or resetting the digital simulation scene.

[0016] Through the above exemplary embodiments, a full-process high-real-time digital simulation system for on-orbit manipulation of dynamic targets provided by the present application deploys a relatively comprehensive dynamics and control method for simulation difficulties such as arm-base coupling, multi-arm coupling, and contact coupling involved in the on-orbit manipulation task of dynamic targets, can effectively and highly real-time reflect the dynamic characteristics of the system, and at the same time, by setting a vision camera in the digital simulation scene, can simulate the full-process on-orbit manipulation task including vision servo, in order to provide more comprehensive and reliable technical support for the simulation analysis of the on-orbit manipulation task of dynamic targets.

[0017] It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By referring to the accompanying drawings and describing its exemplary embodiments in detail, the above and other objectives, features, and advantages of the present application will become more apparent. The drawings described below are only some embodiments of the present application and do not limit the present application.

[0019] Figure 1 Schematic diagram of a full-process high-real-time digital simulation system for on-orbit manipulation of dynamic targets showing an exemplary embodiment;

[0020] Figure 2 Schematic diagram of a task planning and management module showing an exemplary embodiment;

[0021] Figure 3 Schematic diagram of a vision servo module showing an exemplary embodiment;

[0022] Figure 4 Schematic diagram showing the working principle of the vision servo module;

[0023] Figure 5 Schematic diagram of a dynamics and control module showing an exemplary embodiment;

[0024] Figure 6Schematic diagram showing the control implementation strategy unit of an exemplary embodiment;

[0025] Figure 7 Schematic diagram showing the task display module of an exemplary embodiment;

[0026] Figure 8 Interface image for generating a digital simulation scenario during the operation of the digital simulation system task;

[0027] Figure 9A 、 Figure 9B Simulation calculation result for dynamic target deskewing;

[0028] Figure 10 Schematic diagram of CPU usage time per calculation step for dynamic target deskewing. Detailed implementation manners

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. Like reference numerals in the figures denote like or similar parts, and thus their repeated description will be omitted.

[0030] The features, structures, or characteristics described may be combined in one or more embodiments in any suitable manner. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of these specific details, or can be implemented in other ways, components, materials, devices, etc. In these cases, well-known structures, methods, devices, implementations, materials, or operations will not be shown or described in detail.

[0031] The flowcharts shown in the accompanying drawings are only illustrative and not necessarily include all the contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0032] The terms "first", "second", etc. in the specification, claims, and above-mentioned drawings of the present application are used to distinguish different objects and not to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0033] Those skilled in the art can understand that the attached drawings are only schematic diagrams of exemplary embodiments, and the modules or processes in the attached drawings are not necessarily essential for implementing this application, so they cannot be used to limit the protection scope of this application.

[0034] Figure 1 Schematic diagram of a full-process high-real-time digital simulation system for on-orbit manipulation of dynamic targets showing an exemplary embodiment.

[0035] As Figure 1 shown, the full-process high-real-time digital simulation system includes: a mission planning and management module 101, a vision servo module 102, a dynamics and control module 103, and a mission display module 104.

[0036] Among them, the mission planning and management module 101 is used to receive the manipulation task of the dynamic target and generate a path planning scheme according to the requirements of the manipulation task.

[0037] The vision servo module 102 is connected to the mission planning and management module 101 and is used to receive the first control instruction sent by the mission planning and management module to perform visual measurement on the dynamic target and output a path planning scheme.

[0038] The dynamics and control module 103 is respectively connected to the mission planning and management module 101 and the vision servo module 102 and is used to receive the second control instruction sent by the mission planning and management module 101 and generate the motion state data of the manipulation mechanism, the dynamic target, and / or the floating base according to the path planning scheme and / or the manipulation task sent by the mission planning and management module 101, and / or the path planning scheme sent by the vision servo module 102. Among them, the manipulation mechanism is used to manipulate the dynamic target, and the manipulation mechanism is installed on the floating base.

[0039] The mission display module 104 is respectively connected to the mission planning and management module 101, the vision servo module 102, and the dynamics and control module 103 and is used to receive the third control instruction sent by the mission planning and management module 101 to present the digital simulation scene of the dynamic target according to the motion state data generated by the dynamics and control module 103 and send the camera image of the digital simulation scene to the vision servo module 102 for the vision servo module to use for visual measurement.

[0040] Figure 2 Schematic diagram of the mission planning and management module showing an exemplary embodiment.

[0041] The mission planning and management module is used to carry out mission analysis, configuration, and scheduling according to the requirements of the manipulation task and generate a path planning scheme. As Figure 2As shown in the figure, the task planning and management module 101 includes: a Cartesian space path planning unit 1011, a joint space path planning unit 1012, an obstacle mode path planning unit 1013, a joint failure path planning unit 1014, and a task analysis, configuration and scheduling unit 1015.

[0042] Among them, the Cartesian space path planning unit 1011 is used to generate a path planning scheme with a limited speed when the manipulation task includes speed constraint conditions. The Cartesian space path planning unit 1011 uses a velocity-level inverse kinematics algorithm to perform spatial path planning and avoid kinematic and dynamic singularities during the planning.

[0043] The joint space path planning unit 1012 is used to generate a path planning scheme for each joint variable when the manipulation task includes given joint angle constraint conditions. The joint space path planning unit 1012 generates a change curve for each joint variable according to the constraint conditions of the given joint angles (the positions, velocities, accelerations, etc. of the starting point, ending point or intermediate nodes), and usually uses methods such as fifth-order polynomial or cubic spline interpolation to generate a multi-node path planning scheme between point-to-point and continuous path planning.

[0044] The obstacle mode path planning unit 1013 is used to generate a path planning scheme for obstacle avoidance when the manipulation task includes obstacle mode constraint conditions. The obstacle mode path planning unit 1013 uses methods such as discrete configuration space multi-obstacle avoidance and continuous configuration space collision avoidance to ensure that the arm of the redundant manipulation mechanism itself does not collide with dynamic targets and other obstacles.

[0045] The joint failure path planning unit 1014 is used to generate a path planning scheme for joint failure when the manipulation task includes joint failure constraint conditions. The joint failure path planning unit 1014 simulates the situation where some joints of the manipulation mechanism fail and formulates a path planning scheme in the failure state to ensure the completion of the predetermined task.

[0046] The task analysis, configuration and scheduling unit 1015 is used to receive the manipulation task of the dynamic target, and according to the content of the manipulation task, allocate the manipulation task to the Cartesian space path planning unit 1011, the joint space path planning unit 1012, the obstacle mode path planning unit 1013 and / or the joint failure path planning unit 1014, and receive the generated path planning scheme from the Cartesian space path planning unit 1011, the joint space path planning unit 1012, the obstacle mode path planning unit 1013 and / or the joint failure path planning unit 1014, and send it to the dynamics and control module 103. The task analysis, configuration and scheduling unit 1015 analyzes, decomposes and makes decisions on the tasks at each stage, and realizes the full-process simulation of the on-orbit manipulation task by uniformly scheduling each module.

[0047] The task analysis, configuration and scheduling unit 1015 is connected to the visual servo module 102, and is used to send the first control instruction and the category of the dynamic target, and receive the feedback information of the visual servo module, so as to turn on or off the visual servo module 102 at the corresponding task node.

[0048] Figure 3 Schematic diagram of a visual servo module showing an exemplary embodiment.

[0049] As Figure 3 shown, the visual servo module 102 includes: a visual measurement unit 1021 and a motion path planning unit 1022.

[0050] Among them, the visual measurement unit 1021 is used to determine the relative pose of the dynamic target in the digital simulation scene in the coordinate system of the manipulation mechanism, that is, the expected end pose of the manipulation mechanism.

[0051] According to the exemplary embodiment, as Figure 4 shown, when the dynamic target is a cooperative target, the visual measurement unit 1021 is used to extract features from the target image of the cooperative target in the digital simulation scene, so as to obtain the relative pose of the dynamic target in the coordinate system of the manipulation mechanism.

[0052] According to the exemplary embodiment, when the dynamic target is a non-cooperative target, the visual measurement unit 1021 is used to: acquire the pixel points of the non-cooperative target in the digital simulation scene, collect the point cloud data of the pixel points, and perform filtering processing. For the first frame of point cloud data in the point cloud data, the Random Sample Consensus (RANSAC) algorithm is used for rough registration, and the Generalized Iterative Closest Point (GICP) algorithm is used for fine registration to obtain the first frame of pose data. For the non-first frame of point cloud data in the point cloud data, the Normal Iterative Closest Point (NICP) algorithm is used for fine registration to obtain the non-first frame of pose data. The relative pose of the dynamic target in the coordinate system of the manipulation mechanism is determined according to the first frame of pose data and the non-first frame of pose data.

[0053] The motion path planning unit 1022 is used to obtain the actual end pose of the manipulation mechanism from the dynamics and control module 103, and subtract the relative pose of the dynamic target in the coordinate system of the manipulation mechanism, that is, the expected end pose of the manipulation mechanism, from the actual end pose to obtain the end pose error , where represents the position error, Represents the pose error, and based on the end - pose error, an appropriate weight matrix and proportional coefficient are selected to perform end - velocity planning, and the end - velocity is calculated and obtained. And based on the end - velocity, joint - velocity planning is performed, and the end - velocity is converted into joint - velocity through the Jacobian matrix. A path - planning scheme is obtained.

[0054] The dynamics and control module 103 outputs the motor torque via the control model according to the path - planning scheme. Drive the dynamics simulation, and finally, according to the results of the dynamics simulation, the joint state variables are updated in real - time in the digital simulation scene of the task display module 104. The visual servo module 102 runs continuously in iteration until the dynamic target enters the grasping threshold. The visual servo module 102 receives the first control instruction from the task analysis, configuration, and scheduling unit 1015, shuts down the visual servo module 102, and the task of the visual servo module 102 is completed.

[0055] The task analysis, configuration, and scheduling unit 1015 is connected to the dynamics and control module 103, and is used to send the second control instruction, the path - planning scheme, and / or the manipulation task, and receive the feedback information from the dynamics and control module 103.

[0056] Figure 5 Shows a schematic diagram of the dynamics and control module of an exemplary embodiment.

[0057] As Figure 5 shown, the dynamics and control module 103 includes: a control implementation strategy unit 1031 and a dynamics modeling method unit 1032, which have the simulation capabilities for multi - body dynamics of the manipulation mechanism and its coupling with the base, contact dynamics between the manipulation mechanism and the target, and derotation dynamics of the dynamic target derotation, etc., and can deploy the tracking control method for the planned path and the corresponding contact and derotation control methods to simulate the dynamics characteristics in the on - orbit manipulation task in real - time.

[0058] Among them, the control implementation strategy unit 1031 is connected to the task planning and management module 101 and the visual servo module 102, and is used to receive the path - planning scheme and / or the manipulation task sent by the task planning and management module 101, and / or receive the path - planning scheme sent by the visual servo module connection 102, and generate a control scheme.

[0059] According to some embodiments, as Figure 6 shown, the control implementation strategy unit 1031 includes: a tracking control sub - unit, a contact control sub - unit, and a derotation control sub - unit. Among them, the tracking control sub - unit is used to generate a tracking control scheme according to the path - planning scheme. The contact control sub - unit is used to generate a contact control scheme according to the manipulation task. The derotation control sub - unit is used to generate a derotation control scheme according to the manipulation task.

[0060] According to some embodiments, the control implementation strategy unit 1031 can flexibly adjust the simulation environment configuration according to the actual system computing resources by setting the control cycle and the state sampling cycle. By deploying and scheduling various control strategies, such as PD, sliding mode, model-free fixed time, etc., the control implementation strategy unit 1031 can meet the requirements of different operation tasks for the motion speed, stiffness, compliance and robustness of the control mechanism.

[0061] For example, in response to the smooth control requirements of a dynamic target derotation task, the control implementation strategy unit 1031 designs a variable impedance model based on the characteristics of the target grasping and derotation tasks. This model can handle high-frequency vibrations in grasping collisions, reduce the motor torque at the moment of collision, and dissipate the impact force in a compliant manner in all directions, thereby avoiding impact damage to the control mechanism and the target object.

[0062] For example, when facing a multi-arm collaborative operation task, the control implementation strategy unit 1031 can deploy a polymorphic distributed control method to seek the optimal distribution of operating force by taking the total energy consumption of the control mechanism as the objective function and the torque limit of the control mechanism as the constraint condition.

[0063] The dynamic modeling method unit 1032 is connected to the control implementation strategy unit 1031 and the task display module 104 respectively, and is used to generate motion state data of the control mechanism, dynamic target and / or floating base according to the control scheme, and output it to the task display module 104.

[0064] According to some embodiments, the dynamic modeling method unit 1032 is configured to: use a multi-body dynamics algorithm for modeling, and to generate motion state data of a control mechanism, a dynamic target, and / or a floating base according to a tracking control scheme, a contact control scheme, and / or a de-rotation control scheme. The multi-body dynamics algorithm includes a contact dynamics algorithm and a de-rotation dynamics algorithm.

[0065] According to some embodiments, the dynamic modeling method unit 1032 summarizes the transmission characteristics of typical control mechanisms and adopts a parametric modeling method to achieve fast dynamic modeling configuration of multi-arm control mechanisms. Taking into account the modeling requirements such as arm-base coupling, multi-arm coupling, and contact coupling, the dynamic modeling method unit 1032 adopts the Lagrangian method to establish the system equation, and uses rigid body or continuous collision dynamics based on Hertz contact theory to deal with contact collision problems. The geometric relationship of contact collision and the contact force model are combined to perform the de-rotational dynamics simulation of the dynamic target. In order to ensure the real-time calculation, a low-dimensional equivalent dynamic equation with the least number of degrees of freedom is established according to the transmission characteristics of the control mechanism during modeling. The dynamic modeling method unit 1032 adopts a high-frequency system noise reduction method to avoid high-frequency cycles from slowing down the calculation efficiency, thereby improving the calculation speed.

[0066] The task analysis, configuration and scheduling unit 1015 is connected to the task display module 104, and is used to send a third control instruction and receive the feedback information of the task display module 104.

[0067] Figure 7 The schematic diagram of the task display module showing an exemplary embodiment.

[0068] As Figure 7 shown, the task display module 104 includes: a display unit, a scene interaction unit, a vision camera, a network communication unit, a safety detection unit and a task interruption unit.

[0069] Among them, the display unit is used to present the digital simulation scene of the dynamic target according to the motion state data. The scene interaction unit is used to receive external input instructions and adjust the presentation perspective of the digital simulation scene. For example, through the scene interaction function, the user can use the mouse operation to zoom, rotate and move the perspective, so as to observe from multiple angles and flexibly adjust the viewing range.

[0070] The vision camera is used to acquire the image of the digital simulation scene. The network communication unit is used to connect with the dynamics and control module and the visual servo module, receive the motion state data and send the camera image in the digital simulation scene.

[0071] The safety detection unit is used to send an alarm message when the motion state data exceeds the set value for policy adjustment. The task interruption unit is used to interrupt the presentation and / or reset the digital simulation scene.

[0072] The full-process high-real-time digital simulation system provided by this application is applicable to offline and online modes, no-load and loaded conditions.

[0073] When performing an offline no-load task, the general operation process of the full-process high-real-time digital simulation system is as follows:

[0074] 1. The task planning and management module 101 formulates a task plan according to the task requirements and generates a manipulation mechanism joint motion path planning scheme;

[0075] 2. The dynamics and control module 103 adopts a tracking control algorithm, uses the error between the real joint angles (angular velocity, angular acceleration) calculated by the dynamics simulation and the desired joint angles (angular velocity, angular acceleration) of the path planning as the input, and uses the motor torque as the output to drive the manipulation mechanism joint to perform the predetermined task;

[0076] 3. The manipulation mechanism joint angle information obtained by the dynamics simulation is used as the input and transmitted to the task display module 104 to display the simulation task scene.

[0077] When performing an online loaded task such as dynamic target grasping, the general operation process of the full-process high-real-time digital simulation system is as follows:

[0078] 1. The task analysis, configuration and scheduling unit 1015 formulates a task plan according to the relative pose of the dynamic target.

[0079] 2. According to the task plan generated by the task analysis, configuration and scheduling unit 1015, in the remote control stage, the task planning and management module 101 generates a joint motion path planning scheme for the manipulation mechanism, and the dynamics and control module 103 conducts dynamics simulation based on the tracking control method.

[0080] 3. When the manipulation mechanism approaches the dynamic target, it enters the short-range control stage. The dynamics and control module 103 receives the information issued by the task plan and switches to receiving the path planning scheme generated by the visual servo. The visual servo module 102 guides the manipulation mechanism to grasp the target.

[0081] 4. During the target grasping, the dynamics and control module 103 enables the contact and de-rotation control strategy to realize the grasping and de-rotation dynamics simulation of the manipulation mechanism for the dynamic target.

[0082] 5. After the target is grasped and de-rotated, the task planning and management module 101 formulates a task plan again and issues information to execute operation tasks such as multi-arm collaborative maintenance.

[0083] During operation, the task display module 104 receives information data packets such as joint angles and poses obtained by the dynamics and control module 103 in real time through the TCP / IP communication protocol, and transmits the camera image information of the dynamic target to the visual servo module 102. In this way, the digital drive scenario and the scenario feedback task are used to realize the full-process digital simulation of the on-orbit manipulation of the dynamic target.

[0084] The present application provides a full-process high-real-time digital simulation system for on-orbit manipulation of dynamic targets. A relatively comprehensive dynamics and control method is deployed for simulation difficulties such as arm-base coupling, multi-arm coupling, and contact coupling involved in the on-orbit manipulation task of dynamic targets, which can effectively and highly real-time reflect the dynamic characteristics of the system. At the same time, by setting a visual camera in the digital simulation scenario, it can simulate the full-process on-orbit manipulation task including visual servo, in order to provide more comprehensive and reliable technical support for the simulation analysis of the on-orbit manipulation task of dynamic targets.

[0085] In order to test the simulation ability of the full-process high-real-time digital simulation system provided by the present application for the on-orbit manipulation task of dynamic targets, the modeling and simulation analysis is carried out taking the process of grasping and de-rotating the dynamic target as an example. Figure 8Generate an interface image of a digital simulation scenario during the operation of a digital simulation system. In the figure, the manipulation mechanism is installed on the service satellite and performs on-orbit tasks such as repair and maintenance on the dynamically operating satellite in orbit in a two-arm collaborative form. Considering a dynamically operating satellite with a mass of 1000 kg spinning at a relative angular velocity of 2° / s, simulate the grasping and despinning process of the manipulation mechanism on the dynamically operating satellite: In the remote stage, guide the movement of the manipulation mechanism through the space path planning of the mission planning and management module; in the near stage, enable the vision servo module for dynamic guidance; when the dynamically operating satellite reaches the grasping domain, enable the contact and despinning control strategy.

[0086] As Figure 9A and 9B shown, the results of the dynamic simulation show that the dynamically operating satellite is grasped by the manipulation mechanism at 6.9 s, the relative angular velocity drops within the limit of 0.05° / s at 19.7 s, and the despinning period is 12.8 s. The maximum value of the main joint torque of the manipulation mechanism in the despinning task is about 1.3 Nm, which meets the safety operation requirements according to the conventional motor specifications.

[0087] To test the real-time performance of the digital simulation calculation, offline calculation is performed with an integration step of 5 ms. The CPU usage for each step is as Figure 10 shown. The average value of the CPU consumption within each time period of each step is 0.14 ms, and the peak value is 0.54 ms, which never exceeds the integration step, meeting the real-time performance requirements of the digital simulation calculation.

[0088] It should be clearly understood that this application describes how to form and use specific examples, but this application is not limited to any details of these examples. Instead, based on the teachings of the content disclosed in this application, these principles can be applied to many other embodiments.

[0089] In addition, it should be noted that the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of this application, rather than for the purpose of limitation. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the time sequence of these processes. Additionally, it is also easy to understand that these processes can be executed, for example, synchronously or asynchronously in multiple modules.

[0090] The above specifically shows and describes the exemplary embodiments of this application. It should be understood that this application is not limited to the detailed structures, settings, or implementation methods described here; instead, this application is intended to cover various modifications and equivalent settings included within the spirit and scope of the appended claims.

Claims

1. A full-process high-real-time digital simulation system for on-orbit control of dynamic targets, characterized in that: include: A task planning and management module, used to receive the control task of the dynamic target and generate a path planning solution according to the requirements of the control task; A visual servo module is connected to the task planning and management module and is used to receive a first control instruction sent by the task planning and management module to perform visual measurement on the dynamic target and output a path planning solution, wherein the visual servo module includes: A visual measurement unit, used to determine the relative position and posture of the dynamic target in the digital simulation scene in the control mechanism coordinate system, that is, the expected end position and posture of the control mechanism; The dynamics and control module is connected to the task planning and management module and the visual servo module respectively, and is used to receive the second control instruction sent by the task planning and management module, and generate motion state data of the control mechanism, the dynamic target and / or the floating base according to the path planning scheme and / or the control task sent by the task planning and management module, and / or the path planning scheme sent by the visual servo module, wherein the control mechanism is used to control the dynamic target, and the control mechanism is installed on the floating base. The dynamics and control module deploys and schedules control strategies, including: PD, sliding mode and / or model-free fixed time, to meet the requirements of the operation task for the motion speed, stiffness, compliance and robustness of the control mechanism. The dynamics and control module includes: A control implementation strategy unit, which designs a variable impedance model in the case of a compliant control requirement of a dynamic target derotation task; The dynamic modeling method unit, taking into account the modeling requirements of arm-base coupling, multi-arm coupling, and contact coupling, adopts the Lagrangian method to establish the system equation, uses rigid body or continuous collision dynamics based on Hertz contact theory to deal with contact collision problems, and combines the geometric relationship of contact collision with the contact force model to perform the de-racemonic dynamics simulation of the dynamic target; A task display module is connected to the task planning and management module, the visual servo module and the dynamics and control module respectively, and is used to receive a third control instruction sent by the task planning and management module to present a digital simulation scene of the dynamic target according to the motion state data generated by the dynamics and control module, and send a camera image of the digital simulation scene to the visual servo module so that the visual servo module is used for visual measurement.

2. The full-process high-real-time digital simulation system according to claim 1, characterized in that: The mission planning and management module includes: A Cartesian space path planning unit, configured to generate a path planning scheme with a limited speed when the manipulation task includes a speed constraint; A joint space path planning unit, configured to generate a path planning scheme for each joint variable when the manipulation task includes a given joint angle constraint condition; an obstacle mode path planning unit, configured to generate an obstacle avoidance path planning scheme when the control task includes an obstacle mode constraint condition; a joint failure path planning unit, configured to generate a path planning scheme for a joint failure when the manipulation task includes a joint failure constraint; A task analysis, configuration and scheduling unit is used to receive the manipulation task of the dynamic target, and assign the manipulation task to the Cartesian space path planning unit, the joint space path planning unit, the obstacle mode path planning unit and / or the joint fault path planning unit according to the content of the manipulation task, and receive the generated path planning plan from the manipulation task assigned to the Cartesian space path planning unit, the joint space path planning unit, the obstacle mode path planning unit and / or the joint fault path planning unit.

3. The full-process high-real-time digital simulation system according to claim 2, characterized in that: The task analysis, configuration and scheduling unit is connected to the visual servo module, and is used to send the first control instruction and the category of the dynamic target, and receive feedback information from the visual servo module; The task analysis, configuration and scheduling unit is connected to the dynamics and control module, and is used to send the second control instruction and path planning scheme and / or manipulation task, and receive feedback information from the dynamics and control module; The task analysis, configuration and scheduling unit is connected to the task display module and is used to send the third control instruction and receive feedback information from the task display module.

4. The full-process high-real-time digital simulation system according to claim 3, characterized in that: The visual servo module also includes: A motion path planning unit is used to obtain the actual terminal posture of the control mechanism from the dynamics and control module, to subtract the expected terminal posture from the actual terminal posture to obtain a terminal posture error, to calculate the terminal velocity according to the terminal posture error, and to obtain a path planning solution according to the terminal velocity.

5. The full-process high-real-time digital simulation system according to claim 4, characterized in that: In the case where the dynamic target is a cooperative target, the visual measurement unit is used to extract features from the target image of the cooperative target in the digital simulation scene to obtain the relative position and posture of the dynamic target in the control mechanism coordinate system.

6. The full-process high-real-time digital simulation system according to claim 4, characterized in that: In the case where the dynamic target is a non-cooperative target, the visual measurement unit is used to: Acquire pixel points of non-cooperative targets in the digital simulation scene, collect point cloud data of the pixel points, and perform filtering processing; For the first frame of point cloud data in the point cloud data, a random sampling consistency algorithm is used to perform coarse alignment, and a generalized iterative closest point algorithm is used to perform fine alignment to obtain the first frame of pose data; Using a non-rigid iterative closest point algorithm to perform precise registration on non-first frame point cloud data in the point cloud data to obtain non-first frame pose data; The relative posture of the dynamic target in the coordinates of the control mechanism is determined according to the first frame posture data and the non-first frame posture data.

7. The full-process high-real-time digital simulation system according to claim 1, characterized in that: The control implementation strategy unit is connected to the task planning and management module and the visual servo module, and is used to receive a path planning solution and / or a manipulation task, and generate a control solution; The dynamic modeling method unit is connected to the control implementation strategy unit and the task display module respectively, and is used to generate motion state data of the control mechanism, the dynamic target and / or the floating base according to the control scheme, and output it to the task display module.

8. The full-process high-real-time digital simulation system according to claim 7, characterized in that: The control implementation strategy unit includes: A tracking control subunit, used to generate a tracking control plan according to the path planning plan; A contact control subunit, used for generating a contact control scheme according to the manipulation task; The de-rotation control subunit is used to generate a de-rotation control plan according to the control task.

9. The full-process high-real-time digital simulation system according to claim 8, characterized in that: The kinetic modeling method unit is configured as follows: Modeling is performed using a multi-body dynamics algorithm to generate motion state data of the control mechanism, the dynamic target and / or the floating base according to the tracking control scheme, the contact control scheme and / or the de-rotation control scheme; Wherein, the multi-body dynamics algorithm includes a contact dynamics algorithm and a racemic dynamics algorithm.

10. The full-process high-real-time digital simulation system according to claim 1, characterized in that: The task display module includes: A display unit, used for presenting a digital simulation scene of the dynamic target according to the motion state data; A scene interaction unit, used to receive external input instructions and adjust the presentation viewing angle of the digital simulation scene; A visual camera, used to obtain a camera image of the digital simulation scene; A network communication unit, used to connect with the dynamics and control module and the visual servo module, receive the motion state data and send the camera image in the digital simulation scene to the visual servo module; A safety detection unit, used to send an alarm message when the motion state data exceeds a set value; A task interruption unit is used to interrupt the presentation and / or reset the digital simulation scene.

Images