Full-process high-real-time digital simulation system for dynamic target on-orbit control

Abstract

The invention provides a full-process high-real-time digital simulation system for dynamic target on-orbit control, and relates to the technical field of dynamic target on-orbit control. The digital simulation system comprises a task planning and management module used for receiving a control task and generating a path planning scheme; the visual servo module is used for performing visual measurement on a dynamic target and outputting a path planning scheme; the dynamics and control module is used for generating motion state data of a control mechanism, a dynamic target and / or a floating base according to a path planning scheme and / or a control task sent by the task planning and management module and / or the visual servo module; and the task display module is used for presenting a digital simulation scene of the dynamic target according to the motion state data, and sending a camera image of the digital simulation scene to the visual servo module, so that the visual servo module is used for visual measurement. According to the system, the dynamic characteristics of the system can be effectively reflected in real time, and technical support is provided for simulation analysis of an on-orbit control task of a dynamic target.

Description

Technical Field

[0001] The present application relates to the technical field of on-orbit control of dynamic targets, and in particular to a full-process high-real-time digital simulation system for on-orbit control of dynamic targets. Background Art

[0002] Dynamic target on-orbit manipulation plays a key role in tasks such as space station assembly and construction, spacecraft docking and maintenance, and is an important technology for ensuring space safety and promoting space cooperation. With the rapid development of aerospace technology, the design of spacecraft systems has become increasingly complex. Given the long cycle and high cost of space experiments, ground experiments can only test typical working conditions but cannot fully test the on-orbit status. Digital simulation of on-orbit manipulation tasks has become an important part of testing mission indicators and technical feasibility. In on-orbit manipulation tasks, complex space operations such as dynamic target capture are hot research topics internationally. Faced with such challenging tasks, the importance of digital simulation technology has become increasingly prominent.

[0003] On-orbit digital simulation technology has application prospects such as pre-mission simulation analysis, in-mission status monitoring, and on-orbit fault analysis. Over the years, its modeling and development have made certain progress. For example, the American AGI company has developed the STK (satellite tool kit) commercial analysis software, and NASA has led the development of the GMAT (general mission analysis tool) general space mission simulation tool. These software support visual scene simulation of space missions and are suitable for simulating general mission scenarios such as spacecraft design, launch, and orbital motion, but they cannot provide support for process simulation and real-time calculation of complex space operations. If complex space operation processes are simulated, traditional kinematic planning and dynamic simulation usually rely on software platforms such as Matlab and Adams. These software are good at calculations but can only present calculation results in the form of curves or simple animations, and it is difficult to truly display physical scenes; digital simulation scenes can use development tools such as Unity3D and Unreal Engine to achieve highly realistic real-time rendering effects, but these tools are generally used to make functional display animations, and these tools themselves do not have the modeling, calculation, and algorithm debugging capabilities of complex dynamics and control.

[0004] In order to conduct systematic on-orbit digital simulation analysis for specific complex space operations, researchers have developed virtual vision simulation platforms for on-orbit control, rendezvous and docking mission scenarios based on the FMI (functional mock-up interface) standard in recent years. However, the current virtual vision simulation platform mainly provides simulation results of kinematic parameters such as position and velocity, and lacks the support of results that can reflect dynamic characteristics. In addition, in the on-orbit control task of dynamic targets, there are simulation difficulties such as arm-base coupling caused by floating base, multi-arm coupling caused by collaborative operation, and contact coupling caused by dynamic target grasping. The dynamics and control methods integrated in the simulation platform still face challenges. In addition, the current on-orbit digital simulation technology usually transmits data to the simulation scene in a single-threaded form, lacks interactive feedback of the simulation scene to the data represented by visual signals, and fails to simulate the full process of on-orbit control including visual servoing. Summary of the invention

[0005] In order to solve at least one of the above problems, the present application proposes a full-process high-real-time digital simulation system for on-orbit control of dynamic targets.

[0006] According to the first aspect of the present application, at least one embodiment of the present application provides a full-process high-real-time digital simulation system for on-orbit manipulation of dynamic targets, including: a task planning and management module, used to receive the manipulation task of the dynamic target, and generate a path planning solution according to the requirements of the manipulation task; a visual servo module, connected to the task planning and management module, used to receive a first control instruction sent by the task planning and management module to perform visual measurement of the dynamic target and output a path planning solution; a dynamics and control module, respectively connected to the task planning and management module and the visual servo module, used to receive a second control instruction sent by the task planning and management module, and generate a path planning solution according to the requirements of the manipulation task; a visual servo module, connected to the task planning and management module and the visual servo module, used to receive a second control instruction sent by the task planning and management module, and generate a path planning solution according to the requirements of the manipulation task; A path planning scheme and / or a manipulation task, and / or a path planning scheme sent by the visual servo module, generates motion state data of a manipulation mechanism, the dynamic target and / or a floating base; wherein the manipulation mechanism is used to manipulate the dynamic target, and the manipulation mechanism is installed on the floating base; a task display module, which is respectively connected to the task planning and management module, the visual servo module and the dynamics and control module, is used to receive a third control instruction sent by the task planning and management module, so as to present a digital simulation scene of the dynamic target according to the motion state data generated by the dynamics and control module, and send a camera image of the digital simulation scene to the visual servo module, so that the visual servo module is used for visual measurement.

[0007] For example, in some embodiments of the present application, the task planning and management module includes: a Cartesian space path planning unit, which is used to generate a path planning scheme with a limited speed when the manipulation task includes a speed constraint; a joint space path planning unit, which is used to generate a path planning scheme for each joint variable when the manipulation task includes a given joint angle constraint; an obstacle mode path planning unit, which is used to generate an obstacle avoidance path planning scheme when the manipulation task includes an obstacle mode constraint; a joint fault path planning unit, which is used to generate a path planning scheme for joint failure when the manipulation task includes a joint fault constraint; a task analysis, configuration and scheduling unit, which is used to receive the manipulation task of the dynamic target, and assign the manipulation task to the Cartesian space path planning unit, the joint space path planning unit, the obstacle mode path planning unit and / or the joint fault path planning unit according to the content of the manipulation task, and receive the generated path planning scheme from the manipulation task assigned to the Cartesian space path planning unit, the joint space path planning unit, the obstacle mode path planning unit and / or the joint fault path planning unit.

[0008] For example, in some embodiments of the present application, the task analysis, configuration and scheduling unit is connected to the visual servo module to send the first control instruction and the category of the dynamic target, and receive feedback information from the visual servo module; the task analysis, configuration and scheduling unit is connected to the dynamics and control module to send the second control instruction and path planning scheme and / or manipulation task, and receive feedback information from the dynamics and control module; the task analysis, configuration and scheduling unit is connected to the task display module to send the third control instruction and receive feedback information from the task display module.

[0009] For example, in some embodiments of the present application, the visual servo module includes: a visual measurement unit, used to determine the relative pose of the dynamic target in the digital simulation scene in the coordinate system of the control mechanism, that is, the expected end pose of the control mechanism; a motion path planning unit, used to obtain the actual end pose of the control mechanism from the dynamics and control module, subtract the expected end pose from the actual end pose to obtain the end pose error, and used to calculate the end velocity according to the end pose error, and used to obtain a path planning scheme according to the end velocity.

[0010] For example, in some embodiments of the present application, when the dynamic target is a cooperative target, the visual measurement unit is used to extract features of the target image of the cooperative target in the digital simulation scene to obtain the relative position and posture of the dynamic target in the coordinate system of the control mechanism.

[0011] For example, in some embodiments of the present application, when the dynamic target is a non-cooperative target, the visual measurement unit is used to: obtain pixel points of the non-cooperative target in the digital simulation scene, and collect point cloud data of the pixel points for filtering; for the first frame of point cloud data in the point cloud data, use a random sampling consistency algorithm to perform coarse alignment, and use a generalized iterative closest point algorithm to perform fine alignment to obtain first frame pose data; for non-first frame point cloud data in the point cloud data, use a non-rigid iterative closest point algorithm to perform fine alignment to obtain non-first frame pose data; and determine the relative pose of the dynamic target in the coordinates of the control mechanism according to the first frame pose data and the non-first frame pose data.

[0012] For example, in some embodiments of the present application, the dynamics and control module includes: a control implementation strategy unit, which is connected to the task planning and management module and the visual servo module, and is used to receive path planning schemes and / or manipulation tasks, and generate control schemes; a dynamic modeling method unit, which is respectively connected to the control implementation strategy unit and the task display module, and is used to generate motion state data of the manipulation mechanism, the dynamic target and / or the floating base according to the control scheme, and output it to the task display module.

[0013] For example, in some embodiments of the present application, the control implementation strategy unit includes: a tracking control subunit, used to generate a tracking control scheme according to a path planning scheme; a contact control subunit, used to generate a contact control scheme according to a manipulation task; and a de-rotation control subunit, used to generate a de-rotation control scheme according to a manipulation task.

[0014] For example, in some embodiments of the present application, the dynamic modeling method unit is configured to: use a multi-body dynamics algorithm for modeling, for generating motion state data of the control mechanism, the dynamic target and / or the floating base according to the tracking control scheme, the contact control scheme and / or the de-rotation control scheme; wherein the multi-body dynamics algorithm includes a contact dynamics algorithm and a de-rotation dynamics algorithm.

[0015] For example, in some embodiments of the present application, the task display module includes: a display unit, used to present a digital simulation scene of the dynamic target according to the motion state data; a scene interaction unit, used to receive external input instructions and adjust the presentation perspective of the digital simulation scene; a visual camera, used to obtain a camera image of the digital simulation scene; a network communication unit, used to connect with the dynamics and control module and the visual servo module, receive the motion state data and send the camera image in the digital simulation scene to the visual servo module; a safety detection unit, used to send an alarm message when the motion state data exceeds a set value; a task interruption unit, used to interrupt the presentation and / or reset the digital simulation scene.

[0016] Through the above-mentioned example embodiments, the present application provides a full-process high-real-time digital simulation system for on-orbit manipulation of dynamic targets. Aiming at the simulation difficulties such as arm-base coupling, multi-arm coupling, contact coupling, etc. involved in the on-orbit manipulation tasks of dynamic targets, a relatively comprehensive dynamics and control method is deployed, which can effectively and highly real-timely reflect the dynamic characteristics of the system. At the same time, by setting a visual camera in the digital simulation scene, it can simulate the full-process on-orbit manipulation tasks including visual servoing, in order to provide more comprehensive and reliable technical support for the simulation analysis of on-orbit manipulation tasks of dynamic targets.

[0017] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing in detail exemplary embodiments thereof with reference to the accompanying drawings, the above and other objects, features and advantages of the present application will become more apparent. The accompanying drawings described below are only some embodiments of the present application, and are not intended to limit the present application.

[0019] Figure 1 A schematic diagram of a full-process high-real-time digital simulation system for on-orbit control of a dynamic target according to an exemplary embodiment is shown; Figure 2 A schematic diagram illustrating a mission planning and management module of an exemplary embodiment; Figure 3 A schematic diagram showing a visual servo module of an exemplary embodiment; Figure 4 Shows the working principle block diagram of the visual servo module; Figure 5 A schematic diagram showing a dynamics and control module of an exemplary embodiment; Figure 6 A schematic diagram showing a control implementation strategy unit of an exemplary embodiment; Figure 7A schematic diagram showing a task display module of an exemplary embodiment; Figure 8 Generate an interface image of a digital simulation scene when the digital simulation system task is running; Fig. 9A , Fig. 9B The simulation results of the dynamic target deracination are shown below; Fig.10 Schematic diagram of CPU usage time per calculation step for dynamic target deracination. DETAILED DESCRIPTION

[0020] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0021] The described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of these specific details, or other modes, components, materials, devices, etc. may be adopted. In these cases, known structures, methods, devices, implementations, materials or operations will not be shown or described in detail.

[0022] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0023] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0024] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present application, and therefore cannot be used to limit the scope of protection of the present application.

[0025] Figure 1 A schematic diagram of a full-process high-real-time digital simulation system for on-orbit manipulation of a dynamic target according to an exemplary embodiment is shown.

[0026] like Figure 1 As shown, the full-process high-real-time digital simulation system includes: a task planning and management module 101, a visual servoing module 102, a dynamics and control module 103 and a task display module 104.

[0027] The task planning and management module 101 is used to receive a dynamic target manipulation task and generate a path planning solution according to the requirements of the manipulation task.

[0028] The visual servo module 102 is connected to the task planning and management module 101 and is used to receive a first control instruction sent by the task planning and management module to perform visual measurement on a dynamic target and output a path planning solution.

[0029] The dynamics and control module 103 is connected to the task planning and management module 101 and the visual servo module 102 respectively, and is used to receive the second control instruction sent by the task planning and management module 101, and generate motion state data of the control mechanism, dynamic target and / or floating base according to the path planning scheme and / or control task sent by the task planning and management module 101, and / or the path planning scheme sent by the visual servo module 102. Among them, the control mechanism is used to control the dynamic target, and the control mechanism is installed on the floating base.

[0030] The task display module 104 is connected to the task planning and management module 101, the visual servo module 102 and the dynamics and control module 103 respectively, and is used to receive the third control instruction sent by the task planning and management module 101, so as to present a digital simulation scene of the dynamic target according to the motion state data generated by the dynamics and control module 103, and send the camera image of the digital simulation scene to the visual servo module 102, so that the visual servo module can be used for visual measurement.

[0031] Figure 2 A schematic diagram illustrating a mission planning and management module of an exemplary embodiment.

[0032] The task planning and management module is used to carry out task analysis, configuration and scheduling according to the requirements of the control task, and generate a path planning solution. Figure 2 As shown, the task planning and management module 101 includes: a Cartesian space path planning unit 1011, a joint space path planning unit 1012, an obstacle mode path planning unit 1013, a joint fault path planning unit 1014 and a task analysis, configuration and scheduling unit 1015.

[0033] The Cartesian space path planning unit 1011 is used to generate a speed-limited path planning scheme when the control task includes speed constraints. The Cartesian space path planning unit 1011 uses a speed-level inverse kinematics algorithm to perform space path planning and avoid kinematic and dynamic singular points in the planning.

[0034] The joint space path planning unit 1012 is used to generate a path planning scheme for each joint variable when the manipulation task includes a given joint angle constraint. The joint space path planning unit 1012 generates a change curve for each joint variable according to the given joint angle constraint (position, velocity, acceleration, etc. of the starting point, end point or intermediate node), and usually uses a quintic polynomial or cubic spline interpolation method to generate a multi-node path planning scheme between point-to-point and continuous path planning.

[0035] The obstacle mode path planning unit 1013 is used to generate an obstacle avoidance path planning scheme when the control task includes obstacle mode constraints. The obstacle mode path planning unit 1013 uses methods such as discrete configuration space multi-obstacle avoidance and continuous configuration space collision avoidance to ensure that the redundant control mechanism's own arm does not collide with dynamic targets and other obstacles.

[0036] The joint failure path planning unit 1014 is used to generate a path planning scheme for joint failure when the manipulation task includes joint failure constraints. The joint failure path planning unit 1014 simulates the situation where some joints of the manipulation mechanism fail, and formulates a path planning scheme under the failure state to ensure the completion of the predetermined task.

[0037] The task analysis, configuration and scheduling unit 1015 is used to receive the manipulation task of the dynamic target, and according to the content of the manipulation task, allocate the manipulation task to the Cartesian space path planning unit 1011, the joint space path planning unit 1012, the obstacle mode path planning unit 1013 and / or the joint fault path planning unit 1014, and receive the generated path planning scheme from the Cartesian space path planning unit 1011, the joint space path planning unit 1012, the obstacle mode path planning unit 1013 and / or the joint fault path planning unit 1014, and send it to the dynamics and control module 103. The task analysis, configuration and scheduling unit 1015 analyzes, decomposes and makes decisions on the tasks at each stage, and realizes the full process simulation of the on-orbit manipulation task by uniformly scheduling each module.

[0038] The task analysis, configuration and scheduling unit 1015 is connected to the visual servo module 102, and is used to send the first control instruction and the category of the dynamic target, and receive feedback information from the visual servo module to turn on or off the visual servo module 102 at the corresponding task node.

[0039] Figure 3 A schematic diagram showing a visual servo module of an exemplary embodiment.

[0040] like Figure 3 As shown, the visual servo module 102 includes: a visual measurement unit 1021 and a motion path planning unit 1022 .

[0041] The visual measurement unit 1021 is used to determine the relative position and posture of the dynamic target in the digital simulation scene in the control mechanism coordinate system, that is, the expected end position and posture of the control mechanism.

[0042] According to an example embodiment, Figure 4 As shown, in the case where the dynamic target is a cooperative target, the visual measurement unit 1021 is used to extract features from the target image of the cooperative target in the digital simulation scene to obtain the relative position and posture of the dynamic target in the control mechanism coordinate system.

[0043] According to an example embodiment, when the dynamic target is a non-cooperative target, the visual measurement unit 1021 is used to: obtain the pixel points of the non-cooperative target in the digital simulation scene, and collect the point cloud data of the pixel points for filtering. For the first frame of point cloud data in the point cloud data, the random sample consensus algorithm (RANSAC) is used for coarse alignment, and the generalized iterative closest point algorithm (GICP) is used for fine alignment to obtain the first frame pose data. For the non-first frame point cloud data in the point cloud data, the non-rigid iterative closest point algorithm (NICP) is used for fine alignment to obtain the non-first frame pose data. The relative pose of the dynamic target in the control mechanism coordinate system is determined based on the first frame pose data and the non-first frame pose data.

[0044] The motion path planning unit 1022 is used to obtain the real terminal posture of the control mechanism from the dynamics and control module 103, and to obtain the terminal posture error by subtracting the relative posture of the dynamic target in the control mechanism coordinate system, that is, the expected terminal posture of the control mechanism from the real terminal posture. ,in, represents the position error, Represents the attitude error, and according to the terminal posture error, select the appropriate weight matrix and proportional coefficient to plan the terminal velocity and calculate the terminal velocity , and according to the terminal velocity, perform joint velocity planning, and convert the terminal velocity into joint velocity through the Jacobian matrix , and obtain the path planning solution.

[0045] The dynamics and control module 103 outputs the motor torque through the control model according to the path planning scheme Drive dynamics simulation, and finally update joint state variables in real time in the digital simulation scene of task display module 104 according to the dynamics simulation results The visual servo module 102 continues to iterate until the dynamic target enters the grasping threshold, and the visual servo module 102 receives the first control instruction of the task analysis, configuration and scheduling unit 1015, closes the visual servo module 102, and the task of the visual servo module 102 is completed.

[0046] The task analysis, configuration and scheduling unit 1015 is connected to the dynamics and control module 103 and is used to send a second control instruction, a path planning solution and / or a manipulation task, and receive feedback information from the dynamics and control module 103 .

[0047] Figure 5 A schematic diagram of the dynamics and control module of an exemplary embodiment is shown.

[0048] like Figure 5 As shown, the dynamics and control module 103 includes: a control implementation strategy unit 1031 and a dynamics modeling method unit 1032, which has the simulation capabilities of multi-body dynamics of the control mechanism and its coupling with the base, contact dynamics between the control mechanism and the target, and derotation dynamics of dynamic target derotation. It can deploy tracking control methods for planned paths and corresponding contact and derotation control methods to simulate the dynamic characteristics of on-orbit control tasks in real time.

[0049] Among them, the control implementation strategy unit 1031 is connected to the task planning and management module 101 and the visual servo module 102, and is used to receive the path planning plan and / or control task sent by the task planning and management module 101, and / or receive the path planning plan sent by the visual servo module 102, and generate a control plan.

[0050] According to some embodiments, Figure 6 As shown, the control implementation strategy unit 1031 includes: a tracking control subunit, a contact control subunit and a de-rotation control subunit. Among them, the tracking control subunit is used to generate a tracking control plan according to the path planning plan. The contact control subunit is used to generate a contact control plan according to the manipulation task. The de-rotation control subunit is used to generate a de-rotation control plan according to the manipulation task.

[0051] According to some embodiments, the control implementation strategy unit 1031 can flexibly adjust the simulation environment configuration according to the actual system computing resources by setting the control cycle and the state sampling cycle. By deploying and scheduling various control strategies, such as PD, sliding mode, model-free fixed time, etc., the control implementation strategy unit 1031 can meet the requirements of different operation tasks for the motion speed, stiffness, compliance and robustness of the control mechanism.

[0052] For example, in response to the smooth control requirements of a dynamic target derotation task, the control implementation strategy unit 1031 designs a variable impedance model based on the characteristics of the target grasping and derotation tasks. This model can handle high-frequency vibrations in grasping collisions, reduce the motor torque at the moment of collision, and dissipate the impact force in a compliant manner in all directions, thereby avoiding impact damage to the control mechanism and the target object.

[0053] For example, when facing a multi-arm collaborative operation task, the control implementation strategy unit 1031 can deploy a polymorphic distributed control method to seek the optimal distribution of operating force by taking the total energy consumption of the control mechanism as the objective function and the torque limit of the control mechanism as the constraint condition.

[0054] The dynamic modeling method unit 1032 is connected to the control implementation strategy unit 1031 and the task display module 104 respectively, and is used to generate motion state data of the control mechanism, dynamic target and / or floating base according to the control scheme, and output it to the task display module 104.

[0055] According to some embodiments, the dynamic modeling method unit 1032 is configured to: use a multi-body dynamics algorithm for modeling, and to generate motion state data of a control mechanism, a dynamic target, and / or a floating base according to a tracking control scheme, a contact control scheme, and / or a de-rotation control scheme. The multi-body dynamics algorithm includes a contact dynamics algorithm and a de-rotation dynamics algorithm.

[0056] According to some embodiments, the dynamic modeling method unit 1032 summarizes the transmission characteristics of typical control mechanisms and adopts a parametric modeling method to achieve fast dynamic modeling configuration of multi-arm control mechanisms. Taking into account the modeling requirements such as arm-base coupling, multi-arm coupling, and contact coupling, the dynamic modeling method unit 1032 adopts the Lagrangian method to establish the system equation, and uses rigid body or continuous collision dynamics based on Hertz contact theory to deal with contact collision problems. The geometric relationship of contact collision and the contact force model are combined to perform the de-rotational dynamics simulation of the dynamic target. In order to ensure the real-time calculation, a low-dimensional equivalent dynamic equation with the least number of degrees of freedom is established according to the transmission characteristics of the control mechanism during modeling. The dynamic modeling method unit 1032 adopts a high-frequency system noise reduction method to avoid high-frequency cycles from slowing down the calculation efficiency, thereby improving the calculation speed.

[0057] The task analysis, configuration and scheduling unit 1015 is connected to the task display module 104 and is used to send a third control instruction and receive feedback information from the task display module 104 .

[0058] Figure 7 A schematic diagram illustrating a task display module according to an exemplary embodiment.

[0059] like Figure 7 As shown, the task display module 104 includes: a display unit, a scene interaction unit, a visual camera, a network communication unit, a safety detection unit and a task interruption unit.

[0060] The display unit is used to present a digital simulation scene of a dynamic target according to the motion state data. The scene interaction unit is used to receive external input instructions and adjust the presentation perspective of the digital simulation scene. For example, through the scene interaction function, the user can use the mouse to zoom, rotate and move the perspective, so as to observe from multiple angles and flexibly adjust the field of view.

[0061] The visual camera is used to obtain the image of the digital simulation scene. The network communication unit is used to connect with the dynamics and control module and the visual servo module, receive the motion state data and send the camera image in the digital simulation scene.

[0062] The safety detection unit is used to send an alarm message when the motion state data exceeds a set value so as to adjust the strategy. The task interruption unit is used to interrupt the presentation and / or reset the digital simulation scene.

[0063] The full-process high real-time digital simulation system provided in this application is suitable for offline and online modes, no-load and loaded conditions.

[0064] When performing offline no-load tasks, the general operation process of the full-process high-real-time digital simulation system is as follows: 1. The task planning and management module 101 formulates a task plan according to task requirements and generates a joint motion path planning scheme for the control mechanism; 2. The dynamics and control module 103 adopts a tracking control algorithm, takes the error between the actual joint angle (angular velocity, angular acceleration) calculated by dynamics simulation and the expected joint angle (angular velocity, angular acceleration) of path planning as input, and takes the motor torque as output to drive the joint movement of the control mechanism to perform the predetermined task; 3. The joint angle information of the control mechanism obtained by the dynamic simulation is used as input and transmitted to the task display module 104 to display the simulation task scene.

[0065] When performing online load-carrying tasks such as dynamic target grabbing, the general operation process of the full-process high-real-time digital simulation system is as follows: 1. The task analysis, configuration and scheduling unit 1015 formulates the task plan according to the relative position and posture of the dynamic target; 2. According to the task plan generated by the task analysis, configuration and scheduling unit 1015, in the remote control stage, the task planning and management module 101 generates a joint motion path planning scheme for the control mechanism, and performs dynamic simulation based on the tracking control method in the dynamics and control module 103; 3. When the control mechanism approaches the dynamic target, it enters the short-range control stage. The dynamics and control module 103 receives the task planning information and switches to receiving the path planning solution generated by the visual servo. The visual servo module 102 guides the control mechanism to grasp the target. 4. The dynamics and control module 103 uses the contact and de-rotation control strategy to realize the dynamic simulation of the grasping and de-rotation of the dynamic target by the control mechanism during the target grasping; 5. After the target is captured and eliminated, the task planning and management module 101 formulates the task plan again and issues information to perform multi-arm collaborative maintenance and other operational tasks.

[0066] During operation, the task display module 104 receives the information data packets such as joint angles and postures obtained by the dynamics and control module 103 in real time through the TCP / IP communication protocol, and transmits the camera image information of the dynamic target to the visual servo module 102. In this way, the scene is driven digitally and the task is fed back by the scene, so as to realize the full process digital simulation of the on-orbit control of the dynamic target.

[0067] The present application provides a full-process, high-real-time digital simulation system for on-orbit manipulation of dynamic targets. In view of the simulation difficulties such as arm-base coupling, multi-arm coupling, and contact coupling involved in the on-orbit manipulation tasks of dynamic targets, a relatively comprehensive dynamics and control method is deployed, which can effectively and highly real-timely reflect the dynamic characteristics of the system. At the same time, by setting a visual camera in the digital simulation scene, the full-process on-orbit manipulation tasks including visual servoing can be simulated, in order to provide more comprehensive and reliable technical support for the simulation analysis of on-orbit manipulation tasks of dynamic targets.

[0068] In order to test the simulation capability of the full-process high-real-time digital simulation system provided in this application for the dynamic target on-orbit control task, a modeling simulation analysis is carried out using the dynamic target capture and derotation process as an example. Figure 8The interface image of the digital simulation scene is generated when the digital simulation system is running. In the figure, the control mechanism is mounted on the service satellite, and performs on-orbit tasks such as repair and maintenance on the dynamic satellite in orbit in the form of dual-arm collaboration. Considering a dynamic satellite with a mass of 1000kg spinning at a relative angular velocity of 2° / s, the process of the control mechanism grabbing and despinning the dynamic satellite is simulated: in the long-range stage, the space path planning of the mission planning and management module guides the movement of the control mechanism; in the short-range stage, the visual servo module is turned on for dynamic guidance; when the dynamic satellite reaches the grasping domain, the contact and despinning control strategy is enabled.

[0069] like Fig. 9A and 9B As shown in the figure, the dynamic simulation results show that the dynamic satellite was captured by the control mechanism at 6.9s, the relative angular velocity dropped to within the limit of 0.05° / s at 19.7s, and the derotation period was 12.8s. The maximum torque of the main joints of the control mechanism in the derotation task was about 1.3Nm, which met the safety operation requirements according to the conventional motor indicators.

[0070] In order to verify the real-time performance of digital simulation calculations, offline calculations were performed with an integration step length of 5 ms. The CPU usage for each step length is shown in the following figure. Fig.10 As shown in the figure, the average CPU consumption in each time period is 0.14ms, and the peak value is 0.54ms, which never exceeds the integral step length, meeting the real-time requirements of digital simulation calculations.

[0071] It should be clearly understood that the present application describes how to form and use specific examples, but the present application is not limited to any details of these examples. On the contrary, based on the teaching of the content disclosed in the present application, these principles can be applied to many other embodiments.

[0072] In addition, it should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0073] The exemplary embodiments of the present application are specifically shown and described above. It should be understood that the present application is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present application is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A full-process high-real-time digital simulation system for on-orbit control of dynamic targets, characterized in that: include: A task planning and management module, used to receive the control task of the dynamic target and generate a path planning solution according to the requirements of the control task; A visual servo module, connected to the task planning and management module, for receiving a first control instruction sent by the task planning and management module to perform visual measurement on the dynamic target and output a path planning solution; a dynamics and control module, connected to the task planning and management module and the visual servo module respectively, for receiving a second control instruction sent by the task planning and management module, and generating motion state data of a control mechanism, the dynamic target and / or a floating base according to a path planning scheme and / or a control task sent by the task planning and management module, and / or a path planning scheme sent by the visual servo module; wherein the control mechanism is used to control the dynamic target, and the control mechanism is installed on the floating base; A task display module is connected to the task planning and management module, the visual servo module and the dynamics and control module respectively, and is used to receive a third control instruction sent by the task planning and management module to present a digital simulation scene of the dynamic target according to the motion state data generated by the dynamics and control module, and send a camera image of the digital simulation scene to the visual servo module so that the visual servo module is used for visual measurement.

2. The full-process high-real-time digital simulation system according to claim 1, characterized in that: The mission planning and management module includes: A Cartesian space path planning unit, configured to generate a path planning scheme with a limited speed when the manipulation task includes a speed constraint; A joint space path planning unit, configured to generate a path planning scheme for each joint variable when the manipulation task includes a given joint angle constraint condition; an obstacle mode path planning unit, configured to generate an obstacle avoidance path planning scheme when the control task includes an obstacle mode constraint condition; a joint failure path planning unit, configured to generate a path planning scheme for a joint failure when the manipulation task includes a joint failure constraint; A task analysis, configuration and scheduling unit is used to receive the manipulation task of the dynamic target, and assign the manipulation task to the Cartesian space path planning unit, the joint space path planning unit, the obstacle mode path planning unit and / or the joint fault path planning unit according to the content of the manipulation task, and receive the generated path planning plan from the manipulation task assigned to the Cartesian space path planning unit, the joint space path planning unit, the obstacle mode path planning unit and / or the joint fault path planning unit.

3. The full-process high-real-time digital simulation system according to claim 2, characterized in that: The task analysis, configuration and scheduling unit is connected to the visual servo module, and is used to send the first control instruction and the category of the dynamic target, and receive feedback information from the visual servo module; The task analysis, configuration and scheduling unit is connected to the dynamics and control module, and is used to send the second control instruction and path planning scheme and / or manipulation task, and receive feedback information from the dynamics and control module; The task analysis, configuration and scheduling unit is connected to the task display module and is used to send the third control instruction and receive feedback information from the task display module.

4. The full-process high-real-time digital simulation system according to claim 3, characterized in that: The visual servo module comprises: A visual measurement unit, used to determine the relative position and posture of the dynamic target in the digital simulation scene in the control mechanism coordinate system, that is, the expected end position and posture of the control mechanism; A motion path planning unit is used to obtain the actual terminal posture of the control mechanism from the dynamics and control module, to subtract the expected terminal posture from the actual terminal posture to obtain a terminal posture error, to calculate the terminal velocity according to the terminal posture error, and to obtain a path planning solution according to the terminal velocity.

5. The full-process high-real-time digital simulation system according to claim 4, characterized in that: In the case where the dynamic target is a cooperative target, the visual measurement unit is used to extract features from the target image of the cooperative target in the digital simulation scene to obtain the relative position and posture of the dynamic target in the control mechanism coordinate system.

6. The full-process high-real-time digital simulation system according to claim 4, characterized in that: In the case where the dynamic target is a non-cooperative target, the visual measurement unit is used to: Acquire pixel points of non-cooperative targets in the digital simulation scene, collect point cloud data of the pixel points, and perform filtering processing; For the first frame of point cloud data in the point cloud data, a random sampling consistency algorithm is used to perform coarse alignment, and a generalized iterative closest point algorithm is used to perform fine alignment to obtain the first frame of pose data; Using a non-rigid iterative closest point algorithm to perform precise registration on non-first frame point cloud data in the point cloud data to obtain non-first frame pose data; The relative posture of the dynamic target in the coordinates of the control mechanism is determined according to the first frame posture data and the non-first frame posture data.

7. The full-process high-real-time digital simulation system according to claim 1, characterized in that: The dynamics and control module includes: A control implementation strategy unit, connected to the task planning and management module and the visual servo module, for receiving a path planning scheme and / or a manipulation task and generating a control scheme; The dynamic modeling method unit is connected to the control implementation strategy unit and the task display module respectively, and is used to generate motion state data of the control mechanism, the dynamic target and / or the floating base according to the control scheme, and output it to the task display module.

8. The full-process high-real-time digital simulation system according to claim 7, characterized in that: The control implementation strategy unit includes: A tracking control subunit, used to generate a tracking control plan according to the path planning plan; A contact control subunit, used for generating a contact control scheme according to the manipulation task; The de-rotation control subunit is used to generate a de-rotation control plan according to the control task.

9. The full-process high-real-time digital simulation system according to claim 8, characterized in that: The kinetic modeling method unit is configured as follows: Modeling is performed using a multi-body dynamics algorithm to generate motion state data of the control mechanism, the dynamic target and / or the floating base according to the tracking control scheme, the contact control scheme and / or the de-rotation control scheme; Wherein, the multi-body dynamics algorithm includes a contact dynamics algorithm and a racemic dynamics algorithm.

10. The full-process high-real-time digital simulation system according to claim 1, characterized in that: The task display module includes: A display unit, used for presenting a digital simulation scene of the dynamic target according to the motion state data; A scene interaction unit, used to receive external input instructions and adjust the presentation viewing angle of the digital simulation scene; A visual camera, used to obtain a camera image of the digital simulation scene; A network communication unit, used to connect with the dynamics and control module and the visual servo module, receive the motion state data and send the camera image in the digital simulation scene to the visual servo module; A safety detection unit, used to send an alarm message when the motion state data exceeds a set value; A task interruption unit is used to interrupt the presentation and / or reset the digital simulation scene.

Images