A robot motion control method and electronic device based on digital twinning

Abstract

The application discloses a kind of robot motion control method and electronic equipment based on digital twinning, belong to robot motion control technical field.The action control scheme of existing robot, it is difficult to realize complex stage performance, leading to robot dance performance effect is not good.A kind of robot motion control method based on digital twinning of the application, by constructing environment acquisition module, map processing module, digital stage twin module, dance action design module, trajectory planning module, dance synthesis module, carry out data interaction and function cooperation, formed an efficient, scalable dance design scheme, ensure that robot can accurately perceive and adapt to the change of stage environment, so that robot action can accurately match scene environment, avoid robot dance action and environment disjunction, while the dance action of robot can be designed, so complex stage performance can be realized, effectively improve the robot dance performance effect.

Description

TECHNICAL FIELD

[0001] The application relates to a robot motion control method and electronic equipment based on digital twinning, and belongs to the technical field of robot motion control. BACKGROUND

[0002] With the rapid development of robot technology, robot performances have developed from simple action demonstrations to complex stage performances.

[0003] Chinese Patent Application (Publication No. CN119839885A) discloses a robot and a robot control method, which comprises a robot body and a rotating performance piece; the robot body is provided with an arm for assembling the rotating performance piece and a throwing motor for throwing performance props; the rotating performance piece is assembled at the end of the arm, and the throwing motor can provide throwing power to give the performance props kinetic energy to fly outward, so that the robot can at least perform rotating, throwing and other dance actions, the actions are rich, the running demonstration effect is good, the performance performance is good, and the robot can be used for humanoid robot dance performance, especially national dance performance.

[0004] The above scheme can realize the action control of the robot, but cannot realize environment perception and action planning, resulting in disconnection between the robot dance action and the environment, so that the robot action cannot accurately match the scene environment, and therefore it is difficult to realize complex stage performances, the robot dance performance effect is not good, and the robot is not conducive to popularization and use.

[0005] The information disclosed in this background section is only for the purpose of understanding the background of the inventive concept, so it can include information that does not constitute the prior art. SUMMARY

[0006] In view of the above problems or one of the above problems, the purpose of the present application is to provide a robot motion control method and electronic equipment based on digital twinning, which acquires point cloud data and an environment map of the environment by constructing an environment acquisition module, a map processing module, a digital stage twinning module, a dance action design module, a trajectory planning module and a dance synthesis module; and constructs a digital stage model according to the environment map; then designs the dance action of the robot based on the digital stage model to obtain dance action information; and then generates the motion trajectory of the robot in combination with the digital stage model and the kinematic constraint of the robot, so that a complete dance performance scheme can be obtained, accurate environment perception and action planning can be realized, the robot action can accurately match the scene environment, the disconnection between the robot dance action and the environment can be avoided, and complex stage performances can be realized, which is conducive to the popularization and use of the robot.

[0007] To solve the above problems or one of the above problems, the second object of the present application is to provide a robot motion control method and electronic equipment based on digital twinning, which fully considers the characteristics of robot dance performance, interacts data and collaborates functions through a layered architecture with multiple simulation modules, forms an efficient and scalable dance design scheme, ensures that the robot can accurately perceive and adapt to changes in the stage environment, provides accurate positioning reference for robot motion, and thus can build a motion control scheme from environmental perception to action execution, ensures the accurate synchronization of robot dance actions and motion trajectories, and realizes smooth robot dance performance effect.

[0008] To achieve one of the above objects, the first technical solution of the present application is:

[0009] A robot motion control method based on digital twinning, comprising the following contents:

[0010] Obtain point cloud data collected by a laser radar through a pre-constructed environment acquisition module;

[0011] Process the point cloud data using a pre-constructed map processing module to obtain an environment map;

[0012] Based on the environment map, construct a digital stage model based on a pre-constructed digital stage twinning module;

[0013] Based on the digital stage model, design the dance action of the robot using a pre-constructed dance action design module to obtain dance action information;

[0014] Based on the digital stage model, generate the motion trajectory of the robot in combination with the kinematic constraints of the robot according to a pre-constructed trajectory planning module;

[0015] Couple the dance action information and the motion trajectory using a pre-constructed dance synthesis module to generate a complete dance performance scheme, and realize robot motion control based on digital twinning.

[0016] Through continuous exploration and testing, the present application constructs an environment acquisition module, a map processing module, a digital stage twinning module, a dance action design module, a trajectory planning module, and a dance synthesis module to obtain point cloud data and an environment map of the environment, constructs a digital stage model according to the environment map, designs the dance action of the robot based on the digital stage model to obtain dance action information, generates the motion trajectory of the robot in combination with the digital stage model and the kinematic constraints of the robot, and thus obtains a complete dance performance scheme, so as to realize accurate environmental perception and action planning, make the robot action accurately match the scene environment, avoid the disconnection between the robot dance action and the environment, and thus realize complex stage performance, which is conducive to the popularization and use of robots.

[0017] Further, the application fully considers the characteristics of robot dance performance, interacts data and cooperates functions through a layered architecture with multiple simulation modules, forms an efficient and scalable dance design scheme, ensures that the robot can accurately perceive and adapt to changes in the stage environment, provides accurate positioning reference for robot movement, so that a motion control scheme from environment perception to action execution can be constructed, ensuring the accurate synchronization of robot dance movements and motion trajectories, and realizing smooth robot dance performance effect.

[0018] As a preferred technical measure:

[0019] The method for processing point cloud data to obtain an environment map by using a pre-constructed map processing module is as follows:

[0020] Obtain original point cloud data and perform noise filtering processing on the point cloud data to obtain point cloud filtered data;

[0021] Perform geometric transformation on the point cloud filtered data to obtain point cloud geometric data, so that it can be translated, rotated, scaled, sheared and nonlinearly transformed;

[0022] Perform density uniformization processing on the point cloud geometric data to obtain point cloud uniform data;

[0023] Based on the point cloud uniform data, a pyramid multi-resolution point cloud structure is constructed, and an environment map is generated to realize dynamic detail switching based on the line-of-sight.

[0024] As a preferred technical measure:

[0025] The method for performing noise filtering processing on point cloud data to obtain point cloud filtered data is as follows:

[0026] Obtain original point cloud data, which includes a plurality of collection points;

[0027] Calculate the distance of each collection point from the remaining collection points to obtain the near neighbor distance of each collection point;

[0028] Calculate the distance between the collection points to obtain the mean value and standard deviation of the average distance of all points;

[0029] Calculate the distance threshold value according to the mean value, the standard deviation and the threshold coefficient;

[0030] Compare the near neighbor distance of each collection point with the distance threshold value to select noise points;

[0031] Filter out the noise points from the point cloud data to obtain point cloud filtered data.

[0032] As a preferred technical measure:

[0033] The method for density uniformization processing of point cloud geometry data is as follows:

[0034] Obtain point cloud geometry data, which includes a plurality of geometry points;

[0035] Calculate the neighborhood point set of each geometry point within a certain radius range;

[0036] According to the Gaussian weight function, the weight value between the geometry points is calculated;

[0037] According to the weight value of the geometry point and the neighborhood point set, the density value of the geometry point is calculated;

[0038] According to the point cloud density distribution requirement, the lower limit value, the upper limit value and the sensitivity parameter of the voxel size are set;

[0039] Based on the lower limit value, the upper limit value, the sensitivity parameter and the density value of the geometry point, the voxel value of the geometry point is calculated;

[0040] Based on a plurality of voxel values, the point cloud geometry data is processed by density uniformization to obtain point cloud uniform data.

[0041] As a preferred technical measure:

[0042] Based on the pre-constructed digital stage twin module, on the basis of the environment map, the method for constructing the digital stage model is as follows:

[0043] Obtain the environment map, which includes a plurality of coordinate points;

[0044] Based on the adjacent coordinate points, a plurality of coordinate vectors are calculated;

[0045] According to the adjacent coordinate vectors, a plurality of vector angles are calculated;

[0046] According to the vector angle, the attribution value of a coordinate point is calculated;

[0047] Based on the attribution value of the coordinate point, it is judged whether a coordinate point belongs to a polygon, so as to obtain the attribution data of all coordinate points;

[0048] According to the attribution data, the environment map is cut and regionally managed to obtain stage structure data;

[0049] According to the stage size and precision requirement, the stage structure data is processed to obtain a plurality of reference grids, including a main grid and a sub-grid;

[0050] Based on the view zoom level, the display density of the main grid and the sub-grid is adjusted to obtain a digital stage model.

[0051] As a preferred technical measure:

[0052] The method for obtaining the dance action information by using the pre-constructed dance action design module and based on the digital stage model is as follows:

[0053] Based on the robot dance action to be designed, key posture points are obtained;

[0054] According to the key posture points, key frame time sequences and corresponding joint angles are set;

[0055] Based on the key frame time sequences and corresponding joint angles, a spline curve is constructed;

[0056] According to the spline curve, continuity constraint conditions and coefficient solving matrix equations are set;

[0057] The continuity constraint conditions include position continuity, first-order derivative continuity and second-order derivative continuity;

[0058] Based on the continuity constraint conditions, a target function for optimizing the smoothness of the action is set;

[0059] The target function aims to minimize the jerk of the joint;

[0060] According to the target function, the Lagrange multiplier method is used to convert the constraint optimization problem into an unconstrained problem, so as to generate a smooth action curve, i.e., the dance action information.

[0061] As a preferred technical measure:

[0062] According to the pre-constructed trajectory planning module, based on the digital stage model, and combined with the kinematic constraints of the robot, the method for generating the movement trajectory of the robot is as follows:

[0063] Based on the grid coordinate system of the digital stage model, and combined with the kinematic constraints of the robot, key path points are set;

[0064] According to the key path points, and based on the spline curve algorithm, a control point sequence and a basis function are constructed;

[0065] According to the control point sequence and the basis function, an expression of the spline curve is obtained;

[0066] The expression of the spline curve is solved to obtain the movement trajectory of the robot, which includes movement position, movement direction and time information.

[0067] As a preferred technical measure:

[0068] Using the pre-constructed dance synthesis module, the dance action information and the movement trajectory are coupled to generate a complete dance performance scheme as follows:

[0069] A multi-track parallel display design method is adopted to set a timeline track for each robot, so that each robot has an independent timeline; the timeline track is used to add action clips or adjust the trajectory path for one or more robots at any time;

[0070] Based on dance movement information and creation information, the timing choreography module is used to obtain drag operation information, so that multi-robot collaborative choreography can be achieved through drag operations;

[0071] Based on the timeline track, the dance movement information, drag operation information and motion trajectory of each robot are coupled and processed to obtain the movement sequence and duration of each robot. The robot collaborative control algorithm is then combined to ensure the movement synchronization of multiple robots.

[0072] Arrange and combine the movements and trajectories of multiple robots according to the movement sequence and duration of each robot to obtain choreography data;

[0073] With the help of the dance description file generation module, the choreography data is converted into an execution instruction set to generate a complete dance performance plan.

[0074] To achieve one of the above purposes, the second technical solution of the present invention is:

[0075] A robot motion control method based on digital twins includes the following steps:

[0076] Step 1: Obtain point cloud data collected by the lidar through the pre-built environment acquisition module;

[0077] Step 2: Use the pre-built digital stage twin module to process the point cloud data and obtain the digital stage model;

[0078] Step 3: Using the pre-built dance movement design model, dance movement information is generated according to the digital stage model;

[0079] In step 4, a pre-built trajectory planning module is used to generate the robot's motion trajectory based on dance movement information and the digital stage model, combined with the robot's kinematic constraints, to achieve robot motion control based on digital twins.

[0080] The present invention fully considers the characteristics of robot dance performances, and forms an efficient and scalable dance design scheme through data interaction and functional collaboration with a layered architecture with multiple simulation modules. It ensures that the robot can accurately perceive and adapt to changes in the stage environment, and provides an accurate positioning reference for the robot's movement, thereby constructing a motion control scheme from environmental perception to action execution, ensuring the precise synchronization of the robot's dance movements and motion trajectories, and achieving a smooth robot dance performance effect.

[0081] To achieve one of the above-mentioned purposes, the third technical solution of the present application is:

[0082] An electronic device comprises:

[0083] One or more processors;

[0084] Storage means for storing one or more programs;

[0085] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned digital twin-based robot motion control method.

[0086] Compared with the prior art, the present application has the following beneficial effects:

[0087] Through continuous exploration and testing, the present application acquires point cloud data and an environment map of the environment by constructing an environment acquisition module, a map processing module, a digital stage twin module, a dance action design module, a trajectory planning module and a dance synthesis module; and constructs a digital stage model according to the environment map; then designs the dance action of the robot based on the digital stage model to obtain dance action information; and generates the motion trajectory of the robot by combining the digital stage model and the kinematic constraints of the robot, so that a complete dance performance scheme can be obtained, thereby realizing accurate environment perception and action planning, making the robot action accurately match the scene environment, avoiding the disconnection between the robot dance action and the environment, and further realizing complex stage performance, which is beneficial to the popularization and use of the robot.

[0088] Further, the present application fully considers the characteristics of robot dance performance, and realizes data interaction and function cooperation through the hierarchical architecture with multiple simulation modules, forming an efficient and expandable dance design scheme, ensuring that the robot can accurately perceive and adapt to changes in the stage environment, providing accurate positioning reference for robot motion, so that a motion control scheme from environment perception to action execution can be constructed, ensuring the accurate synchronization of robot dance action and motion trajectory, and realizing smooth robot dance performance effect. BRIEF DESCRIPTION OF DRAWINGS

[0089] Figure 1 Fig. 1 is a flowchart of the robot motion control method of the present application;

[0090] Figure 2 Fig. 2 is a data flow diagram of the robot motion control method of the present application. DETAILED DESCRIPTION

[0091] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be given to the present application in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and not to limit the present application.

[0092] On the contrary, the present application covers any substitution, modification, equivalent method and solution defined by the claims within the essence and scope of the present application. Further, in order to make the public have a better understanding of the present application, some specific details are described in the following detailed description of the present application. The present application can also be completely understood without the description of these details by those skilled in the art.

[0093] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing specific examples and are not intended to limit the present application.

[0094] As shown in Figure 1 The first specific embodiment of the robot motion control method based on digital twinning of the present application is as follows:

[0095] A robot motion control method based on digital twinning comprises the following contents:

[0096] Through the pre-constructed environment acquisition module, the point cloud data collected by the laser radar is acquired;

[0097] The point cloud data is processed by using the pre-constructed map processing module to obtain an environment map;

[0098] Based on the pre-constructed digital stage twin module, a digital stage model is constructed based on the environment map;

[0099] A pre-constructed dance action design module is used to design the dance action of the robot based on the digital stage model to obtain dance action information;

[0100] According to the pre-constructed trajectory planning module, the kinematic constraints of the robot are combined to generate the motion trajectory of the robot based on the digital stage model;

[0101] A pre-constructed dance synthesis module is used to couple the dance action information and the motion trajectory to generate a complete dance performance scheme, thereby realizing the robot motion control based on digital twinning.

[0102] The second specific embodiment of the robot motion control method based on digital twinning of the present application is as follows:

[0103] A robot motion control method based on digital twinning comprises the following steps:

[0104] Step one, through the pre-constructed environment acquisition module, the point cloud data collected by the laser radar is obtained;

[0105] Step two, using the pre-constructed digital stage twin module, the point cloud data is processed to obtain a digital stage model;

[0106] Step three, using the pre-constructed dance action design model, dance action information is generated according to the digital stage model;

[0107] Step four, using the pre-constructed trajectory planning module, the motion trajectory of the robot is generated based on the dance action information and the digital stage model, combined with the kinematic constraints of the robot, to realize digital twin-based robot motion control.

[0108] A robot motion system control system applying the method of the application:

[0109] A robot motion system control system based on digital twin, which adopts the robot motion control method based on digital twin and is based on the modular design idea to build a complete robot dance design and control platform. The system mainly consists of six core functional modules, including a map processing module, a digital stage twin module, a dance action design module, a trajectory planning module, a dance synthesis module, and a system control module. The modules interact with each other through standardized interfaces for data exchange and functional cooperation, forming an efficient and expandable dance design ecosystem. The design of the system fully considers the characteristics of robot dance performance, and through a layered architecture, a complete process from environment perception to action execution is constructed.

[0110] In the basic layer of the system, the map processing module undertakes the core task of environment modeling. This module processes the point cloud data collected by the laser radar to construct a high-precision digital stage model. It not only supports basic operations such as rotation and translation of point cloud data, but also can perform point cloud optimization and noise filtering. The module uses data processing algorithms to support multiple point cloud data formats and has real-time data processing capabilities, enabling dynamic updates of the environment model, providing reliable environmental data support for subsequent dance design.

[0111] The digital stage twin module constructs an intuitive visual stage environment based on the point cloud map. This module converts the original point cloud data into a digital stage that is easy to understand and operate, supports visual editing of stage layout, generation and management of reference grids, and dynamic adjustment of stage elements. Through multi-angle stage preview and real-time rendering technology, designers can intuitively understand the stage environment and provide accurate spatial reference for dance action design. The editing interface of the module focuses on user experience, supports dynamic updating of stage elements, and greatly improves the efficiency of dance design.

[0112] The dance motion design module, as the creative layer of the system, provides rich motion editing and choreography functions. The module supports the creation, editing and optimization of motion sequences, including fine adjustment of upper limb motion, precise control of motion timing, and management and reuse of motion library. Through an intuitive graphical interface and rich motion templates, designers can quickly implement complex dance motion design and preview motion effects in real time. The motion optimization algorithm of the module ensures the smoothness and naturalness of the motion, providing professional motion support for robot performances.

[0113] The trajectory planning module is responsible for the motion path planning of the robot on the stage, which is a key link of the system motion control. Based on the digital stage environment and combined with the kinematic constraints of the robot, the module generates safe and smooth motion trajectories. It supports various trajectory generation algorithms, including smooth trajectories based on spline interpolation and obstacle avoidance-based path planning. The real-time obstacle avoidance capability and trajectory optimization function of the module ensure the safe motion of the robot in complex stage environment, providing reliable motion guarantee for dance performances.

[0114] The dance synthesis module integrates the results of motion design and trajectory planning to generate a complete dance performance plan. The module realizes accurate synchronization of motion and trajectory, supports fine-tuning of performance timing, and provides real-time effect preview function. Through precise time synchronization and overall effect preview, designers can intuitively evaluate the dance effect and make necessary adjustments and optimizations. The module supports the export of performance plans, providing complete control instructions for subsequent execution.

[0115] The system control module, as the execution layer of the system, is responsible for converting the dance plan into actual motion of the robot. The module realizes real-time analysis and execution of dance instructions, including motion control, posture adjustment, motion synchronization and other functions. Through precise motion control and perfect state monitoring, the stability and reliability of the performance process are ensured. The exception handling mechanism of the module can respond to various emergencies in time, ensuring the smooth progress of the performance.

[0116] The module relationship of the system is designed with a hierarchical architecture, with clear and explicit data flow. The bottom layer data flows from the map processing module to the digital stage twin module, the middle layer data flows to the dance motion design module and the trajectory planning module, and the upper layer data converges to the dance synthesis module, and finally executes through the system control module. The modules communicate through standardized data interfaces, supporting bidirectional data interaction and real-time data update, ensuring the integrity and scalability of the system. The module relationship and data flow of the system can be referred to Figure 2The map processing module is a basic data layer of the system and is responsible for environment modeling; the digital stage twin module is a visualization layer of the system and is used for subsequent construction of a digital stage environment; the dance motion design module is a creative layer of the system and is responsible for motion choreography, formation and point design; the trajectory planning module is responsible for generating a robot motion trajectory; finally, the dance synthesis module integrates the motion and the trajectory, generates a task queue, and is handed over to different robots for execution.

[0117] In the embodiment, the digital stage twin module adopts a hierarchical architecture design and includes a data preprocessing layer, a point cloud processing layer, an intelligent cutting module, a grid generation module and an interactive control layer.

[0118] The data preprocessing layer can perform quality assessment, noise filtering, density equalization and format standardization processing on original point cloud PCD data.

[0119] The noise filtering algorithm adopts a statistical outlier detection method for noise filtering, and the calculation formula is as follows:

[0120]

[0121]

[0122] wherein, is the average distance of the point to its nearest neighbor, is the mean of the average distance of all points, is the standard deviation, is the jth nearest neighbor point. When , it is determined as a noise point, wherein a is a threshold coefficient, usually 2.0.

[0123] The geometric transformation layer can realize various geometric transformation operations of the point cloud, including translation, rotation, scaling, shearing and complex nonlinear transformation. The expression of the translation transformation is as follows:

[0124]

[0125] wherein, is a translation transformation matrix, is a direction displacement amount, is a direction displacement amount, is a direction displacement amount.

[0126] The rotation transformation algorithm adopts a quaternion for representation to avoid the gimbal lock problem, and the expression is as follows:

[0127]

[0128] in, is the rotation transformation matrix, is a quaternion, is the real part, 、 、 is the imaginary part.

[0129] The expression for the scaling transformation is as follows:

[0130]

[0131] in, is the scaling transformation matrix, is the scaling factor of x, is the scaling factor of y, is the scaling factor of z.

[0132] The final expression of the composite transformation is as follows:

[0133]

[0134] in, is the original point cloud, is the transformed point cloud.

[0135] The point cloud processing layer is constructed based on the adaptive downsampling method, which adaptively adjusts the voxel size according to the point cloud density distribution, ensuring the details of key areas while improving the overall processing efficiency.

[0136] Point density The expression to be calculated is as follows:

[0137]

[0138] in, is the radius The neighborhood point set within is the volume of the neighborhood sphere, is the Gaussian weight function, and its calculation formula is as follows:

[0139]

[0140] in, is the Euclidean distance between two points, is the standard deviation of the Gaussian distribution, which is used to control the "width" or "influence range" of weight decay.

[0141] Calculating voxel size The expression is as follows:

[0142]

[0143] wherein, , is the lower bound of the voxel size, is the sensitivity parameter.

[0144] The multi-level detail model (LOD) supports dynamic detail switching based on the view distance by constructing a pyramid multi-resolution point cloud structure, and its expression is as follows:

[0145]

[0146] wherein, is the point set contained in the LOD of the th level; is the local point cloud density of the point P; is the density threshold of the LOD of the th level.

[0147] The expression of the view distance related multi-level detail model is as follows:

[0148]

[0149] wherein, is the LOD level to be selected currently, is the observation distance, is the reference distance.

[0150] The intelligent clipping module adopts an improved ray projection algorithm to clip complex polygons, and its calculation expression is as follows:

[0151]

[0152]

[0153] wherein, is the two-dimensional point to be judged, is the two-dimensional point P, is the i th vertex of the polygon, is the vector from the point P to the i th vertex of the polygon, is the vector angle.

[0154] The expression of the polygon interior judgment is as follows:

[0155]

[0156] wherein, represents whether the point P is inside the polygon.

[0157] Further, according to the determination result, hierarchical cutting management is performed.

[0158] Further, the multi-level stage area is independently managed, and different visibility and editing permissions can be set for each level.

[0159] The expression of the level mask is as follows:

[0160]

[0161] Among them, is the total level mask (whether in the active layer) of the point , is the active level set, is the mask function of the layer.

[0162] The dynamic reference grid module is generated based on adaptive grid generation, which can automatically generate multi-level reference grids according to the stage size and precision requirements. The expression of the main grid generation is as follows:

[0163]

[0164] Among them, is the set of main grid points, , is the grid index (integer), is the spacing of the main grid (the distance between the main grid lines), represents the three-dimensional coordinates of the main grid point (usually z=0 represents the stage plane).

[0165] The expression of the sub-grid generation is as follows:

[0166]

[0167] Among them, is the set of sub-grid points; is the number of levels that the main grid is subdivided into, used to represent that each main grid spacing is divided into n parts.

[0168] Based on the view zoom level, the grid display density is dynamically adjusted, and the grid visibility is controlled. The expression of the visibility function is as follows:

[0169]

[0170] Among them, is the grid visibility function, which returns the value between them indicates the visibility of the grid, is the current zoom level (usually the zoom factor, such as 1.0 represents 100%), and is the threshold value (zoom level) of the main grid display, Threshold value (scaling level) for sub-grid display.

[0171] Based on the mouse picking algorithm, real-time interactive response is carried out to realize the accurate three-dimensional point cloud picking function.

[0172] Ray The generated expression is as follows:

[0173]

[0174] Wherein, is the starting point of the ray, is the direction of the ray, is the ray parameter, indicating the distance from the starting point along the direction.

[0175] The calculation formula for calculating the nearest point is as follows:

[0176]

[0177] Wherein, is the picked (selected) point, is the ith point in the point cloud, R( ) is the point on the ray closest to the point cloud, is the ray parameter, indicating the position on the ray closest to the point cloud.

[0178] Further, in order to support real-time transformation preview during the dragging process, an incremental update strategy is adopted, and the expression of the incremental transformation matrix is as follows:

[0179]

[0180] Wherein, is the incremental transformation matrix, indicating the transformation from the previous state to the current state, is the current transformation matrix, is the inverse of the transformation matrix at the previous moment.

[0181] In this embodiment, the dance design module is a complete humanoid robot dance action design and arrangement system. Through the graphical interface and intelligent algorithm, the whole process automation from single action design to multi-robot cooperative dance arrangement is realized. The system adopts hierarchical architecture design and integrates core function modules such as action teaching, trajectory planning, timing arrangement and file generation.

[0182] This module is responsible for the design and generation of robot single action, which can record key posture points through artificial teaching mode, and automatically generate smooth action curve combined with intelligent algorithm. It includes the following contents:

[0183] Let the key frame time sequence be , and the corresponding joint angle be , then the i-th segment spline curve is:

[0184]

[0185] in 、 、 、 are the coefficients of the i-th segment spline curve, is the time parameter, is the time point of the i-th key frame, .

[0186] Construct continuity constraints, including position continuity, first-order derivative continuity, and second-order derivative continuity.

[0187] The expression for position continuity is as follows:

[0188]

[0189] The expression for the continuity of the first-order derivative is as follows:

[0190]

[0191] The expression for the continuity of the second-order derivative is as follows:

[0192]

[0193] The expressions for solving the matrix equation with coefficients are as follows:

[0194]

[0195] ; 、 … is the second-order derivative coefficient of the i-th segment of the spline curve.

[0196] By constructing an objective function about minimizing jerk, we optimize the smoothness of the motion and minimize the jerk. The expression is as follows:

[0197]

[0198] in, For joints The jerk (third derivative), is the number of robot degrees of freedom, is the angle of the j-th joint at time t.

[0199] The optimization solution method uses the Lagrange multiplier method to transform the constrained optimization problem into an unconstrained problem. The calculation formula is as follows:

[0200]

[0201] where, is the Lagrange function, is the Lagrange multiplier of the kth constraint, is the kth constraint function.

[0202] In this embodiment, the trajectory generation module includes the following contents:

[0203] In the grid coordinate system of the digital stage, the movement trajectory of the robot is designed, and 2D trajectory data containing position, direction and time information is generated. The core functions include visual trajectory drawing: supporting straight line, curve, composite path; accurate positioning of path points: based on grid-based coordinate capture; speed curve design: trajectory generation with variable speed.

[0204] Given a sequence of control points , and then a spline curve of order n is defined as:

[0205]

[0206] where, The expression of the B-spline basis function is as follows:

[0207]

[0208]

[0209] where, is the i-th order B-spline basis function, is the i-th node value. In this embodiment, the time arrangement module is the core component of the entire dance design system, which provides an intuitive timeline interface similar to professional video editing software, allowing users to accurately arrange and combine the movements and trajectories of multiple robots. The design concept of this module is to simplify the complex multi-robot collaborative arrangement into a visual drag-and-drop operation, so that even non-professional users can quickly get started and create dances.

[0210]

[0211] ​​The system of the present application adopts a multi-track parallel display design scheme, each robot has an independent time axis track, and users can add action clips or adjust the trajectory path for a specific robot at any time. The interface design fully considers the convenience of human-computer interaction, supports multiple operation modes, drag adjustment of action clips, batch selection and editing, and real-time zoom browsing function. The time axis supports up to 64 concurrent tracks, each track can accommodate up to 1000 action clips, which can meet the complex arrangement needs of large-scale robot group dance.

[0212] In the management of action clips, the system realizes intelligent clip fusion technology. When the user sets two action clips to overlap, the system will automatically detect the overlapping area and apply a smooth transition algorithm for fusion. The action mixing algorithm uses the spherical linear interpolation (SLERP) algorithm to ensure smooth transition of joint angles in quaternion space.

[0213] The data structure design of the system of the present application adopts an efficient hierarchical storage mode. The time axis track as the top container contains robot identification, action clip list, trajectory clip list and time boundary information. The action clip records the action identification, time parameter, fade-in and fade-out effect and transition type and other key attributes. This structured design not only facilitates data organization and management, but also provides convenience for subsequent serialization and file export.

[0214] Real-time preview function is an important feature of the timing arrangement module, the system can provide real-time rendering preview of multi-robot cooperative action at a frame rate of not less than 30fps. The preview engine adopts a hierarchical rendering strategy, which independently renders the robot body action, movement trajectory and stage environment, and then combines them into the final preview picture. In order to improve rendering efficiency, the system realizes adaptive level of detail (LOD) management, dynamically adjusts rendering precision according to viewing distance and importance. At the same time, multiple viewing modes are supported during preview: top view for observing overall formation changes, side view for checking action coordination, and free view for comprehensive effect evaluation.

[0215] The system of the present application has built-in beat detection and automatic alignment function, which can automatically prompt key time points according to the beats of background music, helping users to realize accurate synchronization of music and action. The conflict detection algorithm will monitor the potential collision risk between robots in real time, and remind the user to adjust the arrangement scheme through highlighted display. The version management function supports automatic saving and backtracking of arrangement history, users can freely try different arrangement ideas during the creation process without worrying about data loss.

[0216] The system also provides rich programming templates and quick operation functions. Users can save commonly used programming modes as templates for quick reuse in subsequent creation. Batch operation functions support unified adjustment of multiple robots or time periods, greatly improving the work efficiency of large-scale programming. Import and export functions are compatible with multiple standard formats, facilitating data exchange and collaborative development with other professional software.

[0217] In this embodiment, the dance description file generation module is the final output link of the entire design process, and undertakes the key task of converting complex timeline programming data into executable instruction sets for robots. The design goal of this module is to establish a standardized, efficient, and cross-platform dance description language to ensure that design intentions can be accurately conveyed to the robot execution system.

[0218] The module adopts a multi-level data organization architecture, including dance meta-information such as title, total duration, number of participating robots, stage size, and other global parameters at the top level. These information provides important reference for the initialization and resource allocation of the robot system. At the individual robot level, the system establishes independent data blocks for each participant, detailing their complete action sequences and movement trajectories. The action sequence part uses timestamp indexing to organize each action segment, including action identifiers, start times, duration, and specific parameter configurations to ensure accurate execution of actions. The trajectory part records the spatial movement information of the robot in the form of key path points, including position coordinates, orientation angles, and corresponding timestamps.

[0219] To meet the needs of different application scenarios and hardware platforms, the system supports flexible switching of multiple output formats. Using lightweight data format JSON as the main data exchange format, it has good readability and cross-platform compatibility, facilitating development and debugging and system integration. Extensible Markup Language (XML) provides stronger data validation capabilities and hierarchical structure expression, suitable for formal performance occasions with higher data integrity requirements. Binary format is optimized for transmission efficiency and storage space, especially suitable for large-scale robot group dance or network bandwidth limited application environments.

[0220] Therefore, the system has significant technical advantages and innovative features in the field of robot dance design and control. Through modular design and advanced technology implementation, the system exhibits unique advantages in environment modeling, design process, stage adaptation, and dance design.

[0221] In terms of environment modeling, the system adopts high-precision environment modeling technology based on point cloud PCD, realizing accurate digitization of the stage environment. Through advanced point cloud processing algorithms, the system can construct a high-precision environment model to provide accurate positioning reference for the robot. The reusability design of environment data enables the same stage environment to be quickly applied to different performance scenes, greatly improving the practicality and efficiency of the system. At the same time, the system supports real-time updating and dynamic adjustment of the environment model, ensuring that the robot can accurately perceive and adapt to changes in the stage environment.

[0222] The design process of the system adopts a modular architecture, realizing efficient management of the dance design process. Through standardized file system design, the system realizes orderly storage and fast access of various data. The visual editing interface design makes complex dance motion design intuitive and efficient. The system supports fast environment switching function, and the designer can seamlessly switch between different stage scenes, greatly improving the design efficiency. At the same time, the system records all operations during the design process, supports traceability of the design process, and facilitates post-optimization and adjustment.

[0223] In terms of stage adaptation, the system shows strong flexibility and adaptability. Through standardized interface design, the system can quickly adapt to different types of stage environments, including standard stages, special stages and other scenes. The system provides accurate environment adjustment tools to support fine-tuning and optimization of stage layout. The modular design of dance movements makes it easy to reuse and adjust movements in different stage environments. Scene switching function supports fast switching between different stage environments, ensuring the continuity and smoothness of the performance.

[0224] The dance design system of the system is complete and powerful, realizing the close combination of motion design and environment. Through advanced motion design tools and algorithms, the system supports complex stage performance motion design. The complete dance synthesis scheme ensures the accurate synchronization of motion and trajectory, realizing smooth dance performance effect. The modular design of the system ensures good scalability, and new functional modules can be added flexibly according to needs. At the same time, the system provides a complete performance evaluation system to support comprehensive evaluation and optimization of dance performance effect.

[0225] An embodiment of a device applying the method of the present application is as follows:

[0226] An electronic device comprises:

[0227] one or more processors;

[0228] a storage device for storing one or more programs;

[0229] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned robot motion control method based on digital twinning.

[0230] A computer medium embodiment of the method of the application is as follows:

[0231] A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above-mentioned robot motion control method based on digital twinning.

[0232] Those skilled in the art will understand that the embodiments of the present application can be provided as a method, a system, and a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) having computer-usable program code.

[0233] The present application is described in terms of flowcharts or / and block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow or / and block in the flowcharts or / and block diagrams, as well as combinations of flows or / and blocks in the flowcharts or / and block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts or / and block diagrams. Figure 1 one flow or multiple flows or / and blocks Figure 1 an apparatus that performs the functions specified in one or more blocks.

[0234] The module in the present application is an object that constitutes a morphological structure by means of physical or virtual representation, and the object is not equal to the object and is not limited to physical and virtual. It can be a data processing function, a software program, a processing mode, a use method, an operation method, a work flow, an application process, an electronic hardware, a circuit module, a processing system, a system imitation product, or a simulation object.

[0235] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them, although the present application has been described in detail with reference to the above embodiments, the ordinary skilled in the art can still modify or equivalently replace the specific embodiments of the present application without departing from the spirit and scope of the present application, any modification or equivalent replacement which does not depart from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A robot motion control method based on digital twins, characterized by: Includes the following: Obtain point cloud data collected by LiDAR through the pre-built environment acquisition module; Use the pre-built map processing module to process the point cloud data to obtain the environment map; Based on the pre-built digital stage twin module, a digital stage model is constructed on the basis of the environment map; Using the pre-built dance movement design module, the robot's dance movements are designed based on the digital stage model to obtain dance movement information; Generate the robot's motion trajectory based on the pre-built trajectory planning module, the digital stage model, and the robot's kinematic constraints; It includes the following: Based on the grid coordinate system of the digital stage model and combined with the robot's kinematic constraints, key path points are set; According to the key path points and based on the spline curve algorithm, the control point sequence and basis function are constructed; According to the control point sequence and basis function, the expression of the spline curve is obtained; Solve the expression of the spline curve to obtain the robot's motion trajectory, which includes the moving position, moving direction and time information; Using a pre-built dance synthesis module, dance movement information and motion trajectory are coupled to generate a complete dance performance plan, realizing robot motion control based on digital twins.

2. The robot motion control method based on digital twin according to claim 1, characterized in that: The method for processing point cloud data using the pre-built map processing module to obtain an environment map is as follows: Obtain the original point cloud data and perform noise filtering on the point cloud data to obtain point cloud filtered data; Perform geometric transformation on the point cloud filtering data to obtain point cloud geometric data, so that it can be translated, rotated, scaled, sheared and nonlinearly transformed; Perform density homogenization on the point cloud geometric data to obtain uniform point cloud data; Based on the uniform point cloud data, a pyramid-type multi-resolution point cloud structure is constructed, and an environment map is generated to achieve dynamic detail switching based on viewing distance.

3. The robot motion control method based on digital twin according to claim 2, characterized in that: The method for performing noise filtering on point cloud data to obtain point cloud filtered data is as follows: Obtaining original point cloud data, which includes several collection points; Calculate the distance between each collection point and the remaining collection points to obtain the nearest neighbor distance of each collection point; Calculate the distance between the collected points and obtain the mean and standard deviation of the average distance of all points; The distance threshold is calculated based on the mean, standard deviation and threshold coefficient; Compare the nearest neighbor distance of each collection point with the distance threshold to select noise points; The noise points are filtered out from the point cloud data to obtain point cloud filtered data.

4. The robot motion control method based on digital twin according to claim 2, characterized in that: The method for performing density homogenization on point cloud geometric data to obtain uniform point cloud data is as follows: Obtaining point cloud geometric data, which includes a number of geometric points; Calculate the neighborhood point set of each geometric point within a certain radius; Calculate the weight values ​​between geometric points according to the Gaussian weight function; Calculate the density value of the geometric point according to the weight value of the geometric point and the neighborhood point set; According to the point cloud density distribution requirements, set the lower and upper bounds of the voxel size and the sensitivity parameters; Based on the lower bound value, the upper bound value, the sensitivity parameter and the density value of the geometric point, the voxel value of the geometric point is calculated; Based on several voxel values, the point cloud geometric data is density-homogenized to obtain uniform point cloud data.

5. The robot motion control method based on digital twin according to claim 1, characterized in that: Based on the pre-built digital stage twin module and the environment map, the method for building a digital stage model is as follows: Acquire an environment map, which includes a plurality of coordinate points; Based on adjacent coordinate points, multiple coordinate vectors are calculated; According to the adjacent coordinate vectors, several vector angles are calculated; According to the vector angle, calculate the attribute value of a certain coordinate point; Based on the attribution value of the coordinate point, determine whether a coordinate point belongs to a certain polygon, so as to obtain the attribution data of all coordinate points; Based on the attribution data, the environment map is cut and the area management is performed to obtain the stage structure data; According to the stage size and accuracy requirements, the stage structure data is processed to obtain a multi-level reference grid, which includes a main grid and sub-grids; Based on the view zoom level, the display density of the main grid and sub-grids is adjusted to obtain a digital stage model.

6. The robot motion control method based on digital twin according to claim 1, characterized in that: The robot's dance movements are designed using a pre-built dance movement design module based on the digital stage model. The method for obtaining dance movement information is as follows: Based on the robot dance movements to be designed, key posture points are obtained; According to the key posture points, set the key frame time sequence and the corresponding joint angle; Construct a spline curve based on the key frame time series and the corresponding joint angles; According to the spline curve, set the continuity constraints and coefficients to solve the matrix equation; Continuity constraints include position continuity, first-order derivative continuity, and second-order derivative continuity; Based on the continuity constraint, an objective function is set to optimize the smoothness of the action; The objective function aims to minimize the joint jerk; According to the objective function, the Lagrange multiplier method is used to transform the constrained optimization problem into an unconstrained problem, thereby generating a smooth motion curve, namely the dance movement information.

7. The robot motion control method based on digital twin according to claim 1, characterized in that: The method for using the pre-built dance synthesis module to couple dance movement information and motion trajectories to generate a complete dance performance plan is as follows: A multi-track parallel display design method is adopted to set a timeline track for each robot, so that each robot has an independent timeline; the timeline track is used to add action clips or adjust the trajectory path for one or more robots at any time; Based on dance movement information and creation information, the timing choreography module is used to obtain drag operation information, so that multi-robot collaborative choreography can be achieved through drag operations; Based on the timeline track, the dance movement information, drag operation information and motion trajectory of each robot are coupled and processed to obtain the movement sequence and duration of each robot. The robot collaborative control algorithm is then combined to ensure the synchronization of the movements of multiple robots. Arrange and combine the movements and trajectories of multiple robots according to the movement sequence and duration of each robot to obtain choreography data; With the help of the dance description file generation module, the choreography data is converted into an execution instruction set to generate a complete dance performance plan.

8. A robot motion control method based on digital twins, characterized by: The following steps are involved: Step 1: Obtain point cloud data collected by the lidar through the pre-built environment acquisition module; Step 2: Use the pre-built digital stage twin module to process the point cloud data to obtain a digital stage model; this includes the following: Obtain the original point cloud data and perform noise filtering on the point cloud data to obtain point cloud filtered data; Perform geometric transformation on the point cloud filtering data to obtain point cloud geometric data, so that it can be translated, rotated, scaled, sheared and nonlinearly transformed; Perform density homogenization on the point cloud geometric data to obtain uniform point cloud data; Based on uniform point cloud data, a pyramid-shaped multi-resolution point cloud structure is constructed and an environment map is generated to achieve dynamic detail switching based on viewing distance; Based on the environment map, build a digital stage model; Step 3: Using the pre-built dance movement design model, dance movement information is generated according to the digital stage model; In step 4, a pre-built trajectory planning module is used to generate the robot's motion trajectory based on dance movement information and the digital stage model, combined with the robot's kinematic constraints, to achieve robot motion control based on digital twins.

9. An electronic device, characterized in that: It includes: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a robot motion control method based on digital twins as described in any one of claims 1 to 8.

Images