Robot motion control method based on digital twinning and electronic equipment

Abstract

The invention discloses a robot motion control method based on digital twinning and electronic equipment, and belongs to the technical field of robot motion control. According to an existing robot action control scheme, complex stage performance is difficult to achieve, and consequently the dance performance effect of a robot is poor. According to the robot motion control method based on digital twinning, an environment acquisition module, a map processing module, a digital stage twinning module, a dance motion design module, a trajectory planning module and a dance synthesis module are constructed to perform data interaction and function collaboration, so that an efficient and extensible dance design scheme is formed; it is ensured that the robot can accurately sense and adapt to the change of the stage environment, the robot action can accurately match the scene environment, the robot dance action is prevented from being disjointed with the environment, meanwhile, the robot dance action can be designed, therefore, complex stage performance can be achieved, and the robot dance performance effect is effectively improved.

Description

Technical Field

[0001] The present invention relates to a robot motion control method and electronic equipment based on digital twins, and belongs to the technical field of robot motion control. Background Art

[0002] With the rapid development of robotics technology, robot performances have evolved from simple action displays to complex stage performances.

[0003] A Chinese patent application (publication number: CN119839885A) discloses a robot and a robot control method, which include a robot body and a rotating performance piece; the robot body is provided with an arm for assembling the rotating performance piece and a throwing motor for throwing the performance props; the rotating performance piece is assembled at the end of the arm, and the throwing motor can provide throwing power, giving the performance props kinetic energy to fly outward, so that the above-mentioned robot can at least perform dance movements such as rotation and throwing, with rich movements, good operation demonstration effect, and good performance performance. It can be used for humanoid robot dance performances, especially folk dance performances.

[0004] The above scheme can realize the robot's motion control, but it cannot perform environmental perception and motion planning, resulting in the robot's dance movements being out of touch with the environment, making it impossible for the robot's movements to accurately match the scene environment. Therefore, it is difficult to achieve complex stage performances, and the robot's dance performance effect is not good, which is not conducive to the promotion and use of robots.

[0005] The information disclosed in this Background Art is only for understanding the background of the present inventive concept and therefore it may include information that does not constitute prior art. Summary of the Invention

[0006] In response to the above problem or one of the above problems, an object of the present invention is to provide a robot motion control method and electronic device based on digital twins, which obtains the point cloud data and environment map of the environment by constructing an environment acquisition module, a map processing module, a digital stage twin module, a dance movement design module, a trajectory planning module, and a dance synthesis module; and constructs a digital stage model based on the environment map; and then designs the robot's dance movements based on the digital stage model to obtain dance movement information; and then generates the robot's motion trajectory by combining the digital stage model and the robot's kinematic constraints, so as to obtain a complete dance performance plan, thereby achieving accurate environmental perception and motion planning, so that the robot's movements can accurately match the scene environment, avoiding the robot's dance movements from being out of touch with the environment, and then realizing complex stage performances, which is conducive to the promotion and use of robots.

[0007] In response to the above problem or one of the above problems, the second purpose of the present invention is to provide a robot motion control method and electronic equipment based on digital twins, which fully considers the characteristics of robot dance performances, and forms an efficient and scalable dance design scheme through data interaction and functional collaboration with a layered architecture with multiple simulation modules, ensuring that the robot can accurately perceive and adapt to changes in the stage environment, and provide accurate positioning reference for robot movement, so as to construct a motion control scheme from environmental perception to action execution, ensuring the precise synchronization of robot dance movements and motion trajectories, and achieving a smooth robot dance performance effect.

[0008] To achieve one of the above purposes, the first technical solution of the present invention is: A robot motion control method based on digital twins includes the following: Obtain point cloud data collected by LiDAR through the pre-built environment acquisition module; Use the pre-built map processing module to process the point cloud data and obtain the environment map; Based on the pre-built digital stage twin module, a digital stage model is constructed on the basis of the environment map; Using the pre-built dance movement design module, the robot's dance movements are designed based on the digital stage model to obtain dance movement information; Generate the robot's motion trajectory based on the pre-built trajectory planning module, the digital stage model, and the robot's kinematic constraints; Using a pre-built dance synthesis module, dance movement information and motion trajectory are coupled to generate a complete dance performance plan, realizing robot motion control based on digital twins.

[0009] After continuous exploration and experimentation, the present invention obtains point cloud data and an environmental map of the environment by constructing an environment acquisition module, a map processing module, a digital stage twin module, a dance movement design module, a trajectory planning module, and a dance synthesis module; and constructs a digital stage model based on the environmental map; then, based on the digital stage model, the robot's dance movements are designed to obtain dance movement information; and then, the robot's motion trajectory is generated by combining the digital stage model and the robot's kinematic constraints, so as to obtain a complete dance performance plan, thereby achieving accurate environmental perception and movement planning, so that the robot's movements can accurately match the scene environment, avoiding the disconnection between the robot's dance movements and the environment, and thus realizing complex stage performances, which is conducive to the promotion and use of robots.

[0010] Furthermore, the present invention fully considers the characteristics of robot dance performances, and through a layered architecture with multiple simulation modules for data interaction and functional collaboration, forms an efficient and scalable dance design solution, ensuring that the robot can accurately perceive and adapt to changes in the stage environment, and provide accurate positioning reference for robot movement, thereby constructing a motion control solution from environmental perception to action execution, ensuring the precise synchronization of the robot's dance movements and motion trajectories, and achieving a smooth robot dance performance effect.

[0011] As preferred technical measures: The method for processing point cloud data using the pre-built map processing module to obtain an environment map is as follows: Obtain the original point cloud data, and perform noise filtering on the point cloud data to obtain point cloud filtered data; Perform geometric transformation on the point cloud filtering data to obtain point cloud geometric data, so that it can be translated, rotated, scaled, sheared and nonlinearly transformed; Perform density homogenization on the point cloud geometric data to obtain uniform point cloud data; Based on the uniform point cloud data, a pyramid-type multi-resolution point cloud structure is constructed, and an environment map is generated to achieve dynamic detail switching based on viewing distance.

[0012] As preferred technical measures: The method for performing noise filtering on point cloud data to obtain point cloud filtered data is as follows: Obtaining original point cloud data, which includes several collection points; Calculate the distance between each collection point and the remaining collection points to obtain the nearest neighbor distance of each collection point; Calculate the distance between the collected points and obtain the mean and standard deviation of the average distance of all points; The distance threshold is calculated based on the mean, standard deviation and threshold coefficient; Compare the nearest neighbor distance of each collection point with the distance threshold to select noise points; The noise points are filtered out from the point cloud data to obtain point cloud filtered data.

[0013] As preferred technical measures: The method for performing density homogenization on point cloud geometric data to obtain uniform point cloud data is as follows: Obtaining point cloud geometric data, which includes a number of geometric points; Calculate the neighborhood point set of each geometric point within a certain radius; Calculate the weight values ​​between geometric points according to the Gaussian weight function; Calculate the density value of the geometric point according to the weight value of the geometric point and the neighborhood point set; According to the point cloud density distribution requirements, set the lower and upper bounds of the voxel size and the sensitivity parameters; Based on the lower bound value, the upper bound value, the sensitivity parameter and the density value of the geometric point, the voxel value of the geometric point is calculated; Based on several voxel values, the point cloud geometric data is density-homogenized to obtain uniform point cloud data.

[0014] As preferred technical measures: Based on the pre-built digital stage twin module and the environment map, the method for building a digital stage model is as follows: Acquire an environment map, which includes a plurality of coordinate points; Based on adjacent coordinate points, multiple coordinate vectors are calculated; According to the adjacent coordinate vectors, several vector angles are calculated; According to the vector angle, calculate the attribute value of a certain coordinate point; Based on the attribution value of the coordinate point, determine whether a coordinate point belongs to a certain polygon, so as to obtain the attribution data of all coordinate points; Based on the attribution data, the environment map is cut and the area management is performed to obtain the stage structure data; According to the stage size and accuracy requirements, the stage structure data is processed to obtain a multi-level reference grid, which includes a main grid and sub-grids; Based on the view zoom level, the display density of the main grid and sub-grids is adjusted to obtain a digital stage model.

[0015] As preferred technical measures: The robot's dance movements are designed using a pre-built dance movement design module based on the digital stage model. The method for obtaining dance movement information is as follows: Based on the robot dance movements to be designed, key posture points are obtained; According to the key posture points, set the key frame time sequence and the corresponding joint angle; Construct a spline curve based on the key frame time series and the corresponding joint angles; According to the spline curve, set the continuity constraints and coefficients to solve the matrix equation; Continuity constraints include position continuity, first-order derivative continuity, and second-order derivative continuity; Based on the continuity constraint, an objective function is set to optimize the smoothness of the action; The objective function aims to minimize the joint jerk; According to the objective function, the Lagrange multiplier method is used to transform the constrained optimization problem into an unconstrained problem, thereby generating a smooth motion curve, namely the dance movement information.

[0016] As preferred technical measures: According to the pre-built trajectory planning module, based on the digital stage model and combined with the robot's kinematic constraints, the method for generating the robot's motion trajectory is as follows: Based on the grid coordinate system of the digital stage model and combined with the robot's kinematic constraints, key path points are set; According to the key path points and based on the spline curve algorithm, the control point sequence and basis function are constructed; According to the control point sequence and basis function, the expression of the spline curve is obtained; The expression of the spline curve is solved to obtain the robot's movement trajectory, which includes the movement position, movement direction and time information.

[0017] As preferred technical measures: The method for using the pre-built dance synthesis module to couple dance movement information and motion trajectories to generate a complete dance performance plan is as follows: A multi-track parallel display design method is adopted to set a timeline track for each robot, so that each robot has an independent timeline; the timeline track is used to add action clips or adjust the trajectory path for one or more robots at any time; Based on dance movement information and creation information, the timing choreography module is used to obtain drag operation information, so that multi-robot collaborative choreography can be achieved through drag operations; Based on the timeline track, the dance movement information, drag operation information and motion trajectory of each robot are coupled and processed to obtain the movement sequence and duration of each robot. The robot collaborative control algorithm is then combined to ensure the synchronization of the movements of multiple robots. Arrange and combine the movements and trajectories of multiple robots according to the movement sequence and duration of each robot to obtain choreography data; With the help of the dance description file generation module, the choreography data is converted into an execution instruction set to generate a complete dance performance plan.

[0018] To achieve one of the above purposes, the second technical solution of the present invention is: A robot motion control method based on digital twins includes the following steps: Step 1: Obtain point cloud data collected by the lidar through the pre-built environment acquisition module; Step 2: Use the pre-built digital stage twin module to process the point cloud data and obtain the digital stage model; Step 3: Using the pre-built dance movement design model, dance movement information is generated according to the digital stage model; In step 4, a pre-built trajectory planning module is used to generate the robot's motion trajectory based on dance movement information and the digital stage model, combined with the robot's kinematic constraints, to achieve robot motion control based on digital twins.

[0019] The present invention fully considers the characteristics of robot dance performances, and forms an efficient and scalable dance design scheme through data interaction and functional collaboration with a layered architecture with multiple simulation modules. It ensures that the robot can accurately perceive and adapt to changes in the stage environment, and provides an accurate positioning reference for the robot's movement, thereby constructing a motion control scheme from environmental perception to action execution, ensuring the precise synchronization of the robot's dance movements and motion trajectories, and achieving a smooth robot dance performance effect.

[0020] To achieve one of the above purposes, the third technical solution of the present invention is: An electronic device comprising: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned robot motion control method based on digital twins.

[0021] Compared with the existing technical solutions, the present invention has the following beneficial effects: After continuous exploration and experimentation, the present invention obtains point cloud data and an environmental map of the environment by constructing an environment acquisition module, a map processing module, a digital stage twin module, a dance movement design module, a trajectory planning module, and a dance synthesis module; and constructs a digital stage model based on the environmental map; then, based on the digital stage model, the robot's dance movements are designed to obtain dance movement information; and then, the robot's motion trajectory is generated by combining the digital stage model and the robot's kinematic constraints, so as to obtain a complete dance performance plan, thereby achieving accurate environmental perception and movement planning, so that the robot's movements can accurately match the scene environment, avoiding the disconnection between the robot's dance movements and the environment, and thus realizing complex stage performances, which is conducive to the promotion and use of robots.

[0022] Furthermore, the present invention fully considers the characteristics of robot dance performances, and through a layered architecture with multiple simulation modules for data interaction and functional collaboration, forms an efficient and scalable dance design solution, ensuring that the robot can accurately perceive and adapt to changes in the stage environment, and provide accurate positioning reference for robot movement, thereby constructing a motion control solution from environmental perception to action execution, ensuring the precise synchronization of the robot's dance movements and motion trajectories, and achieving a smooth robot dance performance effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic flow chart of the robot motion control method of the present invention; Figure 2 This is a data flow diagram of the robot motion control method of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0025] Rather, the present invention encompasses any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention as defined by the claims. Furthermore, to facilitate a better understanding of the present invention, certain specific details are described in detail below in the detailed description of the present invention. Those skilled in the art will be able to fully understand the present invention without these details.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention.

[0027] like Figure 1 As shown, the first specific embodiment of the robot motion control method based on digital twin of the present invention is: A robot motion control method based on digital twins includes the following: Obtain point cloud data collected by LiDAR through the pre-built environment acquisition module; Use the pre-built map processing module to process the point cloud data and obtain the environment map; Based on the pre-built digital stage twin module, a digital stage model is constructed on the basis of the environment map; Using the pre-built dance movement design module, the robot's dance movements are designed based on the digital stage model to obtain dance movement information; Generate the robot's motion trajectory based on the pre-built trajectory planning module, the digital stage model, and the robot's kinematic constraints; Using a pre-built dance synthesis module, dance movement information and motion trajectory are coupled to generate a complete dance performance plan, realizing robot motion control based on digital twins.

[0028] The second specific embodiment of the robot motion control method based on digital twin of the present invention: A robot motion control method based on digital twins includes the following steps: Step 1: Obtain point cloud data collected by the lidar through the pre-built environment acquisition module; Step 2: Use the pre-built digital stage twin module to process the point cloud data and obtain the digital stage model; Step 3: Using the pre-built dance movement design model, dance movement information is generated according to the digital stage model; In step 4, a pre-built trajectory planning module is used to generate the robot's motion trajectory based on dance movement information and the digital stage model, combined with the robot's kinematic constraints, to achieve robot motion control based on digital twins.

[0029] An embodiment of a robot motion system control system using the method of the present invention: A digital twin-based robotic motion control system utilizes the aforementioned digital twin-based robotic motion control method and, based on a modular design philosophy, constructs a complete robotic dance design and control platform. The system primarily consists of six core functional modules: a map processing module, a digital stage twin module, a dance movement design module, a trajectory planning module, a dance synthesis module, and a system control module. These modules interact with each other through standardized interfaces for data exchange and functional collaboration, forming an efficient and scalable dance design ecosystem. The system design fully considers the characteristics of robotic dance performances, constructing a complete process from environmental perception to movement execution through a layered architecture.

[0030] At the system's foundational layer, the map processing module undertakes the core task of environmental modeling. This module processes point cloud data collected by LiDAR to construct a high-precision digital stage model. It not only supports basic operations such as point cloud rotation and translation, but also performs point cloud optimization and noise filtering. The module utilizes data processing algorithms, supports multiple point cloud data formats, and possesses real-time data processing capabilities, enabling dynamic updates of the environmental model and providing reliable environmental data support for subsequent choreography design.

[0031] The Digital Stage Twin module builds an intuitive visual stage environment based on point cloud maps. This module converts raw point cloud data into an easy-to-understand and easy-to-use digital stage. It supports visual editing of stage layouts, generation and management of reference grids, and dynamic adjustment of stage elements. Through multi-view stage previews and real-time rendering technology, designers gain an intuitive understanding of the stage environment, providing accurate spatial reference for dance movement design. The module's editing interface focuses on user experience and supports dynamic updating of stage elements, significantly improving the efficiency of dance design.

[0032] The dance movement design module, serving as the system's creative layer, provides a wealth of movement editing and choreography capabilities. This module supports the creation, editing, and optimization of movement sequences, including fine-tuning upper limb movements, precise control of movement timing, and the management and reuse of movement libraries. Through an intuitive graphical interface and a rich selection of movement templates, designers can quickly implement complex dance movement designs and preview the effects in real time. The module's movement optimization algorithm ensures smooth and natural movements, providing professional motion support for robot performances.

[0033] The trajectory planning module is responsible for planning the robot's motion path on stage and is a key component of the system's motion control. Based on the digital stage environment and the robot's kinematic constraints, this module generates safe and smooth motion trajectories. It supports a variety of trajectory generation algorithms, including smooth trajectories based on spline interpolation and obstacle avoidance-based path planning. The module's real-time obstacle avoidance and trajectory optimization ensure the robot's safe movement in complex stage environments, providing reliable motion support for dance performances.

[0034] The dance synthesis module organically integrates the results of movement design and trajectory planning to generate a complete dance performance plan. This module achieves precise synchronization between movement and trajectory, supports fine-tuning of performance timing, and provides real-time effect preview. With precise timing synchronization and overall effect preview, designers can intuitively evaluate the dance effect and make necessary adjustments and optimizations. The module supports the export of performance plans, providing complete control instructions for subsequent execution.

[0035] The system control module, serving as the execution layer, is responsible for translating dance plans into actual robot movements. This module implements real-time parsing and execution of dance commands, including motion control, posture adjustment, and movement synchronization. Precise motion control and comprehensive status monitoring ensure the stability and reliability of the performance. The module's exception handling mechanism enables timely response to various emergencies, ensuring a smooth performance.

[0036] The module relationship of the system of the present invention adopts a layered architecture design, and the data flow is clear and unambiguous. The bottom-level data flows from the map processing module to the digital stage twin module, the middle-level data flows to the dance movement design module and the trajectory planning module respectively, and the upper-level data is gathered to the dance synthesis module, and finally executed by the system control module. The modules communicate with each other through standardized data interfaces, supporting two-way data interaction and real-time data updates, ensuring the integrity and scalability of the system. The module relationship and data flow of the system can be found in Figure 2The map processing module is the basic data layer of the system, responsible for environment modeling; the digital stage twin module is the visualization layer of the system, used for the subsequent construction of the digital stage environment; the dance movement design module is the creative layer of the system, responsible for the choreography, formation and point design; the trajectory planning module is responsible for generating the robot's motion trajectory; and finally, the dance synthesis module integrates the movements and trajectories, generates a task queue, and assigns them to different robots for execution.

[0037] In this embodiment, the digital stage twin module adopts a layered architecture design, which includes a data preprocessing layer, a point cloud processing layer, an intelligent cutting module, a grid generation module and an interactive control layer.

[0038] The data preprocessing layer can perform quality assessment, noise filtering, density homogenization and format standardization on the original point cloud PCD data.

[0039] The noise filtering algorithm uses the statistical outlier detection method to filter out noise. The calculation formula is as follows:

[0040]

[0041] in, for point to its The average distance to the nearest neighbors, is the mean of the average distances of all points, is the standard deviation, is the jth neighbor point. When , it is determined to be a noise point, where α is the threshold coefficient, usually 2.0.

[0042] The geometric transformation layer can implement various geometric transformation operations on point clouds, including translation, rotation, scaling, shearing, and complex nonlinear transformations. The expression of translation transformation is as follows:

[0043] in, is the translation transformation matrix, for Directional displacement, for Directional displacement, for Directional displacement.

[0044] The rotation transformation algorithm is represented by quaternion to avoid the gimbal lock problem. Its expression is as follows:

[0045] in, is the rotation transformation matrix, is a quaternion, is the real part, 、 、 is the imaginary part.

[0046] The expression for the scaling transformation is as follows:

[0047] in, is the scaling transformation matrix, is the scaling factor of x, is the scaling factor of y, is the scaling factor of z.

[0048] The final expression of the composite transformation is as follows:

[0049] in, is the original point cloud, is the transformed point cloud.

[0050] The point cloud processing layer is constructed based on the adaptive downsampling method, which adaptively adjusts the voxel size according to the point cloud density distribution, ensuring the details of key areas while improving the overall processing efficiency.

[0051] Point density The expression to be calculated is as follows:

[0052] in, is the radius The neighborhood point set within is the volume of the neighborhood sphere, is the Gaussian weight function, and its calculation formula is as follows:

[0053] in, is the Euclidean distance between two points, is the standard deviation of the Gaussian distribution, which is used to control the "width" or "influence range" of weight decay.

[0054] Calculating voxel size The expression is as follows:

[0055] in, , are the lower and upper bounds of the voxel size, is the sensitivity parameter.

[0056] The multi-level detail model (LOD) supports dynamic detail switching based on viewing distance by constructing a pyramid-like multi-resolution point cloud structure. Its expression is as follows:

[0057] in, For the The point set contained in the layer LOD (Level of Detail); for point The local point cloud density; For the Density threshold for layer LOD.

[0058] The expression of the view distance-dependent multi-level detail model is as follows:

[0059] in, The LOD level that should be selected currently. is the observation distance, is the reference distance.

[0060] The intelligent cutting module uses an improved ray casting algorithm to cut complex polygons. Its calculation expression is as follows:

[0061]

[0062] in, is the two-dimensional point to be judged, A two-dimensional point Regarding the winding number of polygons, is the i-th vertex of the polygon, From point P to the i-th vertex vector, is the vector angle.

[0063] The expression for determining the interior of a polygon is as follows:

[0064] in, Indicates whether point P is inside the polygon.

[0065] Then, according to the judgment results, layered cutting management is carried out.

[0066] Furthermore, multi-level stage areas can be managed independently, and different visibility and editing permissions can be set for each level.

[0067] The expression of the level mask is as follows:

[0068] in, for point The total level mask of (whether it is in the activation layer), To activate a level set, For the The mask function of the layer.

[0069] The dynamic reference grid module is based on adaptive grid generation, which can automatically generate multi-level reference grids according to the stage size and accuracy requirements. The expression for generating the main grid is as follows:

[0070] in, is the set of main grid points, 、 is the grid index (integer), The spacing of the major grid (the distance between major grid lines), Represents the 3D coordinates of the primary grid point (usually z=0 represents the stage plane).

[0071] The expression for subgrid generation is as follows:

[0072] in, is the set of sub-grid points; The number of subdivisions of the main grid, used to indicate that each main grid spacing is divided into n parts.

[0073] Dynamically adjust the grid display density based on the view zoom level to control the grid visibility. The expression of the visibility function is as follows:

[0074] in, Is the grid visibility function, returns The values ​​between indicate the visibility of the grid. is the current zoom level (usually a zoom factor, such as 1.0 for 100%), The threshold (zoom level) for the main grid display, The threshold (zoom level) for subgrid display.

[0075] Based on the mouse picking algorithm, real-time interactive response is performed to achieve accurate 3D point cloud picking function.

[0076] ray The generated expression is as follows:

[0077] in, is the starting point of the ray, is the ray direction, is the ray parameter, which represents the distance from the starting point along the direction.

[0078] The formula for calculating the closest point is as follows:

[0079] in, is the picked (selected) point, is the i-th point in the point cloud, R( ) is the point on the ray closest to the point cloud, is the ray parameter, which indicates the position on the ray closest to the point cloud.

[0080] Furthermore, to support real-time transformation preview during dragging, an incremental update strategy is adopted. The expression of the incremental transformation matrix is ​​as follows:

[0081] in, is the incremental transformation matrix, which represents the transformation from the previous state to the current state. is the current transformation matrix, is the inverse of the transformation matrix at the previous moment.

[0082] In this embodiment, the dance design module is a complete system for designing and choreographing humanoid robot dance movements. Through a graphical interface and intelligent algorithms, it automates the entire process, from single-movement design to multi-robot collaborative choreography. The system utilizes a layered architecture, integrating core functional modules such as movement instruction, trajectory planning, timing choreography, and file generation.

[0083] This module is responsible for the design and generation of the robot's individual movements. It can record key posture points through manual teaching and automatically generate smooth movement curves using intelligent algorithms. It includes the following: Assume the key frame time sequence is , the corresponding joint angle is , then the i-th segment spline curve is:

[0084] in 、 、 、 are the coefficients of the i-th segment spline curve, is the time parameter, is the time point of the i-th key frame, .

[0085] Construct continuity constraints, including position continuity, first-order derivative continuity, and second-order derivative continuity.

[0086] The expression for position continuity is as follows:

[0087] The expression for the continuity of the first-order derivative is as follows:

[0088] The expression for the continuity of the second-order derivative is as follows:

[0089] The expressions for solving the matrix equation with coefficients are as follows:

[0090] ; 、 … is the second-order derivative coefficient of the i-th segment of the spline curve.

[0091] By constructing an objective function about minimizing jerk, we optimize the smoothness of the motion and minimize the jerk. The expression is as follows:

[0092] in, For joints The jerk (third derivative), is the number of robot degrees of freedom, is the angle of the j-th joint at time t.

[0093] The optimization solution method uses the Lagrange multiplier method to transform the constrained optimization problem into an unconstrained problem. The calculation formula is as follows:

[0094] in, is the Lagrangian function, is the Lagrange multiplier of the kth constraint, is the kth constraint function.

[0095] In this embodiment, the trajectory generation module includes the following contents: Design the robot's trajectory within the digital stage's grid coordinate system, generating 2D trajectory data containing position, direction, and time information. Core features include visual trajectory drawing (supporting straight lines, curves, and composite paths); precise path point positioning (grid-based coordinate capture); and speed curve design (variable speed trajectory generation).

[0096] Given a sequence of control points ,and then Degree spline curve Defined as:

[0097] in, The expression for the B-spline basis function is as follows:

[0098]

[0099] in, For the indivual Step Spline basis functions, is the value of the i-th node.

[0100] In this embodiment, the Time Choreography module is the core component of the entire choreography system. It provides an intuitive timeline interface similar to professional video editing software, allowing users to precisely arrange and combine the movements and trajectories of multiple robots. This module's design philosophy simplifies complex multi-robot collaborative choreography into visual drag-and-drop operations, allowing even non-expert users to quickly master choreography.

[0101] The system utilizes a multi-track parallel display design, with each robot having its own independent timeline track. Users can add action clips or adjust trajectory paths for a specific robot at any time. The interface design fully considers the convenience of human-computer interaction, supporting multiple operation modes, drag-and-drop adjustment of action clips, batch selection and editing, and real-time zoom browsing. The timeline supports up to 64 concurrent tracks, each of which can accommodate up to 1,000 action clips, meeting the complex choreography requirements of large-scale robot group dances.

[0102] The system implements intelligent segment fusion technology for managing motion clips. When two motion clips are set to overlap, the system automatically detects the overlapping area and applies a smooth transition algorithm to blend them. This motion blending algorithm uses spherical linear interpolation (SLERP) to ensure smooth transitions between joint angles in quaternion space.

[0103] The data structure design of the system of the present invention utilizes an efficient hierarchical storage model. The timeline track serves as the top-level container, containing the robot identifier, a list of action segments, a list of trajectory segments, and time boundary information. An action segment records key attributes such as the action identifier, timing parameters, fade effects, and transition type. This structured design not only facilitates data organization and management but also facilitates subsequent serialization and file export.

[0104] The real-time preview function is a key feature of the timing choreography module. The system can provide real-time rendering previews of multi-robot coordinated movements at a frame rate of no less than 30fps. The preview engine adopts a layered rendering strategy, independently rendering the robot's movements, movement trajectories, and stage environment, and then synthesizing them into the final preview image. To improve rendering efficiency, the system implements adaptive level of detail (LOD) management, dynamically adjusting rendering accuracy based on viewing distance and importance. The preview process also supports multiple viewing angles: a top-down view for observing overall formation changes, a side view for checking the high degree of movement coordination, and a free view for comprehensive effect evaluation.

[0105] The system features built-in beat detection and automatic alignment, automatically indicating key timings based on the background music's tempo, helping users achieve precise synchronization between music and movement. A collision detection algorithm monitors potential collisions between robots in real time and highlights potential collisions to prompt users to adjust their choreography. A version management feature automatically saves and revisits choreography history, allowing users to freely experiment with different choreography ideas without worrying about data loss.

[0106] The system also offers a rich set of layout templates and quick operation functions. Users can save commonly used layout patterns as templates and quickly reuse them in subsequent creations. The batch operation function supports unified adjustments for multiple robots or time periods, greatly improving the efficiency of large-scale layouts. The import and export function is compatible with multiple standard formats, facilitating data exchange and collaborative development with other professional software.

[0107] In this embodiment, the dance description file generation module is the final output link of the entire design process, responsible for converting complex timeline choreography data into a set of robot-executable instructions. The design goal of this module is to establish a standardized, efficient, and cross-platform dance description language to ensure that design intent can be accurately communicated to the robot execution system.

[0108] The module adopts a multi-level data organization architecture. The top level contains metadata about the dance, such as the title, total duration, number of participating robots, stage dimensions, and other global parameters. This information provides an important reference for the initialization and resource allocation of the robot system. At the individual robot level, the system creates an independent data block for each participant, recording its complete action sequence and movement trajectory in detail. The action sequence part is organized using a timestamp index. Each action segment contains an action identifier, start time, duration, and specific parameter configuration to ensure the precise execution of the action. The trajectory part records the robot's spatial movement information in the form of key path points, including position coordinates, heading angle, and corresponding timestamps.

[0109] To meet the needs of diverse application scenarios and hardware platforms, the system supports flexible switching between multiple output formats. The lightweight JSON data format is used as the primary data exchange format, offering excellent readability and cross-platform compatibility, facilitating development, debugging, and system integration. XML (Extensible Markup Language) provides enhanced data validation capabilities and hierarchical structure representation, making it suitable for formal performances requiring high data integrity. The binary format is optimized for transmission efficiency and storage space, making it particularly well-suited for large-scale robot group dances or applications with limited network bandwidth.

[0110] Therefore, this system has significant technical advantages and innovative features in the field of robot dance design and control. Through modular design and advanced technology, the system has demonstrated unique advantages in environmental modeling, design process, stage adaptation and dance design.

[0111] Regarding environmental modeling, the system utilizes high-precision point cloud PCD-based environmental modeling technology to achieve precise digitization of the stage environment. Through advanced point cloud processing algorithms, the system constructs a highly accurate environmental model, providing an accurate positioning reference for the robot. The reusable design of environmental data allows the same stage environment to be quickly applied to different performance scenarios, significantly improving the system's practicality and efficiency. Furthermore, the system supports real-time updates and dynamic adjustments to the environmental model, ensuring that the robot can accurately perceive and adapt to changes in the stage environment.

[0112] The system's design process utilizes a modular architecture, enabling efficient management of the dance design process. Through a standardized file system, the system enables orderly storage and rapid access to various data types. The visual editing interface makes complex dance movement design intuitive and efficient. The system supports rapid environment switching, allowing designers to seamlessly switch between different stage scenes, significantly improving design efficiency. Furthermore, the system fully records all design operations, supporting traceability and facilitating subsequent optimization and adjustment.

[0113] The system demonstrates exceptional flexibility and adaptability in stage adaptation. Through standardized interface design, it can quickly adapt to various stage environments, including standard and specialized ones. The system provides precise environmental adjustment tools, enabling fine-tuning and optimization of stage layouts. The modular design of dance movements allows for easy reuse and adaptation across diverse stage environments. The scene switching function enables rapid switching between different stage environments, ensuring a continuous and fluid performance.

[0114] The system's comprehensive and powerful dance design framework seamlessly integrates movement design with the environment. Leveraging advanced motion design tools and algorithms, the system supports complex stage performance movement design. A comprehensive dance synthesis solution ensures precise synchronization of movement and trajectory, resulting in a smooth dance performance. The system's modular design ensures excellent scalability, allowing for the flexible addition of new functional modules as needed. Furthermore, the system provides a comprehensive performance evaluation system, enabling comprehensive assessment and optimization of dance performances.

[0115] An embodiment of a device applying the method of the present invention: An electronic device comprising: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned robot motion control method based on digital twins.

[0116] A computer medium embodiment of the method of the present invention: A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned robot motion control method based on digital twins.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, optical storage, etc.) containing computer-usable program code.

[0118] The present application is described in terms of flowcharts or / and block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process or / and block in the flowchart or / and block diagram, as well as the combination of processes or / and blocks in the flowchart or / and block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] The module in this application is an object that objectively describes the morphological structure with the help of physical or virtual representation. The object is not equal to the physical body and is not limited to physical and virtual. It can be a data processing function, software program, processing mode, usage method, operation method, workflow, application process, electronic hardware, circuit module, processing system, system imitation or simulation object.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field can still modify or replace the specific implementation methods of the present invention. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A robot motion control method based on digital twins, characterized by: Includes the following: Obtain point cloud data collected by LiDAR through the pre-built environment acquisition module; Use the pre-built map processing module to process the point cloud data and obtain the environment map; Based on the pre-built digital stage twin module, a digital stage model is constructed on the basis of the environment map; Using the pre-built dance movement design module, the robot's dance movements are designed based on the digital stage model to obtain dance movement information; Generate the robot's motion trajectory based on the pre-built trajectory planning module, the digital stage model, and the robot's kinematic constraints; Using a pre-built dance synthesis module, dance movement information and motion trajectory are coupled to generate a complete dance performance plan, realizing robot motion control based on digital twins.

2. The robot motion control method based on digital twin according to claim 1, characterized in that: The method for processing point cloud data using the pre-built map processing module to obtain an environment map is as follows: Obtain the original point cloud data, and perform noise filtering on the point cloud data to obtain point cloud filtered data; Perform geometric transformation on the point cloud filtering data to obtain point cloud geometric data, so that it can be translated, rotated, scaled, sheared and nonlinearly transformed; Perform density homogenization on the point cloud geometric data to obtain uniform point cloud data; Based on the uniform point cloud data, a pyramid-type multi-resolution point cloud structure is constructed, and an environment map is generated to achieve dynamic detail switching based on viewing distance.

3. The robot motion control method based on digital twin according to claim 2, characterized in that: The method for performing noise filtering on point cloud data to obtain point cloud filtered data is as follows: Obtaining original point cloud data, which includes several collection points; Calculate the distance between each collection point and the remaining collection points to obtain the nearest neighbor distance of each collection point; Calculate the distance between the collected points and obtain the mean and standard deviation of the average distance of all points; The distance threshold is calculated based on the mean, standard deviation and threshold coefficient; Compare the nearest neighbor distance of each collection point with the distance threshold to select noise points; The noise points are filtered out from the point cloud data to obtain point cloud filtered data.

4. The robot motion control method based on digital twin according to claim 2, characterized in that: The method for performing density homogenization on point cloud geometric data to obtain uniform point cloud data is as follows: Obtaining point cloud geometric data, which includes a number of geometric points; Calculate the neighborhood point set of each geometric point within a certain radius; Calculate the weight values ​​between geometric points according to the Gaussian weight function; Calculate the density value of the geometric point according to the weight value of the geometric point and the neighborhood point set; According to the point cloud density distribution requirements, set the lower and upper bounds of the voxel size and the sensitivity parameters; Based on the lower bound value, the upper bound value, the sensitivity parameter and the density value of the geometric point, the voxel value of the geometric point is calculated; Based on several voxel values, the point cloud geometric data is density-homogenized to obtain uniform point cloud data.

5. The robot motion control method based on digital twin according to claim 1, characterized in that: Based on the pre-built digital stage twin module and the environment map, the method for building a digital stage model is as follows: Acquire an environment map, which includes a plurality of coordinate points; Based on adjacent coordinate points, multiple coordinate vectors are calculated; According to the adjacent coordinate vectors, several vector angles are calculated; According to the vector angle, calculate the attribute value of a certain coordinate point; Based on the attribution value of the coordinate point, determine whether a coordinate point belongs to a certain polygon, so as to obtain the attribution data of all coordinate points; Based on the attribution data, the environment map is cut and the area management is performed to obtain the stage structure data; According to the stage size and accuracy requirements, the stage structure data is processed to obtain a multi-level reference grid, which includes a main grid and sub-grids; Based on the view zoom level, the display density of the main grid and sub-grids is adjusted to obtain a digital stage model.

6. The robot motion control method based on digital twin according to claim 1, characterized in that: The robot's dance movements are designed using a pre-built dance movement design module based on the digital stage model. The method for obtaining dance movement information is as follows: Based on the robot dance movements to be designed, key posture points are obtained; According to the key posture points, set the key frame time sequence and the corresponding joint angle; Construct a spline curve based on the key frame time series and the corresponding joint angles; According to the spline curve, set the continuity constraints and coefficients to solve the matrix equation; Continuity constraints include position continuity, first-order derivative continuity, and second-order derivative continuity; Based on the continuity constraint, an objective function is set to optimize the smoothness of the action; The objective function aims to minimize the joint jerk; According to the objective function, the Lagrange multiplier method is used to transform the constrained optimization problem into an unconstrained problem, thereby generating a smooth motion curve, namely the dance movement information.

7. The robot motion control method based on digital twin according to claim 1, characterized in that: According to the pre-built trajectory planning module, based on the digital stage model and combined with the robot's kinematic constraints, the method for generating the robot's motion trajectory is as follows: Based on the grid coordinate system of the digital stage model and combined with the robot's kinematic constraints, key path points are set; According to the key path points and based on the spline curve algorithm, the control point sequence and basis function are constructed; According to the control point sequence and basis function, the expression of the spline curve is obtained; The expression of the spline curve is solved to obtain the robot's movement trajectory, which includes the movement position, movement direction and time information.

8. The robot motion control method based on digital twin according to claim 1, characterized in that: The method for using the pre-built dance synthesis module to couple dance movement information and motion trajectories to generate a complete dance performance plan is as follows: A multi-track parallel display design method is adopted to set a timeline track for each robot, so that each robot has an independent timeline; the timeline track is used to add action clips or adjust the trajectory path for one or more robots at any time; Based on dance movement information and creation information, the timing choreography module is used to obtain drag operation information, so that multi-robot collaborative choreography can be achieved through drag operations; Based on the timeline track, the dance movement information, drag operation information and motion trajectory of each robot are coupled and processed to obtain the movement sequence and duration of each robot. The robot collaborative control algorithm is then combined to ensure the synchronization of the movements of multiple robots. Arrange and combine the movements and trajectories of multiple robots according to the movement sequence and duration of each robot to obtain choreography data; With the help of the dance description file generation module, the choreography data is converted into an execution instruction set to generate a complete dance performance plan.

9. A robot motion control method based on digital twins, characterized by: The following steps are involved: Step 1: Obtain point cloud data collected by the lidar through the pre-built environment acquisition module; Step 2: Use the pre-built digital stage twin module to process the point cloud data and obtain the digital stage model; Step 3: Using the pre-built dance movement design model, dance movement information is generated according to the digital stage model; In step 4, a pre-built trajectory planning module is used to generate the robot's motion trajectory based on dance movement information and the digital stage model, combined with the robot's kinematic constraints, to achieve robot motion control based on digital twins.

10. An electronic device, characterized in that: It includes: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a robot motion control method based on digital twins as described in any one of claims 1 to 9.

Images