Virtual and real action synchronous comprehensive mapping method and device, equipment and medium

Abstract

The invention discloses a virtual and real action synchronous comprehensive mapping method, device and equipment and a medium, and relates to the technical field of intelligent manufacturing digitization, and the method remarkably reduces the workload of software development through an innovative technical means. By automatically recording the key parameters of the virtual space and carrying out statistical processing on the key points of the real-time data, the tedious process of manually setting and adjusting a large number of parameters in a traditional mode is avoided, so that the development process is greatly simplified, and the development efficiency is improved. And meanwhile, the action mapping mode is simplified. A traditional action mapping method often needs complex programming and debugging work, but according to the comprehensive mapping mode, key points of real-time data are automatically mapped to a virtual model through a physical engine, and accurate virtual and real action synchronization can be achieved without complex programming. Therefore, the technical threshold is reduced, and the action mapping is more intuitive and easy to understand.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing digital technology, and in particular to a method, device, equipment and medium for synchronous comprehensive mapping of virtual and real actions. Background Art

[0002] Patent document CN 117725757 A records a virtual-real mapping method and system for the assembly process of a digital twin workshop. The present invention relates to a virtual-real mapping method and system for the assembly process of a digital twin workshop, including: collecting various types of production data in the assembly process of a digital twin workshop, preprocessing the collected data, and transmitting the preprocessed data in real time to provide motion data in the virtual entity assembly process; analyzing the production status and production behavior in the assembly process of the digital twin workshop, and constructing a behavior logic model based on Petri nets; decomposing the complex production behavior of the production equipment used in the assembly process of the digital twin workshop, and replacing the overall movement of the equipment with the composite movement of the sub-components in translation and rotation according to the kinematic relationship between the production equipment and its sub-components; performing simulation mapping based on the behavior logic model, real-time data transmission, and decomposition of the complex production behavior of the production equipment to realize real-time drive of the digital twin workshop. The monitoring efficiency and accuracy of the digital workshop are improved.

[0003] Patent document CN 111797163 B records a virtual-real synchronization system and implementation method based on digital twins. The present invention provides a virtual-real synchronization system and implementation method of digital twins. Digital twin 1 is arranged locally together with the device, connected through a local network, and constitutes a local system; digital twin 2 is arranged at the remote end or in the cloud, in a remote system, and digital twin 2 is connected to digital twin 1 through a wide area network. On the one hand, the present invention takes into account local real-time processing and local security requirements, and on the other hand, it transforms the traditional transmission of "rough" data into information reflected by the transmission of a large amount of data, which greatly reduces the requirements for network bandwidth. Encryption between digital twins 1 and 2, and private protocols ensure security. In the overall implementation method, the device is securely isolated from the digital twin in a wide area network environment, which is conducive to the virtual-real synchronization of the device entity and its digital twin under conditions of high security requirements and limited network capabilities.

[0004] Existing technologies all focus on the overall framework, and lack a comprehensive solution for the synchronization of virtual and real action mapping. Existing technologies usually require developers to develop and debug each action mapping between a virtual model and an actual device separately, which results in a large amount of manpower and time being invested in the development process. There is a lack of a unified mapping method, and developers need to customize the mapping scheme according to the characteristics of different virtual models and actual devices, which not only increases the difficulty of development, but also increases the possibility of errors. Once the mapping scheme is developed and released, it is difficult for non-developers to adjust it. This limits the flexibility and adaptability of the system, requiring a lot of development work to be done again when the device is updated or the environment changes. Summary of the invention

[0005] In view of the above problems, the present invention provides a method, device, equipment and medium for synchronous comprehensive mapping of virtual and real actions for overcoming the above problems or at least partially solving the above problems.

[0006] The present invention provides the following scheme:

[0007] A virtual-real action synchronous comprehensive mapping method, comprising:

[0008] During the operation of the virtual model, the position of the model is dragged, the animation execution speed is adjusted, the binding variable type is recorded, whether the model has a parent object is identified, and the rotation direction and speed are collected, so as to enter the trajectory parameters of each target component in the virtual model;

[0009] Write the entered trajectory parameters into the XML1 file through a script;

[0010] Perform statistical processing on the data received by the real-time data interface, extract the key mapping position values ​​corresponding to each of the target components during the actual equipment operation process and write them into the XML2 file;

[0011] Access the hierarchical address of the data interface to be bound to the model corresponding to each target component in the XML1 file; and query the interface address of the corresponding sub-element in the XML2 file, so as to complete the binding of the virtual model with the real-time data interface according to the hierarchical address and the interface address;

[0012] After determining that the user has completed modifying the trajectory parameters in the XML1 file, the modified trajectory parameters are used in combination with the binding relationship between the virtual model and the real-time data interface to achieve synchronous adjustment of the actions of the virtual model and the actual device.

[0013] Preferably: the target component includes a cylinder, the motion coordinates of the virtual space cylinder drive model component are recorded by position dragging, the motion target position is determined by the Boolean variable value, and linear interpolation is performed between the current position of the model and the target position to achieve smooth movement to obtain the trajectory parameters of the cylinder.

[0014] Preferably: after receiving the time step required to be adjusted according to the actual movement time of the cylinder input by the user, it is determined that the user has completed the modification of the trajectory parameters of the cylinder in the XML1 file.

[0015] Preferably, the target component includes a servo motor, and the trajectory parameters of the servo motor are obtained by obtaining the virtual space coordinates of the model of the servo motor by dragging the model of the servo motor.

[0016] Preferably: receiving a certain axial coordinate value of the servo motor after modification input by the user, and determining that the user has completed modification of the trajectory parameters of the servo motor in the XML1 file.

[0017] Preferably: determine whether there are parent-child relationship objects between the servo motor, receive the parent object coordinates input by the user and obtained in real time through the child object, and the axes bound to the actual data together constitute the real-time target position coordinates of the child object, and determine whether the user has completed the modification of the trajectory parameters of the servo motor in the XML1 file.

[0018] Preferably: the target component includes a robotic arm, and when it is determined that the data points of each joint angle of the robotic arm can be obtained, the real-time data is used to drive the model to rotate, and the model is rotated to the target angle in the form of a quaternion to perform smooth motion frame by frame;

[0019] When only the position coordinates of the end effector are determined, an inverse kinematics solution algorithm is used to solve the rotation angles of the joint axes of the robotic arm;

[0020] When it is determined that there is only one gripper Boolean variable or no data points related to the posture are known at all, the animation system is used to realize action production, and the animation is segmented on the development platform. The state is inferred by obtaining the robotic arm point value or using the point data of upstream and downstream equipment to achieve synchronization of the animation trigger points, and then the synchronization of the virtual model and the actual robotic arm is achieved by adjusting the playback speed of the segmented animation.

[0021] A virtual-real action synchronous comprehensive mapping device, used to execute the virtual-real action synchronous comprehensive mapping method, the device comprising:

[0022] A virtual model parameter recording unit is used to drag the model, adjust the animation execution speed, record the type of bound variables, identify whether the model has a parent object, and collect the rotation direction and speed during the operation of the virtual model, so as to enter the trajectory parameters of each target component in the virtual model;

[0023] An input unit, used for writing the input trajectory parameters into an XML1 file through a script;

[0024] The actual equipment operation data acquisition unit is used to perform statistical processing on the data received by the real-time data interface, extract the key mapping position values ​​corresponding to each of the target components during the actual equipment operation process, and write them into the XML2 file;

[0025] An interface binding unit, used to access the hierarchical address of the data interface to be bound to the model corresponding to each target component in the XML1 file; and query the interface address of the corresponding sub-element in the XML2 file, so as to complete the binding of the virtual model with the real-time data interface according to the hierarchical address and the interface address;

[0026] The synchronous adjustment unit is used to determine that after the user has completed the modification of the trajectory parameters in the XML1 file, the modified trajectory parameters are combined with the binding relationship between the virtual model and the real-time data interface to achieve synchronous adjustment of the actions of the virtual model and the actual device.

[0027] A device for synchronous comprehensive mapping of virtual and real actions, the device comprising a processor and a memory:

[0028] The memory is used to store program code and transmit the program code to the processor;

[0029] The processor is used to execute the above-mentioned virtual and real action synchronous comprehensive mapping method according to the instructions in the program code.

[0030] A computer-readable storage medium is used to store program codes, and the program codes are used to execute the above-mentioned virtual-real action synchronous comprehensive mapping method.

[0031] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0032] The embodiments of the present application provide a method, device, equipment and medium for synchronous comprehensive mapping of virtual and real actions, which significantly reduces the workload of software development through innovative technical means. By automatically recording the key parameters of the virtual space and performing statistical processing on the key points of real-time data, the present invention avoids the tedious process of manually setting and adjusting a large number of parameters in the traditional method, thereby greatly simplifying the development process and improving development efficiency.

[0033] At the same time, the action mapping method is simplified. Traditional action mapping methods often require complex programming and debugging, while the comprehensive mapping method of the present invention automatically maps the key points of real-time data to the virtual model through the physical engine, without complex programming, to achieve accurate virtual and real action synchronization. This not only lowers the technical threshold, but also makes action mapping more intuitive and easy to understand.

[0034] In addition, the adjustability of later release is improved. Since the statistical processing of key parameters and real-time data is automated and written into XML files for storage, developers can easily adjust the mapping parameters as needed to achieve the best synchronous display effect during later release. This flexibility enables the present invention to better adapt to different application scenarios and user needs, thereby enhancing the market competitiveness of the product. By reducing the workload of software development, simplifying the action mapping method, and improving the adjustability of later release, it brings significant convenience and advantages to the development and application of virtual-real synchronous display.

[0035] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 It is a flow chart of a method for synchronous comprehensive mapping of virtual and real actions provided by an embodiment of the present invention;

[0038] Figure 2 It is an overall framework diagram of a method for synchronous comprehensive mapping of virtual and real actions provided by an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of a virtual space key parameter XML file provided by an embodiment of the present invention;

[0040] Figure 4 This is a schematic diagram of a real-time data key parameter xm2 file provided by an embodiment of the present invention;

[0041] Figure 5 is a schematic diagram of a device for synchronous comprehensive mapping of virtual and real actions provided by an embodiment of the present invention;

[0042] Figure 6 It is a schematic diagram of a device for synchronous comprehensive mapping of virtual and real actions provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0044] See also Figure 1 , a virtual-real action synchronous comprehensive mapping method provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0045] S101: during the operation of the virtual model, the position of the model is dragged, the animation execution speed is adjusted, the binding variable type is recorded, whether the model has a parent object is identified, and the rotation direction and speed are collected, so as to enter the trajectory parameters of each target component in the virtual model;

[0046] S102: writing the entered trajectory parameters into the XML1 file through a script;

[0047] S103: Performing statistical processing on the data received by the real-time data interface, extracting the key mapping position values ​​corresponding to each of the target components during the actual device operation process and writing them into the XML2 file;

[0048] S104: Accessing the hierarchical address of the data interface to be bound to the model corresponding to each target component in the XML1 file; and querying the interface address of the corresponding sub-element in the XML2 file, so as to complete the binding of the virtual model with the real-time data interface according to the hierarchical address and the interface address;

[0049] S105: After determining that the user has completed modifying the trajectory parameters in the XML1 file, synchronously adjusting the actions of the virtual model and the actual device by using the modified trajectory parameters in combination with a binding relationship between the virtual model and the real-time data interface.

[0050] The virtual-real action synchronization comprehensive mapping method provided by the present application establishes a virtual space key parameter record and a real-time data key point statistical processing, respectively synchronizes the trajectory of the virtual model and the real-time data processing, obtains the motion trajectory points and constraint positions of the virtual space model, and calculates and processes the actual equipment operation data. With the device name as the association point, a fast mapping between the virtual model and the actual device is achieved, which greatly reduces the workload of developers. At the same time, through a unified mapping method, the complexity of the mapping is reduced, allowing developers to complete the mapping work more quickly and accurately, and improving development efficiency. In addition, non-developers are allowed to adjust the mapping scheme through simple operations, which enhances the flexibility and adaptability of the system. When the equipment is updated or the environment changes, users can quickly make corresponding adjustments without having to do a lot of development work.

[0051] Furthermore, the target component includes a cylinder, and the motion coordinates of the virtual space cylinder driving model component are recorded by position dragging, the motion target position is determined by the Boolean variable value, and linear interpolation is performed between the current position of the model and the target position to achieve smooth movement and obtain the trajectory parameters of the cylinder.

[0052] After receiving the time step that needs to be adjusted according to the actual movement time of the cylinder input by the user, it is determined that the user has completed the modification of the trajectory parameters of the cylinder in the XML1 file.

[0053] The target component includes a servo motor, and the trajectory parameters of the servo motor are obtained by obtaining the virtual space coordinates of the model of the servo motor by dragging the model of the servo motor.

[0054] A certain axial coordinate value of the servo motor after modification input by the user is received, and it is determined that the user has completed modification of the trajectory parameters of the servo motor in the XML1 file.

[0055] Determine whether there are parent-child relationship objects in the servo motor, receive the parent object coordinates obtained in real time through the child object input by the user, and the axis bound to the actual data together constitute the real-time target position coordinates of the child object, and determine that the user has completed the modification of the trajectory parameters of the servo motor in the XML1 file.

[0056] The target component includes a robotic arm. When it is determined that the data points of each joint angle of the robotic arm can be obtained, the model is driven to rotate using real-time data, and the model is rotated to the target angle and smoothly moves frame by frame in the form of a quaternion;

[0057] When only the position coordinates of the end effector are determined, an inverse kinematics solution algorithm is used to solve the rotation angles of the joint axes of the robotic arm;

[0058] When it is determined that there is only one gripper Boolean variable or no data points related to the posture are known at all, the animation system is used to realize action production, and the animation is segmented on the development platform. The state is inferred by obtaining the robotic arm point value or using the point data of upstream and downstream equipment to achieve synchronization of the animation trigger points, and then the synchronization of the virtual model and the actual robotic arm is achieved by adjusting the playback speed of the segmented animation.

[0059] The method provided by this application is introduced in detail below.

[0060] like Figure 2 As shown, the embodiment of the present application provides a method for synchronous comprehensive mapping of virtual and real actions, which aims to achieve accurate synchronous mapping of virtual space and the real world through key parameter recording, real-time data key point statistical processing, and the establishment of a relationship between a virtual model and real-time data. It reduces the workload of software development, simplifies the action mapping method, and improves the adjustability of later releases.

[0061] Real-time motion data classification.

[0062] 1. Cylinder

[0063] Cylinders are widely used in many fields such as industrial manufacturing, mechanical processing and production line control to realize various mechanical actions, such as opening and closing doors, carrying objects, etc. They have the characteristics of small size, light weight, simple structure and high reliability.

[0064] Without adding position sensors, industrial cylinders can only obtain whether they are in place through Boolean variables, and cannot perceive the position information during the operation of the cylinder.

[0065] By recording the motion coordinates of the cylinder-driven model components in the virtual space, the target position of the movement is determined by the Boolean variable value, and linear interpolation is performed between the current position and the target position of the model to achieve smooth movement. According to the actual movement time of the cylinder, the time step is adjusted to synchronize the movements.

[0066] 2. Servo

[0067] The servo motor has a fast response speed and can achieve high speed and high acceleration in a short time. It is suitable for occasions that require high-precision position control and fast response. It has the characteristics of high precision, high speed and high reliability.

[0068] The servo motor can provide real-time feedback of position information to the control system, which is generally floating-point data. The equipment operating position information can be obtained by mapping the actual read value with the limit position value.

[0069] The actual value is mapped to the virtual space coordinate using linear mapping. Currently, most motion trajectories are straight lines. In virtual space mapping, it is only necessary to modify one axial coordinate value. For special cases such as objects with a parent-child relationship, the synchronization method is for the child object to obtain the parent object's coordinates in real time, and the axis bound to the actual data together constitutes the child object's real-time target position coordinates.

[0070] 3. Robotic arms

[0071] The numerical values ​​that the robotic arm can obtain are complex and can be divided into three situations.

[0072] ①The angle of each section can be obtained.

[0073] When the data points of each joint angle can be obtained, the real-time data is used to drive the model rotation, and the model is rotated to the target angle in the form of a quaternion, and the motion is smoothed frame by frame to achieve the synchronization of the virtual robot arm motion with the actual robot arm motion. Quaternion calculation can more accurately and stably represent 3D rotation, and Euler angles will have a universal lock problem.

[0074] ②Only the position of the end effector can be obtained.

[0075] When only the position coordinates of the end effector are available, the inverse kinematics solution algorithm is used to calculate the rotation angles of each joint axis of the robotic arm.

[0076] There are two main methods for solving the inverse kinematics solution: numerical solution and analytical solution. The numerical solution can only find the specific solution of the equation, not all solutions. Its advantage is that the calculation is simple and no matrix operation is required; its disadvantage is that due to the use of iterative methods (such as Newton iteration), the real-time performance is poor, so the analytical solution is used. The analytical solution is divided into algebraic solution and geometric solution, and the joint angle rotation solution is achieved through algebraic solution.

[0077] Inverse Kinematics—Algebraic Solution.

[0078] Pieper's Criteria:

[0079] i. The axes of three adjacent joints intersect at one point;

[0080] ii. The axes of three adjacent joints are parallel to each other.

[0081] If the structure of the serial robot meets one of the above two sufficient conditions, there will be an analytical solution.

[0082] That is, for a serial robot whose six joints are all revolute joints and whose axes of the last three joints intersect at one point, the inverse solution has a closed solution. The closed solution of inverse kinematics can be solved by the following algebraic method, which is called the "Pieper solution for three-axis intersection".

[0083] Reference for relevant calculation formulas.

[0084] When the last three joint axes intersect, the origin of the link coordinate system {4}, {5}, {6} is located at the intersection, and the coordinates of the intersection in the base coordinate system {0} are as follows:

[0085]

[0086] Based on the improved DH parameter connecting rod homogeneous transformation matrix, the transformation formula of the homogeneous transformation matrix is ​​applied:

[0087]

[0088] It can be seen that when i=4,

[0089]

[0090] Applying the homogeneous transformation matrix to You can get f i The expression is:

[0091] f1=a3c3+d4sα3s3+a2

[0092] f2=a3cα2s3-d4sα3cα2c3-d4sα2cα3-d3sα2

[0093] f3=a3sα2s3-d4sα3sα2c3+d4cα2cα3+d3cα2

[0094] right and Apply the second transformation matrix to obtain g1, g2, and g3. The calculation process is as follows:

[0095] g1=c2f1-s2f2+a1

[0096] g2=s2cα1f1+c2cα1f2-sα1f3-d2sα1

[0097] g3=s2sα1f1+c2sα1f2+cα1f3+d2cα1

[0098] Let r = x 2 +y 2 +z 2 , from the above formula we can know:

[0099]

[0100] By writing the equation of the Z-axis component from the above formula, we can get two equations of the system:

[0101] r=(k1c2+k2s2)2a1+k3, z=(k1s2-k2c2)2a2+k4

[0102] For k1, k2, k3, k4, the calculations are as follows:

[0103] k1=f1

[0104] k2=-f2

[0105]

[0106] k4=f3cα1+d2cα1

[0107] Then solve θ3, divided into three cases:

[0108] (1) If α1 = 0, then r = k3, and r is known. The right side of k3 is a function of θ3. Substituting into the following equation, θ3 can be solved from the quadratic equation containing tan(θ3 / 2);

[0109]

[0110] (2) If sα1 = 0, then z = k4, and z is known. Substituting into the above formula, use the above quadratic equation to solve θ3;

[0111] (3) Otherwise, eliminating s2 and c2 from the formula r = (k1c2 + k2s2)2a1 + k3, z = (k1s2 - k2c2)2a2 + k4 yields:

[0112]

[0113] Substitution After solving θ3, we can get a quartic equation, from which we can solve θ3. After solving θ3, we can solve θ2, and then solve θ1 according to the following formula.

[0114] 0 P4=[c1g1-s1g2 s1g1+c1g2 g3 1] T

[0115] The Pieper solution solves the joint variables of the first three axes, which usually has 4 or 8 sets of solutions. Since the joint angles of the rear three axes only affect the orientation of the tool coordinate system, the joint variables of the rear three axes can be solved by the Euler angle solution.

[0116] The posture of the six-degree-of-freedom robot tool coordinate system can be expressed using a set of Euler angles To express it, this angle sequence represents the rotation sequence around the x-axis, y-axis and z-axis: first rotate around the z-axis by an angle of ψ, then rotate around the y-axis of the new coordinate system by an angle of θ, and finally rotate around the z-axis of the new coordinate system. horn.

[0117] Euler angle transformation It can be represented by multiplying three rotation matrices:

[0118]

[0119] If a homogeneous transformation representing an arbitrary rotation is known, that is, the known attitude matrix R, it is expressed by the following formula:

[0120]

[0121] According to the attitude matrix R, the corresponding Euler angle can be obtained:

[0122] Ψ=atan2(a x ,a y )+180°

[0123] θ=atan2(cΨa x +sΨa y ,a z )

[0124]

[0125] For the 6R type six-degree-of-freedom robot, the joint variables of the first three axes have been obtained, and the homogeneous transformation matrix can be calculated

[0126]

[0127] Assume that the Euler angle is established The coordinate system of is {E}, then the homogeneous transformation matrix from the base coordinate system {0} to the coordinate system {E} is:

[0128]

[0129] In the formula, α3 represents the connecting rod torsion angle of the fourth axis in the improved DH parameter table. The calculation method of the state matrix represented by Euler angle is shown in the formula:

[0130]

[0131] oERT represents the transpose of the rotation matrix in the homogeneous transformation matrix OET; Represents the homogeneous transformation matrix The rotation matrix in .

[0132] After obtaining the Euler angle attitude matrix, it can be obtained by

[0133] Ψ=atan2(a x ,a y )+180°

[0134] θ=atan2(cΨa x +sΨay ,a z )

[0135]

[0136] Calculate the corresponding Euler angle. The relationship between the obtained Euler angle and the relevant variables of the rear three axes is:

[0137]

[0138] Due to the flipping of the robot wrist, another set of solutions for the rear three axes can be obtained:

[0139] θ′4=θ4+180°, θ′5=-θ5, θ′6=θ6+180°

[0140] According to this method, multiple solutions will be calculated. In this case, a group of solutions is selected for driving execution by comprehensively considering the geometric limitations of the robot itself, the solution closest to the current state, the fastest, most energy-saving, and obstacle avoidance.

[0141] ③Others

[0142] In other cases, such as when there is only one gripper Boolean variable or no data points related to the posture are available, the animation system is used to realize action production, the animation is segmented on the development platform, and the state is inferred by obtaining the robotic arm point value or using the point data of other upstream and downstream equipment to achieve synchronization of the animation trigger points. The synchronization of the virtual model and the actual robotic arm is achieved by adjusting the playback speed of the segmented animation.

[0143] Record of key parameters of virtual space.

[0144] Based on the Unity platform development, after completing the model grouping, Unity will not save any runtime modifications and changes, and most of the virtual model action adjustments need to be completed in the running state. Therefore, the trajectory parameters are entered by dragging the model position, adjusting the animation execution speed, recording the binding variable type, whether the model has a parent object, the rotation direction and speed, etc. at runtime. The modified parameters are written into the XML1 file through the script, such as Figure 3 shown.

[0145] Real-time data key point statistical processing.

[0146] Perform statistical processing on the data received by the real-time data interface, extract the key mapping position values ​​during the actual device operation process, and write them into the XML2 file, such as Figure 4 shown.

[0147] The relationship between virtual models and real-time data is established.

[0148] By accessing the XML1 file, each model obtains the hierarchical address of the data interface that needs to be bound. In the XML2 file, the corresponding sub-element is queried to obtain the interface address, and the binding of the virtual model and the real-time data interface is completed.

[0149] This method is built based on the unity platform. After the development work is completed, when the application is officially released, the action can be adjusted by modifying the configuration file XML1 file.

[0150] In summary, the virtual-real action synchronization comprehensive mapping method provided by this application significantly reduces the workload of software development through innovative technical means. By automatically recording the key parameters of the virtual space and performing statistical processing on the key points of real-time data, the present invention avoids the tedious process of manually setting and adjusting a large number of parameters in the traditional method, thereby greatly simplifying the development process and improving development efficiency.

[0151] At the same time, the action mapping method is simplified. Traditional action mapping methods often require complex programming and debugging, while the comprehensive mapping method of the present invention automatically maps the key points of real-time data to the virtual model through the physical engine, without complex programming, to achieve accurate virtual and real action synchronization. This not only lowers the technical threshold, but also makes action mapping more intuitive and easy to understand.

[0152] In addition, the adjustability of later release is improved. Since the statistical processing of key parameters and real-time data is automated and written into XML files for storage, developers can easily adjust the mapping parameters as needed to achieve the best synchronous display effect during later release. This flexibility enables the present invention to better adapt to different application scenarios and user needs, thereby enhancing the market competitiveness of the product. By reducing the workload of software development, simplifying the action mapping method, and improving the adjustability of later release, it brings significant convenience and advantages to the development and application of virtual-real synchronous display.

[0153] See also Figure 5 , the embodiment of the present application can also provide a virtual and real action synchronous comprehensive mapping device, such as Figure 5 As shown, the device for executing the above-mentioned virtual and real action synchronous comprehensive mapping method may include:

[0154] A virtual model parameter recording unit 501 is used to drag the model, adjust the animation execution speed, record the type of bound variables, identify whether the model has a parent object, and collect the rotation direction and speed during the operation of the virtual model, so as to enter the trajectory parameters of each target component in the virtual model;

[0155] An input unit 502 is used to write the input trajectory parameters into an XML1 file through a script;

[0156] The actual equipment operation data acquisition unit 503 is used to perform statistical processing on the data received by the real-time data interface, extract the key mapping position values ​​corresponding to each target component in the actual equipment operation process, and write them into the XML2 file;

[0157] The interface binding unit 504 is used to access the hierarchical address of the data interface to be bound to the model corresponding to each target component in the XML1 file; and query the interface address of the corresponding sub-element in the XML2 file, so as to complete the binding of the virtual model and the real-time data interface according to the hierarchical address and the interface address;

[0158] The synchronous adjustment unit 505 is used to determine that after the user has completed the modification of the trajectory parameters in the XML1 file, the modified trajectory parameters are combined with the binding relationship between the virtual model and the real-time data interface to achieve synchronous adjustment of the actions of the virtual model and the actual device.

[0159] The embodiment of the present application may also provide a virtual-real action synchronous comprehensive mapping device, the device comprising a processor and a memory:

[0160] The memory is used to store program code and transmit the program code to the processor;

[0161] The processor is used to execute the steps of the above-mentioned virtual and real action synchronous comprehensive mapping method according to the instructions in the program code.

[0162] like Figure 6 As shown, a virtual-real action synchronous comprehensive mapping device provided by an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.

[0163] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0164] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of the method for synchronous integrated mapping of virtual and real actions.

[0165] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0166] During the operation of the virtual model, the position of the model is dragged, the animation execution speed is adjusted, the binding variable type is recorded, whether the model has a parent object is identified, and the rotation direction and speed are collected, so as to enter the trajectory parameters of each target component in the virtual model;

[0167] Write the entered trajectory parameters into the XML1 file through a script;

[0168] Perform statistical processing on the data received by the real-time data interface, extract the key mapping position values ​​corresponding to each of the target components during the actual equipment operation process and write them into the XML2 file;

[0169] Access the hierarchical address of the data interface to be bound to the model corresponding to each target component in the XML1 file; and query the interface address of the corresponding sub-element in the XML2 file, so as to complete the binding of the virtual model with the real-time data interface according to the hierarchical address and the interface address;

[0170] After determining that the user has completed modifying the trajectory parameters in the XML1 file, the modified trajectory parameters are used in combination with the binding relationship between the virtual model and the real-time data interface to achieve synchronous adjustment of the actions of the virtual model and the actual device.

[0171] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required for at least one function (such as a file creation function, a data reading and writing function), etc.; the data storage area can store data created during use, such as initialization data, etc.

[0172] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0173] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0174] Of course, it should be noted that Figure 6 The structure shown does not constitute a limitation on the virtual-real action synchronization comprehensive mapping device in the embodiment of the present application. In actual applications, the virtual-real action synchronization comprehensive mapping device may include Figure 6 More or fewer components than shown, or combinations of certain components.

[0175] The embodiment of the present application may also provide a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the steps of the above-mentioned virtual-real action synchronous integrated mapping method.

[0176] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0177] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0178] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0179] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for synchronous comprehensive mapping of virtual and real actions, characterized in that: include: During the operation of the virtual model, the position of the model is dragged, the animation execution speed is adjusted, the binding variable type is recorded, whether the model has a parent object is identified, and the rotation direction and speed are collected, so as to enter the trajectory parameters of each target component in the virtual model; Write the entered trajectory parameters into the XML1 file through a script; Perform statistical processing on the data received by the real-time data interface, extract the key mapping position values ​​corresponding to each of the target components during the actual equipment operation process and write them into the XML2 file; Access the hierarchical address of the data interface to be bound to the model corresponding to each target component in the XML1 file; and query the interface address of the corresponding sub-element in the XML2 file, so as to complete the binding of the virtual model with the real-time data interface according to the hierarchical address and the interface address; After determining that the user has completed modifying the trajectory parameters in the XML1 file, the modified trajectory parameters are used in combination with the binding relationship between the virtual model and the real-time data interface to achieve synchronous adjustment of the actions of the virtual model and the actual device.

2. The method for synchronous comprehensive mapping of virtual and real actions according to claim 1, characterized in that: The target component includes a cylinder. The motion coordinates of the virtual space cylinder driving model component are recorded by position dragging, the motion target position is determined by the Boolean variable value, and linear interpolation is performed between the current position of the model and the target position to achieve smooth movement and obtain the trajectory parameters of the cylinder.

3. The method for synchronous comprehensive mapping of virtual and real actions according to claim 2, characterized in that: After receiving the time step that needs to be adjusted according to the actual movement time of the cylinder input by the user, it is determined that the user has completed the modification of the trajectory parameters of the cylinder in the XML1 file.

4. The method for synchronous comprehensive mapping of virtual and real actions according to claim 1, characterized in that: The target component includes a servo motor, and the trajectory parameters of the servo motor are obtained by obtaining the virtual space coordinates of the model of the servo motor by dragging the model of the servo motor.

5. The method for synchronous comprehensive mapping of virtual and real actions according to claim 4, characterized in that: A certain axial coordinate value of the servo motor after modification input by the user is received, and it is determined that the user has completed modification of the trajectory parameters of the servo motor in the XML1 file.

6. The method for synchronous comprehensive mapping of virtual and real actions according to claim 4, characterized in that: Determine whether there are parent-child relationship objects in the servo motor, receive the parent object coordinates obtained in real time through the child object input by the user, and the axis bound to the actual data together constitute the real-time target position coordinates of the child object, and determine that the user has completed the modification of the trajectory parameters of the servo motor in the XML1 file.

7. The method for synchronous comprehensive mapping of virtual and real actions according to claim 1, characterized in that: The target component includes a robotic arm. When it is determined that the data points of each joint angle of the robotic arm can be obtained, the model is driven to rotate using real-time data, and the model is rotated to the target angle and smoothly moves frame by frame in the form of a quaternion; When only the position coordinates of the end effector are determined, an inverse kinematics solution algorithm is used to solve the rotation angles of the joint axes of the robotic arm; When it is determined that there is only one gripper Boolean variable or no data points related to the posture are known at all, the animation system is used to realize action production, and the animation is segmented on the development platform. The state is inferred by obtaining the robotic arm point value or using the point data of upstream and downstream equipment to achieve synchronization of the animation trigger points, and then the synchronization of the virtual model and the actual robotic arm is achieved by adjusting the playback speed of the segmented animation.

8. A device for synchronous comprehensive mapping of virtual and real actions, characterized in that: The device is used to execute the method for synchronous comprehensive mapping of virtual and real actions according to any one of claims 1 to 7, comprising: A virtual model parameter recording unit is used to drag the model, adjust the animation execution speed, record the type of bound variables, identify whether the model has a parent object, and collect the rotation direction and speed during the operation of the virtual model, so as to enter the trajectory parameters of each target component in the virtual model; An input unit, used for writing the input trajectory parameters into an XML1 file through a script; The actual equipment operation data acquisition unit is used to perform statistical processing on the data received by the real-time data interface, extract the key mapping position values ​​corresponding to each of the target components during the actual equipment operation process, and write them into the XML2 file; An interface binding unit, used to access the hierarchical address of the data interface to be bound to the model corresponding to each target component in the XML1 file; and query the interface address of the corresponding sub-element in the XML2 file, so as to complete the binding of the virtual model with the real-time data interface according to the hierarchical address and the interface address; The synchronous adjustment unit is used to determine that after the user has completed the modification of the trajectory parameters in the XML1 file, the modified trajectory parameters are combined with the binding relationship between the virtual model and the real-time data interface to achieve synchronous adjustment of the actions of the virtual model and the actual device.

9. A device for synchronous comprehensive mapping of virtual and real actions, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the virtual-real action synchronous comprehensive mapping method described in any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the method for synchronous comprehensive mapping of virtual and real actions as described in any one of claims 1-7.

Images