Mechanical arm virtual and real position synchronization method, device and equipment based on industrial internet and medium

Abstract

The invention belongs to the technical field of industrial robots, and particularly relates to a mechanical arm virtual and real position synchronization method, device and equipment based on the industrial internet and a medium. The method comprises the steps that according to first joint pose data of a real mechanical arm, pose adjustment is conducted on at least one virtual joint of a virtual mechanical arm, the pose of the virtual mechanical arm corresponds to the pose of the actual mechanical arm; according to the spatial position coordinates of the actual mechanical arm, the first virtual coordinates of the virtual mechanical arm are adjusted till the first virtual coordinates correspond to the spatial position coordinates; and determining a first identification coordinate of at least one actual part forming the actual mechanical arm, and adjusting a second virtual coordinate of at least one virtual part of the virtual mechanical arm according to the first identification coordinate until the second virtual coordinate corresponds to the first identification coordinate. By means of the method, accurate alignment of the virtual space and the physical space is achieved, and therefore the mechanical arm track generated through simulation is suitable for actual operation.

Description

Technical Field

[0001] This application belongs to the technical field of industrial robots, and particularly relates to a method, device, equipment and medium for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet. Background Art

[0002] During the multi-robotic arm assembly process, simulation software can effectively simulate the motion trajectories and assembly processes of robotic arms. However, in practical applications, the trajectories of robotic arms generated in the virtual world often cannot be perfectly translated into the execution routes of real-world robotic arms. This deviation mainly stems from the fact that robotic arms, equipment, and components in the actual environment cannot be precisely aligned with their positions in the virtual space.

[0003] Due to the lack of coordination between the virtual model and the real environment, virtual trajectory points cannot be effectively applied in the operation of real robotic arms. This inconsistency not only affects the assembly efficiency but also may pose operational risks. Therefore, it is urgent to solve the alignment problem between the virtual and real environments to ensure that the simulation results can be reasonably applied in actual operations. Summary of the Invention

[0004] In view of the above problems, this application provides a method, device, equipment and medium for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet.

[0005] In a first aspect, this application provides a method for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet, the method comprising:

[0006] Performing pose adjustment on at least one virtual joint of the virtual robotic arm according to the first joint pose data of the real robotic arm until the pose of the virtual robotic arm corresponds to that of the real robotic arm, wherein there is a corresponding relationship between the real robotic arm and the virtual robotic arm;

[0007] Adjusting the first virtual coordinate of the virtual robotic arm according to the spatial position coordinates of the real robotic arm until the first virtual coordinate corresponds to the spatial position coordinates;

[0008] Determining the first identification coordinates of at least one actual component constituting the real robotic arm, and adjusting the second virtual coordinates of at least one virtual component of the virtual robotic arm according to the first identification coordinates until the second virtual coordinates correspond to the first identification coordinates.

[0009] In a possible implementation manner, wherein the joint pose data includes: the joint angle and the rotation angle corresponding to the joint, and the performing pose adjustment on at least one virtual joint of the virtual robotic arm according to the first joint pose data of the real robotic arm includes:

[0010] Adjust the included angle between at least one virtual joint of the virtual robotic arm and at least one actual joint of the actual robotic arm to be the same as the included angle of the actual joint corresponding to the actual robotic arm;

[0011] Determine whether the rotation angles of at least one actual joint of the actual robotic arm are the same as the rotation angles of at least one virtual joint of the virtual robotic arm;

[0012] If not, determine a first correction parameter for the rotation angle of at least one virtual joint of the virtual robotic arm, and correct at least one virtual joint corresponding to the virtual robotic arm according to the first correction parameter, so that the rotation angle of the at least one virtual joint is the same as the rotation angle of the corresponding at least one actual joint, and the first correction parameter is used to indicate the rotation angle required to correct the virtual joint.

[0013] In a possible implementation, after correcting at least one virtual joint corresponding to the virtual robotic arm, it further includes:

[0014] Adjust the rotation angles of at least one virtual joint of the virtual robotic arm and at least one actual joint of the corresponding actual robotic arm to a preset angle, determine the first rotation direction of the actual robotic arm, and the second rotation direction of the virtual robotic arm;

[0015] Determine whether the first rotation direction and the second rotation direction are the same;

[0016] If not, determine a second correction parameter for the virtual robotic arm, and correct the rotation direction of the virtual joint corresponding to the virtual robotic arm according to the second correction parameter, so that the rotation direction of the virtual joint is the same as the rotation direction of the corresponding actual joint, and the second correction parameter is used to indicate the parameter for correcting the rotation direction of the virtual joint.

[0017] In a possible implementation, before determining the first identification coordinates of at least one actual component constituting the actual robotic arm, it further includes:

[0018] Obtain the initial second identification position coordinates collected by the camera device, where the initial second identification position coordinates are the spatial position of the actual robotic arm determined based on the camera coordinate system, and the camera coordinate system is used to indicate the coordinate system used by the camera device, and the camera device is arranged on the actual robotic arm;

[0019] Determine a rotation matrix according to the spatial position coordinates and the initial second identification position coordinates, where the rotation matrix is used to indicate the direction of the camera coordinate system relative to the actual robotic arm coordinate system, the spatial position coordinates are determined based on the actual robotic arm coordinate system, and the initial second identification position coordinates are determined based on the camera coordinate system;

[0020] Determine a translation vector based on the origin of the camera coordinate system and the origin of the actual robotic arm coordinate system. Then, convert the initial second recognition position coordinates according to the rotation matrix and the translation vector to obtain the second recognition position coordinates. The translation vector is used to indicate the displacement from the origin of the camera coordinate system to the origin of the actual robotic arm coordinate system, and the actual robotic arm coordinate system is different from the camera coordinate system.

[0021] In a possible implementation manner, the determining the first recognition coordinates of at least one actual component that makes up the actual robotic arm includes:

[0022] Obtain the initial first recognition coordinates collected by the imaging device, where the initial first recognition coordinates are used to indicate the spatial positions of the at least one actual component recognized based on the camera coordinate system;

[0023] Convert the initial first recognition coordinates according to the rotation matrix and the translation vector to obtain the first recognition coordinates.

[0024] In a possible implementation manner, the method further includes:

[0025] If robotic arm position change information sent by the client is received, then adjust the actual robotic arm to a first pose so that the imaging device of the actual robotic arm can collect the origin of the actual robotic arm coordinate system in space;

[0026] Obtain the third recognition coordinates of the actual robotic arm collected by the imaging device;

[0027] Adjust the first virtual coordinates of the virtual robotic arm so that the first virtual coordinates of the virtual robotic arm are the same as the third recognition coordinates.

[0028] In a possible implementation manner, the method further includes:

[0029] If robotic arm component position change information sent by the client is received, then adjust the actual robotic arm to a second pose so that the imaging device can capture at least one actual component of the actual robotic arm;

[0030] Obtain the fourth recognition coordinates of at least one actual component of the actual robotic arm collected by the imaging device;

[0031] Adjust the second virtual coordinates of the virtual components of the virtual robotic arm so that the second virtual coordinates are the same as the fourth recognition coordinates.

[0032] In a second aspect, the present application provides a mechanical arm virtual - real position synchronization device based on the industrial Internet. The device includes:

[0033] The first adjustment module is used to adjust the poses of at least one virtual joint of the virtual robotic arm according to the first joint pose data of the actual robotic arm until the pose of the virtual robotic arm corresponds to the pose of the actual robotic arm, where there is a corresponding relationship between the actual robotic arm and the virtual robotic arm;

[0034] In a possible implementation, the first adjustment module is specifically used for:

[0035] According to the joint angles of the actual joints corresponding to the actual robotic arm, adjust the angles between at least one virtual joint of the virtual robotic arm and at least one actual joint corresponding to the actual robotic arm to be the same;

[0036] Determine whether the rotation angles of at least one actual joint of the actual robotic arm are the same as the rotation angles of at least one virtual joint of the virtual robotic arm;

[0037] If not, determine the first correction parameter for the rotation angle of at least one virtual joint of the virtual robotic arm, and correct at least one virtual joint corresponding to the virtual robotic arm according to the first correction parameter so that the rotation angle of at least one virtual joint is the same as the rotation angle of the corresponding at least one actual joint, where the first correction parameter is used to indicate the rotation angle required to correct the virtual joint.

[0038] In a possible implementation, the first adjustment module is specifically used for:

[0039] Adjust the rotation angles of at least one virtual joint of the virtual robotic arm and at least one actual joint of the corresponding actual robotic arm to a preset angle, determine the first rotation direction of the actual robotic arm, and the second rotation direction of the virtual robotic arm;

[0040] Determine whether the first rotation direction is the same as the second rotation direction;

[0041] If not, determine the second correction parameter of the virtual robotic arm, and correct the rotation direction of the virtual joint corresponding to the virtual robotic arm according to the second correction parameter so that the rotation direction of the virtual joint is the same as the rotation direction of the corresponding actual joint, where the second correction parameter is used to indicate the parameter for correcting the rotation direction of the virtual joint.

[0042] The second adjustment module is used to adjust the first virtual coordinate of the virtual robotic arm according to the spatial position coordinates of the actual robotic arm until the first virtual coordinate corresponds to the spatial position coordinates;

[0043] The third adjustment module is used to determine the first identification coordinates of at least one actual component that makes up the actual robotic arm, and adjust the second virtual coordinates of at least one virtual component of the virtual robotic arm according to the first identification coordinates until the second virtual coordinates correspond to the first identification coordinates.

[0044] In a possible implementation, the third adjustment module is further used to:

[0045] Obtain the initial second identification position coordinates collected by the camera device. The initial second identification position coordinates are the spatial position of the actual robotic arm determined based on the camera coordinate system, and the camera coordinate system is used to indicate the coordinate system used by the camera device. The camera device is arranged on the actual robotic arm;

[0046] Determine a rotation matrix according to the spatial position coordinates and the initial second identification position coordinates. The rotation matrix is used to indicate the direction of the camera coordinate system relative to the actual robotic arm coordinate system. The spatial position coordinates are determined based on the actual robotic arm coordinate system, and the initial second identification position coordinates are determined based on the camera coordinate system;

[0047] Determine a translation vector according to the origin of the camera coordinate system and the origin of the actual robotic arm coordinate system. Then, convert the initial second identification position coordinates according to the rotation matrix and the translation vector to obtain the second identification position coordinates. The translation vector is used to indicate the displacement from the origin of the camera coordinate system to the origin of the actual robotic arm coordinate system, and the actual robotic arm coordinate system is different from the camera coordinate system.

[0048] In a possible implementation, the third adjustment module is specifically used to:

[0049] Obtain the initial first identification coordinates collected by the camera device. The initial first identification coordinates are used to indicate the spatial position of the at least one actual component identified based on the camera coordinate system;

[0050] Convert the initial first identification coordinates according to the rotation matrix and the translation vector to obtain the first identification coordinates.

[0051] In a possible implementation, the third adjustment module is further used to:

[0052] If the robotic arm position change information sent by the client is received, then adjust the actual robotic arm to the first pose so that the camera device of the actual robotic arm can collect the origin of the actual robotic arm coordinate system in space;

[0053] Obtain the third identification coordinates of the actual robotic arm collected by the camera device;

[0054] Adjust the first virtual coordinate of the virtual robotic arm so that the first virtual coordinate of the virtual robotic arm is the same as the third recognition coordinate.

[0055] In a possible implementation, the third adjustment module is further configured to:

[0056] If the robotic arm component position change information sent by the client is received, adjust the actual robotic arm to the second pose so that the imaging device can capture at least one actual component of the actual robotic arm;

[0057] Obtain the fourth recognition coordinate of at least one actual component of the actual robotic arm collected by the imaging device;

[0058] Adjust the second virtual coordinate of the virtual component of the virtual robotic arm so that the second virtual coordinate is the same as the fourth recognition coordinate.

[0059] In a third aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in any one of the first aspects.

[0060] In a fourth aspect, the present application provides an electronic device, including: at least one processor and a memory; wherein,

[0061] The memory stores computer-executable instructions;

[0062] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method described in any one of the first aspects.

[0063] In a fifth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the first aspects can be implemented.

[0064] The robotic arm virtual-real position synchronization method, device, equipment and medium provided by the present application ensure the consistent postures of the virtual robotic arm and the actual robotic arm by adjusting the joint postures; make the virtual robotic arm accurately correspond to the position of the actual robotic arm by adjusting the overall spatial coordinates of the virtual robotic arm; ensure the precise alignment at the component level by identifying the spatial positions of the actual robotic arm components and synchronously updating them in the virtual model. Further, the accuracy of recognition is improved through the coordinate system conversion method, that is, the recognition results in the camera coordinate system are converted to the robotic arm coordinate system through the rotation matrix and translation vector, enhancing the positioning reliability in the virtual-real interaction scenario. The precise alignment between the virtual space and the physical space is achieved, so that the robotic arm trajectory generated by the simulation is applicable to actual operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0066] Figure 1 The flowchart of the method for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet provided in the embodiments of the present application Figure 1 ;

[0067] Figure 2 The flowchart of the method for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet provided in the embodiments of the present application Figure 2 ;

[0068] Figure 3 The flowchart of the method for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet provided in the embodiments of the present application Figure 3 ;

[0069] Figure 4 The diagram of the device for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet provided in the embodiments of the present invention;

[0070] Figure 5 The schematic diagram of the hardware of the electronic device provided in the embodiments of the present invention.

[0071] Through the above-mentioned accompanying drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These accompanying drawings and written descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed Embodiments

[0072] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below with reference to the accompanying drawings in the present application. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0073] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present invention and the above-mentioned accompanying drawings are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example.

[0074] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0075] In industrial automation, the applications of robotic arm virtual simulation mainly include the following aspects:

[0076] Process planning and optimization: In the production line layout and process planning stage, the use of robotic arm virtual simulation can help engineers simulate different process flows and layout schemes, evaluate the impact of various factors on production efficiency, and thus optimize the production line design;

[0077] Equipment selection and configuration: When selecting robotic arm equipment and configuration, virtual simulation can help enterprises simulate different robotic arm brands, models and configuration schemes, evaluate their performance and applicability in specific production scenarios, and thus make more appropriate choices;

[0078] Production process simulation: During the production process, robotic arm virtual simulation can simulate parameters such as the movement trajectory, speed, and acceleration of the robotic arm under various operation tasks, help enterprises evaluate the operation efficiency and stability of the production line, and optimize the production process;

[0079] Human-robot collaboration and safety assessment: In industrial production, robotic arms usually need to collaborate with human operators or other equipment. Through virtual simulation, the collaboration mode between the robotic arm and the human operator can be simulated, and its safety and efficiency can be evaluated, thus optimizing the human-robot collaboration plan;

[0080] Fault troubleshooting and maintenance training: Virtual simulation can also be used to simulate the fault conditions and maintenance processes of robotic arms, help engineers and operators familiarize themselves with the structure, working principle and maintenance methods of robotic arms, and improve the efficiency of fault troubleshooting and maintenance;

[0081] In summary, robotic arm virtual simulation has important application values in industrial automation, has many application scenarios, can help enterprises improve production efficiency, reduce costs, and optimize the production process and equipment configuration, thereby enhancing competitiveness and market share.

[0082] In a multi-robot arm assembly scenario, the trajectories generated by simulation software are difficult to directly apply to actual robot arm operations. The main reason is that the positions of the virtual space and the physical space cannot be completely aligned. For example, there are often slight offsets in the actual installation of the robot arm base, equipment, components, etc., while the simulation trajectories are calculated based on ideal positions, ignoring these deviations. Since the actual trajectories need to consider these offsets, without compensation or correction, the actual operating trajectories of the robot arm will not meet the expected requirements, resulting in a decrease in the accuracy and efficiency of the assembly process.

[0083] To address the above problems, this application provides a method for synchronizing the virtual and physical positions of a robot arm based on the industrial Internet. Through a complete process of virtual-real alignment and error compensation, from initialization alignment to dynamic adjustment, it gradually realizes the precise mapping between the virtual and physical spaces, enabling the simulation trajectories to be applied to actual operations. Through virtual-real initial calibration, vision algorithm optimization, dynamic error compensation, and feedback mechanisms, it achieves precise alignment between the virtual space and the physical space, so that the robot arm trajectories generated by simulation are applicable to actual operations. This method ensures virtual-real consistency under dynamic changes in the production line through real-time adjustment, enabling intelligent synchronization of the robot arms in the virtual space and the physical space, thereby improving the assembly accuracy and efficiency.

[0084] It should be noted that in the industrial Internet, efficient data transmission and interaction can be achieved between various devices and servers. This method utilizes this feature to transmit the data of the robot arm's physical space to the virtual space in real time through the industrial Internet, and at the same time, feedback the optimization results of the virtual space to the physical space to achieve virtual-real position synchronization. In addition, the industrial Internet can also provide powerful computing capabilities and data analysis capabilities, providing support for complex calculations such as vision algorithm optimization and dynamic error compensation in this application, ensuring the accuracy and real-time nature of the synchronization process. Therefore, this application can use the industrial Internet as the technical foundation and implementation platform, thus better meeting the requirements for the operation accuracy and efficiency of robot arms in industrial production.

[0085] The following uses specific embodiments to elaborate in detail on the technical solutions of this application and how the technical solutions of this application solve the above technical problems. These several specific embodiments can exist independently or be combined with each other. For the same or similar concepts or processes, they may not be repeated in some embodiments. The following will describe the embodiments of this application in conjunction with the accompanying drawings.

[0086] This embodiment provides a method for synchronizing the virtual and physical positions of a robot arm based on the industrial Internet. Figure 1 For the flow of the method for synchronizing the virtual and physical positions of a robot arm based on the industrial Internet provided by the embodiments of this application Figure 1 . As Figure 1 shown, this method includes:

[0087] S101. Adjust the poses of at least one virtual joint of the virtual robotic arm according to the first joint pose data of the actual robotic arm until the pose of the virtual robotic arm corresponds to that of the actual robotic arm.

[0088] Among them, there is a corresponding relationship between the actual robotic arm and the virtual robotic arm.

[0089] In this step, in order to ensure that the pose of the virtual robotic arm is consistent with the pose of the actual robotic arm in physical space. Since there may be deviations between the virtual and actual robotic arms, the initial virtual robotic arm may not automatically align with the actual robotic arm. Therefore, it is necessary to adjust the joints of the virtual robotic arm according to the joint pose data of the actual robotic arm to ensure that the postures of the virtual robotic arm and the actual robotic arm are consistent.

[0090] Exemplarily, obtaining the joint pose data of the actual robotic arm includes, but is not limited to, joint angles (included angles) and rotation angles. Compare the joint pose data of the virtual robotic arm and the actual robotic arm to ensure that the pose of the virtual robotic arm can correspond one by one to that of the actual robotic arm. Control the joint adjustment of the virtual robotic arm through software until the pose of the virtual robotic arm is completely consistent with that of the actual robotic arm. For example, simulation software can be used to control the actions of the virtual robotic arm. Also, through the inverse kinematics algorithm, calculate the positions of the corresponding joints of the virtual robotic arm based on the joint data of the actual robotic arm and make adjustments. This process can be carried out in an automated manner or manually.

[0091] It should be noted that the premise for the implementation of this embodiment is that a corresponding relationship between the virtual robotic arm and the actual robotic arm has been established and the joint data of the actual robotic arm can be obtained.

[0092] S102. Adjust the first virtual coordinate of the virtual robotic arm according to the spatial position coordinates of the actual robotic arm until the first virtual coordinate corresponds to the spatial position coordinates.

[0093] In this step, the spatial position coordinates of the actual robotic arm are the positions of the robotic arm base or other specified reference points in physical space. The spatial position of the virtual robotic arm needs to be consistent with that of the actual robotic arm to ensure that the virtual robotic arm can accurately represent the spatial position of the actual robotic arm in the virtual environment.

[0094] Exemplarily, obtain the spatial coordinates (x, y, z) of the actual robotic arm base or reference point, and adjust the coordinates of the virtual robotic arm according to these coordinates so that the spatial position of the virtual robotic arm is consistent with that of the actual robotic arm. If there are minor deviations in the actual robotic arm, vision calibration techniques (such as camera recognition) or sensors (such as laser rangefinders) can be used for real-time monitoring and adjustment. For example, coordinate transformation and mapping algorithms can be used to map the position coordinates of the actual robotic arm to the virtual space. Use simulation software to set the coordinates of the virtual robotic arm base to be consistent with the spatial coordinates of the actual robotic arm. If there is an error between the virtual environment and the actual environment, position alignment can be performed through optimization algorithms (such as the least squares method).

[0095] Exemplarily, the spatial position coordinates of the actual robotic arm can be obtained through sensors (such as lidar, infrared sensors, vision systems, etc.) or motion capture devices. These devices can monitor the position and posture of the robotic arm in real time and convert them into spatial coordinates (x, y, z) in the actual robotic arm coordinate system. The first virtual coordinates of the virtual robotic arm can be achieved through an automated adjustment algorithm, usually using a coordinate transformation method to map the spatial coordinates of the actual robotic arm to the virtual space. During specific operations, first receive the spatial position coordinates of the actual robotic arm, and automatically adjust the coordinates of the virtual robotic arm by calculating the spatial transformation relationship (such as rotation matrix and translation vector) between the virtual robotic arm and the actual robotic arm to make it consistent with the position of the actual robotic arm, thereby achieving synchronization between the virtual and physical spaces.

[0096] S103. Determine the first identification coordinates of at least one actual component that makes up the actual robotic arm.

[0097] In this step, the position of each component of the robotic arm (such as fixtures, end effectors, sensors, etc.) may affect the overall trajectory and working ability of the robotic arm. By obtaining the precise positions of these components, the alignment of the virtual robotic arm and the actual robotic arm in every detail can be ensured.

[0098] Exemplarily, cameras or other sensors (such as lidar) can be used to obtain the precise identification coordinates of the components. Set multiple reference points or calibration objects on the actual robotic arm, and these points can be used to identify and measure the spatial positions of the components. Determine the precise coordinates of the components through vision algorithms or laser scanning techniques. For example, a vision recognition system can be used to scan and identify the components on the actual robotic arm to obtain their coordinate information. Use a sensor network to monitor the positions of the components in real time and transmit the coordinates to the computing system for processing.

[0099] Exemplarily, before determining the first identification coordinates of at least one actual component that makes up the actual robotic arm, it further includes:

[0100] Obtain the initial second recognition position coordinates collected by the camera device. The initial second recognition position coordinates are the spatial position of the actual robotic arm determined based on the camera coordinate system, and the camera coordinate system is used to indicate the coordinate system used by the camera device. The camera device is arranged on the actual robotic arm;

[0101] Determine a rotation matrix according to the spatial position coordinates and the initial second recognition position coordinates. The rotation matrix is used to indicate the direction of the camera coordinate system relative to the actual robotic arm coordinate system. The spatial position coordinates are determined based on the actual robotic arm coordinate system, and the initial second recognition position coordinates are determined based on the camera coordinate system;

[0102] Determine a translation vector according to the origin of the camera coordinate system and the origin of the actual robotic arm coordinate system. Then, perform a transformation on the initial second recognition position coordinates according to the rotation matrix and the translation vector to obtain the second recognition position coordinates. The translation vector is used to indicate the displacement from the origin of the camera coordinate system to the origin of the actual robotic arm coordinate system. The actual robotic arm coordinate system is different from the camera coordinate system.

[0103] It should be noted that the camera device located on the actual robotic arm will collect spatial position data based on its own coordinate system (camera coordinate system) during operation. The camera coordinate system and the actual robotic arm coordinate system are usually inconsistent. Therefore, it is necessary to calculate a rotation matrix to represent the direction difference between the camera coordinate system and the actual robotic arm coordinate system. The rotation matrix is a mathematical tool that can describe the rotational change of one coordinate system relative to another. In addition to rotation, there may also be a translational difference between the camera coordinate system and the robotic arm coordinate system, that is, the origins of the two coordinate systems may not coincide. To complete the coordinate transformation, it is also necessary to determine the translation vector. The translation vector describes the displacement from the origin of the camera coordinate system to the origin of the robotic arm coordinate system. With the rotation matrix and the translation vector, the initial second recognition position coordinates in the camera coordinate system can be transformed to the actual robotic arm coordinate system. This process is to achieve the transformation between coordinate systems through the rotation matrix and the translation vector. The transformed coordinates can represent the spatial position of the actual robotic arm.

[0104] Exemplarily, assume that the original coordinate is Pcamera (a point in the camera coordinate system), the rotation matrix is R, and the translation vector is T. Then the formula for coordinate transformation can be expressed as:

[0105] ,

[0106] where, is the position in the robotic arm coordinate system.

[0107] S104. Adjust the second virtual coordinates of at least one virtual component of the virtual robotic arm according to the first identified coordinates until the second virtual coordinates correspond to the first identified coordinates.

[0108] In this step, the positions of the components of the virtual robotic arm in the virtual space need to match the actual positions of the components of the actual robotic arm. This is to ensure that the operations in the virtual simulation are consistent with the movements in the actual physical world. Especially in assembly and operation, the actions and positions of the virtual components must be consistent with those of the actual components. The precise adjustment of the positions of the virtual components can be ensured through the coordinate system transformation in the foregoing example, or position matching algorithms, sensor calibration, etc.

[0109] Exemplarily, the determining the first identified coordinates of at least one actual component constituting the actual robotic arm includes:

[0110] Obtain the initial first identified coordinates collected by the imaging device, where the initial first identified coordinates are used to indicate the spatial positions of the at least one actual component identified based on the imaging coordinate system;

[0111] Convert the initial first identified coordinates according to the rotation matrix and the translation vector to obtain the first identified coordinates.

[0112] It should be noted that the imaging device captures the actual robotic arm, identifies the positions of the components, and represents them in the form of coordinates in the imaging coordinate system, which is called the initial first identified coordinates. Since the imaging coordinate system and the robotic arm coordinate system are usually inconsistent, the initial first identified coordinates need to be converted to the robotic arm coordinate system by using the pre-calculated rotation matrix (describing the direction difference between the two coordinate systems) and translation vector (describing the position difference between the origins of the two coordinate systems) to generate accurate first identified coordinates. To ensure that the virtual model can accurately reflect the spatial positions of the components of the actual robotic arm and improve the accuracy of virtual-real synchronization.

[0113] The method for synchronizing the virtual and real positions of the robotic arm based on the industrial Internet provided in this embodiment ensures the same posture of the virtual robotic arm and the actual robotic arm by adjusting the joint postures; makes the virtual robotic arm accurately correspond to the position of the actual robotic arm by adjusting the overall spatial coordinates of the virtual robotic arm; and ensures the precise alignment at the component level by identifying the spatial positions of the components of the actual robotic arm and synchronously updating them in the virtual model. Further, the accuracy of the identification is improved through the coordinate system transformation method, that is, the identification result in the imaging coordinate system is converted to the robotic arm coordinate system by using the rotation matrix and the translation vector, enhancing the positioning reliability in the virtual-real interaction scenario. It realizes the precise alignment between the virtual space and the physical space, so that the trajectory of the robotic arm generated by the simulation is applicable to actual operations.

[0114] This embodiment provides a method for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet. Figure 2 It is the flow of the method for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet provided by the embodiment of the present application. Figure 2 . As Figure 2 shown, on the basis of the Figure 1 embodiment, the process of performing pose adjustment on at least one virtual joint of the virtual robotic arm is described in detail. Among them, the joint pose data includes: the joint angle and the rotation angle of the corresponding joint. The method includes:

[0115] S201. According to the joint angle of the actual joint of the actual robotic arm, adjust the included angle between at least one virtual joint of the virtual robotic arm and at least one actual joint of the actual robotic arm to be the same.

[0116] In this step, during the multi-robotic arm assembly process, the simulation trajectory is generated based on an idealized virtual space. However, there may be slight deviations in the joint positions of the actual robotic arm. These deviations are usually caused by factors such as installation errors, mechanical errors, or environmental impacts. Therefore, the joint angles of the simulation robotic arm and the actual robotic arm need to be aligned to ensure that the virtual robotic arm and the actual robotic arm are in the same pose and position. In this embodiment, the included angle represents the angle between the mechanical axes on both sides of the joint and can be used to describe the degree of extension or folding of the robotic arm in a plane or space. That is, each joint of the robotic arm will have an included angle, which determines the opening and closing degree of the joint and the position of the end effector of the robotic arm. The rotation angle represents the rotation angle of the joint itself around its rotation axis. It pays more attention to the specific rotation action of a single joint and determines the precise operation of the robotic arm.

[0117] Real-time data can be obtained through feedback devices such as sensors or encoders to acquire the actual included angles of each joint of the actual robotic arm. Then, the joint angles of the virtual robotic arm are adjusted so that the included angles between the virtual joints and the actual joints are kept consistent. The angles of the virtual joints are adjusted through parameter input or control signals in the simulation software. For example, the angle data of the actual joints are acquired through the joint angle sensors of the robotic arm (such as potentiometers or rotary encoders). Based on these angle data, the virtual simulation software adjusts the joint angles of the virtual robotic arm in real time to ensure its alignment with the actual robotic arm. In some other embodiments, the actual robotic arm can also be directly in its initial pose, that is, the pose preset at the factory. At this time, the included angle of the corresponding actual joints can be the angle measured in the physical sense or the theoretical included angle defined by the factory. The theoretical included angle indicates that the physical angle of the possible actual joint is 90°, while the corresponding displayed theoretical included angle is 0°. When the actual joint increases the angle on this basis, the theoretical included angle increases on the basis of 0°. Correspondingly, when realizing synchronous adjustment, the virtual joints of the virtual robotic arm also need to synchronize the definition method of this theoretical included angle.

[0118] S202. Determine whether the rotation angles of at least one actual joint of the actual robotic arm are the same as the rotation angles of at least one virtual joint of the virtual robotic arm.

[0119] S203. If so, no operation is performed.

[0120] S204. If not, determine the first correction parameter of the rotation angle of at least one virtual joint of the virtual robotic arm.

[0121] It should be noted that after adjusting the included angles between the virtual joints of the virtual robotic arm and the actual joints of the actual robotic arm to be the same, then confirming whether their rotation angles are the same is mainly to verify their consistency in the static posture. Whether the rotation angles of the virtual robotic arm and the actual robotic arm are consistent reflects their alignment in this static posture. If the included angles of the virtual joints and the actual joints are the same but the rotation angles are inconsistent, it may indicate that there are geometric or model differences between the virtual model and the actual robotic arm and they cannot be accurately mapped. Or there may also be problems such as sensor errors, mechanical errors, or calculation errors. Confirming whether the rotation angles are consistent can help identify these error sources and take corresponding compensation measures.

[0122] The rotation angle of the actual robotic arm can be collected through sensors (such as rotary encoders, angle sensors, etc.) and compared with the rotation angle of the virtual robotic arm in the simulation software. If the two are consistent, it indicates that the posture of the virtual robotic arm has been synchronized with the actual robotic arm. If the virtual joint angle is different from the actual joint angle, a correction parameter needs to be calculated to adjust the rotation angle of the virtual joint to make it the same as the joint angle of the actual robotic arm. The correction parameter can be an angle deviation value or increment. For example, a mathematical model can be used to calculate the angle error. For example, setting the virtual joint angle as θv and the actual joint angle as θa, the correction parameter is:

[0123] Δθ = θa - θv, and this correction parameter can be updated in real time through an algorithm or a closed-loop control system.

[0124] S205. Correct at least one virtual joint corresponding to the virtual robotic arm according to the first correction parameter, so that the rotation angle of the at least one virtual joint is the same as the rotation angle of the corresponding at least one actual joint, and the first correction parameter is used to indicate the rotation angle required to correct the virtual joint.

[0125] In this step, the angle of the virtual joint can be dynamically adjusted by servo control or directly input into the simulation system. This adjustment can be gradual (such as through a PID control algorithm) to ensure that the adjustment of the virtual joint angle does not cause instability in other systems.

[0126] Exemplarily, after correcting at least one virtual joint corresponding to the virtual robotic arm, it further includes:

[0127] Adjust the rotation angles of at least one virtual joint of the virtual robotic arm and at least one actual joint of the corresponding actual robotic arm to a preset angle, determine the first rotation direction of the actual robotic arm, and the second rotation direction of the virtual robotic arm;

[0128] Determine whether the first rotation direction and the second rotation direction are the same;

[0129] If not, determine the second correction parameter of the virtual robotic arm, and correct the rotation direction of the virtual joint corresponding to the virtual robotic arm according to the second correction parameter, so that the rotation direction of the virtual joint is the same as that of the corresponding actual joint, and the second correction parameter is used to indicate the parameter for correcting the rotation direction of the virtual joint.

[0130] It should be noted that although the virtual robotic arm and the actual robotic arm may be similar in appearance and joint configuration, due to the differences between the virtual system and the actual hardware, the motion synchronization between the two needs to be verified. By comparing whether the rotation directions are the same, it is possible to verify whether the motions of the two are consistent, so as to ensure that under virtual control, the actual robotic arm can perform actions as expected. The rotation angle of the actual robotic arm can be collected through sensors (such as rotary encoders, angle sensors, etc.) and compared with the rotation angle of the virtual robotic arm in the simulation software. If the two are consistent, it indicates that the posture of the virtual robotic arm has been synchronized with the actual robotic arm.

[0131] The rotation direction represents the direction in which the joint rotates around the axis, which may be counterclockwise or clockwise. While adjusting the virtual joint and the actual joint to the preset angle, it is also possible to clarify the direction of their rotation and monitor during the rotation of the virtual joint and the actual joint to determine whether their rotation directions are the same, that is, to monitor whether their dynamic processes are consistent. In addition, it is also possible to determine whether the rotation directions are the same by checking the joint postures after rotation is completed, and there is no limitation here.

[0132] The second correction parameter represents a parameter used to characterize and indicate the correction of the rotation direction of the virtual joint. Specifically, it can include the values or conditions required to adjust the current rotation direction of the virtual joint to be the same as the rotation direction of the actual joint. For example, the rotation direction of a certain joint of the actual robotic arm is clockwise, while the rotation direction of the corresponding joint of the virtual robotic arm is counterclockwise. In this case, the second correction parameter will indicate that the rotation direction of the virtual joint needs to be rotated counterclockwise by a certain angle so that the two finally agree.

[0133] The method for synchronizing the virtual and real positions of the robotic arm based on the industrial Internet provided in this embodiment ensures the consistency of the motion trajectories in the virtual and physical spaces by calibrating the joint angles and rotation directions of the virtual robotic arm, thus avoiding the decline in assembly accuracy and efficiency caused by small offsets. This method can effectively correct positions and angles in static calibration, ensure the synchronization of virtual and physical robotic arms in a complex production environment, and significantly improve the accuracy, reliability, and production efficiency of the assembly process.

[0134] This embodiment provides a method for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet. Figure 3 is the flow of the method for synchronizing the virtual and real positions of the robotic arm based on the industrial Internet provided in the embodiments of this application Figure 3 . As Figure 3 shown, based on the Figure 2 embodiment, this embodiment elaborates in detail on the process of dynamic adjustment. The method includes:

[0135] S301. Synchronize the poses, spatial positions, and component positions of the virtual robotic arm and the corresponding actual robotic arm.

[0136] The synchronization process of this step has been described in the foregoing embodiments and will not be elaborated here.

[0137] S302. If the robotic arm position change information sent by the client is received, adjust the actual robotic arm to the first pose so that the imaging device of the actual robotic arm can collect the origin of the actual robotic arm coordinate system in space.

[0138] S303. Obtain the third identification coordinate of the actual robotic arm collected by the imaging device.

[0139] S304. Adjust the first virtual coordinate of the virtual robotic arm so that the first virtual coordinate of the virtual robotic arm is the same as the third identification coordinate.

[0140] It should be noted that in some application scenarios, such as in the early production line adjustment, the position of the actual robotic arm needs to be adjusted multiple times, and then the overall assembly process is simulated in the simulation space until the overall assembly process meets the requirements. Therefore, if the actual robotic arm moves, all the involved models such as the corresponding virtual robotic arm or virtual components in the simulation space need to move accordingly.

[0141] When the client sends the robotic arm position change information, the information can be transmitted to the robotic arm control system through the network protocol. The position change information can include the identification of the changed robotic arm, or can also include the instruction information for resynchronizing the virtual and real robotic arms. Through the adjustment of the first pose, the imaging device can identify the position of the robotic arm based on the origin and determine the position from the origin to the actual robotic arm. The first pose can be obtained by inverse kinematics calculation combined with the target position to get the angle adjustment required for each joint of the robotic arm; or by using the real-time data of the sensor and combining the position of the components recognized by the vision system to dynamically adjust the pose of the robotic arm. In addition, it can also rely on the information input by the user to manually specify the pose, or generate a smooth motion trajectory through the path planning algorithm to achieve the pose conversion. These methods can be used alone or in combination to meet the requirements of specific operation tasks.

[0142] Exemplarily, the origin of the actual robotic arm coordinate system can be determined by setting a calibration object in the physical space, and the determination of the first pose can be based on the position of the calibration object in the image collected by the imaging device. Or the determination of the first pose can be achieved by combining the data of the imaging device and other sensors (such as lidar, depth sensor, etc.) and using multi-sensor fusion technology to determine the origin. For example, the lidar provides the point cloud data of the space, and the imaging device provides the image data. By aligning the data of the two through the algorithm, the origin can be more accurately located, thereby determining the first pose.

[0143] S305. If the position change information of the robotic arm components sent by the client is received, adjust the actual robotic arm to the second pose so that the imaging device can capture at least one actual component of the actual robotic arm.

[0144] S306. Obtain the fourth recognition coordinates of at least one actual component of the actual robotic arm collected by the imaging device.

[0145] S307. Adjust the second virtual coordinates of the virtual components of the virtual robotic arm so that the second virtual coordinates are the same as the fourth recognition coordinates.

[0146] It should be noted that in some scenarios, such as equipment upgrade, task requirement change, increased precision requirement, maintenance, recalibration, new product manufacturing requirement, safety hazard investigation, or increased flexibility, etc., it is usually necessary to adjust the positions of the actual robotic arm components, so as to optimize the working performance of the robotic arm and improve production efficiency. Therefore, when the above situations occur and the information from the client is received, the virtual components can be synchronized automatically. The determination of the second pose and the determination of the actual component positions are similar to the operation principles when the position of the actual robotic arm changes as described above, and will not be elaborated here.

[0147] The method for synchronizing the virtual and real positions of the robotic arm based on the industrial Internet provided in this embodiment eliminates the deviation between the virtual and physical spaces by dynamically synchronizing the positions of the virtual and actual robotic arms, thereby ensuring that the trajectory generated by the simulation can be accurately applied to the actual assembly operation, and improving the precision and efficiency in the production process.

[0148] This embodiment also provides a device for synchronizing the virtual and real positions of the robotic arm based on the industrial Internet. Figure 4 As shown in the figure for a device for synchronizing the virtual and real positions of the robotic arm provided in an embodiment of the present invention, Figure 4 the device 40 includes:

[0149] A first adjustment module 401, configured to adjust the poses of at least one virtual joint of the virtual robotic arm according to the first joint pose data of the actual robotic arm until the pose of the virtual robotic arm corresponds to the pose of the actual robotic arm, where there is a corresponding relationship between the actual robotic arm and the virtual robotic arm;

[0150] In a possible implementation manner, the first adjustment module 401 is specifically configured to:

[0151] Adjust the included angles between at least one virtual joint of the virtual robotic arm and at least one actual joint of the actual robotic arm to be the same as the included angles of the actual joints corresponding to the actual robotic arm;

[0152] Determine whether the rotation angles of at least one actual joint of the actual robotic arm are the same as the rotation angles of at least one virtual joint of the virtual robotic arm;

[0153] If not, determine a first correction parameter for the rotation angles of at least one virtual joint of the virtual robotic arm, and correct at least one virtual joint corresponding to the virtual robotic arm according to the first correction parameter, so that the rotation angles of the at least one virtual joint are the same as the rotation angles of the corresponding at least one actual joint, and the first correction parameter is used to indicate the rotation angle required to correct the virtual joint.

[0154] In a possible implementation manner, the first adjustment module 401 is specifically configured to:

[0155] Adjust the rotation angles of at least one virtual joint of the virtual robotic arm and the rotation angles of at least one actual joint of the corresponding actual robotic arm to a preset angle, determine a first rotation direction of the actual robotic arm, and a second rotation direction of the virtual robotic arm;

[0156] Determine whether the first rotation direction and the second rotation direction are the same;

[0157] If not, determine a second correction parameter for the virtual robotic arm, and correct the rotation direction of the virtual joint corresponding to the virtual robotic arm according to the second correction parameter, so that the virtual joint has the same rotation direction as the corresponding actual joint, and the second correction parameter is used to indicate a parameter for correcting the rotation direction of the virtual joint.

[0158] The second adjustment module 402 is configured to adjust a first virtual coordinate of the virtual robotic arm according to the spatial position coordinates of the actual robotic arm until the first virtual coordinate corresponds to the spatial position coordinates;

[0159] The third adjustment module 403 is configured to determine a first identification coordinate of at least one actual component that makes up the actual robotic arm, and adjust a second virtual coordinate of at least one virtual component of the virtual robotic arm according to the first identification coordinate until the second virtual coordinate corresponds to the first identification coordinate.

[0160] In a possible implementation manner, the third adjustment module 403 is further configured to:

[0161] Obtain an initial second identification position coordinate collected by an imaging device, where the initial second identification position coordinate is the spatial position of the actual robotic arm determined based on an imaging coordinate system, and the imaging coordinate system is used to indicate the coordinate system used by the imaging device, and the imaging device is disposed on the actual robotic arm;

[0162] Determine a rotation matrix according to the spatial position coordinates and the initial second recognition position coordinates, where the rotation matrix is used to indicate the direction of the camera coordinate system relative to the actual robotic arm coordinate system, the spatial position coordinates are determined based on the actual robotic arm coordinate system, and the initial second recognition position coordinates are determined based on the camera coordinate system;

[0163] Determine a translation vector according to the origin of the camera coordinate system and the origin of the actual robotic arm coordinate system, and then convert the initial second recognition position coordinates according to the rotation matrix and the translation vector to obtain the second recognition position coordinates, where the translation vector is used to indicate the displacement from the origin of the camera coordinate system to the origin of the actual robotic arm coordinate system, and the actual robotic arm coordinate system is different from the camera coordinate system.

[0164] In a possible implementation manner, the third adjustment module 403 is specifically configured to:

[0165] Obtain the initial first recognition coordinates collected by the camera device, where the initial first recognition coordinates are used to indicate the spatial positions of the at least one actual component recognized based on the camera coordinate system;

[0166] Convert the initial first recognition coordinates according to the rotation matrix and the translation vector to obtain the first recognition coordinates.

[0167] In a possible implementation manner, the third adjustment module 403 is further configured to:

[0168] If receiving the robotic arm position change information sent by the client, adjust the actual robotic arm to the first pose so that the camera device of the actual robotic arm can collect the origin of the actual robotic arm coordinate system in space;

[0169] Obtain the third recognition coordinates of the actual robotic arm collected by the camera device;

[0170] Adjust the first virtual coordinate of the virtual robotic arm so that the first virtual coordinate of the virtual robotic arm is the same as the third recognition coordinates.

[0171] In a possible implementation manner, the third adjustment module 403 is further configured to:

[0172] If receiving the robotic arm component position change information sent by the client, adjust the actual robotic arm to the second pose so that the camera device can capture at least one actual component of the actual robotic arm;

[0173] Obtain the fourth recognition coordinates of at least one actual component of the actual robotic arm collected by the camera device;

[0174] Adjust the second virtual coordinates of the virtual components of the virtual robotic arm so that the second virtual coordinates are the same as the fourth recognition coordinates.

[0175] The robotic arm virtual-real position synchronization device based on the industrial Internet provided in this embodiment can execute the robotic arm virtual-real position synchronization method based on the industrial Internet provided in the above method embodiment. Their implementation principles and technical effects are similar, and will not be elaborated here in this embodiment.

[0176] Figure 5 It is a hardware schematic diagram of the electronic device provided in the embodiment of the present invention. As Figure 5 shown, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. The device 50 also includes a communication component 503. Among them, the processor 501, the memory 502, and the communication component 503 are connected through a bus 504.

[0177] In the specific implementation process, at least one processor 501 executes the computer execution instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0178] The specific implementation process of the processor 501 can refer to the above method embodiment. Their implementation principles and technical effects are similar, and will not be elaborated here in this embodiment.

[0179] In the above Figure 5 shown embodiment, it should be understood that the processor may be a central processing unit (English: Central Processing Unit, abbreviated: CPU), or other general-purpose processors, digital signal processors (English: Digital Signal Processor, abbreviated: DSP), application specific integrated circuits (English: Application SpecificIntegrated Circuit, abbreviated: ASIC), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0180] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0181] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience in representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0182] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method as described above.

[0183] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disc. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0184] An exemplary readable storage medium is coupled to the processor, enabling the processor to read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0185] The division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be an indirect coupling or communication connection through some interfaces, devices or units, and can be in an electrical, mechanical or other form.

[0186] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] In addition, in each embodiment of the present invention, each functional unit may be integrated in a processing unit, may exist separately physically for each unit, or two or more units may be integrated in one unit.

[0188] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0189] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0190] So far, the technical solution of this application has been described in combination with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of this application is obviously not limited to these specific embodiments. The above embodiments are only used to illustrate the technical solution of this application, rather than to limit it; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of each embodiment of this application.

Claims

1. A method for synchronizing the virtual and real positions of a robotic arm based on the industrial Internet, characterized in that: The method comprises: According to the first joint posture data of the actual robotic arm, adjusting the posture of at least one virtual joint of the virtual robotic arm until the posture of the virtual robotic arm corresponds to the posture of the actual robotic arm, wherein there is a corresponding relationship between the actual robotic arm and the virtual robotic arm; According to the spatial position coordinates of the actual robotic arm, adjusting the first virtual coordinates of the virtual robotic arm until the first virtual coordinates correspond to the spatial position coordinates; Determine a first identification coordinate of at least one actual component constituting the actual robotic arm, and adjust a second virtual coordinate of at least one virtual component of the virtual robotic arm according to the first identification coordinate until the second virtual coordinate corresponds to the first identification coordinate.

2. The method according to claim 1, characterized in that in, The joint posture data includes: the joint angle and the rotation angle of the corresponding joint. The posture adjustment of at least one virtual joint of the virtual robotic arm according to the first joint posture data of the actual robotic arm includes: According to the joint angle of the actual joint corresponding to the actual mechanical arm, adjusting the angle of at least one virtual joint of the virtual mechanical arm to be the same as the angle of at least one actual joint corresponding to the actual mechanical arm; determining whether a rotation angle of at least one actual joint of the actual robotic arm is the same as a rotation angle of at least one virtual joint of the virtual robotic arm; If not, determine a first correction parameter of the rotation angle of at least one virtual joint of the virtual robotic arm, and correct at least one virtual joint corresponding to the virtual robotic arm according to the first correction parameter so that the rotation angle of the at least one virtual joint is the same as the rotation angle of the corresponding at least one actual joint, and the first correction parameter is used to indicate the rotation angle required to correct the virtual joint.

3. The method according to claim 2, characterized in that After the at least one virtual joint corresponding to the virtual mechanical arm is corrected, the method further includes: Adjusting the rotation angle of at least one virtual joint of the virtual robotic arm and the rotation angle of at least one actual joint of the corresponding actual robotic arm to a preset angle, determining a first rotation direction of the actual robotic arm and a second rotation direction of the virtual robotic arm; determining whether the first rotation direction and the second rotation direction are the same; If not, determine the second correction parameter of the virtual robotic arm, and correct the rotation direction of the virtual joint corresponding to the virtual robotic arm according to the second correction parameter so that the rotation direction of the virtual joint is the same as the corresponding actual joint, and the second correction parameter is used to indicate the parameter for correcting the rotation direction of the virtual joint.

4. The method according to claim 1, characterized in that Before determining the first identification coordinate of at least one actual component constituting the actual mechanical arm, the method further includes: Acquire the initial second identification position coordinates collected by the camera device, wherein the initial second identification position coordinates are based on the spatial position of the actual robotic arm determined by a camera coordinate system, wherein the camera coordinate system is used to indicate a coordinate system used by the camera device, and the camera device is arranged on the actual robotic arm; Determine a rotation matrix according to the spatial position coordinates and the initial second recognition position coordinates, wherein the rotation matrix is ​​used to indicate the direction of the camera coordinate system relative to the actual mechanical arm coordinate system, the spatial position coordinates are determined based on the actual mechanical arm coordinate system, and the initial second recognition position coordinates are determined based on the camera coordinate system; A translation vector is determined based on the origin of the camera coordinate system and the origin of the actual robotic arm coordinate system. Then, based on the rotation matrix and the translation vector, the initial second identification position coordinates are transformed to obtain the second identification position coordinates. The translation vector is used to indicate the displacement from the origin of the camera coordinate system to the origin of the actual robotic arm coordinate system. The actual robotic arm coordinate system is different from the camera coordinate system.

5. The method according to claim 4, characterized in that The determining of a first identification coordinate of at least one actual component constituting the actual robotic arm comprises: Acquire initial first identification coordinates acquired by the camera device, wherein the initial first identification coordinates are used to indicate the spatial position of the at least one actual component identified based on the camera coordinate system; The initial first recognition coordinates are transformed according to the rotation matrix and the translation vector to obtain the first recognition coordinates.

6. The method according to claim 4, characterized in that The method further comprises: If the robot arm position change information sent by the user end is received, the actual robot arm is adjusted to the first posture so that the camera device of the actual robot arm can capture the origin of the actual robot arm coordinate system in space; Acquire a third identification coordinate of the actual robotic arm captured by the camera device; The first virtual coordinate of the virtual robotic arm is adjusted so that the first virtual coordinate of the virtual robotic arm is the same as the third recognition coordinate.

7. The method according to claim 4, characterized in that The method further comprises: If the position change information of the robot arm component sent by the user end is received, the actual robot arm is adjusted to a second posture so that the camera device can capture at least one actual component of the actual robot arm; Acquire a fourth identification coordinate of at least one actual component of the actual robotic arm captured by the camera device; The second virtual coordinate of the virtual component of the virtual robot arm is adjusted so that the second virtual coordinate is the same as the fourth recognition coordinate.

8. A virtual and real position synchronization device for a robotic arm based on the industrial Internet, characterized in that: The device comprises: A first adjustment module is used to adjust the posture of at least one virtual joint of the virtual robotic arm according to the first joint posture data of the actual robotic arm, until the posture of the virtual robotic arm corresponds to the posture of the actual robotic arm, wherein there is a corresponding relationship between the actual robotic arm and the virtual robotic arm; A second adjustment module, configured to adjust the first virtual coordinates of the virtual robotic arm according to the spatial position coordinates of the actual robotic arm until the first virtual coordinates correspond to the spatial position coordinates; The third adjustment module is used to determine the first identification coordinates of at least one actual component constituting the actual robotic arm, and adjust the second virtual coordinates of at least one virtual component of the virtual robotic arm according to the first identification coordinates until the second virtual coordinates correspond to the first identification coordinates.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: at least one processor and memory; wherein, The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 7.

Images