A virtual-real interaction digitalized drill method based on a mixed reality technology

Abstract

The application discloses a device virtual-real interaction digitalized drilling method based on a mixed reality technology, and comprises the following steps: 1, a virtual scene three-dimensional space model is built M and a virtual three-dimensional model of a robot m ; original and ending position points, starting and ending working position points, each working position point and a charging position point in m are marked M ; solidified interference, non-solidified interference and wave band interference fields existing in M are marked; 2, the total time of the robot in the path between each position point is calculated; 3, the robot action is classified to obtain a working operation action set, a real robot is used to simulate and drill the virtual scene or a virtual robot is used to simulate and drill the real scene, and a working simulation drilling time is obtained; and 4, the real robot with the same drilling speed is placed in the real scene to simulate and drill, and drilling action time is verified.

Description

Technical Field

[0001] This invention relates to the field of virtual robot technology, and in particular to a digital simulation method for virtual-real interaction based on mixed reality technology. Background Technology

[0002] Robots face the following challenges when performing tasks under specific working conditions within an application environment: 1) High timeliness requirements: Robots must be able to quickly plan solutions and conduct drills for unexpected situations within the scenario to improve response speed. 2) High reliability requirements: Robots must be able to conduct frequent and repeated drills for the specific working conditions within the scenario; however, physical drills are costly. 3) High safety requirements: Robots must ensure their own safety during task execution or simulation drills, eliminating risks such as collisions and overspeeding. Therefore, there is an urgent need for a digital rapid drill method for robots, employing a virtual-real interaction approach to improve the timeliness, reliability, and safety of drills. Summary of the Invention

[0003] The purpose of this invention is to address the technical deficiencies in the existing technology by providing a digital simulation method for virtual-real interaction based on mixed reality technology.

[0004] The technical solution adopted to achieve the purpose of this invention is:

[0005] A method for digital simulation of virtual-real interaction based on mixed reality technology includes the following steps:

[0006] Step 1: Build a 3D spatial model of the virtual scene Virtual 3D model of the robot ,mark exist Mark the initial working position, start working position, intermediate working position, end working position, end working position, and charging position. Solidified interference, non-solidified interference, and band interference fields exist in the environment;

[0007] Step 2: Calculate the total time for the robot to travel between each location point. The total time is the sum of the time for the robot to travel through each path segment, the time to overcome solid or non-solid interference after adding the safety layer model, and the time to overcome the band interference field. Also, calculate the charging and transfer time of the robot at the working position.

[0008] Step 3: Classify the robot's actions to form a set of robot operation actions. Use a real robot to conduct a work simulation exercise in a virtual scene or use a virtual robot to conduct a work simulation exercise in a real scene. Calculate the time for the robot to complete the action in the simulation exercise, and add it to the total time in Step 2 and the charging and transfer time to obtain the simulation exercise time.

[0009] Step 4: Place the real robot in a real scene for a real-world exercise. The robot's speed is the same during the real exercise and the simulated exercise. By comparing the real exercise time with the simulated exercise time in Step 3, the feasibility of the exercise method is verified.

[0010] In the above technical solution, the ratio of the difference between the actual drill time in step 4 and the simulated drill time in step 3 is calculated using the following formula:

[0011] ;

[0012] When the difference ratio does not exceed 2%, the exercise method is feasible; when the actual exercise time is greater than the simulated exercise time and the difference ratio is greater than 2%, the simulated exercise action is not refined enough; when the actual exercise time is less than the simulated exercise time and the difference ratio is greater than 2%, the simulated exercise process has missing actions.

[0013] In the above technical solution, in step 2, the total time taken by the robot to travel the path between each location point... The calculation formula is:

[0014] ;

[0015] In the formula, This is the total time it takes for the robot to traverse each segment of the path. For the time it takes for the robot to overcome the movement of solidified interference objects, The time required for the robot to overcome interference fields in the specified frequency band.

[0016] In the above technical solution, the total time taken for the robot to traverse each path segment is... The calculation formula is as follows:

[0017] ;

[0018] In the formula, The time it takes for the robot to travel from its original location to the first solidified obstacle. The time it takes for the robot to travel from the first solidified obstacle to the second solidified obstacle. For the robot to pass the first The first solidified interferon to the first The time required for the solidification of interfering substances For the robot to pass the first The time it takes for a solidified interfering substance to reach its final position. This refers to the quantity of solidified interfering substances;

[0019] The time it takes for the robot to travel any segment of the path The calculation formula is:

[0020] ;

[0021] In the formula, This is the total time it takes for the robot to traverse each interference-free path. The time required for the robot to overcome non-solidified interference;

[0022] The total time it takes for the robot to traverse each interference-free path The calculation formula is:

[0023] ;

[0024] In the formula, Indicates the starting point of the path and the first... The time it takes for the robot to move normally within a section of road between non-fixed interference objects. Indicates in and The time it takes for a robot to move normally within a road segment between two adjacent non-solidified interfering objects; Indicates in and The time it takes for a robot to move normally within a road segment between two adjacent non-solidified interfering objects; Indicates in The time it takes for the robot to move normally within the segment of the path from a non-fixed interference object to the end of the path.

[0025] Time taken by the robot to overcome non-solidified interference The calculation formula is as follows:

[0026] ;

[0027] In the formula, To help robots overcome non-solidified interference The duration of exercise, To help robots overcome non-solidified interference The duration of exercise, To help robots overcome non-solidified interference The duration of exercise;

[0028] The calculation formula is:

[0029] ;

[0030] In the formula, To enable robots to bypass non-solidified interference The time taken Waiting for non-solidified interference for the robot The time it took for them to leave on their own. For non-curing interference The set time redundancy coefficient.

[0031] In the above technical solution, the time it takes for the robot to overcome solidified interference is... The calculation formula is:

[0032] ;

[0033] In the formula, To help robots overcome the first x The time it takes for a solidified interfering substance to move. The time required for the robot to overcome the movement of the first solidified interference object. For the time it takes for the robot to overcome the movement of the second solidified interference, To help robots overcome the first The time it takes for a solidified interfering object to move;

[0034] The time it takes for the robot to overcome the xth solidified interference. The calculation formula is:

[0035] ;

[0036] In the formula, The distance the robot needs to travel to bypass the xth solidified obstruction is given by... The speed at which the robot navigates around the xth solidified obstruction;

[0037] The time it takes for the robot to overcome interference from the band interference field The calculation formula is as follows:

[0038] ;

[0039] In the formula, Distance delay caused by interference in each band This indicates the magnification or reduction factor. t Duration of the interference field.

[0040] In the above technical solution, the robot is at its working position. charging and transfer time The calculation formula is:

[0041] ;

[0042] In the formula, For work location To the charging location in the charging field The charging and transfer time, For work location To charging location The charging and transfer time, For work location To charging location The charging and transfer time, This represents the number of charging locations.

[0043] In the above technical solution, the action set of the robot operation in step 3 This includes translational motion within a two-dimensional plane at the end effector of the robotic arm. The end effector of the robotic arm rotates around a certain joint axis. Performing discontinuous complex motions in three-dimensional space at the end effector of the robotic arm Three categories;

[0044] From Action Set Select robot in work position The actions to be performed are formed based on the work location. Set of work actions ,in, In order to be in The position undergoes its first translational motion within the two-dimensional plane. In order to be in Position proceeds to the first Sub-two-dimensional translational motion In order to be in The position initiates the first rotational motion of the robotic arm's end effector around a certain joint axis. In order to be in Position proceeds to the first The end effector of the robotic arm rotates around a certain joint axis. In order to be in The position is where the robotic arm's end effector performs its first discontinuous complex motion in three-dimensional space. In order to be in Position proceeds to the first The end effector of the robotic arm performs non-continuous complex motions in three-dimensional space.

[0045] A second aspect of the present invention is an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the device virtual-real interaction digital simulation method based on mixed reality technology.

[0046] A third aspect of the present invention is a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the device virtual-real interaction digital simulation method based on mixed reality technology.

[0047] A fourth aspect of the present invention is a computer program product, characterized in that the computer program product includes computer-executable instructions, which, when executed, are used to implement the device virtual-real interaction digital simulation method based on mixed reality technology.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] 1. This invention analyzes the equipment operation plan before planning and determines whether the items to be practiced and the participating equipment can reach the designated location. It then plans, tests, verifies, and practices the operation plan, and finally conducts actual equipment operation drills to verify the feasibility of this digital rapid drill method. This method provides a solution for unexpected situations in a scenario, assisting operators in quickly practicing operation plans and actions. It features high safety, high reliability, high drill efficiency, low drill cost, and strong practicality.

[0050] 2. This invention, based on Mixed Reality (MR) technology, designs a set of digital simulation methods for virtual-real interaction of equipment using MR technology. This method can be used for simulated operation training of robots in special environments. By constructing a virtual 3D scene and a virtual robot, and classifying obstacles within the scene, the feasibility of the virtual robot entering the virtual scene is verified, laying the foundation for robot action rehearsal. Depending on the robot and environmental scene, appropriate strategies can be selected for rapid simulation training (virtual robot in a real scene or real robot in a virtual environment). The training is highly efficient and practical. This method can be used for rapid training of work plans and actions in special environments, ensuring that the robot can promptly and effectively formulate work plans and conduct safe and reliable simulation training when facing a sudden situation in the application scenario. Attached Figure Description

[0051] Figure 1 The diagram shows the process flow of the device virtual-real interaction digital simulation method based on mixed reality technology according to the present invention. Detailed Implementation

[0052] The present invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0053] Example 1

[0054] like Figure 1As shown, a method for digital simulation of virtual-real interaction based on mixed reality (MR) technology includes the following steps.

[0055] Step 1: Build a 3D spatial model of the virtual scene Virtual 3D model of the robot ,mark exist Mark the initial working position, start working position, various working positions, end working position, end position, and charging position. The solidified interference, non-solidified interference, and band interference fields present in the sample are as follows:

[0056] A three-dimensional spatial model of the virtual environment to be practiced (this virtual environment can be a special environment or a battlefield) is built. This involves adding elements such as materials and lighting to the 3D spatial model of a virtual scene to improve the fit between the scene model and the real scene. For the length of the virtual scene's three-dimensional spatial model, For the width of the 3D spatial model of the virtual scene, The height of the virtual scene's 3D spatial model. Then the virtual scene's 3D spatial model... The volume is This volume represents the robot's three-dimensional virtual motion space.

[0057] For known robots, build a virtual 3D model of the robot. ,in, For the length of the robot's virtual 3D model, For the width of the robot's virtual 3D model, To enhance the virtual 3D model of the robot, add elements such as materials and lighting to the virtual 3D model of the robot, improve the fit between the virtual robot and the real robot, and add the function of the robot moving in any direction and any posture on the ground.

[0058] Virtual 3D model of the robot Placed in a virtual 3D space model In the process, it is intuitively determined whether the robot can enter the training scenario to perform operations. Specifically, if the robot's virtual 3D model... Can be included in the three-dimensional spatial model of a virtual scene ,Right now Intuitively, the robot's virtual 3D model In the virtual scene 3D space model If there are no protruding parts inside, it can be determined that the virtual robot can enter the training scenario to carry out the operation.

[0059] Using the constructed virtual 3D model of the robot In the virtual scene 3D space model The original location that the robot can reach Mark the original location. This can be the starting point for the robot to begin working, or it can be a point in a non-working area before the starting point (where the robot does not perform any work, but only serves as a transfer and placement area for the robot); and the virtual scene's 3D spatial model should be marked. The scope of work performed by the robot, i.e., the robot's workspace. , For the long working space of the robot, For the robot's working space to be wide, Let the height of the robot's workspace be , then the volume of the robot's workspace is . Specify the robot's starting work point position. The robot begins its operation from this location (moving from this point to the first work position). The path the robot needs to travel before commencing its work is obtained. , From arrive The displacement vector.

[0060] Three-dimensional virtual motion space Interference objects / fields present within The interference was classified and labeled into three categories: solidified interference (Class I), non-solidified interference (Class II), and band interference fields (Class III). These interferences were then visualized in the robot's three-dimensional virtual motion space. Marking to form an interference field ,in, For solidifying interfering objects (Class I) fields, For non-curing interference (Class II) fields, This is a band interference field (Class III).

[0061] Then there is , , ;in, It is the first type of solidification interference (Class I). It is the second type of curing interference (Class I). For the first One solidified interfering substance (Class I); For three-dimensional virtual motion space The number of internally cured interfering substances, The first non-curing interfering substance ( kind), The second non-curing interfering substance ( kind), For the first One non-solidified interfering substance ( kind), For three-dimensional virtual motion space The number of non-cured interfering substances inside, For the first band interference field ( kind), For the second band interference field ( kind), For the first k Interference fields in each band ( kind), k For three-dimensional virtual motion space The number of interference fields in the inner band.

[0062] The path a robot needs to move before it can perform its task. The interfering field present in it is .in For path In Ⅰ Solidified interference, For path In Non-curable interference substances For path In Band-like interference field. Among them, ,in, For path In Ⅰ The number of solidification-like interfering substances, ,in For the first indivual Ⅰ Solidified interference substances; in, For the first indivual Non-curable interference substances For path The number of non-curing interfering substances in the mixture. ; ,in, For the first indivual Band-like interference field, For path In The number of band-like interference fields, .

[0063] Three-dimensional virtual motion space Internal solidification interference ( Ⅰ (Class) and non-curing interferences ⅡThe process involves preprocessing the object (of type) by adding a safety layer model. The solidified or non-solidified interfering object is then divided into parts. A rectangular safety layer is created around the longest side of each part, resulting in a cubic safety layer that completely surrounds the part. This process is repeated to surround all parts of the interfering object, forming its safety layer model. This results in a solidified interfering object with an external safety layer. , For three-dimensional virtual motion space The first solidified interference in the outer security layer. For three-dimensional virtual motion space The solidified interference in the second outer security layer. For three-dimensional virtual motion space The Middle Cured interference in the external security layer, and non-cured interference in the external security layer. , For three-dimensional virtual motion space The first non-cured interference in the outer safety layer. For three-dimensional virtual motion space The second outer safety layer contains non-cured interference. For three-dimensional virtual motion space The Middle A non-cured interference component with an external safety layer; among which, the path The solidified interference in the external security layer is , For path The first one that exists in A solidified interference layer on the outer safety layer, The non-cured interference in the external safety layer is , For path The first one that exists in A non-cured interference substance with an external safety layer.

[0064] In the virtual scene 3D space model Determine the robot's working range, i.e., the starting point of the work. To the end of the work Within this space, the designated work location points that the robot needs to reach are determined and marked, thus obtaining the work location field. , contains One work location; This is the first working location. This is the second working location. For the first Each work location For virtual scene 3D spatial model The number of working positions; to ensure the robot can continuously and safely perform its tasks, the charging positions of the robot in the virtual environment are marked. ,common One charging location This is the first charging location. This is the second charging location. For the first One charging location For virtual scene 3D spatial model The number of charging locations.

[0065] Step 2: Calculate the total time for the robot to travel the path between each location point. This total time includes the sum of the time taken for the robot to traverse each path segment, the time taken to overcome solidified or non-solidified interference after adding the safety layer model, and the time taken to overcome band interference fields. Also, calculate the charging and transfer time for the robot at the working position, specifically:

[0066] The path a robot needs to move before it can perform its task. Based on the solidified interference field of the external safety layer, it is divided into segments, which are divided into: part( (and mark the coordinates of each interfering object to obtain the location field of the solidified interfering objects in the outer security layer) ; For the first Location of solidified interference in the outer safety layer For the first Location of solidified interference in the outer safety layer For the first The location of solidified interference within the outer safety layer; the computational robot's movement path. The time, that is, the total time for the robot to travel the path between each location point. The The calculation formula is:

[0067] ;

[0068] In the formula, This is the total time it takes for the robot to traverse each path segment. To help robots overcome solidified interference ( Ⅰ The time of movement (of a class), The time required for the robot to overcome interference fields in the specified frequency band.

[0069] The calculation formula is:

[0070] ;

[0071] In the formula, The time it takes for the robot to overcome the movement of the xth solidified interference object. The time required for the robot to overcome the movement of the first solidified interference object. For the time it takes for the robot to overcome the movement of the second solidified interference, To help robots overcome the first The time it takes for a solidified interfering substance to move.

[0072] The time it takes for the robot to overcome the xth solidified interference. The calculation formula is:

[0073] ;

[0074] In the formula, The distance the robot needs to travel to bypass the xth solidified obstruction is given by... Let x be the speed at which the robot moves around the x-th solidified obstacle (assuming uniform motion).

[0075] The distance the robot needs to travel to bypass the xth solidified obstruction. The calculation formula is:

[0076] ;

[0077] In the formula, To enlarge the distance from the vertex to the center point of the xth solidified interference safety layer model. Double the circle radius, To set the safety redundancy factor ( ), for The corresponding central angle (in radians). From arrive The displacement vector;

[0078] For circle With the robot's motion path Of the two intersecting points, the one closer to the starting point is... The intersection, For circle With the robot's motion path Of the two intersecting points, the one closer to the termination point The intersection point.

[0079] The calculation formula is:

[0080] ;

[0081] In the formula, Let x be the distance from the vertex of the solidified interference safety layer model to the center point.

[0082] The calculation formula is:

[0083] , for , Distance between two points;

[0084] The calculation formula is as follows:

[0085] ;

[0086] In the formula, The disturbance delay caused by interference in each band This indicates the magnification or reduction factor. t Duration of the interference field.

[0087] The calculation formula is as follows:

[0088] ;

[0089] In the formula, The time it takes for the robot to travel from its original location to the first solidified obstacle. The time it takes for the robot to travel from the first solidified obstacle to the second solidified obstacle. For the robot to pass the first The first solidified interferon to the first The time required for the solidification of interfering substances For the robot to pass the first The time it takes for a solidified interfering substance to reach its final position.

[0090] The time it takes for the robot to travel any segment of the path The calculation formula is:

[0091] ;

[0092] In the formula, This is the total time it takes for the robot to traverse each interference-free path. To help robots overcome non-solidified interference ( Ⅱ Time taken for (class).

[0093] The calculation formula is:

[0094] ;

[0095] In the formula, Indicates the starting point of the path and the first... The time it takes for the robot to move normally within a section of road between non-fixed interference objects. Indicates in and The time it takes for a robot to move normally within a road segment between two adjacent non-solidified interfering objects; Indicates in and The time it takes for a robot to move normally within a road segment between two adjacent non-solidified interfering objects; Indicates in The time it takes for the robot to move normally within the segment of the path from a non-fixed interference object to the end of the path.

[0096] The calculation formula is as follows:

[0097] ;

[0098] In the formula, To help robots overcome non-solidified interference The duration of exercise, To help robots overcome non-solidified interference The duration of exercise, To help robots overcome non-solidified interference The duration of exercise;

[0099] The calculation formula is:

[0100] ;

[0101] In the formula, To enable robots to bypass non-solidified interference The time taken Waiting for non-solidified interference for the robot The time it took for them to leave on their own. For non-curing interference The set time redundancy coefficient. Among them, The calculation method and The calculation method is the same.

[0102] In calculation At that time, if a certain path ( Non-curing interfering substances appeared in ) Ⅱ kind), From arrive The displacement vector is then used to segment the path according to the non-cured interference of the external safety layer, resulting in a total of 10 segments. part( (and mark the coordinates of each interfering object to obtain the non-cured interfering object location field of the outer security layer) ; For the first Location of non-cured interference in the outer safety layer For the first Location of non-cured interference in the outer safety layer For the first The location of non-solidified interference in the outer safety layer; the calculation of the robot's travel time across this segment. ,in, This is the total time it takes for the robot to traverse each interference-free path. To help robots overcome non-solidified interference ( Ⅱ Time taken for (class).

[0103] Based on the total time of the robot's path between all locations. Calculation method for starting point To the initial working position Between each working position, and at the end working position To the end point The transfer time between them will determine the start and end points of the work. To the end of the work Three types of interference objects / fields are marked between each work position to form three types of interference fields. A safety layer model is added for Type I and Type II interference objects. The transfer time between each work position is calculated. The calculation method is the same as that used by the robot to perform pre-work motion path calculation. time Similarly, the robot's work transfer time field is obtained. . For the robot to start from the work point Transfer time to the first work location The transfer time for the robot from the first work position to the second work position. For the robot from the first The work location point to the Transfer time between work locations For the robot from the first From work location to work completion point The transit time.

[0104] And according to the total time of the robot's path between each location point. The calculation method for the initial To the initial charging position Between charging locations, and at the end charging location To the end point The transit time between them is calculated, taking a certain work location as an example. For example, calculate charging and transfer time ,calculate The shortest travel time to each charging point is selected as the [time / time]. , The calculation expression is as follows:

[0105] ;

[0106] in, For work location To charging location The charging and transfer time, For work location To charging location The charging and transfer time, For work location To the charging point in the charging field The charging and transfer time. This represents the number of charging locations.

[0107] The charging and transfer time field of the robot was calculated. , For the robot to start from the starting point To charging location Transit time, From the first working position to the charging position of the robot Transit time, For the robot from the first n From the working location to the charging location Transit time, For the robot to finish its work To charging location The transit time.

[0108] Step 3: Classify the robot's actions to form a set of robot operation actions. Simulate the operation using a real robot in a virtual scenario or use a virtual robot in a real-world scenario. Calculate the time it takes for the robot to complete the actions in the simulation, and sum this time with the total time from Step 2 and the charging / transfer time to obtain the simulation time. Specifically:

[0109] Based on the complexity of the tasks the robot needs to perform, the end-effector actions of its robotic arm are divided into three categories; the first... ⅰ This type of motion is a two-dimensional translational motion performed at the end effector of the robotic arm (denoted as...). ), No. ⅱ This type of motion is a rotational movement of the robotic arm's end effector around a certain joint axis (denoted as...). ), No. ⅲ This type of motion refers to the non-continuous complex motion performed by the end effector of a robotic arm in three-dimensional space. ); forming a set of actions for robot operations. ;

[0110] in, , This is the first translational motion within the two-dimensional plane. This is the second translational motion within the two-dimensional plane. For the first Sub-two-dimensional translational motion;

[0111] , For the first rotational motion of the robotic arm's end effector around a certain joint axis, For the second time, the end effector of the robotic arm rotates around a certain joint axis. For the first The end effector of the robotic arm rotates around a certain joint axis;

[0112] , This is the first instance of a non-continuous complex motion in three-dimensional space performed by the end effector of the robotic arm. This is the second time the robotic arm's end effector performs a non-continuous complex motion in three-dimensional space. For the first The end effector of the robotic arm performs non-continuous complex motions in three-dimensional space.

[0113] Based on the robot and environmental scenario, an appropriate strategy is selected for virtual-real interaction simulation exercises. Specifically, if the real robot is readily available and functional, but the actual environment is inconvenient for the robot to enter or the actual scenario is not fully constructed, a virtual scenario is projected onto an empty space, and a real robot performs a work simulation exercise on the virtual scenario. Following the sequence of work positions, the robot's operational actions are selected sequentially, focusing on the actions required for each work position. For example ( Select the actions to be performed at the work location to form a set of work actions based on that work location. :

[0114] ;

[0115] in In order to be in The position undergoes its first translational motion within the two-dimensional plane. In order to be in Position proceeds to the first Sub-two-dimensional translational motion In order to be in The position initiates the first rotational motion of the robotic arm's end effector around a certain joint axis. In order to be in Position proceeds to the first The end effector of the robotic arm rotates around a certain joint axis. In order to be in The position is where the robotic arm's end effector performs its first discontinuous complex motion in three-dimensional space. In order to be in Position proceeds to the first The end effector of the robotic arm performs non-continuous complex motions in three-dimensional space.

[0116] Among them, action set Each action belongs to the action set of the robot's operation. Set the robot's operating speed to complete the task. For example, set the execution speed of the robot's actions at this work position within the set of actions. The computational robot performs the tasks at this work location. The distance required to move The computational robot performs the tasks at this work location. Duration .

[0117] Computational robot at a certain work position The duration of each action is used to determine the robot's position at a given work location. Operation action time field This forms the operation time field for each working position of the robot. .

[0118] If a real robot cannot reach the actual environment or some functions (non-operational functions) are yet to be tested and developed, but the actual scene is accessible, a virtual robot is projected into the real scene environment to simulate operations in the real scene. The virtual robot is then projected into the actual scene to simulate operations in the real environment. According to the order of the work positions, the actions to be performed at each work position are selected from the robot's set of actions, and the robot's running speed for completing the action is set. Furthermore, the time to execute the action is calculated, and then the time to complete the work position is calculated.

[0119] If the real robot is readily available and functioning well, and the actual scene is accessible, you can choose to use the real robot to simulate the operation in the virtual scene or use the virtual robot to simulate the operation in the actual scene.

[0120] Step 4: Place the real robot in a real scene for a real-world exercise. The robot's speed is the same during the real exercise and the simulated exercise. By comparing the real exercise time with the simulated exercise time in Step 3, the feasibility of the exercise method is verified.

[0121] Calculate the ratio of the difference between the actual drill time in step 4 and the simulated drill time in step 3. The formula for calculating the ratio of the difference is:

[0122] ;

[0123] When the difference ratio does not exceed 2%, the exercise method is feasible; when the actual exercise time is greater than the simulated exercise time and the difference ratio is greater than 2%, the simulated exercise action is not refined enough; when the actual exercise time is less than the simulated exercise time and the difference ratio is greater than 2%, the simulated exercise process has missing actions.

[0124] Example 2

[0125] A method for digital simulation of virtual-real interaction based on mixed reality (MR) technology, with the following specific steps:

[0126] Step 1: When the robot arrives at the three-dimensional virtual motion space The robot is capable of reaching its original position within this environmental scenario. To the designated starting point position The first segment of the journey (i.e., the path the robot needs to travel before it can perform its tasks) There is a solidified interfering substance in it. Ⅰ kind) And it is known Assume that there is only one work location within the site's work area. Starting point location If the origin of the operation is... At the robot's starting point position Arrive at the work location path ( From arrive Within the displacement vector, there exists a band of interference field ( Ⅲ (class), whose total coverage path length is The interference mode is linear time interference, meaning the actual travel time through the interference field needs to be multiplied by a certain factor compared to the theoretical travel time (the interference redundancy coefficient is set according to the site conditions); at this location, the robot needs to pick up a residual material. (for each side length) (cube); distance within the site closest (by The radius of the circle's center (Inside) charging position Located at the boundary of the work site The position, that is At the robot's working position Reaching this charging location path ( From arrive In the displacement vector, there exists a non-solidified interfering object ( Ⅱ kind) The interfering object is another piece of equipment that needs to remain in place for 15 seconds.

[0127] Build a virtual scene 3D space model With robot model By placing the virtual robot in a virtual scene, it can be intuitively determined whether it can enter the scene to be practiced and perform the operation.

[0128] Step 2: Determine the path the robot needs to move before it can begin its work. ,get .

[0129] Determine the path the robot needs to move before it can perform its task. Interference field present in The solidified interfering substance (Class I) was obtained. .

[0130] For solidifying interfering substances ( Ⅰ Add a security layer model to form a solidified interference object with an external security layer. Ⅰ (Class). Obtain the solidified interference of the existing external security layer. The security layer model volume is .

[0131] Computational robot via motion path Time First, calculate how the robot overcomes solidified interference ( Ⅰ (Class) Time of Movement Because there is an obstacle in this section of the path, namely According to the following formula:

[0132] ;

[0133] ;

[0134] ;

[0135] ;

[0136] The distance from the vertex to the center point of the interference security layer model for Set the safety redundancy factor. for Then the circle radius for ,get ( for arrive The central angle (in radians) corresponding to the displacement vector. for To obtain the distance the robot needs to travel to bypass the obstacle. for Let the speed at which the robot navigates around the obstacle be... The time it takes for the robot to navigate around the obstacle is obtained. for .

[0137] Assume the robot follows the motion path Speed ​​on normal road sections for ,get .

[0138] Obtain the robot's movement path time .

[0139] Calculate the starting point To the work location in the case time ,exist ( From arrive The total length of the displacement vector There is a length in the journey. Interference field in the band, interference redundancy coefficient Based on the site conditions, it was set as follows: Let the speed over the entire distance be... ,get .

[0140] Computational robot from work position point To the nearest charging location path ( From arrive The motion time of the displacement vector .

[0141] In the path There is a non-cured interfering substance in it. Ⅱ kind) The interfering object is another piece of equipment that needs to remain in place for 15 seconds, resulting in the robot waiting for the non-fixed interfering object. Time taken to leave on one's own It is 15 seconds, that is .

[0142] Furthermore, computational robots can bypass non-solidified interference. Time used The interfering object is surrounded by a safety layer model, resulting in a non-cured interfering object with an external safety layer. The distance from the vertex to the center point of the security layer model of the interfering object is obtained. for Set the safety redundancy factor. for Then the circle radius for ,get The corresponding central angle (in radians) for , From arrive The displacement vector gives the distance the robot needs to travel to get around the obstacle. for Let the speed at which the robot navigates around the obstacle be... The time it takes for the robot to navigate around the obstacle is obtained. for Assuming this applies to non-curing interfering substances... The set time redundancy factor is The value is 5, so we get ,get .

[0143] Let the motion path be Speed ​​on normal road sections for The computational robot obtains information from its working position. To the nearest charging location path exercise time .

[0144] Step 3: Based on the complexity of the robot's actions within the scenario, divide the robotic arm's end effector into different actions. This is because the work position within the scenario... The robot needs to pick up a piece of residual material. (for each side length) The cube's end motion is picking up, which is a translational motion belonging to the first category, denoted as... .

[0145] Based on the robot and the environmental scenario, an appropriate strategy was selected for simulating virtual-real interaction. Since the robot is fully functional and performs well, a real robot was chosen to practice its actions in a virtual environment.

[0146] Calculate the time it takes for the robot to complete the task at that work location.

[0147] Robot working position supplies distance (Unit: mm) From arrive The displacement vector is calculated. .set up for The time required for the robot to complete the task at that work location is obtained. .

[0148] Step 4: Place the real robot in a real-world scenario, simulate the robot's movements in a virtual environment, and verify the simulation. Verify the robot's movement path. Time The actual time is 65 minutes, starting point To the work location in the case Time The actual time is 20 minutes, from the work location point. To the nearest charging location Time The actual duration was 45 minutes, while the actual time for the drill at the work position was 5 seconds, thus verifying the feasibility of this method. This method is applicable to a sudden working situation occurring in a special environment, and innovatively solves the problem of rapid response drills for sudden working situations in this context.

[0149] The above description is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for digital simulation of virtual-real interaction based on mixed reality technology, characterized in that, Includes the following steps: Step 1: Build a 3D spatial model of the virtual scene Virtual 3D model of the robot ,mark exist Mark the initial working position, start working position, intermediate working position, end working position, end working position, and charging position. Solidified interference, non-solidified interference, and band interference fields exist in the environment; Step 2: Calculate the total time for the robot to travel between each location point. The total time is the sum of the time for the robot to travel through each path segment, the time to overcome solid or non-solid interference after adding the safety layer model, and the time to overcome the band interference field. Also, calculate the charging and transfer time of the robot at the working position. The charging transfer time is the time from the working position to the charging position point in the charging field. When adding a safety layer model, the solidified or non-solidified interference object is divided into parts. A rectangular safety layer is formed by enclosing the longest side of a part so that the resulting cubic safety layer completely encloses the part. All parts of the interference object are enclosed in the same way to form the safety layer model of the interference object. Step 3: Classify the robot's actions to form a set of robot operation actions. Use a real robot to conduct a work simulation exercise in a virtual scene or use a virtual robot to conduct a work simulation exercise in a real scene. Calculate the time for the robot to complete the action in the simulation exercise, and add it to the total time in Step 2 and the charging and transfer time to obtain the simulation exercise time. Step 4: Place the real robot in a real scene for a real-world exercise. The robot's speed is the same during the real exercise and the simulated exercise. By comparing the real exercise time with the simulated exercise time in Step 3, the feasibility of the exercise method is verified.

2. The device virtual-real interaction digital drill method based on mixed reality technology as described in claim 1, characterized in that, Calculate the ratio of the difference between the actual drill time in step 4 and the simulated drill time in step 3. The formula for calculating the ratio of the difference is: ； When the difference ratio does not exceed 2%, the exercise method is feasible; when the actual exercise time is greater than the simulated exercise time and the difference ratio is greater than 2%, the simulated exercise action is not refined enough; when the actual exercise time is less than the simulated exercise time and the difference ratio is greater than 2%, the simulated exercise process has missing actions.

3. The device virtual-real interaction digital drill method based on mixed reality technology as described in claim 1, characterized in that, In step 2, the total time for the robot to traverse the path between each location point. The calculation formula is: ； In the formula, This is the total time it takes for the robot to traverse each segment of the path. For the time it takes for the robot to overcome the movement of solidified interference objects, The time required for the robot to overcome interference fields in the specified frequency band.

4. The device virtual-real interaction digital drill method based on mixed reality technology as described in claim 3, characterized in that, The total time it takes for the robot to traverse each path segment The calculation formula is as follows: ； In the formula, The time it takes for the robot to travel from its original location to the first solidified obstacle. The time it takes for the robot to travel from the first solidified obstacle to the second solidified obstacle. For the robot to pass the first The first solidified interferon to the first The time required for the solidification of interfering substances For the robot to pass the first The time it takes for a solidified interfering substance to reach its final position. This refers to the quantity of solidified interfering substances; The time it takes for the robot to travel any segment of the path The calculation formula is: ； In the formula, This is the total time it takes for the robot to traverse each interference-free path. The time required for the robot to overcome non-solidified interference; The total time taken for the robot to traverse each interference-free path The calculation formula is: ； In the formula, Indicates the starting point of the path and the first... The time it takes for the robot to move normally within a section of road between non-fixed interference objects. Indicates in and The time it takes for a robot to move normally within a road segment between two adjacent non-solidified interfering objects; Indicates in and The time it takes for a robot to move normally within a road segment between two adjacent non-solidified interfering objects; Indicates in The time it takes for the robot to move normally within the segment from a non-solidified interfering object to the end of the path. Time taken by the robot to overcome non-solidified interference The calculation formula is as follows: ； In the formula, To help robots overcome non-solidified interference The duration of exercise, To help robots overcome non-solidified interference The duration of exercise, To help robots overcome non-solidified interference The duration of exercise; The calculation formula is: ； In the formula, To enable robots to bypass non-solidified interference The time taken Waiting for non-solidified interference for the robot The time it took for them to leave on their own. For non-curing interference The set time redundancy coefficient.

5. The device virtual-real interaction digital drill method based on mixed reality technology as described in claim 4, characterized in that, The time it takes for the robot to overcome solidified interference. The calculation formula is: ； In the formula, To help robots overcome the first x The time it takes for a solidified interfering substance to move. The time required for the robot to overcome the movement of the first solidified interference object. For the time it takes for the robot to overcome the movement of the second solidified interference, To help robots overcome the first The time it takes for a solidified interfering object to move; The time it takes for the robot to overcome the xth solidified interference. The calculation formula is: ； In the formula, The distance the robot needs to travel to bypass the xth solidified obstruction is given by... The speed at which the robot navigates around the xth solidified obstruction; The time it takes for the robot to overcome interference from the band interference field The calculation formula is as follows: ； In the formula, The disturbance delay caused by interference in each band This indicates the magnification or reduction factor. t Duration of the interference field.

6. The device virtual-real interaction digital drill method based on mixed reality technology as described in claim 1, characterized in that, Robot at work position charging and transfer time The calculation formula is: ； In the formula, For work location To the charging location in the charging field The charging and transfer time, For work location To charging location The charging and transfer time, For work location To charging location The charging and transfer time, This represents the number of charging locations.

7. The device virtual-real interaction digital drill method based on mixed reality technology as described in claim 1, characterized in that, The set of actions for robot operation described in step 3 This includes translational motion within a two-dimensional plane at the end effector of the robotic arm. The end effector of the robotic arm rotates around a certain joint axis. The robotic arm end effector performs discontinuous complex motions in three-dimensional space. Three categories; From Action Set Select robot in work position The actions to be performed are formed based on the work location. Set of work actions ,in, In order to be in The position undergoes its first translational motion within the two-dimensional plane. In order to be in Position proceeds to the first Sub-two-dimensional translational motion In order to be in The position initiates the first rotational motion of the robotic arm's end effector around a certain joint axis. In order to be in Position proceeds to the first The end effector of the robotic arm rotates around a certain joint axis. In order to be in The position is where the robotic arm's end effector performs its first discontinuous complex motion in three-dimensional space. In order to be in Position proceeds to the first The end effector of the robotic arm performs non-continuous complex motions in three-dimensional space.

8. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the device virtual-real interaction digital simulation method based on mixed reality technology as described in claim 1.

9. A computer-readable storage medium, characterized in that, It stores computer-executable instructions, which, when executed, are used to implement the device virtual-real interaction digital drill method based on mixed reality technology as described in claim 1.

10. A computer program product, characterized in that, The aforementioned computer program product includes computer-executable instructions, which, when executed, are used to implement the device virtual-real interaction digital simulation method based on mixed reality technology as described in claim 1.

Images