Adjusting virtual relative position in virtual robot workcell

Abstract

A method (200) for adjusting the position of a component of a robot work unit (30) in a virtual robot work unit (40) simulating the robot work unit (30). The method (200) comprises: acquiring relative positioning information (202) between a robot (31) and an object carrier (32) from a controller (10) for the robot (31) in a robot work unit (30); and adjusting the relative position between a virtual robot (41) corresponding to the robot (31) in the virtual robot work unit (40) and a virtual object carrier (42) corresponding to the object carrier (32) in the virtual robot work unit (40) on the basis of the relative position information. Relative positioning information between the robot (31) and the object carrier (32) can thus be automatically obtained and digitally analyzed.

Description

Technical Field

[0001] Example embodiments of the present disclosure relate generally to robotic work cell simulation and, more particularly, to methods, apparatus, systems, and computer-readable media for adjusting virtual relative positions in a virtual robotic work cell. Background Art

[0002] A digital twin is a digital representation of a physical object, process, or service. A digital twin can be a digital replica of an object in the physical world. Digital twin technology can be used to replicate processes to collect data to predict how they will perform, allowing for the prevention of costly failures in physical objects. Therefore, it is well suited for use in the robotics industry.

[0003] For example, in the process of building a robot work cell. First, in the design phase, a virtual robot work cell will be established in a computer program to simulate the robot work cell to be built. When the simulation is completed, the physical robot work cell will be built and tested. In the testing phase, the size and position of the components in the robot work cell may be adjusted. In this case, the parameters in the virtual robot work cell may be different from those of the adjusted robot work cell. In this regard, in order to simulate the constructed robot work cell, the parameters in the virtual robot work cell must be synchronized with the constructed robot work cell. Therefore, it is desirable to propose an effective solution for synchronization between the constructed robot work cell and its digital copy. Summary of the invention

[0004] Example embodiments of the present disclosure provide a solution for synchronizing the locations of components in real and virtual environments.

[0005] In a first aspect, an example embodiment of the present disclosure provides a method for adjusting a virtual relative position in a virtual robot work cell that simulates a robot work cell. The method includes acquiring the relative position between the robot and an object carrier in the robot work cell from a controller for controlling the robot in the robot work cell. The method also includes: in the virtual robot work cell that simulates the robot work cell, based on the relative position, adjusting the virtual relative position between the virtual robot corresponding to the robot and the virtual object carrier corresponding to the object carrier. In such an embodiment, the relative position can be read from the physical controller of the robot, for example, through a specific interface. The relative position between the robot and the object carrier can therefore be automatically acquired and digitally analyzed, thereby avoiding manual measurement, thereby improving the efficiency and accuracy of position synchronization between the physical work cell and the virtual work cell.

[0006] In some example embodiments, adjusting the virtual relative position includes: determining a base height of a virtual object carrier based on the relative position and a virtual robot height of the virtual robot; determining a height compensation based on a cell configuration of the robot work cell; and adjusting the height of the virtual object carrier based on the base height and the height compensation. In such embodiments, the cell configuration may involve the use of visual detection equipment such as a camera or sensor, and the shape of the object carrier. The cell configuration may affect the selection of a position reference in the virtual work cell. Therefore, by determining the height compensation taking into account the cell configuration, the accuracy of the position adjustment may be improved.

[0007] In some example embodiments, adjusting the relative position further comprises: determining a virtual target horizontal position of the virtual object carrier based on the relative position and the virtual horizontal position of the virtual robot; and adjusting the virtual position of the virtual object carrier to the virtual target horizontal position. In such an embodiment, by separating the vertical and horizontal position adjustments, the adjustment process can be simplified.

[0008] In some example embodiments, determining the base height includes: obtaining a virtual robot height of the virtual robot; determining the relative height based on the relative position; and determining the difference between the virtual robot height and the relative height as the base height. Generally, the relative position may indicate a robot reference point on the robot and a carrier reference point on the surface of the object carrier to describe the relative position. In such an embodiment, by considering the virtual robot as a basis for position adjustment, the position of the virtual robot may be maintained without any adjustment, thereby simplifying the adjustment process.

[0009] In some example embodiments, determining the height compensation includes: obtaining an original virtual surface height of the surface of the virtual object carrier; obtaining a virtual object height of the virtual object at a point where the virtual object appears on the virtual object carrier; and determining the difference between the original virtual surface height and the target virtual height as the height compensation. In such an embodiment, when the object does not appear on the surface of the object carrier for simulation reasons, the height can be compensated so that the height adjustment can be more precise.

[0010] In some example embodiments, determining the height compensation includes: in response to determining that the robotic work cell is configured with a camera above the surface of the object carrier, determining that the height compensation is zero. In such embodiments, when the camera is configured to provide visual data of the object, the object is typically located on the surface of the carrier. Therefore, there is no need to compensate for the base height.

[0011] In some example embodiments, the method further includes: adjusting the position of the virtual camera based on the virtual target horizontal position. The relative position of the camera and the object bracket is fixed. In such an embodiment, after determining the target horizontal position, the virtual camera as a peripheral device can be adjusted, for example, based on the adjustment distance.

[0012] In some example embodiments, determining the height compensation includes: in response to determining that the robot work cell is configured with a sensor for sensing an object on a surface of the object carrier, obtaining a virtual sensor height of a virtual sensor corresponding to the sensor in the virtual robot work cell; obtaining a virtual object height of the virtual object at a point where the virtual object appears on the virtual object carrier; and determining the difference between the virtual sensor height and the virtual object height as the height compensation. The sensor is typically located at an edge of the object carrier to detect the position of an object traveling through a detection range of the sensor. In such an embodiment, the height of the sensor corresponds to the original height of the virtual object carrier before adjustment and can be used to calculate the height compensation.

[0013] In some example embodiments, the method further includes adjusting the position of the virtual sensor based on the virtual target horizontal position. The relative position between the sensor and the object carrier is also usually fixed. In such an embodiment, after determining the target horizontal position, the virtual sensor as a peripheral device can be adjusted, for example, based on the adjustment distance of the object carrier.

[0014] In some example embodiments, the method further comprises: in response to determining that the object carrier is linear, obtaining a virtual robot orientation of the virtual robot; and adjusting the virtual orientation of the virtual object carrier based on the virtual robot orientation and the relative orientation between the robot and the object carrier. The relative orientation generally indicates the relative orientation between the object carrier and the robot on multiple axes. In such embodiments, the orientation of the object carrier can also be adjusted relative to the virtual robot.

[0015] In some example embodiments, the method further includes: in response to determining that the object carrier is circular, determining the center of the virtual object carrier as the origin of a virtual object carrier coordinate system of the virtual object carrier; and adjusting the orientation of the virtual object carrier coordinate system based on the virtual robot orientation and the relative orientation between the robot and the object carrier. In such an embodiment, when the object carrier is circular, the center of the object carrier will be the origin of the virtual object carrier coordinate system of the virtual object carrier, and the virtual object carrier coordinate system can be adjusted.

[0016] In some example embodiments, the method further comprises: in response to determining that the robot work cell further comprises an additional robot, obtaining an additional relative position between the object carrier and the additional robot from an additional controller for controlling the additional robot; and adjusting a virtual position of the virtual additional robot corresponding to the additional robot in the virtual robot work cell based on the relative position and the additional relative position. In such an embodiment, the positions of all components in the robot work cell will be adjusted relative to the first robot, which may be the robot closest to the loading position of the object.

[0017] In a second aspect, an exemplary embodiment of the present disclosure provides a device for adjusting a virtual relative position in a virtual robot work cell. The device includes: a relative position acquisition unit configured to acquire a relative position between a robot and an object carrier in the robot work cell from a controller for controlling the robot in the robot work cell. The device also includes a virtual relative position adjustment unit configured to adjust a virtual relative position between a virtual robot corresponding to the robot and a virtual object carrier corresponding to the object carrier in a virtual robot work cell simulating the robot work cell based on the relative position.

[0018] In a third aspect, an example embodiment of the present disclosure provides a system for adjusting a virtual relative position in a virtual robot work cell. The system includes: a computer processor coupled to a computer readable memory unit, the memory unit including instructions that, when executed by the computer processor, implement a method for adjusting a virtual relative position in a virtual robot work cell.

[0019] In a fourth aspect, an exemplary embodiment of the present disclosure provides a computer-readable medium having instructions stored thereon, which, when executed on at least one processor, causes the at least one processor to perform a method for adjusting virtual relative positions in a virtual robot work unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other purposes, features and advantages of the exemplary embodiments disclosed herein will become more easily understood through the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several exemplary embodiments disclosed herein are shown by way of example and not limitation, in which:

[0021] Figure 1 shows a schematic block diagram of an example robotic working system in which example embodiments of the present disclosure may be implemented;

[0022] Figure 2 A schematic flow chart of a method for adjusting the position of a component of a robot work cell in a virtual robot work cell according to an embodiment of the present disclosure is shown;

[0023] Figures 3A to 3C A schematic flow chart of a method for adjusting the height of a component of a robot work cell in a virtual robot work cell according to an embodiment of the present disclosure is shown;

[0024] Figure 4A 4C show schematic block diagrams of virtual robot work cells with different configurations according to embodiments of the present disclosure, wherein the object carrier is linear;

[0025] Figures 5A to 5C Schematic block diagrams of virtual robot work cells with different configurations according to embodiments of the present disclosure are shown, wherein the object carrier is circular;

[0026] Figure 6 A schematic block diagram of a virtual robot work cell with two robots according to an embodiment of the present disclosure is shown;

[0027] Figure 7 A schematic diagram showing an apparatus for adjusting the position of a component of a robot work cell in a virtual robot work cell according to an embodiment of the present disclosure is shown;

[0028] Figure 8 A schematic diagram of a system for implementing a method according to an embodiment of the present disclosure is shown.

[0029] Throughout the drawings, the same or similar reference numbers are used to refer to the same or similar elements. DETAILED DESCRIPTION

[0030] The principles of the present disclosure will now be described with reference to several example embodiments shown in the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that these embodiments are described only to facilitate those skilled in the art to better understand and thereby implement the present disclosure, rather than to limit the scope of the present disclosure in any way.

[0031] As mentioned above, in order to simulate a constructed robotic work cell, the parameters need to be synchronized with the constructed robotic work cell. Traditionally, users manually adjust the size, position, and orientation of components such as conveyors, robots, and cameras to align them with the real cell. However, the size of the components in the constructed robotic work cell is difficult to measure manually. At the same time, manual measurements may have relatively low accuracy.

[0032] In order to at least partially solve the above and other potential problems, according to an embodiment of the present disclosure, a new method for automatically adjusting the relative positions of components of a robot work cell in a virtual robot work cell is provided. Typically, the size and position of the corresponding components in the virtual robot work cell can be automatically updated (or partially updated) with information derived from the relative position between the robot and the object carrier, which can be read from the real controller, for example, by calling a controller API. In this way, manual measurement can be avoided, thereby improving the efficiency and accuracy of position synchronization between the physical work cell and the virtual work cell.

[0033] Figure 1 1 is a schematic block diagram of an exemplary robot working system 100, in which an exemplary embodiment of the robot working system is disclosed. Figure 1In the figure, the robotic work system 100 includes a robotic work cell 30. The robotic work cell 30 generally includes a robot 31 and an object carrier 32 under the robot 31. In the illustrated embodiment, the robot 31 can be a pick-and-place robot, and the object carrier 32 can be a conveyor. The robot 31 is coupled to the controller 10 and controlled by the controller 10 to pick up an object 33 on the surface of the object carrier 32. To facilitate the operation of the robot 33, other visual detection devices, such as a camera 34 or a sensor 35, may optionally be provided. The controller 10 is also coupled to a computing device 20. The computing device 20 can communicate with the controller 10 to obtain information about the robotic work cell 30. In addition, the computing device 20 can perform a simulation of the robotic work cell 30 and display the simulation in real time on a display of the computing device 20. Figure 1 As shown, a virtual robot work cell 40 is shown. The virtual robot work cell 40 includes a virtual robot 41 corresponding to the robot 31, and a virtual object carrier 42 corresponding to the object carrier 32. The virtual robot work cell 40 may also include a virtual camera 44 or a virtual sensor 45. In addition, the virtual robot work cell 40 may be referred to as a digital twin of the robot work cell 30. The size and position of the components in the virtual robot work cell 40 should be synchronized with the size and position of the components in the robot work cell 30.

[0034] It should be understood that the robot can be any other industrial robot, and the number of robots can be more than one. In addition, the object carrier can also be any other work platform supporting an object, and the number of object carriers can also be more than one.

[0035] In the following, reference will be made to Figures 2 to 6 for more details on the method for adjusting virtual relative positions in a virtual robotic work cell according to the present disclosure.

[0036] Figure 2 1 shows a schematic flow chart of a method 200 for adjusting the position of a component of a robot work cell in a virtual robot work cell according to an embodiment of the present disclosure. The method 200 may be Figure 1 It is implemented by the computing device 20 in.

[0037] At 202, the computing device 20 obtains the relative position between the robot 31 and the object carrier 32 in the robot work cell 30 from the controller 10 for controlling the robot 31 in the robot work cell 30. In the robot operation, one of the important parameters is the relative position between the robot and the object carrier surface. The correlation is usually defined and stored in the controller.

[0038] At 204, the computing device 20 adjusts the virtual relative position between the virtual robot 41 corresponding to the robot 31 and the virtual object carrier 42 corresponding to the object carrier 32 in the virtual robot work cell 40 simulating the robot work cell 30 based on the relative position. In this embodiment, the relative position between the virtual robot 41 and the virtual object carrier 42 includes the relative distance in the vertical direction and the horizontal direction. Figures 3A to 3C Describe the determination of relative height in detail.

[0039] In some example embodiments, the virtual robot 41 may be used as a basis for adjustment. That is, the virtual robot 41 is fixed, and the virtual object carrier 42 is adjusted to obtain a relative position. For example, if the relative distance is greater than the current distance, the corresponding size of the virtual object carrier 42 may be reduced to increase the current distance to achieve the relative distance. Alternatively, the virtual object carrier 42 may be used as a basis for adjustment. That is, the virtual object carrier 42 is fixed and the virtual robot 41 is adjusted to achieve the relative position.

[0040] In this way, the relative position between the virtual robot 41 and the virtual object carrier 42 can be automatically updated, thereby improving the accuracy and efficiency of the work unit synchronization.

[0041] Figure 3A 3 shows a schematic flow chart of a method 301 for adjusting the height of a component of a robot work cell in a virtual robot work cell according to an embodiment of the present disclosure. The method 301 may be Figure 1 It is implemented by the computing device 20 in.

[0042] At 302, the computing device 20 determines the base height of the virtual object carrier 42 based on the relative position of the virtual robot 41 and the virtual robot height. The base height may indicate the distance from the reference point of the virtual object carrier 42 to the floor of the virtual robot work cell 40. Figure 3B A method for determining the susceptor height is described in detail.

[0043] Figure 3B FIG. 3 is a schematic flow chart of a method 303 for determining a base height according to an embodiment of the present disclosure. The method 303 may be performed by Figure 1 The computing device 20 in FIG. 308 obtains the virtual robot height of the virtual robot 41. In the present embodiment, the virtual robot 41 is used as a basis for adjustment. The virtual robot height is the distance from a reference point on the virtual robot to the floor surface of the virtual robot work cell 40. In some embodiments, the virtual robot 41 may be configured with a base coordinate system. The origin of the coordinate system of the virtual robot 41 may be used as a reference point.

[0044] At 310, the computing device 20 determines a relative height based on the relative position. The relative height may be a vertical component of the relative position. At 312, the computing device 20 determines the difference between the virtual robot height and the relative height as the base height. In this way, the distance from the reference point of the virtual object carrier 42 to the floor of the virtual robot work cell 40 may be determined.

[0045] The relative position may indicate the distance from the reference point of the robot 31 to the surface of the object holder 32. However, when the reference point of the virtual object carrier 42 is not located on the surface of the virtual object carrier 42, height compensation needs to be used to adjust the base height.

[0046] Back to Figure 3A At 304, the computing device 20 determines a height compensation based on a cell configuration of the robot work cell. In some example embodiments, the cell configuration may include a shape of the object carrier, visual detection equipment such as cameras or sensors, and the number of robots. The cell configuration may be read from the configuration information of the virtual work cell. The height compensation is associated with the cell configuration and will be referenced to Figure 3C Detailed description.

[0047] Figure 3C 1 shows a schematic flow chart of a method 305 for determining height compensation according to an embodiment of the present disclosure. The method 305 may be Figure 1 The computing device 20 in FIG. 314 determines whether the robotic work cell 30 is configured with a camera 34 located above the surface of the object carrier 32. If the computing device 20 determines that the robotic work cell 30 is configured with a camera 34, the computing device 20 determines at 316 that the height compensation is zero. In the case of using a camera, the height of the object carrier will be identified as the distance from the surface of the virtual object carrier 42 to the work cell floor. In this case, there is no need to compensate for the base height.

[0048] If the computing device 20 determines that the robot work cell 30 is not configured with the camera 34, then at 318, the computing device 20 determines whether the robot work cell 30 is configured with the sensor 35 for sensing the object on the surface of the object carrier 31. If the computing device 20 determines that the robot work cell 30 is configured with the sensor 35, then at 320, the computing device 20 obtains the virtual sensor height of the virtual sensor 45 corresponding to the sensor 35 in the virtual robot work cell 40. In the case of using a sensor, the sensor is located on the edge of the surface. Therefore, the virtual sensor height of the virtual sensor 45 can indicate the original height of the surface of the virtual object carrier 42.

[0049] At 322, the computing device 20 obtains the virtual object height of the virtual object 43 at the point where the virtual object 43 appears on the virtual object carrier. In the virtual work unit 40, the initial position where the object appears can be predefined. Meanwhile, in this case, the reference point of the virtual object carrier 42 is defined at the same height of the initial position. However, due to some simulation preferences, the initial position may not be predefined on the surface of the virtual object carrier 42. Therefore, the reference point of the virtual object carrier 42 is also not on the surface of the virtual object carrier 42.

[0050] At 324, computing device 20 determines the difference between the virtual sensor height and the virtual object height as height compensation. As described above, when the reference point of virtual object carrier 42 is also not on the surface of virtual object carrier 42, the height compensation can be determined based on the difference between the virtual sensor height and the virtual object height.

[0051] If the computing device 20 determines that the robotic work cell 30 is not yet configured with the sensor 35, then at 326, the computing device 20 obtains the original virtual surface height of the surface of the virtual object carrier 42. The original virtual surface height represents the distance from the surface of the virtual object carrier 42 to the floor of the virtual robotic work cell 40 before adjusting the part.

[0052] At 328 , computing device 20 obtains the virtual object height ( 33 ) of the virtual object at the point where the object initially appears on the virtual object carrier. The virtual target height and the original virtual target carrier height may be obtained from the configuration of the virtual robotic work cell 40 .

[0053] At 330, the computing device 20 determines the difference between the original virtual object carrier height and the object virtual height as height compensation. As described above, the difference between the original virtual object carrier height and the object virtual height may also indicate the distance from the reference point of the virtual object carrier 42 to the surface of the virtual object carrier 42. At this point, the determination process of the height compensation in different situations is described.

[0054] return Figure 3A After determining both the base height and the height compensation, the computing device 20 adjusts the height of the virtual object carrier 42 to the target virtual height based on the base height and the height compensation at 306. In this way, by considering the unit configuration during the determination of the target height, an accurate target virtual height can be achieved.

[0055] Figure 4A 4C shows a schematic block diagram of a virtual robot work unit 40 with different configurations according to an embodiment of the present disclosure, wherein the object carrier is linear. All components in the figure have been adjusted.

[0056] Specifically, Figure 4AA schematic block diagram 401 of a virtual robot work cell 40 without a visual inspection device is shown. Figure 4A As shown, the virtual robot work unit 40 includes a virtual robot 41 configured with a robot coordinate system 402 and a virtual object carrier 42 configured with a carrier coordinate system 404. The virtual object 43 is configured to appear at an initial point on the virtual object carrier 42. The origin is also the origin of the object coordinate system 406. As shown in the figure, the object coordinate system 406 is below the surface 421 of the virtual object carrier 42. Therefore, the carrier coordinate system 404 and the object coordinate system 406 are in the same horizontal plane. Therefore, the object coordinate system 406 is also below the surface 421.

[0057] In this embodiment, the relative position may be included in the relative position information from the controller 10. This information may include six values. That is, the relative position of the three orientations and three relative orientations of the holder coordinate system of the object holder 32 relative to the robot coordinate system of the robot 31. For example, in the robot operating system (ROS), the coordinate system may be referred to as a "base frame". In order to synchronize the positions of the components in the actual work cell and the virtual work cell, the relative positions in three directions will be determined. In the illustrated embodiment, the relative height derived from the relative position information is shown as H1. The height of the virtual robot 41, that is, the height from the origin of the robot coordinate system 402 to the floor of the virtual robot work cell 40, is represented as H robot Therefore, the base height H2 can be determined as follows:

[0058] H2=H robot -H1 (1)

[0059] In order to determine the target height of the surface 421, the height compensation H3 must be determined, that is, the height of the surface 421 to the carrier coordinate system 404. Since the height compensation H3 is the same as the height compensation before adjusting the surface, it can be based on the original surface height H original_surface and the original object height H of the object coordinate system 406 which can be read from the configuration of the virtual robot work cell 40 original_object To determine H3. The height compensation can be determined as follows:

[0060] H3=H original_surface -H original_object (2)

[0061] Finally, the target height H of the surface 421 target It can be determined as follows:

[0062] H target =H2+H3=H robot -H1+H original_surface -H original_object (3)

[0063] In some embodiments, after adjusting the height, the position in the x and y directions will be directly adjusted based on the relative position in the x and y directions according to the relative position. In addition, the orientation can also be adjusted based on the relative orientation in the relative position information.

[0064] Figure 4B A schematic block diagram 403 of a virtual robot work cell 40 with a camera 44 is shown. Figure 4B As shown, the virtual robot work unit 40 is configured with a virtual camera 44. The virtual camera 44 is located directly above the carrier coordinate system 404. In this embodiment, since the virtual camera 44 is provided, the carrier coordinate system 404 is on the surface of the virtual object carrier 42. Therefore, the height compensation is zero. Therefore, the target height H target is equal to the base height H2 and can be determined according to equation (1) as follows:

[0065] H target =H2=H robot -H1 (4)

[0066] In addition to adjusting the positions of other components as described above, after adjusting the height, the virtual camera 44 will be adjusted to a position just above the carrier coordinate system 404, such as at its original height.

[0067] Figure 4C A schematic block diagram 405 of a virtual robot work cell 40 with a virtual sensor 45 is shown. Figure 4C As shown, the virtual sensor 45 is located at the edge of the surface 421 and is configured with a sensor coordinate system 410. The object coordinate system 406 and the carrier coordinate system 404 are in the same horizontal plane and below the surface 421 of the virtual object carrier 42. In the embodiment shown, the relative height derived from the relative position is denoted as H1. The height of the virtual robot 41 is denoted as H robot Therefore, the base height H2 can be determined according to equation (1). In order to determine the target height of the surface 421, the height compensation H3 must be determined. Figure 4C , can be based on the sensor height H of the virtual sensor 45 sensor and the object height H of the object coordinate system 406 which can be read from the configuration of the virtual robot work cell 40 object To determine the height compensation H3. The height compensation can be determined as follows:

[0068] H3=H sensor -H object (5)

[0069] Finally, the target height H of the surface 421 target It can be determined as follows:

[0070] H target =H2+H3=H robot -H1+H sensor -H object (6)

[0071] In addition to adjusting the positions of other components as described above, after adjusting the height, the H in the relative position information will be target and other relative positions and orientations to adjust the virtual sensor 45.

[0072] Figures 5A to 5C Schematic block diagrams of virtual robot work cells with different configurations according to embodiments of the present disclosure are shown, wherein the object carrier is circular. Figure 5A -5C shows the height adjustment and Figure 4A The height adjustment in the embodiment shown in FIG4C is basically the same. Therefore, the description of the height adjustment will be omitted here.

[0073] Specifically, Figure 5A A schematic block diagram 501 of a virtual robot work cell 40 without a visual inspection device is shown. Figure 5A As shown, the virtual robot work unit 50 includes a virtual robot 51 configured with a robot coordinate system 502 and a circular virtual object carrier 52 configured with a carrier coordinate system 504. The carrier coordinate system 504 is located at the center of the circular virtual object carrier 52. The virtual object 53 is configured to appear at the initial point on the virtual object carrier 52. The origin is also the origin of the object coordinate system 506. As shown in the figure, the y direction of the object coordinate system 506 and the x direction of the carrier coordinate system 504 are oriented in the same direction.

[0074] In this embodiment, after adjusting the height, the position of the circular virtual object carrier 52 in the x and y directions will be directly adjusted based on the relative positions in the x and y directions. The position of the object coordinate system 506 is adjusted based on the adjustment made on the circular virtual object carrier 52. For example, the adjustment distance of the circular virtual object carrier 52 can be determined as an adjustment vector from the original position to the target position. Then, the position of the object coordinate system 506 can be adjusted using the adjustment vector.

[0075] Figure 5B A schematic block diagram 503 of a virtual robot work cell 50 with a camera 44 is shown. Figure 5B As shown, the virtual robot working unit 50 is equipped with a virtual camera 54. A projection 541 of the virtual camera 54 on the surface 521 of the circular virtual object carrier 52 is located on a radius of the carrier coordinate system 504 in the x direction.

[0076] After adjusting the positions of the circular virtual object carrier 52 in the x and y directions, the position of the virtual camera 54 is adjusted so that the projection 541 is still on the radius of the carrier coordinate system 504 in the x direction.

[0077] Figure 5C A schematic block diagram 505 of a virtual robot work cell 50 with a virtual sensor 55 is shown. Figure 5C As shown, the virtual sensor 55 is located at the inner edge of the surface 521 and is configured with a sensor coordinate system 510. The y direction of the sensor coordinate system 510 is parallel to the x direction of the carrier coordinate system 504.

[0078] After adjusting the position and orientation of the circular virtual object carrier 52 . Since the position and orientation between the virtual sensor 55 and the circular virtual object carrier 52 are fixed, the position of the virtual sensor 55 is adjusted in parallel with the adjustment of the circular virtual object carrier 52 .

[0079] Figure 6 A schematic block diagram 600 of a virtual robot work cell 60 with two robots according to an embodiment of the present disclosure is shown. It should be understood that the two robots are merely exemplary. The virtual robot work cell 60 may have any number of robots.

[0080] The virtual robot work unit 60 corresponds to a real robot work unit including two robots and a linear object carrier 62. Each of the two robots is controlled by a controller. Therefore, the virtual robot work unit 60 includes a first virtual robot 61 and a second virtual robot 66 corresponding to the two robots and a virtual linear object carrier 62. In this embodiment, the first virtual robot 61 is used as the basis for position and direction adjustment. That is, the position and direction of the components in the work unit will be updated relative to the first virtual robot 61, which is the virtual robot closest to the starting point of the virtual linear object carrier 62. The position and direction adjustment of the virtual linear object carrier 62 and other optional components can be performed according to the above method. In the following, the position and direction adjustment of the second virtual robot 66 will be described.

[0081] In also Figure 6 In the most common case of the embodiment shown in , the two carrier coordinate systems in the two controllers are defined at the same position and orientation in the work unit, although they are represented by different values ​​in different controllers. Therefore, the relative position information M is read from the two different controllers as two position matrices u and relative position information M I2 . M I1 represents the relative position of the object carrier relative to the first robot, M I2In order to adjust the position and orientation of the second robot relative to the first robot, the relative position information M between the second robot and the first robot is I3 Determined by:

[0082] M I3 =M I1 *inv(M I2 ) (7)

[0083] Where inv(M I2 ) indicates M I2 The reciprocal of M I2 represents the relative position and orientation of the second robot relative to the object holder. That is, first, the coordinate system of the object holder relative to the second robot is converted to the coordinate system of the second robot relative to the object holder. Then, the coordinate system of the second robot relative to the object holder is converted to the coordinate system of the second robot relative to the first robot through the coordinate system of the object holder relative to the first robot. I3 Once determined, the position and orientation of the second virtual robot 66 may be adjusted based thereon.

[0084] The previous paragraphs have described the detailed steps of methods 200 and 300. In some embodiments of the present disclosure, methods 200 and 300 may be implemented by corresponding devices. Figure 7 A schematic diagram of a position adjustment device 700 for adjusting a virtual relative position in a virtual robot work cell 40 is shown. The device includes a relative position acquisition unit 710, which is configured to acquire a relative position between the robot 31 and the object carrier 32 in the robot work cell 30 from a controller 10 for controlling the robot 31 in the robot work cell 30. The device also includes a virtual relative position adjustment unit 720, which is configured to adjust a virtual relative position between a virtual robot 41 corresponding to the robot 31 and a virtual object carrier 42 corresponding to the object carrier 32 in the virtual robot work cell 40 simulating the robot work cell 30 based on the relative position.

[0085] In some example embodiments, the relative position of the acquisition unit 710 includes: a base height determination module, which is configured to determine the base height of the virtual object carrier 42 based on the relative position of the virtual robot 41 and the virtual robot height; a height compensation determination module, which is configured to determine the height compensation based on the unit configuration of the robot working unit 30; and a height adjustment unit, which adjusts the height of the virtual object carrier 42 based on the base height and the height compensation.

[0086] In some example embodiments, the relative position adjustment acquisition unit 710 also includes: a virtual target horizontal position determination module, which is configured to determine the virtual target horizontal position of the virtual object carrier 42 based on the relative position and the virtual horizontal position of the virtual robot 41; and a virtual position adjustment module, which adjusts the virtual position of the virtual object carrier 42 to the virtual target horizontal position.

[0087] In some example embodiments, the base height determination module includes: a virtual robot height acquisition component configured to acquire the virtual robot height of the virtual robot 41; a relative height determination component configured to determine the relative height based on the relative position; and a first difference determination component configured to determine the difference between the virtual robot height and the relative height as the base height.

[0088] In some example embodiments, the height compensation determination module includes: an original height acquisition component, which is configured to acquire the original virtual surface height of the surface of the virtual object carrier 42; a virtual object height acquisition component, which is configured to acquire the virtual object height of the virtual object 43 at the point where the virtual object 43 appears on the virtual object carrier 42; and a second difference determination component, which is configured to use the difference between the original virtual surface height and the target virtual height as the height compensation.

[0089] In some example embodiments, the height compensation determination module includes a determination component configured to determine that the height compensation is zero in response to determining that the robotic work cell 30 is configured with the camera 34 above the surface of the object carrier 32 .

[0090] In some example embodiments, the apparatus further includes: a position adjustment section that adjusts the position of the virtual camera 44 based on the horizontal position of the virtual target.

[0091] In some example embodiments, the height compensation determination module includes: a virtual sensor height acquisition component, which is configured to acquire a virtual sensor height of a virtual sensor 45 corresponding to the sensor 35 in the virtual robot work unit 40 in response to determining that the robot work unit 30 is configured with a sensor 35 for sensing an object on the surface of the object carrier 32; a virtual object height acquisition component, which is configured to acquire the virtual object height of the virtual object 43 at the point where the virtual object 43 appears on the virtual object carrier 42; and a third difference determination component, which is configured to determine the difference between the virtual sensor height and the virtual object height as the height compensation.

[0092] In some example embodiments, the apparatus further comprises a position adjustment unit to adjust the position of the virtual sensor 45 based on the horizontal position of the virtual target.

[0093] In some example embodiments, the device also includes: a virtual robot orientation acquisition unit, which is configured to acquire a virtual robot orientation of the virtual robot 41 in response to determining that the object carrier 32 is linear; and a virtual orientation adjustment unit, which is configured to adjust the virtual orientation of the virtual object carrier 42 based on the virtual robot orientation and the relative orientation between the robot 31 and the object carrier 32.

[0094] In some example embodiments, the device further includes: a center determination unit configured to determine the center of the virtual object carrier 42 as the origin of the virtual object carrier coordinate system of the virtual object carrier 42 in response to determining that the object carrier 32 is circular; and an orientation adjustment unit configured to adjust the orientation of the virtual object carrier coordinate system based on the virtual robot orientation and the relative orientation between the robot 31 and the object carrier 32.

[0095] In some example embodiments, the device further includes: an additional relative position acquisition unit, which is configured to acquire an additional relative position between the object carrier 32 and the additional robot from an additional controller for controlling the additional robot in response to determining that the robot work unit 40 also includes an additional robot; and a virtual position adjustment module, which is configured to adjust the virtual position of the virtual additional robot corresponding to the additional robot in the virtual robot work unit 40 based on the relative position and the additional relative position.

[0096] In some embodiments of the present disclosure, a system for implementing the above methods 200 , 301 , 303 and 305 is provided. Figure 8 A schematic diagram of a system 800 for implementing a method according to an embodiment of the present disclosure is shown. The system 800 comprises a computer processor 810 coupled to a computer readable memory unit 820, the memory unit 820 comprising instructions 822 that implement the method 200, 301, 303 or 305 when executed by the computer processor 810.

[0097] In some embodiments of the present disclosure, a computer-readable medium for managing a camera system is provided. The computer-readable medium has instructions stored thereon, and the instructions, when executed on at least one processor, may cause the at least one processor to perform a method for managing a camera system as described in the previous paragraphs, and the details will be omitted below.

[0098] In general, various embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software, which may be executed by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of the present disclosure are shown and described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof, as non-limiting examples.

[0099] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer readable storage medium. The computer program product includes computer executable instructions such as those included in program modules that are executed in a device on a target real or virtual processor to perform the above-referenced Figure 2 and 3A -3C described process or method. Generally speaking, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The functions of program modules can be combined or separated between program modules as desired in various embodiments. The machine executable instructions of program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0100] The program code for executing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine, partially on a remote machine, or entirely on a remote machine or server.

[0101] The above program code may be contained on a machine-readable medium, which may be any tangible medium that may contain or store a program used by or in conjunction with an instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0102] In addition, although operations are described in a specific order, this should not be understood as requiring the specific order shown or to perform these operations in sequence, or to perform all the operations shown, to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these details should not be interpreted as limitations on the scope of the present disclosure, but should be interpreted as descriptions of features peculiar to a particular embodiment. Certain features described in the context of a separate embodiment may also be implemented in combination in a single embodiment. In other aspects, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination.

[0103] Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method (200) for adjusting a virtual relative position in a virtual robot work cell (40), the method comprising: Obtaining, from a controller (10) for controlling a robot (31) in a robot work cell (30), a relative position between the robot (31) and an object carrier (32) in the robot work cell (30); and In the virtual robot work unit (40) simulating the robot work unit (30), the virtual relative position between the virtual robot (41) corresponding to the robot (31) and the virtual object carrier (42) corresponding to the object carrier (32) is adjusted based on the relative position.

2. The method (200) of claim 1, wherein adjusting the virtual relative position comprises: Determining a base height of the virtual object carrier (42) based on the relative position and a virtual robot height of the virtual robot (41); determining a height compensation based on a cell configuration of the robotic work cell (30); and The height of the virtual object carrier (42) is adjusted based on the base height and the height compensation.

3. The method (200) according to claim 2, wherein adjusting the virtual relative position further comprises: Determining a virtual target horizontal position of the virtual object carrier (42) based on the relative position and the virtual horizontal position of the virtual robot (41); as well as The virtual position of the virtual object carrier (42) is adjusted to the virtual target horizontal position.

4. The method (200) of claim 2 or 3, wherein determining the base height comprises: Obtaining the virtual robot height of the virtual robot (41); determining a relative height based on the relative position; as well as The difference between the virtual robot height and the relative height is determined as the base height.

5. The method (200) of claim 2 or 3, wherein determining the height compensation comprises: Obtaining an original virtual surface height of the surface of the virtual object carrier (42); Acquiring a virtual object height of a virtual object (43) at a point where the virtual object (43) appears on the virtual object carrier (42); as well as The difference between the original virtual surface height and the object virtual height is determined as the height compensation.

6. The method (200) of claim 3, wherein determining the height compensation comprises: In response to determining that the robotic work cell (30) is configured with a camera (34) above a surface of the object carrier (32), the height compensation is determined to be zero.

7. The method (200) of claim 6, further comprising: The position of the virtual camera (44) is adjusted based on the horizontal position of the virtual target.

8. The method (200) of claim 2, wherein determining the height compensation comprises: In response to determining that the robot work cell (30) is configured with a sensor (35) for sensing an object on a surface of the object carrier (32), obtaining a virtual sensor height of a virtual sensor (45) corresponding to the sensor (35) in the virtual robot work cell (40); obtaining a virtual object height of the virtual object at a point where the virtual object appears on the virtual object carrier (42); as well as The height compensation is determined as a difference between the virtual sensor height and the virtual object height.

9. The method (200) of claim 8, further comprising: The position of the virtual sensor (45) is adjusted based on the horizontal position of the virtual target.

10. The method (200) of claim 1, further comprising: In response to determining that the object carrier (32) is linear, obtaining a virtual robot orientation of the virtual robot (41); as well as The virtual orientation of a virtual object carrier (42) is adjusted based on the virtual robot orientation and the relative orientation between the robot (31) and the object carrier (32).

11. The method (200) of claim 1, further comprising: In response to determining that the object carrier (32) is circular, determining the center of the virtual object carrier (42) as the origin of a virtual object carrier coordinate system of the virtual object carrier (42); and The orientation of the virtual object carrier coordinate system is adjusted based on the virtual robot orientation and the relative orientation between the robot (31) and the object carrier (32).

12. The method (200) of claim 1, further comprising: In response to determining that the robot work cell (40) further includes an additional robot, obtaining an additional relative position between the object carrier (32) and the additional robot from an additional controller for controlling the additional robot; and Based on the relative position and the further relative position, a virtual position of a further virtual robot corresponding to the further robot is adjusted in the virtual robot work cell (40).

13. A device for adjusting a virtual relative position in a virtual robot work cell (40), the device comprising: A relative position acquisition unit (710) configured to acquire a relative position between the robot (31) and an object carrier (32) in the robot work cell (30) from a controller (10) for controlling the robot (31) in the robot work cell (30); and A virtual relative position adjustment unit (720) is configured to adjust the virtual relative position between a virtual robot (41) corresponding to the robot (31) and a virtual object carrier (42) corresponding to the object carrier (32) based on the relative position in the virtual robot working unit (40) simulating the robot working unit (30).

14. A system for adjusting a virtual relative position in a virtual robot work cell (40), comprising: A computer processor coupled to a computer readable memory unit, the memory unit comprising instructions which, when executed by the computer processor, implement the method according to any one of claims 1 to 12.

15. A computer readable medium having stored thereon instructions which, when executed on at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 12.

Images