Control method of robot arm, shelf operating device, and storage medium
By acquiring the pose relationship between the shelf label and the target storage location and performing image processing, the pose of the target storage location relative to the picking and placing point of the robotic arm is determined, which solves the problem of low accuracy caused by shelf deformation and realizes precise material handling by the robotic arm on the shelf.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN YOUIBOT ROBOTICS CO LTD
- Filing Date
- 2023-10-31
- Publication Date
- 2026-05-12
AI Technical Summary
In existing technologies, when a robotic arm performs array picking and placing operations on materials on a shelf, the deformation of the shelf leads to low accuracy.
By acquiring the positional relationship between the labels on the shelf and the target storage location, and combining this with the image captured by the robotic arm of the labels, the positional relationship of the target storage location relative to the robotic arm's pick-up and place-down points is determined, thereby controlling the robotic arm to accurately pick up and place materials.
It improves the array retrieval accuracy of the rack operation equipment, ensuring that the robotic arm can accurately pick up and place materials on deformable racks.
Smart Images
Figure CN117602253B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of warehousing technology, and in particular to control methods for robotic arms, rack operating equipment, and storage media. Background Technology
[0002] Currently, to address the technical challenge of robotic arms performing array-based material handling on shelves (such as electronic material racks), mainstream robotic arm manufacturers typically use an application package called "palletizing / depalletizing." This involves using visual labels on the top layer of the electronic material rack to guide users to the top reference pick-and-place position. Based on the deviation data between the reference pick-and-place position and other target pick-and-place positions in the computer-aided design (CAD) model corresponding to the dimensions of the electronic material rack, the coordinates of the target pick-and-place position are calculated, thus enabling the pick-and-place operation. However, shelves are usually machined from sheet metal, making them prone to deformation. Deformation, in particular, can cause discrepancies between the actual deviations between the target pick-and-place positions and those obtained from the CAD model, affecting the accuracy of material handling. Summary of the Invention
[0003] The main objective of this application is to provide a robotic arm control method, a shelf operation device, and a storage medium, aiming to solve the technical problem of low precision when a robotic arm performs array picking and placing operations on materials on a shelf in the prior art.
[0004] In a first aspect, this application provides a control method for a robotic arm, the control method comprising the following steps:
[0005] Obtain the pose relationship between the labels on the shelf and the target storage location on the shelf;
[0006] The image of the label taken by the robotic arm is obtained, and the pose relationship between the label and the robotic arm base is determined based on the image.
[0007] Based on the pose relationship between the tag and the target storage location, and the current pose relationship between the tag and the robotic arm base, the pick-up / placement point pose of the target storage location relative to the robotic arm is determined;
[0008] Based on the target storage location's position relative to the robotic arm's pick-up / placement point, the robotic arm is controlled to move its end effector toward the pick-up / placement point position.
[0009] Secondly, this application also provides a shelf operating device, the shelf operating device comprising:
[0010] The shelving operation device includes a robotic arm, a processor, and a memory. The robotic arm is used for picking and placing materials, and a camera is mounted on it. The processor can be mounted on a moving platform or on the robotic arm, or the shelving operation device may include multiple processors, with at least one processor mounted on the moving platform and at least another processor mounted on the robotic arm. A computer program stored in the memory and executable by the processor, and a data bus for communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the steps of any of the robotic arm control methods provided in this specification.
[0011] Thirdly, this application also provides a storage medium for computer-readable storage, characterized in that the storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the robotic arm control method as provided in any of the embodiments of this application.
[0012] This application provides a robotic arm control method, a shelf operation device, and a storage medium. The robotic arm control method includes: acquiring the pose relationship between a label on a shelf and a target storage location on the shelf; acquiring an image of the label captured by the robotic arm; determining the pose relationship between the label and the robotic arm's base based on the captured image; determining the pick-and-place pose of the target storage location relative to the robotic arm based on the pose relationship between the label and the target storage location, and the current pose relationship between the label and the robotic arm's base; and controlling the robotic arm to move its end effector towards the pick-and-place pose based on the pick-and-place pose of the target storage location relative to the robotic arm's base. This application can determine the pick-and-place pose of the target storage location relative to the robotic arm's base based on the pose relationship between the label and the robotic arm's base, thereby completing the pick-and-place operation of the robotic arm on the target object and improving the array retrieval accuracy of the shelf operation device. Attached Figure Description
[0013] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0014] Figure 1 This is a flowchart illustrating a control method for a robotic arm provided in an embodiment of this application;
[0015] Figure 2 This is a flowchart illustrating the pose relationship between a label on a shelf and a target storage location on the shelf, as described in one embodiment.
[0016] Figure 3 This is a pose diagram of the various coordinate systems used in a robotic arm according to an embodiment of this application;
[0017] Figure 4 This is a flowchart illustrating the process of determining the position of the target storage location relative to the pick-up / place-down point of the robotic arm in one implementation method.
[0018] Figure 5 This is a schematic block diagram of the structure of a shelf operation device provided in an embodiment of this application. Detailed Implementation
[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0020] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.
[0021] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0022] This application provides a robotic arm control method, a shelf operating device, and a storage medium. The robotic arm control method can be applied to a terminal, which can be the shelf operating device.
[0023] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0024] Please see Figure 1 , Figure 1 This is a flowchart illustrating a robotic arm control method provided in an embodiment of this application. It should be noted that the robotic arm control method provided in this embodiment can be used in rack handling equipment. The rack handling equipment includes at least a robotic arm, a processor, and a memory, and may also include a mobile platform. The robotic arm can be a robotic arm used in wafer handling projects.
[0025] like Figure 1 As shown, the control method of the robotic arm includes steps S100 to S400.
[0026] Step S100: Obtain the pose relationship between the label on the shelf and the target storage location on the shelf.
[0027] In some embodiments, the shelf includes multiple storage locations, and the target storage location is any one of the multiple storage locations; obtaining the pose relationship between a label on the shelf and the target storage location on the shelf includes: obtaining the pose relationship between the label and the target storage location from a preset pose relationship between the label and each of the multiple storage locations.
[0028] In some embodiments, the shelf is provided with multiple labels, each label corresponding to multiple storage locations, and at least two labels are located in different storage locations; alternatively, the shelf may have only one label. The aforementioned acquisition of the pose relationship between the labels on the shelf and the target storage location on the shelf includes: acquiring the pose relationship between the label on the target storage location and the target storage location on the shelf. By acquiring the pose relationship between the label on the target storage location and the target storage location on the shelf, the pose information of the target storage location relative to the label on the target storage location can be obtained quickly and accurately.
[0029] During the project deployment phase, one or more tags can be bound to each of the multiple storage locations on a shelf to determine the pose relationship between the tag and each storage location, thus preparing for obtaining the pose relationship between the tag and the target storage location. Please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a flowchart of one embodiment for determining the pose relationship between the tag and the target storage location, including:
[0030] S101: Obtain the first pose relationship between the tag and the robotic arm base.
[0031] For example, the robotic arm base and the tag exist in the same space but different coordinate systems, and they do not have a clear pose relationship. Using the robotic arm as a medium, the camera on the robotic arm takes a picture of the tag, and an image of the tag can be obtained. Based on the intrinsic parameters calibrated by the camera, the hand-eye calibration parameters of the robotic arm, the image of the tag, and the rigid body kinematics model, the pose relationship between the tag and the robotic arm base can be obtained, denoted as...
[0032] Wherein, the coordinate system where the label is located and the coordinate system where the robotic arm base is located are the label coordinate system and the robotic arm base coordinate system, respectively. For example... Figure 3As shown, the robotic arm base coordinate system describes the coordinate system of the base of the robotic arm mounted on the top of the mobile platform in the shelf operation equipment. This coordinate system will also move as the mobile platform moves, and is represented by Base. The label coordinate system is a coordinate system established on the visual label. The label is affixed to the static shelf surface, so the relative pose relationship between the label coordinate system and each storage location remains fixed, and is represented by mark.
[0033] S102: When the end of the robotic arm is located in the target pose of the target storage location, the second pose relationship between the end of the robotic arm and the robotic arm base is obtained.
[0034] The robotic arm is controlled so that its end effector is located in the target pose of the target storage location. Based on the pose of the robotic arm base, a second pose relationship between the end effector of the robotic arm and the robotic arm base is determined.
[0035] Optionally, the end effector of the robotic arm is controlled to pick up materials. When the end effector is in the target pose of the target storage location, the material is placed on the target storage location sto1, and the second pose relationship between the end effector and the base of the robotic arm is recorded at this time, denoted as .
[0036] The coordinate systems of the robotic arm end effector and the storage location sto1 are the robotic arm end effector coordinate system and the storage location coordinate system, respectively. For example... Figure 3 As shown, the end effector coordinate system of the robotic arm is the coordinate system of the end flange of the robotic arm, which is the working center of the tool installed at the end of the robotic arm flange. The origin and direction of this coordinate system can be set manually and are represented by tcp. The storage position coordinate system is the coordinate system established on the storage position and is represented by sto. When the robotic arm picks up material to the storage position sto1, the specific meaning of the pose of the end effector of the robotic arm is the pose of the storage position sto1. At this time, the end effector coordinate system of the robotic arm coincides with the storage position coordinate system.
[0037] In some embodiments, after the robotic arm captures an image of the label, since the material is exactly within the grasping range of the robotic arm in its first pose, the robotic arm directly grasps the object based on the first pose and places it on the target storage location. Therefore, the second pose relationship between the end effector of the robotic arm and the robotic arm base can be: obtaining the first pose relationship between the label and the robotic arm base when the robotic arm is in its first pose; and the second pose relationship between the end effector of the robotic arm and the robotic arm base when the robotic arm is in its first pose and the end effector of the robotic arm is located in the target pose of the target storage location.
[0038] For example, when controlling a robotic arm to pick up materials, the robotic arm is controlled so that it is in the first pose, and the end effector of the robotic arm is in the target pose of the target storage location. The pose of the robotic arm base is Base1. At this time, the second pose relationship between the end effector of the robotic arm and the robotic arm base can be denoted as...
[0039] S103: Determine the third pose relationship between the target storage location and the robotic arm base based on the second pose relationship between the end of the robotic arm and the robotic arm base.
[0040] Optionally, when determining the third pose relationship between the target storage location and the robotic arm base based on the second pose relationship between the end effector of the robotic arm and the robotic arm base, since the end effector of the robotic arm is located at the target pose of the target storage location, i.e., the end effector of the robotic arm and the target storage location are in an overlapping state, when material is picked up and placed in the target storage location sto1, the specific meaning of the end effector of the robotic arm is the target storage location sto1. The pose relationship between the end effector of the robotic arm and the robotic arm base is the pose relationship between the target storage location and the robotic arm base, and the second pose relationship between the end effector of the robotic arm and the robotic arm base is the third pose relationship between the target storage location and the robotic arm base, i.e., the third pose relationship is...
[0041] S104: Determine the pose relationship between the tag and the target storage location based on the first pose relationship and the third pose relationship.
[0042] For example, after controlling the robotic arm to take pictures and locate the labels affixed to the shelf, in the array retrieval scenario where the robotic arm retrieves the materials at storage location sto1, the pose relationship chain of each element in the array retrieval scenario is sorted out, and the corresponding coordinate system of the pose of multiple elements is established. Then, based on the rigid body motion model, the first pose relationship, the second pose relationship, and the pose relationship between the label and the target storage location are established.
[0043] In some embodiments, the end effector of the robotic arm is located at the target pose of the target storage location. When controlling the robotic arm to pick up materials, the pose relationship between the first pose relationship, the third pose relationship, and the pose relationship between the label and the target storage location can be:
[0044]
[0045] Formula 1 can be transformed using matrix operations:
[0046]
[0047] This allows us to determine the pose relationship between the tag and the target storage location. for:
[0048]
[0049] Based on this, the pose transformation relationship of different storage locations stoi (where i is the storage location number on the shelf) can be obtained, and the equation for calculating the pose relationship between different storage locations and the label can be obtained:
[0050]
[0051] For example, the pose relationship between the label and one of the multiple storage locations in the shelf can be obtained according to Formula 4. This pose relationship is obtained by independently binding the label to one of the multiple storage locations in the shelf. This pose relationship is independent of the pose relationships between the label and the other storage locations in the multiple storage locations in the shelf and does not interfere with each other.
[0052] In some embodiments, the pose relationship includes six degrees of freedom, including both translational and rotational information. The six degrees of freedom include translational degrees of freedom along the three Cartesian coordinate axes and rotational degrees of freedom about the three Cartesian coordinate axes.
[0053] The pose relationship obtained in step S104 includes the pose relationship between a label and a target storage location on the shelf. By taking a storage location on the shelf as the target storage location each time, and determining the pose relationship between the label and the target storage location through steps S101 to S104, the pose relationship between the label and each of the multiple storage locations can be obtained by executing the steps multiple times.
[0054] For example, the pose relationship is stored in a storage medium in the form of a configuration file. When the robotic arm control method reaches step S100, the configuration file is called to use the obtained pose relationship between the tag and the target storage location on the shelf as known information, and then participate in the subsequent steps of the method.
[0055] S200: Obtain an image of the label taken by the robotic arm, and determine the current pose relationship between the label and the robotic arm base based on the captured image.
[0056] For example, taking the robotic arm as an example, the robotic arm is equipped with a camera, which can be used for image capture, positioning and recognition, etc.; the shelf is affixed with labels, which are made using QR codes of a specific shape and size, and these labels can serve as guide labels for material handling. By controlling the camera on the robotic arm, the labels on the shelf can be photographed to obtain images of the labels, which are then used as guide labels.
[0057] In some embodiments, the label may be affixed to a corresponding area of a storage location on the top shelf of the shelf, or to a corresponding area of a storage location on another shelf.
[0058] For example, in order to obtain the pose relationship between the label and the robot arm base, a robot arm base coordinate system is established with the robot arm base and a label coordinate system is established with the label. The pose relationship between the robot arm base coordinate system and the label coordinate system can be determined.
[0059] In some embodiments, the camera has been calibrated with internal parameters and hand-eye calibration before acquiring the image of the tag taken by the robotic arm.
[0060] Based on the image of the tag captured by the camera, the pose relationship between the robot arm base coordinate system and the tag coordinate system, and combined with the camera intrinsic parameter calibration and hand-eye calibration data, the pose relationship between the tag and the robot arm base can be calculated according to the rigid body kinematics model.
[0061] For example, an object has six degrees of freedom in space, namely, the degree of freedom of movement along the three rectangular coordinate axes x, y, and z of the spatial coordinate system in which the object is located, and the degree of freedom of rotation about these three coordinate axes. The calculated pose relationship between the label and the robotic arm base has the above six degrees of freedom. This pose relationship includes position information and attitude information, wherein the position information includes translation data information, and the attitude information includes rotation data information.
[0062] S300. Based on the pose relationship between the tag and the target storage location, and the current pose relationship between the tag and the robotic arm base, determine the pick-up / placement point pose of the target storage location relative to the robotic arm base.
[0063] For example, the robotic arm can only be controlled to pick up and place materials after the pose of the target storage location relative to the pick-up and place-down point of the robotic arm is known.
[0064] Please see Figure 4 In some embodiments, Figure 4 The specific steps for determining the orientation of the target storage location relative to the pick-up and put-down point of the robotic arm are given, including steps S301 to S302.
[0065] S301. Determine the pose relationship between the robotic arm base and the target storage location based on the pose relationship between the tag and the target storage location, and the current pose relationship between the tag and the robotic arm base.
[0066] In some embodiments, determining the pose relationship between the robotic arm base and the target storage location can specifically be as follows:
[0067] 1) Based on the rigid body motion model, establish the equations for solving the pose relationship between the tag and the target storage location during the process from taking a picture to picking it up.
[0068] According to Formula 1, the relationship equation for the pose of the robotic arm during the process of taking a picture of the tag and retrieving the target storage location is as follows:
[0069]
[0070] 2) Substitute the pose relationship between the current tag and the target storage location stox, and the pose relationship between the current tag and the robot arm base into Formula 5 to obtain the pick-up / placement point pose of the target storage location stox relative to the robot arm base.
[0071] For example, based on the pose relationship between the tag and each of the multiple storage locations on the shelf obtained in step S100, the pose relationship between the current tag and the target storage location can be obtained. Simultaneously, the pose relationship between the tag obtained in step 200 and the robotic arm base is established. The known conditions can be substituted into Formula 5, and the pose relationship between the robotic arm base and the target storage location can be obtained by using the matrix multiplication rule.
[0072] S302. Based on the pose relationship between the robotic arm base and the target storage location, determine the pick-up / placement point pose of the target storage location relative to the robotic arm base.
[0073] In some embodiments, when the end of the robotic arm is located at the target pose of the target storage location, a coordinate system of the robotic arm base is established with the pose of the robotic arm base as the origin. Based on the pose relationship between the robotic arm base and the target storage location, the pick-up and put-down point pose of the target storage location relative to the robotic arm can be determined.
[0074] S400: Based on the pick-up / placement point pose of the target storage location relative to the robotic arm base, control the robotic arm to move its end effector toward the pick-up / placement point pose.
[0075] For example, after obtaining the pick-up / placement point pose of the target storage location relative to the robotic arm, the robotic arm controls itself to move its end effector toward the pick-up / placement point pose. Once the end effector reaches the pick-up / placement point pose, a pick-up or place-down operation is determined based on the current control parameters of the robotic arm.
[0076] This application provides a robotic arm control method, including acquiring the pose relationship between a label on a shelf and a target storage location on the shelf; acquiring an image of the label captured by the robotic arm; determining the pose relationship between the label and the robotic arm's base based on the captured image; determining the pick-and-place point pose of the target storage location relative to the robotic arm based on the pose relationship between the label and the target storage location, and the current pose relationship between the label and the robotic arm base; and controlling the robotic arm to move its end effector towards the pick-and-place point pose based on the pick-and-place point pose of the target storage location relative to the robotic arm. Compared to existing technologies that use the coordinates of the target pick-and-place location obtained from a reference pick-and-place location to pick up and place materials, this application controls the robotic arm to perform material pick-and-place operations based on the pick-and-place point pose of the target storage location relative to the robotic arm. Even when operating on shelves prone to deformation, the robotic arm can accurately pick up and place materials, effectively improving the array picking accuracy of shelf operation equipment.
[0077] Please see Figure 5 , Figure 5 This is a schematic block diagram of the structure of a shelf operation device provided in an embodiment of this application.
[0078] like Figure 5 As shown, the shelf operation equipment includes at least a robotic arm, a processor, and a memory.
[0079] In some embodiments, the racking operation equipment may also include a mobile platform, wherein the robotic arm may be mounted on the top or side wall of the mobile platform.
[0080] The robotic arm is equipped with a camera, which is used to photograph the label.
[0081] The processor can be mounted on a mobile platform or on a robotic arm, or the shelf operation device may include multiple processors, with at least one processor mounted on the mobile platform and at least another processor mounted on the robotic arm.
[0082] The memory and processor can be connected via a system bus. The memory may include storage media and internal memory.
[0083] The storage medium can store the operating system and computer programs. When the computer program is executed, it enables the processor to perform any control method for the robotic arm.
[0084] The processor provides computing and control capabilities to support the operation of the entire rack-mounted equipment.
[0085] The internal memory provides an environment for the execution of computer programs stored in the storage medium. When the computer program is executed by the processor, it enables the processor to execute any control method of the robotic arm.
[0086] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the rack operating equipment to which the present application is applied. Specific rack operating equipment may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0087] It should be understood that a processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other convertible logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.
[0088] In one embodiment, the processor is configured to execute a computer program and, when executing the computer program, perform the following steps:
[0089] Obtain the pose relationship between the labels on the shelf and the target storage location on the shelf;
[0090] The image of the label taken by the robotic arm is obtained, and the pose relationship between the label and the robotic arm base is determined based on the image.
[0091] Based on the pose relationship between the tag and the target storage location, and the current pose relationship between the tag and the robotic arm base, the pick-up / placement point pose of the target storage location relative to the robotic arm is determined;
[0092] Based on the target storage location's position relative to the robotic arm's pick-up / placement point, the robotic arm is controlled to move its end effector toward the pick-up / placement point position.
[0093] In one embodiment, when obtaining the pose relationship between the tag and the target storage location on the shelf, the processor is further configured to:
[0094] Obtain the first pose relationship between the tag and the robot arm base of the robot arm;
[0095] When the end effector of the robotic arm is located at the target storage location, the second pose relationship between the end effector of the robotic arm and the base of the robotic arm is obtained;
[0096] Based on the second pose relationship between the end of the robotic arm and the robotic arm base, the third pose relationship between the target storage location and the robotic arm base is determined;
[0097] Based on the first pose relationship and the third pose relationship, the pose relationship between the tag and the target storage location is determined.
[0098] In one embodiment, when obtaining the first pose relationship between the tag and the robotic arm's base, the processor is further configured to:
[0099] The process of obtaining the relationship between the tag and the robot arm base when the robot arm is in the first pose; and obtaining the relationship between the end of the robot arm and the robot arm base when the end of the robot arm is in the target pose of the target storage location, includes: obtaining the relationship between the end of the robot arm and the robot arm base when the robot arm is in the first pose and the end of the robot arm is in the target pose of the target storage location.
[0100] In one embodiment, when obtaining the first pose relationship between the tag and the robotic arm's base, the processor is further configured to:
[0101] The process of obtaining the first pose relationship between the tag and the robotic arm base when the robotic arm is in a first pose; and obtaining the second pose relationship between the end effector of the robotic arm and the robotic arm base when the end effector of the robotic arm is in a target pose at the target storage location, includes: obtaining the pose relationship between the end effector of the robotic arm and the robotic arm base corresponding to the second pose when the robotic arm is in a second pose and the end effector of the robotic arm is in a target pose at the target storage location; and determining the second pose relationship between the end effector of the robotic arm and the robotic arm base corresponding to the first pose based on the second pose, the first pose, and the pose relationship between the end effector of the robotic arm and the robotic arm base corresponding to the second pose.
[0102] In one embodiment, when determining the pose relationship between the tag and the target storage location based on the first pose relationship and the third pose relationship, the processor is further configured to:
[0103] Based on a preset pose transformation formula, the pose relationship between the tag and the target storage location is determined according to the first pose relationship and the third pose relationship.
[0104] In one embodiment, when determining the pick-up / placement point pose of the target storage location relative to the robotic arm based on the pose relationship between the tag and the target storage location, and the current pose relationship between the tag and the robotic arm base, the processor is further configured to:
[0105] Based on the pose relationship between the tag and the target storage location, and the current pose relationship between the tag and the robotic arm base, the pose relationship between the robotic arm base and the target storage location is determined.
[0106] Based on the pose relationship between the robotic arm base and the target storage location, the pick-up and put-down point pose of the target storage location relative to the robotic arm is determined.
[0107] The specific principles and implementation methods of the shelf operation equipment provided in this application embodiment are similar to those of the methods in the foregoing embodiments, and will not be repeated here.
[0108] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method implemented can be referred to in various embodiments of the robotic arm control method of this application.
[0109] The computer-readable storage medium can be an internal storage unit of the shelf operation equipment described in the foregoing embodiments, such as the hard drive or memory of the near-eye display device. Alternatively, the computer-readable storage medium can be an external storage device of the near-eye display device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the near-eye display device.
[0110] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0111] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0112] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A control method for a robotic arm, characterized in that, The method includes: Obtain the first pose relationship between the label on the shelf and the robotic arm's base; When the end effector of the robotic arm is located at the target storage location on the shelf, the second pose relationship between the end effector of the robotic arm and the robotic arm base is obtained. Based on the second pose relationship between the end of the robotic arm and the robotic arm base, the third pose relationship between the target storage location and the robotic arm base is determined; Based on a preset pose transformation formula, the pose relationship between the tag and the target storage location is determined according to the first pose relationship and the third pose relationship. The image of the label taken by the robotic arm is obtained, and the pose relationship between the label and the robotic arm base is determined based on the image. Based on the pose relationship between the tag and the target storage location, and the current pose relationship between the tag and the robotic arm base, determine the pick-up / placement point pose of the target storage location relative to the robotic arm base; Based on the pick-up / placement point pose of the target storage location relative to the robotic arm base, control the robotic arm to move its end effector toward the pick-up / placement point pose.
2. The control method for the robotic arm according to claim 1, characterized in that, The step of obtaining the first pose relationship between the tag and the robot arm base of the robot arm includes: obtaining the first pose relationship between the tag and the robot arm base of the robot arm when the robot arm is in the first pose; When obtaining the target pose of the end of the robotic arm located at the target storage location, the second pose relationship between the end of the robotic arm and the robotic arm base includes: obtaining the second pose relationship between the end of the robotic arm and the robotic arm base when the robotic arm is in the first pose and the end of the robotic arm is located at the target storage location.
3. The control method for the robotic arm according to claim 1, characterized in that, The shelf is provided with multiple labels, and at least two of the labels are located in different storage locations; obtaining the positional relationship between the labels on the shelf and the target storage location on the shelf includes: Obtain the positional relationship between the label on the target storage location and the target storage location on the shelf.
4. The control method for the robotic arm according to any one of claims 1-3, characterized in that, The shelf includes multiple storage locations, and the target storage location is any one of the multiple storage locations; Obtaining the pose relationship between the labels on the shelf and the target storage location on the shelf includes: The pose relationship between the tag and the target storage location is obtained from the preset pose relationship between the tag and each of the plurality of storage locations.
5. The control method for the robotic arm according to any one of claims 1-3, characterized in that, Based on the pose relationship between the tag and the target storage location, and the current pose relationship between the tag and the robotic arm base, the pose of the pick-up / placement point of the target storage location relative to the robotic arm base is determined, including: Based on the pose relationship between the tag and the target storage location, and the current pose relationship between the tag and the robotic arm base, the pose relationship between the robotic arm base and the target storage location is determined. Based on the pose relationship between the robotic arm base and the target storage location, the pick-up and drop-off point pose of the target storage location relative to the robotic arm base is determined.
6. The control method for the robotic arm according to any one of claims 1-3, characterized in that, The pose relationship includes translational degrees of freedom and rotational degrees of freedom.
7. A rack handling device, characterized in that, The shelf operation equipment includes a robotic arm, a memory, and a processor; The robotic arm is used for picking up and placing materials, and a camera is installed on the robotic arm; The memory is used to store computer programs; The processor is used to execute the computer program and, in executing the computer program, implement the control method of the robotic arm as described in any one of claims 1 to 6.
8. A computer-readable storage medium, characterized in that, When the computer-readable storage medium is executed by one or more processors, the one or more processors perform the steps of the control method for the robotic arm as described in any one of claims 1 to 6.