Automatic control method and device for gripping control of clamp, electronic equipment and medium

Abstract

The application discloses an automatic control method and device of a clamp, electronic equipment and a medium. The automatic control method of the clamp comprises: acquiring image data comprising at least one to-be-grabbed article; processing the image data to obtain an orientation feature of the to-be-grabbed article related to an orientation; and controlling the clamp to perform grabbing of the at least one to-be-grabbed article based on at least the orientation feature of the to-be-grabbed article. The grabbing scheme of the application considers the orientation feature of the to-be-grabbed article when controlling the clamp to perform grabbing, and can more accurately determine the difficulty of grabbing articles with different orientations, and reduce the possibility of grabbing failure, compared with existing schemes. In particular, in an industrial scene where a large number of articles are scattered and stacked, the existing scheme does not consider the influence of the orientation feature of the article on grabbing, resulting in poor robot operation effect in such a scene, while the application can greatly improve the robot grabbing effect in such a scene.

Description

Technical Field

[0001] This application relates to the fields of automatic control and program control of robotic arms or grippers (B25J), and more specifically, to automatic control methods, devices, electronic equipment, and media for grippers. Background Technology

[0002] Currently, robots are increasingly being used in shopping malls, supermarkets, and logistics to replace manual labor in tasks such as sorting, handling, and placing goods. However, traditional robots are limited by their ability to operate in a predetermined manner or with limited intelligence, resulting in stringent requirements for the location and placement of items in these scenarios. For example, in a supermarket sorting task, the requirement is to remove items placed in a material crate and move them to a designated location. In this task, the robot visually identifies the location of each item in the material crate and removes it to the designated position. To ensure the robot can successfully grasp each item, existing solutions require workers to first neatly arrange the items in the material crate, and each item in the crate needs to be placed in a specific posture. For example, canned drinks, boxed food, and bagged food all need to be placed with their openings facing upwards. Then, the material crate containing a large number of neatly arranged items is transported to the robot's working area, where the robot performs the grasping operation.

[0003] In scenarios requiring the sorting and gripping of large quantities of items to designated locations, traditional solutions, after identifying all items, typically determine the gripping order based on the items' height or the size of the gripper configured to grip each item. Based on this determined order, the gripper is then controlled to grip the items. Some solutions also consider factors such as whether items are stacked when determining the gripping order. However, in the aforementioned gripping scenario, if a large number of items are not arranged neatly but are piled together in a disorderly and scattered manner, existing gripping solutions may unexpectedly push items over, carry them away, or even fail to grip them successfully. This is especially true when the items to be gripped are located next to the material frame wall or other particularly tall objects or obstacles. These obstacles hinder the movement of the gripper and the gripping process, and existing gripping control solutions do not consider such operational scenarios, resulting in poor gripping performance. Therefore, a gripping control scheme with a high success rate is needed to solve various problems that may arise in such scenarios where densely packed items are scattered and grippers are used to pick them up. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to overcome or at least partially solve the above problems. Specifically, the gripping scheme of the present invention takes into account the orientation characteristics of the object to be gripped when controlling the gripper to grip. Compared with existing schemes, it can more accurately determine the difficulty of gripping objects with different orientations, reducing the possibility of gripping failure. Especially when performing gripping in industrial scenarios where a large number of objects are scattered and piled up, existing schemes do not consider the influence of the orientation characteristics of the objects on gripping, resulting in poor robot operation in such scenarios. The present invention can greatly improve the robot gripping effect in such scenarios.

[0005] All solutions disclosed in the claims and specification of this application possess one or more of the aforementioned innovative features, and accordingly, are able to solve one or more of the aforementioned technical problems. Specifically, this application provides an automatic control method, apparatus, electronic device, and storage medium for a clamp.

[0006] The automatic control method for the fixture according to the embodiments of this application includes:

[0007] Acquire image data including at least one item to be grabbed;

[0008] The image data is processed to obtain orientation features related to the orientation of the object to be grasped;

[0009] The gripper is controlled to perform gripping of at least one item based at least on the orientation characteristics of the item to be gripped.

[0010] In some implementations, the at least one object to be grasped includes a graspable area of ​​the at least one object to be grasped.

[0011] In some implementations, controlling the gripper to perform the gripping of at least one item to be gripped includes determining a gripping order of at least one item to be gripped and controlling the gripper to perform the gripping of at least one item to be gripped in the gripping order.

[0012] In some implementations, the image data is processed to obtain positional features of at least one object to be grasped, and the gripper is controlled to perform grasping of at least one object to be grasped, based at least on the orientation and positional features of the object to be grasped.

[0013] In some implementations, the orientation feature is obtained based on the rotation matrix of the object to be grasped.

[0014] In some implementations, the reference orientation of the rotation matrix is ​​the orientation of the graspable area of ​​the object to be grasped when it is perpendicular to the Z-axis.

[0015] In some implementations, the rotation matrix is ​​an Euler angle-based rotation matrix.

[0016] The automatic control device for the clamp according to the embodiments of this application includes:

[0017] An image data acquisition module is used to acquire image data including at least one item to be grabbed.

[0018] An orientation feature acquisition module is used to process the image data to obtain orientation features related to the orientation of the object to be grasped.

[0019] A gripping control module is used to control a gripper to perform gripping of at least one item based at least on the orientation characteristics of the item to be gripped.

[0020] In some implementations, the at least one object to be grasped includes a graspable area of ​​the at least one object to be grasped.

[0021] In some implementations, the gripping control module is specifically used to determine the gripping order of at least one item to be gripped, and to control the gripper to perform gripping of at least one item to be gripped in the gripping order.

[0022] In some embodiments, a location feature acquisition module is used to process the image data to obtain location-related features of at least one object to be grasped; the grasping control module is used to control the gripper to perform grasping of at least one object to be grasped based at least on the orientation features and location features of the object to be grasped.

[0023] In some implementations, the orientation feature is obtained based on the rotation matrix of the object to be grasped.

[0024] In some implementations, the reference orientation of the rotation matrix is ​​the orientation of the graspable area of ​​the object to be grasped when it is perpendicular to the Z-axis.

[0025] In some implementations, the rotation matrix is ​​an Euler angle-based rotation matrix.

[0026] The electronic device of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the automatic control method of the fixture of any of the above embodiments.

[0027] The computer-readable storage medium of the embodiments of this application stores a computer program thereon, which, when executed by a processor, implements the automatic control method of the fixture of any of the above embodiments.

[0028] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0029] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0030] Figure 1 This is a schematic diagram illustrating the relationship between the items to be picked up and the material basket in the item sorting scenario of this application;

[0031] Figure 2 This is a flowchart illustrating a grasping control method based on the orientation of an object according to certain embodiments of this application;

[0032] Figure 3 This is a schematic diagram of the material frame parameters in some embodiments of this application;

[0033] Figure 4 This is a schematic diagram of the pitch axis, roll axis, and yaw axis related to the rotation matrix;

[0034] Figure 5 This is a schematic diagram of the structure of a gripping control device based on the orientation of an item according to certain embodiments of this application;

[0035] Figure 6 This is a schematic diagram of the structure of an electronic device according to certain embodiments of this application. Detailed Implementation

[0036] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0037] In the description of the specific embodiments, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0038] Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0039] This invention can be applied to vision-based industrial robot control scenarios. A typical vision-based industrial robot control scenario includes devices for acquiring images, hardware and control devices such as production line PLCs for the production line, robot components for performing tasks, and operating systems or software for controlling these devices. The devices for acquiring images can include 2D or 3D intelligent / non-intelligent industrial cameras, which, depending on different functions and application scenarios, can include area scan cameras, line scan cameras, monochrome cameras, color cameras, CCD cameras, CMOS cameras, analog cameras, digital cameras, visible light cameras, infrared cameras, ultraviolet cameras, etc.; the production line can include packaging production lines, sorting production lines, logistics production lines, processing production lines, etc., that require the use of robots; the robot components used in the industrial scenario for performing tasks can be biomimetic robots, such as humanoid robots or dog-like robots, or traditional industrial robots, such as robotic arms; the industrial robots can be manipulating robots, programmable robots, teach-and-playback robots, numerically controlled robots, sensory control robots, adaptive control robots, learning control robots, or intelligent robots. Robotic arms can be categorized by their working principle into ball-mounted robotic arms, multi-joint robotic arms, Cartesian coordinate robotic arms, cylindrical coordinate robotic arms, polar coordinate robotic arms, etc. Based on their function, they can be used as gripping robotic arms, palletizing robotic arms, welding robotic arms, or industrial robotic arms. The end effector of a robotic arm can be equipped with an end effector, which, depending on the task requirements, can be a robot gripper, robot gripper, robot tool quick-change device, robot collision sensor, robot rotary connector, robot pressure tool, compliant device, robot spray gun, robot deburring tool, robot arc welding torch, robot electric welding torch, etc. Robotic grippers can be various general-purpose grippers. General-purpose grippers refer to grippers with standardized structures and a wide range of applications, such as three-jaw and four-jaw chucks for lathes, and flat-jaw vises and indexing heads for milling machines. For example, based on the clamping power source used, clamps can be classified into manual clamping clamps, pneumatic clamping clamps, hydraulic clamping clamps, pneumatic-hydraulic linkage clamping clamps, electromagnetic clamps, vacuum clamps, etc., or other biomimetic devices capable of picking up objects. Devices used for image acquisition, hardware and control devices such as PLCs used in production lines, robot components used for performing tasks, and operating systems or software used to control these devices can communicate based on TCP, HTTP, and GRPC (Google Remote Procedure Call Protocol) protocols to transmit various control instructions or commands.The operating system or software can be installed in any electronic device, such as industrial computers, personal computers, laptops, tablets, and mobile phones. These devices can communicate with other devices or systems via wired or wireless means. Furthermore, the term "grabbing" in this invention refers to any grasping action that can control an object to change its position, not just the narrow definition of "grabbing." In other words, grasping actions such as sucking, lifting, and clamping are also within the scope of this invention. The objects to be grasped in this invention can be cardboard boxes, cartons, soft plastic packaging (including but not limited to snack packaging, Tetra Pak milk cartons, plastic milk packaging, etc.), cosmetic bottles, cosmetic products, and / or irregularly shaped toys, etc. These objects can be placed on the ground, on a pallet, on a conveyor belt, and / or in a material basket.

[0040] The inventors discovered that in scenarios where a large number of items are densely packed and scattered, such as when multiple items to be grabbed are piled up in a material box, existing solutions become less effective because each item has a unique orientation, such as... Figure 1 As shown, suppose multiple items need to be grasped from a deep material box. Some of these items have their graspable areas facing the box opening, while others have their graspable areas facing the box wall. Obviously, items facing the box opening are easier to grasp, especially when an item is near the box edge and its graspable area faces the adjacent box wall; in such cases, the gripper might not even be able to grasp the item. Existing solutions, when determining the grasping order, only consider the impact of height, stacking, and suction cup size on the ease of grasping, neglecting the item's pose, or more specifically, the impact of the item's orientation. Therefore, the grasping effect is poor in scenarios with densely packed, haphazardly stacked items, because in such scenarios, orientation significantly affects grasping. Therefore, the inventors discovered that the solution to this technical problem is to control the grasping based at least on the orientation characteristics of the items.

[0041] Figure 2 A flowchart illustrating a method for controlling the grasping of an object based on the pose and orientation of the object to be grasped, according to an embodiment of the present invention, is shown. Figure 2 As shown, the method includes:

[0042] Step S100: Obtain image data including at least one item to be grabbed;

[0043] Step S110: Process the image data to obtain orientation features related to the orientation of the object to be grasped;

[0044] Step S120: Control the gripper to perform gripping of at least one item based at least on the orientation characteristics of the item to be gripped.

[0045] For step S100, this embodiment does not limit the type of image data or the acquisition method. As an example, the acquired image data may include point clouds or RGB color images. Point cloud information can be acquired through a 3D industrial camera. A 3D industrial camera is generally equipped with two lenses, which capture the group of objects to be grasped from different angles. After processing, a three-dimensional image of the object can be displayed. The group of objects to be grasped is placed below the vision sensor, and the two lenses capture images simultaneously. Based on the relative pose parameters of the two images, a general binocular stereo vision algorithm is used to calculate the X, Y, and Z coordinate values ​​of each point on the glass to be coated, as well as the coordinate orientation of each point, and then converts it into point cloud data of the group of objects to be grasped. In specific implementations, laser detectors, visible light detectors such as LEDs, infrared detectors, and radar detectors can also be used to generate point clouds. This invention does not limit the specific implementation method.

[0046] The point cloud data obtained through the above methods is three-dimensional data. In order to filter out the data corresponding to the dimensions that have little impact on grasping, reduce the amount of data processing, and thus accelerate the data processing speed and improve efficiency, the obtained three-dimensional point cloud data of the object to be grasped can be orthographically projected onto a two-dimensional plane.

[0047] As an example, a depth map corresponding to the orthographic projection can also be generated. A two-dimensional color image corresponding to the 3D object region and a depth map corresponding to the two-dimensional color image can be obtained along a direction perpendicular to the depth of the object. The two-dimensional color image corresponds to the image of a planar region perpendicular to the preset depth direction; each pixel in the depth map corresponding to the two-dimensional color image corresponds one-to-one with each pixel in the two-dimensional color image, and the value of each pixel is its depth value.

[0048] Items to be grasped are typically stacked in boxes and transported to the site. These boxes are commonly called material frames. During the grasping process, the robotic arm or gripper may encounter the material frame during its movement. Therefore, the material frame and the placement of the items within it have a significant impact on the grasping operation. As a preferred embodiment, various parameters of the material frame can be obtained. For example... Figure 3 As shown, the material frame data can be processed to extract or generate auxiliary parameters that affect the gripping process. These parameters include the height, width, and length of the material frame, as well as a grid obtained by dividing the width and length of the material frame. It should be understood that the height, width, and length are fixed values, while the grid division method and number are determined by those skilled in the art based on the actual situation, such as the fixture used, the gripping method, and the characteristics of the item to be gripped. Using a grid facilitates the calibration of the position of the item to be gripped. The material frame data can be preset or acquired through a camera.

[0049] It should be understood that, during actual gripping, the clamp needs to perform gripping within the grippable area of ​​the item; non-gripable areas have no substantial effect on gripping. Therefore, the item in this invention can also be the grippable area of ​​the item. The grippable area of ​​an item refers to the part of the item's surface that can be gripped by the clamp. In industrial scenarios, items to be gripped may be placed in a neat and orderly manner, in which case the grippable area of ​​each item is basically the same, and the method for determining the grippable area is relatively simple; or they may be piled together in a chaotic and disorderly manner, in which case the grippable area of ​​each item is random, requiring a complex method to determine the grippable area. This embodiment does not limit the specific application scenario or the specific method for determining the grippable area, as long as the grippable area can be obtained.

[0050] The image, pose, rotation matrix, orientation, and position of the object in this invention can all be the image, pose, rotation matrix, orientation, and position of the graspable area of ​​the object. The following descriptions will not specifically focus on the "grasping area of ​​the object," and those skilled in the art will understand which "objects" appearing in the embodiments of this invention can be replaced with "grasping area of ​​the object."

[0051] For step S110, as Figure 1 As shown, when an object is facing directly upwards, it is easiest for the clamp to grasp it. The more the object's orientation deviates from the XY plane, the more difficult it is to grasp. The object's orientation characteristic reflects the degree to which its orientation deviates from the XY plane. Features reflecting the object's direction or rotation can be used as orientation characteristics, such as angles or specific projection values, etc., which are not limited in this embodiment. As a preferred implementation, the object's orientation characteristic can be obtained based on the object's rotation matrix. When an object with a specific orientation undergoes a certain rotation, it will transform into another specific orientation. The rotation matrix is ​​used to express how the object has rotated. Essentially, the rotation matrix reflects the transformation relationship between coordinates in one coordinate system and those in another coordinate system.

[0052] In one implementation, it is assumed that the reference object pose has a face-up orientation, meaning the graspable area of ​​the object is oriented perpendicular to the Z-axis, and the pose of the object to be grasped is obtained by rotating the reference pose. Assume the rotation matrix from the reference pose to the current object pose is... The orientation feature of the object to be grasped can then be obtained based on R. In one implementation, the orientation feature of the object can be (X... vector Y vector Z vector ), where X vector Y vector Z vector These are the values ​​of the first, second, and third columns of the third row of the rotation matrix, respectively, i.e., X. vector=x3,Y vector =y3,Z vector =z3.

[0053] Various forms of rotation matrices exist in the prior art, and this invention does not limit itself to any particular form. Optionally, the rotation matrix of this invention can be a rotation matrix based on Euler angles. Any rotation can be represented as a combination of three angles rotated sequentially around three rotation axes; these three angles are called Euler angles. Figure 4 As shown, the rotation of an object is described by three rotation components, which can be understood as the X-axis, Y-axis, and Z-axis in a Cartesian coordinate system. The X-axis is the pitch axis, and the angle of clockwise rotation along the X-axis is the pitch angle, denoted as α; the Y-axis is the yaw axis, and the angle of clockwise rotation along the Y-axis is the yaw angle, denoted as β; the Z-axis is the roll axis, and the angle of clockwise rotation along the Z-axis is the roll angle, denoted as γ. Any rotation can be considered as a combination of these three rotation methods. For example, if an object rotates in an XYZ pattern, it means that the object first rotates clockwise along the X-axis by α, then clockwise along the Y-axis by β, and finally rotates clockwise along the Z-axis by γ. Different rotation methods have different rotation matrices, resulting in a total of 12 rotation methods. Preferably, the object can be rotated from the reference direction to the current state in a ZYX pattern. Correspondingly, the rotation matrix of the object to be grasped can be...

[0054]

[0055] For step S120, in one embodiment, the gripper's posture during grasping can be calculated based on the orientation characteristics of the object and the gripper used, including the gripper's rotation angle and orientation, to control the gripper to grasp the object at a certain angle or orientation within the graspable area of ​​the object. In another embodiment, calculating the orientation feature value of at least one object to be grasped can also be used to rank multiple objects to be grasped according to their grasping difficulty; that is, all objects to be grasped are ranked based on the obtained orientation feature values, and the gripper is controlled to grasp them according to the ranking order. Preferably, when an object has an orientation feature (X... vector Y vector Z vector When ), the orientation feature value of the object can be Max{X}. vector Y vector Z vector}

[0056] In addition, it should be noted that although each embodiment of the present invention has a specific combination of features, further combinations and cross-combinations of these features between embodiments are also possible.

[0057] Figure 5 A gripping control device according to yet another embodiment of the present invention is shown, the device comprising:

[0058] The image data acquisition module 500 is used to acquire image data including at least one item to be grabbed, that is, to implement step S100;

[0059] Orientation feature acquisition module 510 is used to process the image data to obtain orientation features related to the orientation of the object to be grasped, that is, to implement step S110;

[0060] The gripping control module 520 is used to control the gripper to perform gripping of at least one item based at least on the orientation features of the item to be gripped, i.e., to implement step S120.

[0061] It should be understood that, in the above-mentioned... Figure 5 In the illustrated device embodiments, only the main functions of the modules are described. All functions of each module correspond to the corresponding steps in the method embodiments, and the working principles of each module can also be referred to the descriptions of the corresponding steps in the method embodiments. Furthermore, although the above embodiments define the correspondence between the functions of the functional modules and the methods, those skilled in the art will understand that the functions of the functional modules are not limited to the above correspondence; that is, a specific functional module can also implement other method steps or a portion of method steps.

[0062] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method of any of the above embodiments. It should be noted that the computer program stored in the computer-readable storage medium of this application can be executed by a processor of an electronic device. Furthermore, the computer-readable storage medium can be a storage medium built into an electronic device or a storage medium that can be plugged into an electronic device. Therefore, the computer-readable storage medium of this application has high flexibility and reliability.

[0063] Figure 6 The diagram illustrates the structure of an electronic device according to an embodiment of the present invention. The electronic device may be a control system / electronic system configured in a car, a mobile terminal (e.g., a smartphone), a personal computer (PC, e.g., a desktop computer or a laptop computer), a tablet computer, or a server, etc. The specific embodiments of the present invention do not limit the specific implementation of the electronic device.

[0064] like Figure 6 As shown, the electronic device may include: a processor 1202, a communications interface 1204, a memory 1206, and a communications bus 1208.

[0065] in:

[0066] The processor 1202, communication interface 1204, and memory 1206 communicate with each other via communication bus 1208.

[0067] The communication interface 1204 is used to communicate with other network elements such as clients or other servers.

[0068] The processor 1202 is used to execute program 1210, specifically the relevant steps in the above method embodiments.

[0069] Specifically, program 1210 may include program code that includes computer operation instructions.

[0070] Processor 1202 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The electronic device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0071] Memory 1206 is used to store program 1210. Memory 1206 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0072] Program 1210 can be downloaded and installed from a network via communication interface 1204, and / or installed from a removable medium. When the program is executed by processor 1202, processor 1202 can perform the various operations in the above method embodiments.

[0073] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0074] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0075] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0076] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0077] It should be understood that various parts of the embodiments of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0078] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0079] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0080] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0081] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. An automatic control method for a fixture, characterized in that, include: Acquire image data of a graspable area including at least one item to be grasped; The image data is processed to obtain orientation features related to the orientation of the graspable area of ​​the object to be grasped; The gripper is controlled to perform gripping of at least one item based at least on the orientation characteristics of the grippable area of ​​the item to be gripped; The control of the gripper to perform gripping of at least one item based at least on the orientation features of the grippable area of ​​the item to be gripped includes: Based on the orientation characteristics of the object and the fixture used, the pose of the fixture when performing the gripping is calculated, and the object is gripped within the gripping area. The method of controlling the gripper to perform gripping of at least one item based at least on the orientation features of the grippable area of ​​the item to be gripped further includes: Calculate the orientation feature value based on the orientation feature; The grasping order of at least one item to be grasped is determined based on the orientation feature value, and the clamp is controlled to grasp at least one item to be grasped in the grasping order.

2. The automatic control method for the fixture according to claim 1, characterized in that, Also includes: The image data is processed to obtain positional features of at least one object to be grasped, and the gripper is controlled to perform grasping of at least one object to be grasped, based at least on the orientation and positional features of the object to be grasped.

3. The automatic control method for the fixture according to claim 1, characterized in that: The orientation feature is obtained based on the rotation matrix of the object to be grasped.

4. The automatic control method for the fixture according to claim 3, characterized in that: The reference orientation of the rotation matrix is ​​the orientation of the graspable area of ​​the object to be grasped when it is perpendicular to the Z-axis.

5. The automatic control method for the fixture according to claim 3, characterized in that: The rotation matrix is ​​a rotation matrix based on Euler angles.

6. An automatic control device for a clamp, characterized in that, include: The image data acquisition module is used to acquire image data including at least one graspable area of ​​an object to be grasped; An orientation feature acquisition module is used to process the image data to obtain orientation features related to the orientation of the graspable area of ​​the object to be grasped. A gripping control module is used to control a gripper to perform gripping of at least one item based at least on the orientation features of the grippable area of ​​the item to be gripped. The control of the gripper to perform gripping of at least one item based at least on the orientation features of the grippable area of ​​the item to be gripped includes: Based on the orientation characteristics of the object and the fixture used, the pose of the fixture when performing the gripping is calculated, and the object is gripped within the gripping area. The method of controlling the gripper to perform gripping of at least one item based at least on the orientation features of the grippable area of ​​the item to be gripped further includes: Calculate the orientation feature value based on the orientation feature; The grasping order of at least one item to be grasped is determined based on the orientation feature value, and the clamp is controlled to grasp at least one item to be grasped in the grasping order.

7. The automatic control device for the clamp according to claim 6, characterized in that, Also includes: The location feature acquisition module is used to process the image data to obtain at least one location-related location feature of the object to be grabbed. The gripping control module is used to control the gripper to perform gripping of at least one item based on at least the orientation and position features of the item to be gripped.

8. The automatic control device for the clamp according to claim 6, characterized in that: The orientation feature is obtained based on the rotation matrix of the object to be grasped.

9. The automatic control device for the clamp according to claim 8, characterized in that: The reference orientation of the rotation matrix is ​​the orientation of the graspable area of ​​the object to be grasped when it is perpendicular to the Z-axis.

10. The automatic control device for the clamp according to claim 9, characterized in that: The rotation matrix is ​​a rotation matrix based on Euler angles.

11. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the automatic control method for the fixture according to any one of claims 1 to 5.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the automatic control method of the fixture according to any one of claims 1 to 5.

Images