Grabbing process configuration method and device, equipment and storage medium

Abstract

The invention provides a configuration method and device for a grabbing process, equipment and a storage medium. The configuration method comprises the steps that two or more to-be-grabbed materials are determined from an image shot by a visual system; performing grabbing planning on each to-be-grabbed material on the grabbing setting interface to obtain a plurality of grabbing strategies; and according to field process requirements, determining a grabbing strategy, and generating a corresponding grabbing process configuration file. Therefore, in one grabbing beat, a plurality of to-be-grabbed materials can be identified and positioned, grabbing strategies of two or more to-be-grabbed materials are determined at the same time, the generated grabbing strategies are screened according to field process requirements, the grabbing process configuration file is generated, the grabbing process configuration file is easy to transfer and edit, and the grabbing efficiency is improved. The method can be suitable for various complex grabbing scenes, so that the robot continuously executes multiple grabbing operations after one-time visual recognition, and the grabbing efficiency and success rate are effectively improved.

Description

Background Art

[0002] At present, the sorting of materials still mainly relies on manual operation. This primitive manual sorting method is inefficient and difficult to adapt to the needs of large-scale sorting. Using a robot to sort materials instead of manual operation can effectively speed up work efficiency and reduce errors in manual sorting. The prerequisite for using a robot to replace manual operation is that the material needs to be accurately identified and located. After the material is located, the robot arm is controlled to perform grabbing and placement.

[0003] The existing grasping process generally determines the location of multiple materials through the visual system first, then selects the location of the material with the highest priority according to the optimization algorithm, and controls the robot arm to perform the grasping operation on the material. Then repeat the above process until the grasping task of all the materials to be grasped is completed.

[0004] However, the above method needs to identify and locate the position of the material every time a grasping task is executed, which is inefficient and cannot complete the grasping strategy planning for multiple materials within one grasping cycle. Summary of the invention

[0005] In view of the defects in the prior art, the object of the present invention is to provide a configuration method, device, equipment and storage medium for a crawling process.

[0006] In a first aspect, an embodiment of the present application provides a method for configuring a crawling process, including:

[0007] Determine two or more materials to be grasped from the images captured by the vision system;

[0008] Carry out grabbing planning for each material to be grabbed in the grabbing setting interface to obtain multiple grabbing strategies;

[0009] Determine the grabbing strategy based on on-site process requirements and generate the corresponding grabbing process configuration file.

[0010] Optionally, two or more materials to be grasped are determined from the image captured by the vision system, including:

[0011] Identify the images captured by the visual system and mark two or more materials to be grabbed;

[0012] Determine the position of each material to be grasped, and calculate the occlusion rate of each material to be grasped.

[0013] Optionally, before determining the grabbing strategy according to the on-site process requirements, the method further includes:

[0014] The materials to be grasped are filtered according to the occlusion rate of the materials to be grasped, and the materials to be grasped whose occlusion rate is greater than a preset threshold are removed from the grasping list.

[0015] Optionally, in the grabbing setting interface, grabbing planning is performed for each material to be grabbed to obtain multiple grabbing strategies, including:

[0016] Determine a grasping method according to any one or more factors of the type, size, weight, material, and occlusion rate of the material to be grasped, wherein the grasping method includes: clamping, adsorption, and lifting;

[0017] According to the grasping method, the posture of the actuator of the robot arm when each material to be grasped is grasped is determined respectively; wherein the execution posture of the robot arm includes: the rotation angle and / or height of the flange of the robot arm when grasping;

[0018] According to the position and posture of the robot arm actuator, the grabbing order of each material to be grabbed is determined.

[0019] Optionally, after determining the grabbing strategy according to the on-site process requirements and generating a corresponding grabbing process configuration file, the method further includes:

[0020] Jump from the grabbing setting interface to the placement setting interface, and select the placement pose source and the grab and release combination method according to the determined grab strategy;

[0021] Select grab and place workspace and run the test;

[0022] View the running results.

[0023] Optionally, after determining the grabbing strategy according to the on-site process requirements and generating a corresponding grabbing process configuration file, the method further includes:

[0024] Establishing the association between the material to be grasped and the grasping process configuration file and the on-site process requirements;

[0025] When executing the grabbing task for the same type of materials, or operating under the same on-site process requirements, the grabbing process configuration file is directly imported or recommended.

[0026] Optionally, the grabbing process configuration file is divided into: selecting a pre-processing strategy, generating a grabbing plan, and selecting a post-processing strategy according to different stages of grabbing;

[0027] After determining the grabbing strategy according to the on-site process requirements and generating the corresponding grabbing process configuration file, the method further includes:

[0028] In the crawl setting interface, different stages of crawling are edited and processed respectively, wherein the editing process includes any one or more of the following methods:

[0029] Modify the grabbing method of all or part of the materials to be grabbed;

[0030] Add or delete materials to be grabbed in the grab list;

[0031] Modify the grabbing order of all or part of the materials to be grabbed.

[0032] In a second aspect, an embodiment of the present application provides a configuration device for a capture process, including:

[0033] A determination module, used to determine two or more materials to be grasped from the images captured by the visual system;

[0034] The grabbing planning module is used to carry out grabbing planning for each material to be grabbed in the grabbing setting interface and obtain multiple grabbing strategies;

[0035] The grabbing process configuration module is used to determine the grabbing strategy according to the on-site process requirements and generate the corresponding grabbing process configuration file.

[0036] Optionally, the determining module is specifically configured to:

[0037] Identify the images captured by the visual system and mark two or more materials to be grabbed;

[0038] Determine the position of each material to be grasped, and calculate the occlusion rate of each material to be grasped.

[0039] Optionally, the device further includes: a filtering module for filtering the materials to be grasped according to the occlusion rate of the materials to be grasped before determining the grasping strategy according to the on-site process requirements, and removing the materials to be grasped whose occlusion rate is greater than a preset threshold from the grasping list.

[0040] Optionally, the grasping planning module is specifically used to:

[0041] Determine a grasping method according to any one or more factors of the type, size, weight, material, and occlusion rate of the material to be grasped, wherein the grasping method includes: clamping, adsorption, and lifting;

[0042] According to the grasping method, the posture of the actuator of the robot arm when each material to be grasped is grasped is determined respectively; wherein the execution posture of the robot arm includes: the rotation angle and / or height of the flange of the robot arm when grasping;

[0043] According to the position and posture of the robot arm actuator, the grabbing order of each material to be grabbed is determined.

[0044] Optionally, the device further comprises: an operation test module, which is used to jump from the grabbing setting interface to the placement setting interface after determining the grabbing strategy according to the on-site process requirements and generating the corresponding grabbing process configuration file, and select the placement posture source and the grabbing and releasing combination mode according to the determined grabbing strategy;

[0045] Select grab and place workspace and run the test;

[0046] View the running results.

[0047] Optionally, the device further comprises: a storage module for establishing an association relationship between the material to be grasped and the grasping process profile and the on-site process requirements after determining the grasping strategy according to the on-site process requirements and generating a corresponding grasping process profile;

[0048] When executing the grabbing task for the same type of materials, or operating under the same on-site process requirements, the grabbing process configuration file is directly imported or recommended.

[0049] Optionally, the grabbing process configuration file is divided into: selecting a pre-processing strategy, generating a grabbing plan, and selecting a post-processing strategy according to different stages of grabbing;

[0050] The device further includes: a grabbing process editing module, which is used to edit different stages of grabbing in a grabbing setting interface after determining a grabbing strategy according to on-site process requirements and generating a corresponding grabbing process configuration file, wherein the editing process includes any one or more of the following methods:

[0051] Modify the grabbing method of all or part of the materials to be grabbed;

[0052] Add or delete materials to be grabbed in the grab list;

[0053] Modify the grabbing order of all or part of the materials to be grabbed.

[0054] In a third aspect, an embodiment of the present application provides a configuration device for a capture process, including: a processor and a memory, wherein the memory stores executable program instructions, and when the processor calls the program instructions in the memory, the processor is used to:

[0055] Execute the steps of the method for configuring the crawling process as described in any one of the first aspects.

[0056] In a fourth aspect, an embodiment of the present application provides a robot, comprising: a robot body, a drive system, and a robotic arm, wherein the robot body is provided with a processor and a memory, wherein the memory stores executable program instructions, and when the processor calls the program instructions in the memory, the processor is used to control the drive system to drive the robotic arm to implement the steps of the configuration method of the grasping process as described in any one of the first aspects.

[0057] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a program, which, when executed, implements the steps of the method for configuring the crawling process as described in any one of the first aspects.

[0058] In a sixth aspect, an embodiment of the present application provides a program product, which includes a computer program, and the computer program is stored in a readable storage medium. At least one processor of a robot can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the robot implements the configuration method of the grasping process in the first aspect.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] In this application, two or more materials to be grasped are determined from the images taken by the visual system; grasping planning is performed for each material to be grasped in the grasping setting interface to obtain multiple grasping strategies; grasping strategies are determined according to on-site process requirements, and corresponding grasping process configuration files are generated. In this way, the identification and positioning of multiple materials to be grasped can be completed within one grasping cycle, and grasping strategies for two or more materials to be grasped can be determined at the same time. According to on-site process requirements, the generated grasping strategies are screened to generate a grasping process configuration file, which is easy to transfer and edit, and can be applied to various complex grasping scenarios, so that the robot can continuously perform multiple grasping operations after one visual recognition, effectively improving the efficiency and success rate of grasping. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following descriptions are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0062] Figure 1 Schematic diagram of the setting interface of the crawling process provided in the embodiment of the present application Figure 1;

[0063] Figure 2 Schematic diagram of the setting interface of the crawling process provided in the embodiment of the present application Figure 2 ;

[0064] Figure 3 A schematic diagram of a method for configuring a capture process provided in an embodiment of the present application Figure 1 ;

[0065] Figure 4 A schematic diagram of the principle of the grabbing strategy provided in the embodiment of the present application;

[0066] Figure 5 A schematic diagram of a method for configuring a capture process provided in an embodiment of the present application Figure 2 ;

[0067] Figure 6 A schematic diagram of a method for configuring a capture process provided in an embodiment of the present application Figure 3 ;

[0068] Figure 7 A schematic diagram of a method for configuring a capture process provided in an embodiment of the present application Figure 4 ;

[0069] Figure 8 A schematic diagram of the structure of a configuration device for a capture process provided in an embodiment of the present application;

[0070] Fig. 9 A schematic diagram of the structure of a configuration device for a capture process provided in an embodiment of the present application;

[0071] Fig.10 It is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0073] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there can be a central component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there can be a central component at the same time.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0075] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0076] The following specific embodiments are used to describe in detail the technical solution of the present invention and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0077] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0078] The embodiments of the present application provide a configuration method for a grasping process, which can be applied to various types of robots, such as mobile logistics robots, robots on large-scale production lines, simple transport robots, etc., but the embodiments of the present application are not limited thereto.

[0079] Figure 1 Schematic diagram of the setting interface of the crawling process provided in the embodiment of the present application Figure 1 , Figure 2 Schematic diagram of the setting interface of the crawling process provided in the embodiment of the present application Figure 2 ,like Figure 1 , Figure 2As shown, the entire grabbing setting can be divided into three stages: selecting the pre-processing strategy, generating the grabbing plan, and selecting the post-processing strategy. In the first step, you can select the pre-processing strategy and select the preset processing strategy in the drop-down box (the default is no processing). In the second step, select the material to be grabbed in the grab registration set in the drop-down box and generate the grabbing plan. In the third step, you can select the post-processing strategy and select the preset processing strategy in the drop-down box. After the grabbing setting is completed, the background will generate the corresponding grabbing process configuration file, which can be directly called the next time a similar material is grabbed, or saved as a user-defined template and shared with other users.

[0080] Furthermore, after completing the grabbing settings, you can also make placement settings. First, set the placement pose source, such as selecting the slot of the workspace as the placement pose source. The first step is to select the processing strategy for the slot of the workspace. The second step is to determine the grabbing and releasing combination method, such as giving priority to placing the corner points of the workspace. Finally, perform a test run to view the running results.

[0081] This embodiment can flexibly configure the material grabbing process and subdivide the grabbing process. Before and after the grabbing planning, you can choose to add grabbing strategies from the preset strategies. After the grabbing settings are completed, you can also set the placement process, so as to run the complete grabbing and placing process of the robot arm to ensure the success rate of the robot arm in performing grabbing operations in the real environment.

[0082] Figure 3 A schematic diagram of a method for configuring a capture process provided in an embodiment of the present application Figure 1 ,like Figure 3 As shown, the method in this embodiment may include:

[0083] Step S301: determine two or more materials to be grasped from images captured by the visual system.

[0084] In this embodiment, an image of the material to be grasped can be captured by a visual system arranged on the robotic arm, or a visual system arranged in the working space of the robotic arm (for example, a material frame is photographed to obtain an image containing multiple materials to be grasped), and then the image is recognized to mark two or more materials to be grasped; the position of each material to be grasped is determined, and the occlusion rate of each material to be grasped is calculated.

[0085] Step S302: Perform grabbing planning for each material to be grabbed in the grabbing setting interface to obtain multiple grabbing strategies.

[0086] In this embodiment, the grasping method is determined according to any one or more factors including the type, size, weight, material, and occlusion rate of the material to be grasped, and the grasping methods include: clamping, adsorption, and lifting; according to the grasping method, the position and posture of the robot arm actuator when each material to be grasped is grasped is determined respectively; wherein, the execution posture of the robot arm includes: the rotation angle and / or height of the flange of the robot arm when executing the grasping; according to the posture of the robot arm actuator, the grasping order of each material to be grasped is determined.

[0087] Step S303: Determine the grabbing strategy according to the on-site process requirements and generate a corresponding grabbing process configuration file.

[0088] In this embodiment, for different on-site process requirements, the best strategy is selected from the generated multiple grasping strategies, and a grasping process configuration file is generated. The grasping process configuration file records in detail the grasping method of the material to be grasped under the on-site process requirements, and the method for determining the material grasping order (for example, setting multiple grasping priorities, giving priority to grasping the material with the smallest occlusion rate, and then grasping the material with the highest robot arm flange height, or the material with the smallest robot arm flange rotation angle).

[0089] In an optional implementation, the grabbing method can be determined based on the type, size, weight, material, etc. of the material. For example, a suction cup can be used to grab light and large materials, a clamp can be used to grab heavy, small, or irregular materials, and a lifting method can be used to grab neatly stacked and heavy materials. Furthermore, factors to consider in the grabbing order (such as occlusion rate, robot arm flange height, robot arm flange rotation angle, etc.) are added, and then the weight of each factor is set to calculate the priority of each material to be grabbed.

[0090] In another optional implementation, before determining the grabbing strategy according to the on-site process requirements, the materials to be grabbed are first filtered according to the occlusion rate of the materials to be grabbed, and the materials to be grabbed with an occlusion rate greater than a preset threshold are removed from the grabbing list. This method can perform a preliminary screening of the materials to be grabbed, remove materials with high occlusion rates, and improve the success rate of the robot arm's grabbing.

[0091] Figure 4 The schematic diagram of the principle of the grabbing strategy provided in the embodiment of the present application is as follows: Figure 4As shown in the figure, numbers 1 to 12 represent the grasping order of cylindrical workpieces, and the workpieces shown by the dotted lines in the figure are the workpieces filtered out from the grasping list. For example, an upper limit on the number of workpieces participating in the sequence arrangement can be set, for example, it is stipulated that the grasping sequence planning is only performed for 2 or more and less than 15 materials to be grasped at a time. This is because the occlusion rate of the material will change as the grasping operation proceeds. Limiting the number can simplify the difficulty of grasping sequence planning. As the number of grasping times of the robot arm increases, the probability of the posture of the workpiece in the material frame being affected will gradually increase. If the workpiece posture initially identified is still used for grasping, the probability of grasping failure is likely to increase. Therefore, by limiting the number of materials participating in the grasping sequence planning each time, this problem can be well solved, while ensuring the grasping efficiency of the robot arm, maintaining a high grasping success rate.

[0092] In this embodiment, two or more materials to be grasped are determined from the images taken by the visual system; grasping planning is performed for each material to be grasped in the grasping setting interface to obtain multiple grasping strategies; grasping strategies are determined according to on-site process requirements, and corresponding grasping process configuration files are generated. Thus, within one grasping cycle, the identification and positioning of multiple materials to be grasped can be completed, and grasping strategies for two or more materials to be grasped can be determined at the same time. According to on-site process requirements, the generated grasping strategies are screened to generate a grasping process configuration file, which is easy to transfer and edit, and can be applied to various complex grasping scenarios, so that the robot can continuously perform multiple grasping operations after one visual recognition, effectively improving the efficiency and success rate of grasping.

[0093] Figure 5 A schematic diagram of a method for configuring a capture process provided in an embodiment of the present application Figure 2 ,like Figure 5 As shown, the method in this embodiment may include:

[0094] Step S501: determine two or more materials to be grasped from images captured by a visual system.

[0095] Step S502: Perform grabbing planning for each material to be grabbed in the grabbing setting interface to obtain multiple grabbing strategies.

[0096] Step S503: Determine the grabbing strategy according to the on-site process requirements and generate a corresponding grabbing process configuration file.

[0097] For the specific implementation and technical effects of steps S501 to S503 in this embodiment, please refer to Figure 3 The relevant descriptions of steps S301 to S303 are not repeated here.

[0098] Step S504: Jump from the grabbing setting interface to the placement setting interface, and select the placement pose source and the grab and release combination method according to the determined grabbing strategy.

[0099] Step S505: Select the grab and place workspace and perform a run test.

[0100] Step S506: Check the operation results.

[0101] In this embodiment, after completing the configuration of the grabbing process, you can further set the placement process by jumping from the grabbing setting interface to the placement setting interface, selecting the placement pose source and the grabbing and placing combination mode according to the determined grabbing strategy; selecting the grabbing and placing workspaces, and performing a running test; and viewing the running results. In this way, the previously configured grabbing process can be combined with the placement process to verify the feasibility and success rate as a whole.

[0102] Figure 6 A schematic diagram of a method for configuring a capture process provided in an embodiment of the present application Figure 3 ,like Figure 6 As shown, the method in this embodiment may include:

[0103] Step S601: determine two or more materials to be grasped from images captured by the visual system.

[0104] Step S602: Perform grabbing planning for each material to be grabbed in the grabbing setting interface to obtain multiple grabbing strategies.

[0105] Step S603: Determine the grabbing strategy according to the on-site process requirements and generate a corresponding grabbing process configuration file.

[0106] Step S604: Jump from the grabbing setting interface to the placement setting interface, and select the placement pose source and the grabbing and releasing combination method according to the determined grabbing strategy.

[0107] Step S605: Select the grab and place workspace and perform a run test.

[0108] Step S606: Check the operation results.

[0109] For specific implementation and technical effects of steps S601 to S606 in this embodiment, please refer to Figure 5 The relevant descriptions of steps S501 to S506 are not repeated here.

[0110] Step S607: Establishing the association between the material to be grasped, the grasping process configuration file, and the on-site process requirements.

[0111] In this embodiment, a correspondence may be established between the generated grabbing process configuration file and the on-site process requirements, and the corresponding grabbing process configuration file may be saved to facilitate subsequent direct search for the corresponding grabbing process configuration file according to the process requirements.

[0112] Step S608: When executing the grabbing task for the same type of materials or operating under the same on-site process requirements, directly import or recommend the grabbing process configuration file.

[0113] In this embodiment, the generated grabbing process configuration file can be uploaded and shared, or it can be directly imported and used when the next grabbing plan is performed, avoiding repeated settings and simplifying the operation process. When executing the grabbing task of the same type of material, or running under the same on-site process requirements, the background can recommend the most suitable grabbing process configuration file for the user to use.

[0114] Figure 7 A schematic diagram of a method for configuring a capture process provided in an embodiment of the present application Figure 4 ,like Figure 7 As shown, the method in this embodiment may include:

[0115] Step S701: determine two or more materials to be grasped from images captured by the visual system.

[0116] Step S702: Perform grabbing planning for each material to be grabbed in the grabbing setting interface to obtain multiple grabbing strategies.

[0117] Step S703: Determine the grabbing strategy according to the on-site process requirements and generate a corresponding grabbing process configuration file.

[0118] Step S704: Jump from the grabbing setting interface to the placement setting interface, and select the placement pose source and the grab and release combination method according to the determined grabbing strategy.

[0119] Step S705: Select the grab and place workspace and perform a run test.

[0120] Step S706: Check the operation results.

[0121] For specific implementation and technical effects of steps S701 to S706 in this embodiment, please refer to Figure 5 The relevant descriptions of steps S501 to S506 are not repeated here.

[0122] Step S707: Edit different stages of the capture in the capture setting interface.

[0123] In this embodiment, the grabbing process configuration file is divided into: selecting a pre-processing strategy, generating a grabbing plan, and selecting a post-processing strategy according to different stages of grabbing. After determining the grabbing strategy according to the on-site process requirements and generating the corresponding grabbing process configuration file, the different stages of grabbing can be edited in the grabbing setting interface, wherein the editing process includes any one or more of the following methods:

[0124] Modify the grabbing method of all or part of the materials to be grabbed;

[0125] Add or delete materials to be grabbed in the grab list;

[0126] Modify the grabbing order of all or part of the materials to be grabbed.

[0127] In this embodiment, by editing the crawling process configuration file, the crawling strategy can be flexibly modified, thereby greatly improving the applicability to complex crawling scenarios and achieving higher crawling efficiency.

[0128] Figure 8 A schematic diagram of the structure of the configuration device of the capture process provided in the embodiment of the present application, such as Figure 8 As shown, the configuration device 800 of the grasping process in this embodiment may include: a determination module 801, used to determine two or more materials to be grasped from the images taken by the visual system; a grasping planning module 802, used to perform grasping planning for each material to be grasped in the grasping setting interface to obtain multiple grasping strategies; a grasping process configuration module 803, used to determine the grasping strategy according to the on-site process requirements, and generate a corresponding grasping process configuration file.

[0129] Exemplarily, the determination module 801 is specifically configured to:

[0130] Identify the images captured by the visual system and mark two or more materials to be grabbed;

[0131] Determine the position of each material to be grasped, and calculate the occlusion rate of each material to be grasped.

[0132] Exemplarily, the device also includes: a filtering module 804, which is used to filter the materials to be grasped according to the occlusion rate of the materials to be grasped before determining the grasping strategy according to the on-site process requirements, and remove the materials to be grasped whose occlusion rate is greater than a preset threshold from the grasping list.

[0133] Exemplarily, the crawl planning module 802 is specifically used for:

[0134] Determine the grasping method according to any one or more factors of the type, size, weight, material, and occlusion rate of the material to be grasped. The grasping methods include: clamping, adsorption, and lifting;

[0135] According to the grasping mode, the position and posture of the actuator of the robot arm when each material to be grasped is grasped are determined respectively; wherein the execution position and posture of the robot arm includes: the rotation angle and / or height of the flange of the robot arm when grasping;

[0136] According to the position and posture of the robot arm actuator, the grabbing order of each material to be grabbed is determined.

[0137] Exemplarily, the device also includes: an operation test module 805, which is used to jump from the grasping setting interface to the placement setting interface after determining the grasping strategy according to the on-site process requirements and generating the corresponding grasping process configuration file, and select the placement posture source and grasping and placing combination method according to the determined grasping strategy; select the grasping and placement workspace, and perform an operation test; and view the operation results.

[0138] Exemplarily, the device also includes: a storage module 806, which is used to establish an association between the material to be grasped, the grasping process profile, and the on-site process requirements after determining the grasping strategy according to the on-site process requirements and generating the corresponding grasping process profile; when executing the grasping task for the same type of material, or running under the same on-site process requirements, directly importing or recommending the grasping process profile.

[0139] Exemplarily, the grabbing process configuration file is divided into: selecting a pre-processing strategy, generating a grabbing plan, and selecting a post-processing strategy according to different stages of grabbing; the device also includes: a grabbing process editing module 807, which is used to edit different stages of grabbing in the grabbing setting interface after determining the grabbing strategy according to the on-site process requirements and generating the corresponding grabbing process configuration file, wherein the editing process includes any one or more of the following methods:

[0140] Modify the grabbing method of all or part of the materials to be grabbed;

[0141] Add or delete materials to be grabbed in the grab list;

[0142] Modify the grabbing order of all or part of the materials to be grabbed.

[0143] The configuration device 800 of the capture process of this embodiment can execute Figure 3 , Figures 5 to 7 For the technical solution in the method shown, its specific implementation process and technical principles can be found in Figure 3 , Figures 5 to 7 The relevant description in the demonstration method will not be repeated here.

[0144] Fig. 9This is a structural diagram of a configuration device for a capture process provided in an embodiment of the present application. The configuration device 900 for the capture process in this embodiment may include: a processor 901 and a memory 902.

[0145] The memory 902 is used to store programs; the memory 902 may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory 902 is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories 902. And the above computer programs, computer instructions, data, etc. can be called by the processor 901.

[0146] The above-mentioned computer programs, computer instructions, etc. may be stored in partitions in one or more memories 902 . And the above-mentioned computer programs, computer instructions, data, etc. may be called by the processor 901 .

[0147] The processor 901 is used to execute the computer program stored in the memory 902 to implement each step of the method involved in the above embodiment.

[0148] For details, please refer to the relevant description in the previous method embodiment.

[0149] The processor 901 and the memory 902 may be independent structures or integrated structures. When the processor 901 and the memory 902 are independent structures, the memory 902 and the processor 901 may be coupled and connected via a bus 903 .

[0150] The configuration device 900 of the crawling process of this embodiment can execute Figure 3 , Figures 5 to 7 For the technical solution in the method shown, its specific implementation process and technical principles can be found in Figure 3 , Figures 5 to 7 The relevant descriptions in the method shown will not be repeated here.

[0151] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0152] In addition, an embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When at least one processor of a user device executes the computer-executable instructions, the user device executes the above-mentioned various possible methods.

[0153] Among them, computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general or special-purpose computer. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also be present in a communication device as discrete components.

[0154] The present application also provides a program product, which includes a computer program. The computer program is stored in a readable storage medium. At least one processor of the server can read the computer program from the readable storage medium. At least one processor executes the computer program so that the server implements any method of the above-mentioned embodiments of the present invention.

[0155] Those skilled in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc. Various media that can store program codes.

[0156] Fig.10 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Fig.10As shown, a program product 1000 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0157] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0158] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0159] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0160] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0161] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for configuring a crawling process, characterized in that: include: Determine two or more materials to be grasped from the images captured by the vision system; Carry out grabbing planning for each material to be grabbed in the grabbing setting interface to obtain multiple grabbing strategies; Determine the grabbing strategy based on on-site process requirements and generate the corresponding grabbing process configuration file.

2. The method for configuring the crawling process according to claim 1, characterized in that: Determine two or more materials to be picked up from the images taken by the vision system, including: Identify the images captured by the visual system and mark two or more materials to be grabbed; Determine the position of each material to be grasped, and calculate the occlusion rate of each material to be grasped.

3. The method for configuring the crawling process according to claim 2, characterized in that: Before determining the grabbing strategy according to the on-site process requirements, the method further includes: The materials to be grasped are filtered according to the occlusion rate of the materials to be grasped, and the materials to be grasped whose occlusion rate is greater than a preset threshold are removed from the grasping list.

4. The method for configuring the crawling process according to claim 1, characterized in that: In the grabbing setting interface, grabbing plans are made for each material to be grabbed, and multiple grabbing strategies are obtained, including: Determine a grasping method according to any one or more factors of the type, size, weight, material, and occlusion rate of the material to be grasped, wherein the grasping method includes: clamping, adsorption, and lifting; According to the grasping method, the posture of the actuator of the robot arm when each material to be grasped is grasped is determined respectively; wherein the execution posture of the robot arm includes: the rotation angle and / or height of the flange of the robot arm when grasping; According to the position and posture of the robot arm actuator, the grabbing order of each material to be grabbed is determined.

5. The method for configuring the crawling process according to claim 1, characterized in that: After determining the grabbing strategy according to the on-site process requirements and generating the corresponding grabbing process configuration file, the method further includes: Jump from the grabbing setting interface to the placement setting interface, and select the placement pose source and the grab and release combination method according to the determined grab strategy; Select grab and place workspace and run the test; View the running results.

6. The method for configuring a grabbing process according to any one of claims 1 to 5, characterized in that: After determining the grabbing strategy according to the on-site process requirements and generating the corresponding grabbing process configuration file, the method further includes: Establishing the association between the material to be grasped and the grasping process configuration file and the on-site process requirements; When executing the grabbing task for the same type of materials, or operating under the same on-site process requirements, the grabbing process configuration file is directly imported or recommended.

7. The method for configuring a grabbing process according to any one of claims 1 to 5, characterized in that: The crawling process configuration file is divided into: selecting a pre-processing strategy, generating a crawling plan, and selecting a post-processing strategy according to different stages of crawling; After determining the grabbing strategy according to the on-site process requirements and generating the corresponding grabbing process configuration file, the method further includes: In the crawl setting interface, different stages of crawling are edited and processed respectively, wherein the editing process includes any one or more of the following methods: Modify the grabbing method of all or part of the materials to be grabbed; Add or delete materials to be grabbed in the grab list; Modify the grabbing order of all or part of the materials to be grabbed.

8. A configuration device for a capture process, characterized in that: include: A determination module, used to determine two or more materials to be grasped from the images captured by the visual system; The grabbing planning module is used to carry out grabbing planning for each material to be grabbed in the grabbing setting interface and obtain multiple grabbing strategies; The grabbing process configuration module is used to determine the grabbing strategy according to the on-site process requirements and generate the corresponding grabbing process configuration file.

9. A configuration device for a capture process, characterized in that: include: A processor and a memory, wherein the memory stores executable program instructions, and when the processor calls the program instructions in the memory, the processor is used to: Execute the steps of the method for configuring the crawling process described in any one of claims 1 to 7.

10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed, the steps of the method for configuring the crawling process described in any one of claims 1 to 7 are implemented.

Images