Industrial part grabbing method and equipment and medium

Abstract

The invention discloses an industrial part grabbing method and device and a medium, and relates to the technical field of robot control in industrial automation, and the method comprises the steps: obtaining an RGB image and a depth image including an industrial part; inputting the RGB image and the depth image into a pre-trained category recognition model to obtain a part category output by the category recognition model; inputting the RGB image, the depth image and the part category into a pre-trained pose estimation model to obtain a 6D pose of the industrial part output by the pose estimation model; and on the basis of the part category, the part attribute corresponding to the part category and the 6D pose, the grabbing requirement is determined, and the robot is controlled to grab the industrial part on the basis of the grabbing requirement. The industrial part grabbing method and device are used for solving the problems that in the prior art, when part grabbing is carried out without considering the characteristics of industrial parts, safety is poor, and operation efficiency is low, accurate recognition and safe grabbing of the industrial parts are achieved, and operation efficiency and reliability are improved.

Description

Technical Field

[0001] The present application relates to the field of robot control technology in industrial automation, and in particular to a method, device, and medium for grasping industrial parts. Background Art

[0002] In the manufacturing industry, grasping complex industrial parts is a critical step in the production process. However, existing robotic grasping methods rely solely on the part's type and position, potentially damaging the part during grasping and compromising safety throughout the entire grasping process. Furthermore, damaged parts increase production costs and impact the efficiency and reliability of automated production lines. Summary of the Invention

[0003] In response to the above-mentioned problems and technical needs, the applicant has proposed a method, equipment and medium for grasping industrial parts to solve the problems of poor safety and low operating efficiency that occur when grasping parts without considering the characteristics of industrial parts in the existing technology, and to achieve accurate identification and safe grasping of industrial parts to improve operation efficiency and reliability.

[0004] The application embodiment provides a method for grasping industrial parts, the method comprising:

[0005] Acquire RGB images and depth images containing industrial parts;

[0006] Input the RGB image and the depth image into a pre-trained category recognition model to obtain the part category output by the category recognition model, wherein the category recognition model is trained based on the RGB image samples, the depth image samples, and the part category samples;

[0007] Input the RGB image, depth image, and part category into a pre-trained pose estimation model to obtain the 6D pose of the industrial part output by the pose estimation model, wherein the pose estimation model is trained based on the RGB image samples, depth image samples, part category samples, and 6D pose samples;

[0008] Based on the part category, the part attributes corresponding to the part category, and the 6D pose, the grasping requirements are determined, and the robot is controlled to grasp the industrial parts based on the grasping requirements. The grasping requirements include: grasping position, grasping force, and target fixture.

[0009] According to the grasping method of industrial parts provided in the embodiment of the present application, the part attributes include: physical attributes and part characteristics, and the physical attributes include: part shape, part size and part center of gravity;

[0010] Determine the grasping requirements based on the part category, its corresponding part attributes, and its 6D pose, including:

[0011] Determine the initial grasping position and initial grasping force of industrial parts based on physical properties and part categories;

[0012] Based on the part characteristics and 6D pose, adjust any one or more of the initial grasping position, initial grasping force and target fixture to obtain the final grasping requirements.

[0013] According to the grasping method for industrial parts provided in the embodiments of the present application, based on the part characteristics and 6D pose, any one or more of the initial grasping position, initial grasping force, and target fixture are adjusted, including:

[0014] Obtaining a target range that meets the characteristics of the industrial part in the body of the industrial part, wherein the target range is obtained based on the part category;

[0015] Adjust any one or more of the initial grip position, initial grip force, and target fixture based on part characteristics, target range, and 6D pose.

[0016] According to the grasping method of industrial parts provided in the embodiments of the present application, the part characteristics include fragility;

[0017] Adjust any one or more of the initial grip position, initial grip force, and target fixture based on part characteristics, target range, and 6D pose, including:

[0018] When it is determined that the entire target range is the body of an industrial part that is fragile, the target fixture corresponding to the part characteristics is replaced, and the initial grasping position and initial grasping force are adjusted based on the part characteristics and 6D posture; or, the initial grasping force is reduced, and the initial grasping position is adjusted based on the part characteristics and 6D posture.

[0019] According to the grasping method of industrial parts provided in the embodiment of the present application, the part characteristics include sensitivity and fragility;

[0020] Adjust any one or more of the initial grip position, initial grip force, and target fixture based on part characteristics, target range, and 6D pose, including:

[0021] When it is determined that the target range is a sensitive part of the body of the industrial part, the initial grasping position is adjusted to a non-sensitive area, and the initial grasping force is adjusted based on the part characteristics, the adjusted grasping position and the 6D pose;

[0022] When it is determined that the target range is a part of the body of an industrial part that is fragile, the initial grasping force is reduced and the initial grasping position is adjusted based on the part characteristics and 6D posture; or, the initial grasping position is adjusted to a non-fragile area, and the initial grasping force is adjusted based on the part characteristics, the adjusted grasping position and the 6D posture; or, the target fixture corresponding to the part characteristics is replaced, and the initial grasping position and initial grasping force are adjusted based on the part characteristics and the 6D posture.

[0023] According to the grasping method for industrial parts provided in the embodiments of the present application, based on the part characteristics, target range and 6D pose, any one or more of the initial grasping position, initial grasping force and target fixture are adjusted, including:

[0024] When it is determined that the target range is a portion of the body of the industrial part that is fragile, determine the proportion of the fragile area in the industrial part and the location of the fragile area in the industrial part;

[0025] Determine whether the specific gravity is less than the preset specific gravity;

[0026] If the specific gravity is less than the preset specific gravity, determine whether the belonging position and the initial grasping position overlap; if the belonging position and the initial grasping position overlap, adjust the initial grasping position to a non-fragile area; if the belonging position and the initial grasping position do not overlap, reduce the initial grasping force;

[0027] When it is determined that the specific gravity is greater than or equal to the preset specific gravity, a target fixture corresponding to the part characteristics is replaced.

[0028] According to the industrial parts grasping method provided in the embodiment of the present application, after controlling the robot to grasp the industrial parts based on the grasping requirements, the method further includes:

[0029] Get the actual grasping position and actual grasping force of the robot;

[0030] Determine whether the actual grasping position is consistent with the grasping position, and whether the actual grasping force is consistent with the grasping force, and obtain a judgment result;

[0031] When any one or more of the judgment results are not satisfied, the actual grasping position and / or the actual grasping force are adjusted to meet the grasping requirements.

[0032] According to the industrial parts grabbing method provided in the embodiment of the present application, after obtaining the part category output by the category recognition model, the method further includes:

[0033] Determine the number of part categories;

[0034] When it is determined that the number is equal to 1, performing a step of determining grasping requirements based on the part category, the part attributes corresponding to the part category, and the 6D pose;

[0035] When it is determined that the number is greater than 1, the current industrial part to be grasped is determined based on the operation scenario, and the steps of determining the grasping requirements are performed based on the part category, the part attributes corresponding to the part category, and the 6D pose; the next industrial part to be grasped is determined based on the operation scenario until all industrial parts are grasped.

[0036] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of any of the above industrial parts grasping methods are implemented.

[0037] An embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of any of the above industrial parts grasping methods are implemented.

[0038] The industrial parts grasping method, device and medium provided in the embodiments of the present application obtain RGB images and depth images of industrial parts; input the RGB images and depth images into a pre-trained category recognition model to obtain the part category output by the category recognition model. The present application uses the pre-trained category recognition model to accurately obtain the part category of the industrial parts, providing an effective data basis for determining the subsequent grasping requirements; input the RGB images, depth images and part categories into a pre-trained pose estimation model to obtain the 6D pose of the industrial parts output by the pose estimation model. The present application uses the pre-trained The pose estimation model accurately obtains the 6D pose of industrial parts, providing an effective data basis for determining the subsequent grasping requirements; based on the part category, the part attributes corresponding to the part category and the 6D pose, the grasping requirements are determined, and the robot is controlled to grasp the industrial parts based on the grasping requirements, wherein the grasping requirements include: grasping position, grasping force and target fixture. This application combines the part attributes and 6D pose of industrial parts to determine the corresponding grasping requirements, thereby realizing the accurate identification and safe grasping of industrial parts, thereby saving production costs and improving the operating efficiency and reliability of the automated production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is one of the flow charts of the method for grasping industrial parts provided in the embodiment of the present application;

[0041] Figure 2 This is the second flow chart of the method for grasping industrial parts provided in the embodiment of the present application;

[0042] Figure 3 This is the third flow chart of the method for grasping industrial parts provided in the embodiment of the present application;

[0043] Figure 4 This is the fourth flow chart of the method for grasping industrial parts provided in the embodiment of the present application;

[0044] Figure 5 This is the fifth flow chart of the method for grasping industrial parts provided in the embodiment of the present application;

[0045] Figure 6 This is the sixth flow chart of the method for grasping industrial parts provided in the embodiment of the present application;

[0046] Figure 7 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] To make the purpose, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0048] The embodiment of the present application provides a method for grasping industrial parts. The method can be applied to smart terminals and can also be applied to servers. The present application uses the method applied to a server as an example for illustration. This is for illustration only and is not intended to limit the scope of protection of the present application. Some other descriptions in the embodiment are also for illustration only and will not be described one by one later. The specific implementation of the method is as follows: Figure 1 As shown:

[0049] Step 101: Acquire an RGB image and a depth image containing industrial parts.

[0050] The RGB image and the depth image both include at least one type of industrial parts, and the RGB image and the depth image are aligned.

[0051] Step 102: Input the RGB image and the depth image into a pre-trained category recognition model to obtain the part category output by the category recognition model.

[0052] Among them, the category recognition model is trained based on RGB image samples, depth image samples and part category samples.

[0053] Specifically, the category recognition model extracts visual features from RGB images and depth features from depth images; and then obtains and outputs the part category through visual features and depth features.

[0054] Among them, visual features include: color features, edge features and texture features.

[0055] In step 103 , the RGB image, the depth image, and the part category are input into a pre-trained pose estimation model to obtain a 6D pose of the industrial part output by the pose estimation model.

[0056] Among them, the pose estimation model is trained based on RGB image samples, depth image samples, part category samples and 6D pose samples.

[0057] Among them, the pose estimation model includes the FoundationPose model, which is used for 6D pose estimation and tracking.

[0058] Step 104 : Determine the grasping requirements based on the part category, the part attributes corresponding to the part category, and the 6D pose, and control the robot to grasp the industrial part based on the grasping requirements.

[0059] Among them, the grasping requirements include: grasping position, grasping force and target fixture.

[0060] Among them, the grasping requirements also include: grasping posture.

[0061] Part attributes include physical attributes and part characteristics. Physical attributes include part shape, part size, and part center of gravity. Part characteristics include any one or more of sensitivity and fragility.

[0062] The industrial parts grasping method provided in the embodiment of the present application obtains an RGB image and a depth image including the industrial parts; inputs the RGB image and the depth image into a pre-trained category recognition model to obtain the part category output by the category recognition model. The present application uses the pre-trained category recognition model to accurately obtain the part category of the industrial parts, providing an effective data basis for determining subsequent grasping requirements; inputs the RGB image, the depth image and the part category into a pre-trained pose estimation model to obtain the 6D pose of the industrial parts output by the pose estimation model. The present application uses the pre-trained pose estimation model to accurately obtain the 6D pose of the industrial parts, providing an effective data basis for determining subsequent grasping requirements; determines the grasping requirements based on the part category, the part attributes corresponding to the part category and the 6D pose, and controls the robot to grasp the industrial parts based on the grasping requirements, wherein the grasping requirements include: grasping position, grasping force and target fixture. The present application combines the part attributes and 6D pose of the industrial parts to determine the corresponding grasping requirements, thereby realizing accurate identification and safe grasping of industrial parts, thereby saving production costs and improving the operation efficiency and operation reliability of the automated production line.

[0063] In a specific embodiment, based on the part category, the part attributes corresponding to the part category and the 6D pose, the specific implementation of the grasping requirement is determined as follows: Figure 2 As shown:

[0064] Step 201 : Determine an initial grasping position and initial grasping force of an industrial part based on physical properties and part category.

[0065] Step 202 : Based on the part characteristics and 6D pose, adjust any one or more of the initial grasping position, initial grasping force, and target fixture to obtain the final grasping requirements.

[0066] Specifically, the initial grasping position and initial grasping force are determined based on the part shape, part size and part center of gravity.

[0067] Specifically, based on the physical properties and part category, a target fixture corresponding to the industrial part is also determined.

[0068] Specifically, a pre-created mapping relationship between physical properties and part categories, initial gripping position, and initial gripping force is used to determine the initial gripping position and initial gripping force corresponding to the physical properties and part categories. This mapping relationship also characterizes the relationship between the physical properties and part categories and the target fixture.

[0069] Specifically, after obtaining the initial grasping position and initial grasping force of the industrial part, determine whether the part characteristics exist. If so, execute step 202. Otherwise, based on the 6D posture, adjust any one or more of the initial grasping position and initial grasping force, and determine the robot's grasping posture.

[0070] The presence of part characteristics indicates that the part characteristics include any one or more of fragility and sensitivity.

[0071] In a specific embodiment, based on the part characteristics and 6D posture, the specific implementation of adjusting any one or more of the initial grasping position, initial grasping force and target fixture is as follows: Figure 3 As shown:

[0072] Step 301: Obtain a target range in the body of an industrial part that meets the part characteristics.

[0073] Among them, the target range is obtained based on the part category.

[0074] Step 302 : Based on the part characteristics, target range, and 6D pose, adjust any one or more of the initial grasping position, initial grasping force, and target fixture.

[0075] Specifically, a mapping relationship between part categories and part characteristics of industrial parts corresponding to ranges in the industrial part body is created in advance, and a target range of the industrial part is determined based on the mapping relationship.

[0076] Specifically, whether the main body of the industrial part all belongs to the part characteristics is judged based on the target range, and based on the judgment result, part characteristics and 6D posture, any one or more of the initial grasping position, initial grasping force and target fixture are adjusted.

[0077] Specifically, whether the body of an industrial part belongs to the part characteristics includes whether the properties of the material of the industrial part belong to the part characteristics, whether the manufacturing process of the industrial part will cause the industrial part to belong to the part characteristics, etc.

[0078] This application adopts different adjustment strategies to adjust any one or more of the initial grasping position, initial grasping force and target fixture according to whether all industrial parts belong to the part characteristics (whether the body of the industrial part belongs to the part characteristics), and the adjustment method is more flexible.

[0079] The following describes the specific implementation of adjusting any one or more of the initial grasping position, initial grasping force, and target fixture for two situations.

[0080] In a specific embodiment, when it is determined that all industrial parts belong to part characteristics, adjusting any one or more of the initial grasping position, the initial grasping force, and the target fixture includes:

[0081] If the entire target range of an industrial part is determined to be fragile, the target fixture corresponding to the part's characteristics is replaced, and the initial grasping position and initial grasping force are adjusted based on the part's characteristics and 6D pose to obtain a grasping position and grasping force that correspond to the target fixture, part characteristics, and 6D pose. Of course, the grasping posture of the target fixture can also be obtained.

[0082] Alternatively, the initial gripping force can be reduced and the initial gripping position can be adjusted based on the part characteristics and 6D pose to obtain a gripping position and gripping force that correspond to the part characteristics and 6D pose. Of course, the gripping posture of the fixture can also be obtained.

[0083] In a specific embodiment, when it is determined that the industrial part is partially a part characteristic, adjusting any one or more of the initial grasping position, the initial grasping force, and the target fixture includes:

[0084] If the target range is determined to be a sensitive area of the industrial part's body, the initial grasping position is adjusted to a non-sensitive area, and the initial grasping force is adjusted based on the part's characteristics, the adjusted grasping position, and the 6D pose. Of course, the grasping position in this process is determined within the non-sensitive area. The grasping position is determined first, and then the initial grasping force is adjusted to obtain the final grasping force. Of course, the gripping posture of the fixture can also be determined.

[0085] If the target area is determined to be a fragile part of the industrial part body, the initial gripping force is reduced and the initial gripping position is adjusted based on the part characteristics and 6D pose to obtain the final gripping position and gripping force. Of course, the gripping posture of the fixture can also be obtained.

[0086] Alternatively, the initial grasping position can be adjusted to a non-fragile area, and the initial grasping force can be adjusted based on the part characteristics, the adjusted grasping position, and the 6D pose. Of course, the grasping position in this process is determined within the non-fragile area. The grasping position is determined first, and then the initial grasping force is adjusted to determine the final grasping force. Of course, the gripper's grasping posture can also be determined.

[0087] Alternatively, the target fixture corresponding to the part characteristics can be replaced, and the initial grasping position and initial grasping force can be adjusted based on the part characteristics and 6D pose. Specifically, the initial grasping position and initial grasping force are adjusted based on the part characteristics and 6D pose to obtain a grasping position and grasping force that correspond to the target fixture, part characteristics, and 6D pose. Of course, the grasping posture of the target fixture can also be obtained.

[0088] In a specific embodiment, based on the part characteristics, target range and 6D pose, the specific implementation of adjusting any one or more of the initial grasping position, initial grasping force and target fixture is as follows: Figure 4 As shown:

[0089] Step 401 : When it is determined that the target range is a portion of the body of the industrial part that is fragile, the proportion of the fragile area in the industrial part and the location of the fragile area in the industrial part are determined.

[0090] Step 402 , determining whether the specific gravity is less than a preset specific gravity, if so, executing step 403 , otherwise executing step 406 .

[0091] Step 403 , determine whether the belonging position and the initial grasping position overlap, if so, execute step 404 , otherwise execute step 405 .

[0092] Step 404 , adjusting the initial grasping position to a non-fragile area, and determining a new grasping force based on the adjusted grasping position and physical properties.

[0093] Step 405: Reduce the initial grasping force.

[0094] Step 406: Replace the target fixture corresponding to the part characteristics.

[0095] Among them, the correspondence between part features and target fixtures is created in advance.

[0096] In addition, in step 404, after adjusting the initial grasping position to the non-fragile area, it is also necessary to determine whether the non-fragile area can ensure the stability of grasping. If stable grasping can be guaranteed, the new grasping force is determined based on the adjusted grasping position and physical properties; if stable grasping cannot be guaranteed, execute step 406.

[0097] This application determines whether to replace the target fixture based on the proportion of fragile areas in industrial parts, and when the proportion is less than the preset proportion, the grasping position and grasping force are determined based on the positional relationship between the fragile area and the initial grasping position, thereby ensuring the safety of industrial parts.

[0098] In a specific embodiment, after controlling the robot to grasp the industrial parts based on the grasping requirements, in order to ensure the safe and stable grasping of the industrial parts, it is also necessary to adjust the grasping position and / or grasping force, as shown in the following example: Figure 5 As shown:

[0099] Step 501: Acquire the actual grasping position and actual grasping force of the robot.

[0100] Step 502 , determining whether the actual grasping position is consistent with the grasping position, and whether the actual grasping force is consistent with the grasping force. If so, executing step 503 , otherwise executing step 504 .

[0101] The case of yes is that the actual grasping position is consistent with the grasping position, and the actual grasping force is consistent with the grasping force; the case of no is that the actual grasping position is inconsistent with the grasping position, and / or the actual grasping force is inconsistent with the grasping force.

[0102] Step 503: Control the robot to move the industrial part to a preset placement position.

[0103] Step 504 , adjusting the actual grasping position and / or the actual grasping force to meet the grasping requirement, and executing step 503 after the grasping requirement is met.

[0104] Specifically, if the actual grasping position and the grasping position are inconsistent, adjust the actual grasping position to the grasping position; if the actual grasping force and the grasping force are inconsistent, adjust the actual grasping force to the grasping force; if the actual grasping position and the grasping position are inconsistent and the actual grasping force and the grasping force are inconsistent, adjust the actual grasping position to the grasping position and adjust the actual grasping force to the grasping force.

[0105] This application monitors the robot's actual grasping position and actual grasping force in real time, and makes timely adjustments in abnormal situations, thereby improving the accuracy and safety of grasping.

[0106] In a specific embodiment, after obtaining the part category output by the category recognition model, different grasping strategies are executed based on the number of part categories, specifically as follows: Figure 6 As shown:

[0107] Step 601, determine the number of part categories.

[0108] Step 602 , when it is determined that the number is equal to 1, the step of determining the grasping requirement based on the part category and the part attributes and 6D pose corresponding to the part category is executed.

[0109] Step 603: When it is determined that the number is greater than 1, the current industrial part to be grasped is determined based on the operation scenario, and the grasping requirements are determined based on the part category, the part attributes corresponding to the part category, and the 6D pose; the next industrial part to be grasped is determined based on the operation scenario, until all industrial parts are grasped.

[0110] Specifically, the number of part categories varies, requiring different grasping strategies. If the number is 1, representing one type of industrial part, even if there are multiple parts, there's no need to consider the grasping order; grasping can be performed in any order. If the number is greater than 1, different work scenarios, or different work steps within the same work scenario, may correspond to different part categories. In this case, the actual work scenario is used to determine the grasping order for industrial parts of each part category, and grasping is performed sequentially and iteratively based on this grasping order.

[0111] This application ensures the safety and stability of industrial parts grasping by accurately identifying and grasping each industrial part, further saving production costs and improving the operating efficiency and reliability of the automated production line.

[0112] Figure 7 An example of a physical structure diagram of an electronic device is shown below. Figure 7 As shown, the electronic device may include: a processor 701, a communications interface 702, a memory 703, and a communication bus 704. The processor 701, the communications interface 702, and the memory 703 communicate with each other via the communication bus 704. The processor 701 may call logic instructions in the memory 703 to execute a method for grasping an industrial part.

[0113] In addition, the logic instructions in the above-mentioned memory 703 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0114] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the industrial parts grasping methods provided by the above methods.

[0115] On the other hand, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the industrial parts grasping method provided by the above embodiments.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0117] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0118] Finally, it should be noted that the above description is merely a preferred embodiment of the present application and the present application is not limited to the above embodiments. It is understood that other improvements and variations directly derived or conceived by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included within the scope of protection of the present application.

Claims

1. A method for grasping industrial parts, characterized in that: The method comprises: Acquire RGB images and depth images containing industrial parts; Input the RGB image and the depth image into a pre-trained category recognition model to obtain the part category output by the category recognition model, wherein the category recognition model is trained based on the RGB image samples, the depth image samples, and the part category samples; Input the RGB image, depth image, and part category into a pre-trained pose estimation model to obtain the 6D pose of the industrial part output by the pose estimation model, wherein the pose estimation model is trained based on the RGB image samples, depth image samples, part category samples, and 6D pose samples; Based on the part category, the part attributes corresponding to the part category, and the 6D pose, the grasping requirements are determined, and the robot is controlled to grasp the industrial parts based on the grasping requirements. The grasping requirements include: grasping position, grasping force, and target fixture.

2. The method for grasping industrial parts according to claim 1, characterized in that: The part attributes include: physical attributes and part characteristics, and the physical attributes include: part shape, part size and part center of gravity; Determine the grasping requirements based on the part category, its corresponding part attributes, and its 6D pose, including: Determine the initial grasping position and initial grasping force of industrial parts based on physical properties and part categories; Based on the part characteristics and 6D pose, adjust any one or more of the initial grasping position, initial grasping force and target fixture to obtain the final grasping requirements.

3. The method for grasping industrial parts according to claim 2, characterized in that: Adjust any one or more of the initial grip position, initial grip force, and target fixture based on part characteristics and 6D pose, including: Obtaining a target range that meets the characteristics of the industrial part in the body of the industrial part, wherein the target range is obtained based on the part category; Adjust any one or more of the initial grip position, initial grip force, and target fixture based on part characteristics, target range, and 6D pose.

4. The method for grasping industrial parts according to claim 3, characterized in that: Part characteristics include fragility; Adjust any one or more of the initial grip position, initial grip force, and target fixture based on part characteristics, target range, and 6D pose, including: When it is determined that the entire target range is the body of an industrial part that is fragile, the target fixture corresponding to the part characteristics is replaced, and the initial grasping position and initial grasping force are adjusted based on the part characteristics and 6D posture; or, the initial grasping force is reduced, and the initial grasping position is adjusted based on the part characteristics and 6D posture.

5. The method for grasping industrial parts according to claim 3, characterized in that: Part characteristics include sensitivity and fragility; Adjust any one or more of the initial grip position, initial grip force, and target fixture based on part characteristics, target range, and 6D pose, including: When it is determined that the target range is a sensitive part of the body of the industrial part, the initial grasping position is adjusted to a non-sensitive area, and the initial grasping force is adjusted based on the part characteristics, the adjusted grasping position and the 6D pose; When it is determined that the target range is a part of the body of an industrial part that is fragile, the initial grasping force is reduced and the initial grasping position is adjusted based on the part characteristics and 6D posture; or, the initial grasping position is adjusted to a non-fragile area, and the initial grasping force is adjusted based on the part characteristics, the adjusted grasping position and the 6D posture; or, the target fixture corresponding to the part characteristics is replaced, and the initial grasping position and initial grasping force are adjusted based on the part characteristics and the 6D posture.

6. The method for grasping industrial parts according to claim 5, characterized in that: Adjust any one or more of the initial grip position, initial grip force, and target fixture based on part characteristics, target range, and 6D pose, including: When it is determined that the target range is a portion of the body of the industrial part that is fragile, determine the proportion of the fragile area in the industrial part and the location of the fragile area in the industrial part; Determine whether the specific gravity is less than the preset specific gravity; If the specific gravity is less than the preset specific gravity, determine whether the belonging position and the initial grasping position overlap; if the belonging position and the initial grasping position overlap, adjust the initial grasping position to a non-fragile area; if the belonging position and the initial grasping position do not overlap, reduce the initial grasping force; When it is determined that the specific gravity is greater than or equal to the preset specific gravity, a target fixture corresponding to the part characteristics is replaced.

7. The method for grasping industrial parts according to any one of claims 1 to 6, characterized in that: After controlling the robot to grasp industrial parts based on grasping requirements, it also includes: Get the actual grasping position and actual grasping force of the robot; Determine whether the actual grasping position is consistent with the grasping position, and whether the actual grasping force is consistent with the grasping force, and obtain a judgment result; When any one or more of the judgment results are not satisfied, the actual grasping position and / or the actual grasping force are adjusted to meet the grasping requirements.

8. The method for grasping industrial parts according to any one of claims 1 to 6, characterized in that: After obtaining the part category output by the category recognition model, it also includes: Determine the number of part categories; When it is determined that the number is equal to 1, performing a step of determining grasping requirements based on the part category, the part attributes corresponding to the part category, and the 6D pose; When it is determined that the number is greater than 1, the current industrial part to be grasped is determined based on the operation scenario, and the steps of determining the grasping requirements are performed based on the part category, the part attributes corresponding to the part category, and the 6D pose; the next industrial part to be grasped is determined based on the operation scenario until all industrial parts are grasped.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the industrial parts grasping method according to any one of claims 1 to 8 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for grasping industrial parts according to any one of claims 1 to 8 are implemented.

Images