Visual template generation, target detection method and device, and robot system

By marking and calculating position deviations in the drawings of the target workpiece, a visual template carrying deviation information is generated, which solves the problem of inaccurate generation of visual templates in complex scenarios, and improves the efficiency of target detection and grabbing of industrial robots in complex scenarios. .

CN114549821BActive Publication Date: 2024-11-22SANY CONSTR ROBOT (XIAN) RES INST CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210045026.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-14
Publication Date
2024-11-22
Estimated Expiration
2042-01-14

AI Technical Summary

Technical Problem

The prior art is difficult to generate accurate visual templates in complex scenarios, which makes it difficult for industrial robots to accurately determine the material grasping point of the workpiece to be grasped during the grab operation, affecting the operation efficiency.

Method used

By obtaining the drawings of the target workpiece, marking and calculating the position deviation between the material grabbing point and the center point, a visual template carrying deviation information is generated, which is used to perform target detection and determine the material grabbing point in complex scenarios.

Benefits of technology

Generate visual templates suitable for more scenarios, which improves the accuracy of object detection and grabbing efficiency of industrial robots in complex scenarios, and ensures the accurate capture and placement of workpieces to be grasped.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114549821B_ABST
    Figure CN114549821B_ABST
Patent Text Reader

Abstract

The present invention provides a method and device for generating a visual template, detecting a target, and a robot system, and relates to the technical field of engineering machinery. The method for generating a visual template includes: obtaining a drawing of a target workpiece; wherein the grabbing point of the target workpiece is marked in the drawing of the target workpiece; based on the drawing of the target workpiece, obtaining a first position deviation between the grabbing point and the center point of the target workpiece; based on the drawing of the target workpiece, generating a visual template of the target workpiece, and writing the first position deviation into the visual template of the target workpiece. The method and device for generating a visual template, detecting a target, and a robot system provided by the present invention can generate visual templates applicable to more scenarios, can perform more accurate target detection in complex scenarios, and when controlling an industrial robot to perform a grabbing operation, can more accurately and efficiently determine the position information of the grabbing point of the workpiece to be grabbed when determining that the workpiece to be grabbed is the target workpiece, and can improve the operating efficiency of the industrial robot.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of engineering machinery, and in particular to a visual template generation, target detection method and device, and a robot system. Background Art

[0002] Under the development trend of intelligent manufacturing, industrial robots based on visual servoing can automatically complete target detection and perform related operations based on the results of target detection, thereby improving the level of intelligent manufacturing and improving production efficiency. It has become one of the indispensable automation equipment for intelligent manufacturing.

[0003] Industrial robots based on visual servoing can achieve target detection through template matching methods, and the visual template will directly affect the accuracy of target detection.

[0004] The existing visual template generation method generates a visual template based on the difference image between the image of the same scene without the workpiece and the image of the same scene with the workpiece. However, in complex scenes, it is difficult to accurately detect the target based on the visual template generated by the existing visual template generation method. Moreover, when controlling the industrial robot to perform a grasping operation, it is difficult to determine the grasping point of the workpiece to be grasped based on the visual template. Summary of the invention

[0005] The present invention provides a visual template generation, target detection method and device and a robot system, which are used to solve the defects in the prior art that, in complex scenes, the visual template generated based on the existing visual template generation method is difficult to perform accurate target detection, and when controlling the industrial robot to perform grasping operations, it is difficult to determine the grasping point of the workpiece to be grasped based on the above-mentioned visual template. The present invention can generate visual templates suitable for more scenes and when controlling the industrial robot to perform grasping operations, the grasping point of the workpiece to be grasped can be determined based on the above-mentioned visual template.

[0006] The present invention provides a visual template generation method, comprising:

[0007] Obtaining a drawing of a target workpiece; wherein the drawing of the target workpiece has marked the material grabbing point of the target workpiece;

[0008] Based on the drawing of the target workpiece, obtaining a first position deviation between the grabbing point and the center point of the target workpiece;

[0009] Based on the drawing of the target workpiece, a visual template of the target workpiece is generated, and the first position deviation is written into the visual template of the target workpiece.

[0010] According to a visual template generation method provided by the present invention, the drawing of the target workpiece has marked the relative discharge point of the target workpiece;

[0011] Accordingly, after obtaining the drawing of the target workpiece, the method further includes:

[0012] Based on the drawing of the target workpiece, obtaining a second position deviation between the relative discharge point and the grabbing point of the target workpiece;

[0013] Correspondingly, after generating the visual template of the target workpiece based on the drawing of the target workpiece, the method further includes:

[0014] The second position deviation is written into the vision template of the target workpiece.

[0015] According to a visual template generation method provided by the present invention, generating the visual template of the target workpiece based on the drawing of the target workpiece specifically includes:

[0016] Based on the preset angle range corresponding to the target workpiece, a visual template of the target workpiece corresponding to the target angle within the preset angle range is generated.

[0017] According to a visual template generation method provided by the present invention, generating the visual template of the target workpiece based on the drawing of the target workpiece specifically includes:

[0018] Based on the drawing of the target workpiece, a visual template of each preset scale of the target workpiece is generated.

[0019] According to a visual template generation method provided by the present invention, generating a visual template of each preset scale of the target workpiece based on the drawing of the target workpiece specifically includes:

[0020] Based on the drawing of the target workpiece and the preset angle range corresponding to the target workpiece, a visual template of the target workpiece of each preset scale corresponding to the target angle within the preset angle range is generated.

[0021] The present invention also provides a visual template generating device, comprising:

[0022] A drawing acquisition module is used to acquire a drawing of a target workpiece, wherein the drawing of the target workpiece has marked the material grabbing point of the target workpiece;

[0023] a deviation determination module, configured to obtain a first position deviation between the material grabbing point and a center point of the target workpiece based on a drawing of the target workpiece;

[0024] The template generation module is used to generate a visual template of the target workpiece based on the drawing of the target workpiece, and write the first position deviation into the visual template of the target workpiece.

[0025] The present invention also provides a target detection method, comprising:

[0026] Obtaining a visual template of a target workpiece;

[0027] Performing target detection on the workpiece to be grasped based on the visual template of the target workpiece, and determining the position information of the grasping point of the workpiece to be grasped based on the first position deviation carried by the visual template of the target workpiece when the workpiece to be grasped is determined to be the target workpiece;

[0028] The visual template of the target workpiece is generated based on the visual template generation method as described above; the first position deviation is used to describe the position deviation between the center point of the target workpiece and the gripping point of the target workpiece.

[0029] According to a target detection method provided by the present invention, after determining the position information of the grasping point of the workpiece to be grasped based on the first position deviation carried by the visual template of the target workpiece, the method further includes:

[0030] Determining position information of the workpiece to be grasped relative to a material placement point based on a second position deviation carried by the visual template of the target workpiece;

[0031] The second position deviation is used to describe the position deviation between the relative discharge point of the target workpiece and the grabbing point.

[0032] The present invention also provides a target detection device, comprising:

[0033] A template acquisition module is used to acquire the visual template of the target workpiece;

[0034] A target detection module is used to perform target detection on the workpiece to be grasped based on the visual template of the target workpiece, and when it is determined that the workpiece to be grasped is the target workpiece, determine the position information of the grasping point of the workpiece to be grasped based on the first position deviation carried by the visual template of the target workpiece;

[0035] The visual template of the target workpiece is generated based on the visual template generation method as described above; the first position deviation is used to describe the position deviation between the center point of the target workpiece and the gripping point of the target workpiece.

[0036] The present invention also provides a robot system, comprising: the target detection device as described above, an industrial robot and an image sensor.

[0037] A robot system provided according to the present invention further includes: the visual template generating device as described above.

[0038] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the visual template generation methods described above or any of the target detection methods described above are implemented.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the visual template generation methods described above, or any of the target detection methods described above.

[0040] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of any of the visual template generation methods described above, or any of the target detection methods described above.

[0041] The visual template generation, target detection method and device and robot system provided by the present invention obtain the first position deviation between the gripping point of the target workpiece and the center point of the target workpiece in the drawing of the target workpiece based on the drawing of the target workpiece on which the gripping point of the target workpiece is marked, generate a visual template of the target workpiece based on the drawing of the target workpiece, and write the above-mentioned first position deviation into the visual template of the target workpiece. Visual templates suitable for more scenarios can be generated, and more accurate target detection can be performed in complex scenarios. When controlling an industrial robot to perform a gripping operation, the position information of the gripping point of the workpiece to be gripped can be determined more accurately and efficiently when the workpiece to be gripped is determined to be the target workpiece, thereby improving the operating efficiency of the industrial robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flowchart of the visual template generation method provided by the present invention;

[0044] Figure 2 It is a flow chart of the target detection method provided by the present invention;

[0045] Figure 3 It is a structural schematic diagram of the visual template generating device provided by the present invention;

[0046] Figure 4 It is a structural schematic diagram of the target detection device provided by the present invention;

[0047] Figure 5 It is one of the structural schematic diagrams of the robot system provided by the present invention;

[0048] Figure 6 This is the second structural schematic diagram of the robot system provided by the present invention;

[0049] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0052] Generally, when controlling an industrial robot to perform a grasping operation, target detection can be performed on the workpiece to be grasped based on a visual template. After obtaining the target detection result, the position information of the grasping point of the workpiece to be grasped can be obtained based on the target detection result and the traditional visual positioning grasping algorithm. Based on the position information of the grasping point of the workpiece to be grasped, the industrial robot can grasp the workpiece to be grasped from the grasping point of the workpiece to be grasped, and can drive the workpiece to be grasped to move and place the workpiece to be grasped at a preset discharge point.

[0053] However, when the structure of the workpiece to be grasped is complex or special, for example, the workpiece to be grasped has a hole in the center, the workpiece to be grasped is U-shaped, etc., it is difficult to accurately obtain the position information of the grasping point of the workpiece to be grasped based on the traditional visual positioning grasping algorithm, and the industrial robot may miss the grasping point. Moreover, after the target detection of the workpiece to be grasped, the position information of the grasping point of the workpiece to be grasped is still required based on the above-mentioned visual positioning grasping algorithm. The process is cumbersome and affects the operating efficiency of the industrial robot.

[0054] Therefore, the present invention provides a visual template generation method. The visual template generated based on the visual template generation method provided by the present invention can be applicable to more scenarios, and when controlling an industrial robot machinery to perform grasping operations, the visual template generated based on the visual template generation method provided by the present invention can determine the position information of the grasping point of the workpiece to be grasped.

[0055] Figure 1 Schematic diagram of the flow of the visual template generation method provided by the present invention. Figure 1 The visual template generation method of the present invention is described. Figure 1 As shown, the method includes: step 101, obtaining a drawing of a target workpiece; wherein the drawing of the target workpiece has marked the grabbing point of the target workpiece.

[0056] It should be noted that the execution subject of the embodiment of the present invention is a visual template generating device.

[0057] Specifically, each workpiece that has a target detection requirement in the actual operation can be used as a target workpiece in turn. For example, before controlling the industrial robot to perform a grasping operation, each workpiece to be grasped can be used as a target workpiece in turn. Based on the visual template generation method provided by the present invention, a visual template of the target workpiece can be generated, so that the visual template of each workpiece that has a target detection requirement can be obtained. Based on the visual templates of the above-mentioned workpieces, target detection of the workpiece can be achieved and the type of the above-mentioned workpiece can be determined. When controlling the industrial robot to perform a grasping operation, the type of the workpiece to be grasped, the placement direction and the position information of the grasping point of the workpiece to be grasped can be determined based on the visual templates of the above-mentioned workpieces.

[0058] It should be noted that the visual template generated by the visual template generation method in the embodiment of the present invention can be used for target detection of the workpiece to be grasped when the industrial robot performs a grasping operation, and when it is determined that the workpiece to be grasped is the target workpiece, further determine the position information of the grasping point of the workpiece to be grasped.

[0059] Specifically, the drawing of the target workpiece with the gripping points of the target workpiece marked can be obtained in a variety of ways. For example, based on prior knowledge, after marking the gripping points of the target workpiece on the original drawing of the target workpiece, the drawing of the target workpiece can be obtained through an image sensor or a screenshot tool.

[0060] It should be noted that the drawing of the target workpiece may be in picture format.

[0061] Optionally, the grabbing point of the target workpiece can be marked in the drawing of the target workpiece by a first graphic of a preset shape. For example, the grabbing point of the target workpiece can be marked in the drawing of the target workpiece by a solid circle of a preset diameter, that is, the position of the solid circle is the position of the grabbing point of the target workpiece.

[0062] It should be noted that when the grabbing point of the target workpiece is marked by a first graphic of a preset shape in the drawing of the target workpiece, it is necessary to avoid duplication of the above-mentioned first graphic with the original structural features of the target workpiece. For example, if a circular fastener is embedded on the upper surface of the target workpiece, the above-mentioned first graphic should not be a circle or a circle with a diameter the same as the diameter of the above-mentioned circular fastener.

[0063] Step 102: Based on the drawing of the target workpiece, a first position deviation between the grabbing point and the center point of the target workpiece is obtained.

[0064] Specifically, after obtaining the drawing of the target workpiece, the drawing of the target workpiece may be parsed to determine the position of the material grabbing point of the target workpiece and the position of the center point of the target workpiece in the drawing of the target workpiece.

[0065] Optionally, based on a first graphic of a preset shape used to mark the grabbing point of the target workpiece, various graphic features in the drawing of the target workpiece can be screened, the above-mentioned first graphic can be determined in the drawing of the target workpiece, and the position of the above-mentioned first graphic can be obtained as the position of the grabbing point of the target workpiece, so that the position of the grabbing point of the target workpiece can be determined in the drawing of the target workpiece. For example, various graphic features in the drawing of the target workpiece can be screened based on a solid circle of a preset diameter, and the position of the solid circle of the preset diameter can be obtained as the position of the grabbing point of the target workpiece.

[0066] Optionally, based on the drawing of the target workpiece, the position of the center point of the target workpiece can be determined in the drawing of the target workpiece by means of geometric analysis and numerical calculation.

[0067] After the position of the grabbing point of the target workpiece and the position of the center point of the target workpiece are determined in the drawing of the target workpiece, a first position deviation between the grabbing point and the center point may be obtained by numerical calculation.

[0068] Step 103 : Generate a visual template of the target workpiece based on the drawing of the target workpiece, and write the first position deviation into the visual template of the target workpiece.

[0069] Specifically, based on the drawing of the target workpiece, a visual template of the target workpiece can be generated by image transformation, edge extraction, etc. In the visual template of the target workpiece, the background information is empty. When performing target detection based on the visual template of the target workpiece, the requirements for the scene are not high, and it can be applied to target detection in complex scenes.

[0070] After the visual template of the target workpiece is generated, the acquired first position deviation may be written into the visual template of the target workpiece.

[0071] When the industrial robot is controlled to perform a grasping operation, the workpiece to be grasped is located on the loading platform, and a top view image of the workpiece to be grasped can be obtained by an image sensor arranged above the loading platform. Based on the visual template of the target workpiece and the top view image of the workpiece to be grasped, target detection can be performed on the workpiece to be grasped. When the workpiece to be grasped is determined to be the target workpiece, the position information of the center point of the workpiece to be grasped can be obtained based on the top view image of the workpiece to be grasped.

[0072] After the position information of the center point of the workpiece to be grasped is obtained, the position information of the grasping point of the workpiece to be grasped can be obtained based on the first position deviation.

[0073] Based on the position information of the gripping point of the workpiece to be gripped, the industrial robot can be controlled to grip the workpiece to be gripped from the gripping point of the workpiece to be gripped.

[0074] It is understandable that the drawing of the target workpiece may include a view of a preset perspective, and the visual template of the target workpiece generated based on the drawing of the target workpiece may be used to perform target detection on the workpiece to be grasped when the surface corresponding to the preset perspective of the workpiece to be grasped is placed upward. For example: the drawing of the target workpiece may include a top view of the target workpiece, and the visual template of the target workpiece generated based on the drawing of the target workpiece may be used to perform target detection on the workpiece to be grasped when the top surface of the workpiece to be grasped is placed upward.

[0075] The drawing of the target workpiece may also include a front view, a top view, a bottom view, a rear view, a left view and a right view of the target workpiece. The visual template of the target workpiece generated based on the drawing of the target workpiece can be used to perform target detection on the workpiece to be grasped when any side of the workpiece to be grasped is placed upward.

[0076] The embodiment of the present invention obtains a first position deviation between the gripping point of the target workpiece and the center point of the target workpiece in the drawing of the target workpiece based on a drawing of the target workpiece on which the gripping point of the target workpiece has been marked, generates a visual template of the target workpiece based on the drawing of the target workpiece, and writes the above-mentioned first position deviation into the visual template of the target workpiece. The visual template applicable to more scenarios can be generated, and more accurate target detection can be performed in complex scenarios. When controlling an industrial robot to perform a gripping operation, the position information of the gripping point of the workpiece to be gripped can be determined more accurately and efficiently when it is determined that the workpiece to be gripped is the target workpiece, thereby improving the operating efficiency of the industrial robot.

[0077] Based on the contents of the above embodiments, the drawing of the target workpiece has marked the relative discharge point of the target workpiece.

[0078] Normally, when an industrial robot performs a grasping operation, after grasping the workpiece to be grasped from the grasping point of the workpiece to be grasped, the workpiece to be grasped can be moved to a predetermined preset discharge point, and the workpiece to be grasped can be placed at the preset discharge point to complete the grasping operation. However, in the case where the size of the workpiece to be grasped is large, placing the workpiece to be grasped at the preset discharge point may cause the workpiece to be grasped to exceed the boundary of the discharge platform, which may cause the workpiece to be grasped to fall or damage the boundary of the discharge platform. It is necessary to correct the preset discharge point and determine the position information of the workpiece to be grasped relative to the discharge point, so that based on the position information of the relative discharge point, the industrial robot can be controlled to move the workpiece to be grasped to the relative discharge point and place the workpiece to be grasped at the relative discharge point.

[0079] In the embodiment of the present invention, the drawing of the target workpiece is marked with a relative discharge point. The relative discharge point is predetermined based on the shape and size of the target workpiece and the discharge platform. The second position deviation between the relative discharge point and the gripping point can be determined based on the drawing of the target workpiece. Based on the second position deviation and the position information of the preset discharge point, the position information of the relative discharge point can be determined, thereby preventing the target workpiece from falling or damaging the boundary of the discharge platform.

[0080] Optionally, the relative discharge point of the target workpiece can be marked in the drawing of the target workpiece by a second graphic of a preset shape. For example, the relative discharge point of the target workpiece can be marked in the drawing of the target workpiece by a solid equilateral triangle with a preset side length, that is, the position of the above-mentioned solid equilateral triangle is the position of the relative discharge point of the target workpiece.

[0081] It should be noted that when the relative discharge point of the target workpiece is marked by a second graphic of a preset shape in the drawing of the target workpiece, it is necessary to avoid duplication of the second graphic with the original structure of the target workpiece. For example, if a triangular fastener is embedded on the upper surface of the target workpiece, the second graphic should not be a triangle or a triangle with a side length the same as that of the triangular fastener.

[0082] Correspondingly, after obtaining the drawing of the target workpiece, the method further comprises: obtaining a second position deviation between the relative discharge point and the grasping point of the target workpiece based on the drawing of the target workpiece.

[0083] Specifically, after obtaining the drawing of the target workpiece, the drawing of the target workpiece may be parsed to determine the position of the relative discharge point of the target workpiece and the position of the grabbing point of the target workpiece in the drawing of the target workpiece.

[0084] Optionally, based on a second graphic of a preset shape used to mark the relative discharge point of the target workpiece, various graphic features in the drawing of the target workpiece can be screened, the above-mentioned second graphic can be determined in the drawing of the target workpiece, and the position of the above-mentioned second graphic can be obtained as the position of the relative discharge point of the target workpiece, so as to determine the position of the relative discharge point of the target workpiece in the drawing of the target workpiece. For example, various graphic features in the drawing of the target workpiece can be screened based on a solid equilateral triangle with a preset side length, and the position of the solid equilateral triangle with the preset side length can be obtained as the position of the relative discharge point of the target workpiece.

[0085] After the position of the relative discharge point of the target workpiece and the position of the gripping point of the target workpiece are determined in the drawing of the target workpiece, a second position deviation between the relative discharge point and the gripping point can be obtained by numerical calculation.

[0086] Correspondingly, after generating the visual template of the target workpiece based on the drawing of the target workpiece, the method further includes: writing the second position deviation into the visual template of the target workpiece.

[0087] After acquiring the second position deviation, the acquired second position deviation may be written into the visual template of the target workpiece.

[0088] When controlling the industrial robot to perform a grasping operation, the industrial robot can be controlled to grasp the target workpiece at the above grasping point based on the visual template of the target workpiece, and the position information of the relative discharge point of the workpiece to be grasped can be obtained based on the second position deviation.

[0089] After obtaining the position information of the relative discharge point of the workpiece to be grasped, the industrial robot can be controlled to place the workpiece to be grasped at the relative discharge point based on the position information of the relative discharge point of the workpiece to be grasped, thereby preventing the workpiece to be grasped from exceeding the boundary of the discharge platform.

[0090] The embodiment of the present invention obtains a second position deviation between the relative discharge point of the target workpiece and the grasping point of the target workpiece in the drawing of the target workpiece based on the drawing of the target workpiece on which the relative discharge point of the target workpiece is marked. After generating a visual template of the target workpiece based on the drawing of the target workpiece, the second position deviation is written into the visual template of the target workpiece, so that a visual template of the target workpiece carrying the second position deviation can be generated. When controlling the industrial robot to perform a grasping operation, the position information of the relative discharge point of the workpiece to be grasped can be determined more accurately and efficiently when it is determined that the workpiece to be grasped is the target workpiece, so that the workpiece to be grasped can be prevented from exceeding the boundary of the discharge platform when the size of the workpiece is large, and the operation safety of the industrial robot can be improved.

[0091] Based on the contents of the above embodiments, a visual template of the target workpiece is generated based on the drawing of the target workpiece, specifically including: based on the preset angle range corresponding to the target workpiece and the drawing of the target workpiece, a visual template of the target workpiece corresponding to the target angle within the preset angle range is generated.

[0092] Specifically, when the industrial robot is controlled to perform a grasping operation, after obtaining a top view image of the workpiece to be grasped through an image sensor disposed above the loading platform, the matching degree between the visual template of the target workpiece and the top view image of the workpiece to be grasped can be obtained, and when the matching degree is greater than a preset matching degree threshold, the workpiece to be grasped can be determined to be the target workpiece.

[0093] However, usually, the position of the workpiece to be grasped on the loading platform is not fixed, and the rotation angle of the workpiece to be grasped relative to the lower bottom edge of the loading platform is different. In the top view image of the workpiece to be grasped, the rotation angle of the workpiece to be grasped relative to the lower bottom edge of the above top view image is also different. In addition, some workpieces to be grasped are mutually symmetrical and / or similar in shape. When performing target detection based on the visual template generated by the traditional visual template generation method, target detection errors are prone to occur. For example: workpieces A and B are mutually symmetrical and similar in shape. Among them, the workpiece to be grasped is workpiece A, which is placed on the loading platform. However, when performing target detection based on the visual template generated by the traditional visual template generation method, the matching degree between the top view of the workpiece to be grasped and the visual template of workpiece A and the visual template of workpiece B is greater than the preset matching degree threshold, then the workpiece to be grasped may be mistakenly determined as workpiece B.

[0094] In the embodiment of the present invention, the preset angle range corresponding to the target workpiece refers to the angle range in which the target workpiece can be detected based on the visual template of the target workpiece. If the rotation angle of the workpiece to be grasped relative to the lower bottom edge of the loading platform exceeds the above preset angle range, it is impossible to determine that the workpiece to be grasped is the target workpiece based on the visual template of the target workpiece.

[0095] The preset angle range corresponding to the target workpiece can be determined according to the shape of the target workpiece. For example, the preset angle range corresponding to the target workpiece can be 0 to 180°. That is, when the rotation angle of the workpiece to be grasped relative to the lower bottom edge of the loading table exceeds 0 to 180°, it is impossible to determine that the workpiece to be grasped is the target workpiece based on the visual template of the target workpiece.

[0096] Based on preset rules, multiple target angles can be determined within a preset angle range corresponding to the target workpiece. For example, the starting angle of the above preset angle range can be used as the first target angle, and a target angle can be determined every preset step until the end angle of the above preset angle range is reached.

[0097] Based on the drawing of the target workpiece and the preset angle range corresponding to the target workpiece, a visual template of the target workpiece corresponding to each target angle within the preset angle range can be generated by image transformation, edge extraction, etc. Wherein, in the visual template of the target workpiece corresponding to any target angle, the rotation angle of the target workpiece relative to the lower bottom edge of the visual template is the above target angle.

[0098] When controlling the industrial robot to perform grasping operations, a top-view image of the workpiece to be grasped on the loading table can be obtained, and based on the visual template of the target workpiece corresponding to each target angle within the preset angle range corresponding to the target workpiece and the top-view image of the workpiece to be grasped, target detection can be performed on the workpiece to be grasped to obtain the matching degree between the top-view image of the workpiece to be grasped and the visual template of the target workpiece corresponding to each target angle.

[0099] Obtain a visual template of the target workpiece corresponding to the target angle with the highest matching degree with the overhead image of the workpiece to be grasped, and when the above matching degree is greater than a preset matching degree threshold, determine that the workpiece to be grasped is the target workpiece, and determine that the rotation angle of the workpiece to be grasped relative to the lower bottom edge of the loading table is the above target angle.

[0100] The embodiment of the present invention generates a visual template of the target workpiece corresponding to the target angle within the preset angle range based on a drawing of the target workpiece and a preset angle range corresponding to the predetermined target workpiece. This can avoid target detection errors for workpieces that are symmetrical and / or similar in shape, and can perform more accurate target detection based on the visual template.

[0101] Based on the contents of the above embodiments, generating a visual template of the target workpiece based on the drawing of the target workpiece specifically includes: generating a visual template of each preset scale of the target workpiece based on the drawing of the target workpiece.

[0102] Specifically, when controlling the industrial robot to perform a grasping operation, the focal lengths of the image sensors disposed above the loading platform are different, and the scales of the workpiece to be grasped in the top-view image of the workpiece to be grasped are different. In the case where the workpiece to be grasped is the target workpiece, if the scale of the workpiece to be grasped in the top-view image of the workpiece to be grasped is larger or smaller, it may result in a low degree of match between the top-view image of the workpiece to be grasped and the visual template of the target workpiece, resulting in failure to successfully detect that the workpiece to be grasped is the target workpiece.

[0103] In the embodiment of the present invention, a plurality of preset dimensions may be determined based on actual conditions, and a visual template of each preset dimension of the target workpiece may be generated based on a drawing of the target workpiece by image conversion or the like.

[0104] The embodiment of the present invention generates a visual template of each preset scale of the target workpiece based on a drawing of the target workpiece, thereby improving the accuracy of target detection based on the visual template.

[0105] Based on the contents of the above embodiments, a visual template of the target workpiece is generated based on the drawing of the target workpiece, specifically including: based on the drawing of the target workpiece and the preset angle range corresponding to the target workpiece, a visual template of the target workpiece of each preset scale corresponding to the target angle within the preset angle range is generated.

[0106] Specifically, in the embodiment of the present invention, the preset angle range corresponding to the target workpiece can be obtained in advance based on the shape of the target workpiece, and based on the drawing of the target workpiece, the visual template of the target workpiece of each preset scale corresponding to each target angle within the preset angle range can be generated by image transformation, edge extraction, etc. For example: the preset angle range corresponding to the target workpiece is 0-180°, the target angles within the preset angle range are 0°, 1°, 2°, ..., 179° and 180° respectively, and the preset scale includes a first preset scale, a second preset scale and a third preset scale. Then, based on the drawing of the target workpiece and the preset angle range 0-180° corresponding to the target workpiece, a visual template of the target workpiece of the first preset scale corresponding to 1°, a visual template of the target workpiece of the second preset scale corresponding to 1°, a visual template of the target workpiece of the third preset scale corresponding to 1°, and a visual template of the target workpiece of the first preset scale corresponding to 2° can be generated, and by analogy, a visual template of the target workpiece of each preset scale corresponding to each target angle can be obtained.

[0107] The embodiment of the present invention generates a visual template of the target workpiece of each preset scale corresponding to each target angle within the preset angle range based on the drawing of the target workpiece and the preset angle range corresponding to the target workpiece, thereby further improving the accuracy of target detection using the visual template.

[0108] Figure 2 Schematic diagram of the target detection method provided by the present invention. Figure 2 The target detection method of the present invention is described. Figure 2 As shown, the method includes: step 201, obtaining a visual template of the target workpiece; wherein the visual template of the target workpiece is generated based on the visual template generation method as described above; a first position deviation, used to describe the position deviation between the center point of the target workpiece and the grabbing point of the target workpiece.

[0109] It should be noted that the execution subject of the embodiment of the present invention is a target detection device.

[0110] It should be noted that the target detection method in the embodiment of the present invention can be used to detect the target workpiece to be grasped based on the visual template of the target workpiece when controlling the industrial robot to perform a grasping operation, and determine whether the workpiece to be grasped is the target workpiece. When it is determined that the workpiece to be grasped is the target workpiece, the position information of the grasping point of the workpiece to be grasped is further obtained.

[0111] Specifically, before performing target detection on the target to be grasped, a visual template of the target workpiece may be generated based on the visual template generation method in the above embodiments.

[0112] The specific steps of generating the visual template of the target workpiece can be found in the contents of the above embodiments, which will not be described in detail here.

[0113] It should be noted that the visual template of the target workpiece carries a first position deviation, which can be used to describe the position deviation between the center point of the target workpiece and the material grabbing point of the target workpiece.

[0114] Step 202 : performing target detection on the workpiece to be grasped based on the visual template of the target workpiece, and when the workpiece to be grasped is determined to be the target workpiece, determining the position information of the grasping point of the workpiece to be grasped based on the first position deviation carried by the visual template of the target workpiece.

[0115] Specifically, when the industrial robot is controlled to perform a grasping operation, the workpiece to be grasped is located on the loading platform, and a top view image of the workpiece to be grasped can be obtained by an image sensor disposed above the loading platform. Based on the visual template of the target workpiece and the top view image of the workpiece to be grasped, target detection can be performed on the workpiece to be grasped. When the workpiece to be grasped is determined to be the target workpiece, the position information of the center point of the workpiece to be grasped can be obtained based on the top view image of the workpiece to be grasped.

[0116] After the position information of the center point of the workpiece to be grasped is obtained, the position information of the grasping point of the workpiece to be grasped can be obtained based on the first position deviation.

[0117] Based on the position information of the gripping point of the workpiece to be gripped, the industrial robot can be controlled to grip the workpiece to be gripped from the gripping point of the workpiece to be gripped.

[0118] The embodiment of the present invention obtains a visual template of a target workpiece with a first position deviation, and then performs target detection on the workpiece to be grasped based on the visual template of the target workpiece. When the workpiece to be grasped is determined to be the target workpiece, the position information of the grasping point of the workpiece to be grasped is determined based on the first position deviation. The visual template generated by the visual template generation method provided by the present invention can perform more accurate target detection in complex scenes. When controlling an industrial robot to perform a grasping operation, the position information of the grasping point of the workpiece to be grasped can be determined more accurately and efficiently when the workpiece to be grasped is determined to be the target workpiece, thereby improving the operating efficiency of the industrial robot.

[0119] Based on the contents of the above embodiments, after determining the position information of the grabbing point of the workpiece to be grasped based on the first position deviation carried by the visual template of the target workpiece, the above method also includes: determining the position information of the relative discharge point of the workpiece to be grasped based on the second position deviation carried by the visual template of the target workpiece; wherein the second position deviation is used to describe the position deviation between the relative discharge point and the grabbing point of the target workpiece.

[0120] It should be noted that the visual template of the target workpiece carries a second position deviation, which can be used to describe the deviation between the position of the grabbing point of the target workpiece and the position of the relative placing point of the target workpiece.

[0121] Normally, when an industrial robot performs a grasping operation, after grasping the workpiece to be grasped from the grasping point of the workpiece to be grasped, the workpiece to be grasped can be moved to a predetermined preset discharge point to place the workpiece to be grasped, thereby completing the grasping operation. However, if the size of the workpiece to be grasped is large, placing the workpiece to be grasped at the preset discharge point may cause the workpiece to be grasped to exceed the boundary of the discharge platform, which may cause the workpiece to be grasped to fall or damage the boundary of the discharge platform. It is necessary to correct the preset discharge point to determine the position information of the workpiece to be grasped relative to the discharge point.

[0122] Based on the second position deviation carried by the visual template of the target workpiece, the position information of the relative placing point of the workpiece to be grasped can be obtained.

[0123] After obtaining the position information of the relative discharge point of the workpiece to be grasped, the industrial robot can be controlled to place the workpiece to be grasped at the relative discharge point based on the position information of the relative discharge point of the workpiece to be grasped, thereby preventing the workpiece to be grasped from exceeding the boundary of the discharge platform.

[0124] The embodiment of the present invention obtains the position information of the relative discharge point of the workpiece to be grasped by based on the second position offset carried by the visual template of the target workpiece, so as to more accurately and efficiently determine the position information of the relative discharge point of the workpiece to be grasped, avoid the workpiece to be grasped from exceeding the boundary of the discharge platform when the size of the workpiece is large, and improve the operating safety of the industrial robot.

[0125] Figure 3 Schematic diagram of the structure of the visual template generation device provided by the present invention. Figure 3 The visual template generation device provided by the present invention is described. The visual template generation device described below and the visual template generation method provided by the present invention described above can be referred to each other. Figure 3 As shown, the device includes: a drawing acquisition module 301, a deviation determination module 302 and a template generation module 303.

[0126] The drawing acquisition module 301 is used to acquire the drawing of the target workpiece; wherein the grabbing points of the target workpiece are marked in the drawing of the target workpiece.

[0127] The deviation determination module 302 is used to obtain a first position deviation between the material grabbing point and the center point of the target workpiece based on the drawing of the target workpiece.

[0128] The template generation module 303 is used to generate a visual template of the target workpiece based on the drawing of the target workpiece, and write the first position deviation into the visual template of the target workpiece.

[0129] Specifically, a drawing acquisition module 301 , a deviation determination module 302 and a template generation module 303 .

[0130] The drawing acquisition module 301 can acquire the drawing of the target workpiece with the grabbing points of the target workpiece marked in various ways. For example, based on prior knowledge, after marking the grabbing points of the target workpiece on the original drawing of the target workpiece, the drawing of the target workpiece can be acquired through an image sensor or a screenshot tool.

[0131] The deviation determination module 302 can parse the drawing of the target workpiece and determine the position of the grabbing point of the target workpiece and the position of the center point of the target workpiece in the drawing of the target workpiece. After determining the position of the grabbing point of the target workpiece and the position of the center point of the target workpiece in the drawing of the target workpiece, a first position deviation between the grabbing point and the center point can be obtained by numerical calculation.

[0132] The template generation module 303 can generate a visual template of the target workpiece based on the drawing of the target workpiece by image transformation, edge extraction, etc. In the visual template of the target workpiece, the background information is empty. When performing target detection based on the visual template of the target workpiece, the requirements for the scene are not high, and it can be applied to target detection in complex scenes.

[0133] After the template generating module 303 generates the visual template of the target workpiece, the acquired first position deviation may be written into the visual template of the target workpiece.

[0134] Optionally, the deviation determination module 302 may also be used to obtain a second position deviation between the relative placing point and the grabbing point of the target workpiece based on a drawing of the target workpiece.

[0135] Optionally, the template generation module 303 may also be used to write the second position deviation into the visual template of the target workpiece.

[0136] Optionally, the template generating module 303 may also be specifically configured to generate a visual template of the target workpiece corresponding to a target angle within a preset angle range based on a preset angle range corresponding to the target workpiece.

[0137] The template generation module 303 may also be specifically configured to generate a visual template of each preset scale of the target workpiece based on the drawing of the target workpiece.

[0138] The template generation module 303 may also be specifically configured to generate a visual template of the target workpiece of each preset scale corresponding to a target angle within a preset angle range based on the drawing of the target workpiece and the preset angle range corresponding to the target workpiece.

[0139] The embodiment of the present invention obtains a first position deviation between the gripping point of the target workpiece and the center point of the target workpiece in the drawing of the target workpiece based on a drawing of the target workpiece on which the gripping point of the target workpiece has been marked, generates a visual template of the target workpiece based on the drawing of the target workpiece, and writes the above-mentioned first position deviation into the visual template of the target workpiece. The visual template applicable to more scenarios can be generated, and more accurate target detection can be performed in complex scenarios. When controlling an industrial robot to perform a gripping operation, the position information of the gripping point of the workpiece to be gripped can be determined more accurately and efficiently when it is determined that the workpiece to be gripped is the target workpiece, thereby improving the operating efficiency of the industrial robot.

[0140] Figure 4 Schematic diagram of the structure of the target detection device provided by the present invention. Figure 4 The visual template generation device provided by the present invention is described. The visual template generation device described below and the visual template generation method provided by the present invention described above can be referred to each other. Figure 4As shown, the device includes: a template acquisition module 401 and a target detection module 402.

[0141] The template acquisition module 401 is used to acquire the visual template of the target workpiece.

[0142] The target detection module 402 is used to perform target detection on the workpiece to be grasped based on the visual template of the target workpiece. When the workpiece to be grasped is determined to be the target workpiece, the position information of the grasping point of the workpiece to be grasped is determined based on the first position deviation carried by the visual template of the target workpiece.

[0143] The visual template of the target workpiece is generated based on the visual template generation method as described above; the first position deviation is used to describe the position deviation between the center point of the target workpiece and the grabbing point of the target workpiece.

[0144] Specifically, the template acquisition module 401 and the target detection module 402 are electrically connected.

[0145] The template acquisition module 401 generates a visual template of the target workpiece based on the visual template generation method in the above embodiments.

[0146] It should be noted that the template acquisition module 401 may include a visual template generating device.

[0147] When the industrial robot is controlled to perform a grasping operation, the workpiece to be grasped is located on the loading platform, and a top view image of the workpiece to be grasped can be obtained by an image sensor arranged above the loading platform. The target detection module 402 can perform target detection on the workpiece to be grasped based on the visual template of the target workpiece and the top view image of the workpiece to be grasped. When it is determined that the workpiece to be grasped is the target workpiece, the position information of the center point of the workpiece to be grasped can be obtained based on the top view image of the workpiece to be grasped. After obtaining the position information of the center point of the workpiece to be grasped, the position information of the grasping point of the workpiece to be grasped can be obtained based on the first position deviation.

[0148] Optionally, the target detection module 402 can also be used to determine the position information of the relative placement point of the workpiece to be grasped based on the second position deviation carried by the visual template of the target workpiece; wherein the second position deviation is used to describe the position deviation between the relative placement point and the grasping point of the target workpiece.

[0149] The embodiment of the present invention obtains a visual template of a target workpiece with a first position deviation, and then performs target detection on the workpiece to be grasped based on the visual template of the target workpiece. When the workpiece to be grasped is determined to be the target workpiece, the position information of the grasping point of the workpiece to be grasped is determined based on the first position deviation. The visual template generated by the visual template generation method provided by the present invention can perform more accurate target detection in complex scenes. When controlling an industrial robot to perform a grasping operation, the position information of the grasping point of the workpiece to be grasped can be determined more accurately and efficiently when the workpiece to be grasped is determined to be the target workpiece, thereby improving the operating efficiency of the industrial robot.

[0150] Figure 5 This is one of the structural diagrams of the robot system provided by the present invention. Figure 5 The robot system provided by the present invention is described. Figure 5 As shown, the robot system includes: the target detection device 501 as described above, an industrial robot 502 and an image sensor 503.

[0151] Specifically, the industrial robot 502 may be fixed on a base 504 .

[0152] Optionally, the industrial robot 502 is an industrial robot that can perform grasping operations. The industrial robot 502 can be a six-axis industrial robot, a truss arm, or a four-axis industrial robot, etc. The specific type of the industrial robot 502 is not limited in the embodiment of the present invention.

[0153] An image sensor 503 is provided at the end of the mechanical arm of the industrial robot 502. The loading platform 505 and the unloading platform 507 are arranged within the arm span range of the mechanical arm of the industrial robot 502, and the mechanical arm of the industrial robot 502 drives the image sensor 503 to move directly above the loading platform 505. The image sensor 503 can be used to obtain a top view image of a workpiece 506 to be grasped placed on the loading platform 505.

[0154] The image sensor 503 may be a line laser sensor or other types of image sensors. The specific type of the image sensor 503 is not limited in the embodiment of the present invention.

[0155] It should be noted that the image sensor 503 is connected to the target detection device 501. The target detection device 501 is connected to the industrial robot 502. The installation position of the target detection device 501 can be determined according to actual conditions. Figure 5 The installation position of the object detection device 501 shown is only an example.

[0156] Figure 6 This is the second structural diagram of the robot system provided by the present invention. Figure 6As shown, the industrial robot 502 performs the grasping action based on the PLC program. The target detection device 501 can perform target detection on the workpiece to be grasped based on the visual template provided by the visual template generation device 601, and provide control data to the industrial robot 502. The image sensor 503 provides hardware support for the input of image data of the workpiece to be grasped.

[0157] It should be noted that the robot system in the embodiment of the present invention does not include the visual template generating device 601.

[0158] The specific process of the target detection device 501 detecting the target workpiece 506 based on the visual template of the target workpiece, and the specific process of the visual template generation device 601 generating the visual template of the target workpiece can be referred to the contents of the above embodiments, which will not be repeated in the embodiments of the present invention.

[0159] The robot system in the embodiment of the present invention includes a target detection device, an industrial robot and an image sensor. After acquiring a visual template of a target workpiece carrying a first position deviation, target detection is performed on the workpiece to be grasped based on the visual template of the target workpiece. When it is determined that the workpiece to be grasped is the target workpiece, the position information of the grasping point of the workpiece to be grasped is determined based on the first position deviation. The visual template generated by the visual template generation method provided by the present invention can be used to perform more accurate target detection in complex scenes. When controlling the industrial robot to perform grasping operations, when it is determined that the workpiece to be grasped is the target workpiece, the position information of the grasping point of the workpiece to be grasped can be determined more accurately and efficiently, thereby improving the operating efficiency of the industrial robot.

[0160] Based on the contents of the above embodiments, the robot system further includes: the visual template generating device 601 as described above.

[0161] The robot system in the embodiment of the present invention also includes a visual template generating device, which can obtain the visual template of the target workpiece more simply and efficiently, and can improve the operating efficiency of the industrial robot.

[0162] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor (processor) 710, a communication interface (Communications Interface) 720, a memory (memory) 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call the logic instructions in the memory 730 to execute the visual template generation method, which includes: obtaining a drawing of the target workpiece; wherein the gripping point of the target workpiece has been marked in the drawing of the target workpiece; based on the drawing of the target workpiece, obtaining a first position deviation between the gripping point and the center point of the target workpiece; based on the drawing of the target workpiece, generating a visual template of the target workpiece, and writing the first position deviation into the visual template of the target workpiece; and a target detection method, which includes: obtaining a visual template of the target workpiece; performing target detection on the workpiece to be gripped based on the visual template of the target workpiece, and when it is determined that the workpiece to be gripped is the target workpiece, determining the position information of the gripping point of the workpiece to be gripped based on the first position deviation carried by the visual template of the target workpiece; wherein the visual template of the target workpiece is generated based on the visual template generation method as described above; the first position deviation is used to describe the position deviation between the center point of the target workpiece and the gripping point of the target workpiece.

[0163] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0164] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the visual template generation method provided by the above-mentioned methods, which includes: obtaining a drawing of a target workpiece; wherein the grabbing point of the target workpiece is marked in the drawing of the target workpiece; based on the drawing of the target workpiece, obtaining a first position deviation between the grabbing point and the center point of the target workpiece; based on the drawing of the target workpiece, generating a visual template of the target workpiece, and writing the first position deviation into the visual template of the target workpiece; and a target detection method, which includes: obtaining a visual template of the target workpiece; performing target detection on the workpiece to be grasped based on the visual template of the target workpiece, and when it is determined that the workpiece to be grasped is the target workpiece, determining the position information of the grabbing point of the workpiece to be grasped based on the first position deviation carried by the visual template of the target workpiece; wherein the visual template of the target workpiece is generated based on the visual template generation method as described above; the first position deviation is used to describe the position deviation between the center point of the target workpiece and the grabbing point of the target workpiece.

[0165] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the visual template generation method provided by the above-mentioned methods, the method comprising: obtaining a drawing of a target workpiece; wherein the grabbing point of the target workpiece is marked in the drawing of the target workpiece; based on the drawing of the target workpiece, obtaining a first position deviation between the grabbing point and the center point of the target workpiece; based on the drawing of the target workpiece, generating a visual template of the target workpiece, and writing the first position deviation into the visual template of the target workpiece; and a target detection method, the method comprising: obtaining a visual template of the target workpiece; performing target detection on the workpiece to be grasped based on the visual template of the target workpiece, and determining the position information of the grabbing point of the workpiece to be grasped based on the first position deviation carried by the visual template of the target workpiece when determining that the workpiece to be grasped is the target workpiece; wherein the visual template of the target workpiece is generated based on the visual template generation method as described above; the first position deviation is used to describe the position deviation between the center point of the target workpiece and the grabbing point of the target workpiece.

[0166] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0167] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a visual template, characterized in that: include: Obtaining a drawing of a target workpiece; wherein the drawing of the target workpiece has marked the material grabbing point of the target workpiece; Based on the drawing of the target workpiece, obtaining a first position deviation between the grabbing point and the center point of the target workpiece; Based on the drawing of the target workpiece, a visual template of the target workpiece is generated, and the first position deviation is written into the visual template of the target workpiece, so as to determine the position information of the grabbing point of the target workpiece based on the first position deviation carried by the visual template of the target workpiece. The visual template is used to perform target detection on the target workpiece and determine the type of the target workpiece. When controlling the industrial robot to perform grabbing operations, the type of the target workpiece, the placement direction and the position information of the grabbing point of the target workpiece can be determined based on the visual template.

2. The method for generating a visual template according to claim 1, characterized in that: The drawing of the target workpiece has marked the relative discharge point of the target workpiece; Accordingly, after obtaining the drawing of the target workpiece, the method further includes: Based on the drawing of the target workpiece, obtaining a second position deviation between the relative material placing point and the material grabbing point; Correspondingly, after generating the visual template of the target workpiece based on the drawing of the target workpiece, the method further includes: The second position deviation is written into the vision template of the target workpiece.

3. The method for generating a visual template according to claim 1 or 2, characterized in that: The generating the visual template of the target workpiece based on the drawing of the target workpiece specifically includes: Based on the preset angle range corresponding to the target workpiece, a visual template of the target workpiece corresponding to the target angle within the preset angle range is generated.

4. The method for generating a visual template according to claim 1 or 2, characterized in that: The generating the visual template of the target workpiece based on the drawing of the target workpiece specifically includes: Based on the drawing of the target workpiece, a visual template of each preset scale of the target workpiece is generated.

5. The method for generating a visual template according to claim 4, characterized in that: The step of generating a visual template of each preset scale of the target workpiece based on the drawing of the target workpiece specifically includes: Based on the drawing of the target workpiece and the preset angle range corresponding to the target workpiece, a visual template of the target workpiece of each preset scale corresponding to the target angle within the preset angle range is generated.

6. A target detection method, characterized in that: include: Obtaining a visual template of a target workpiece; Performing target detection on the workpiece to be grasped based on the visual template of the target workpiece, and determining the position information of the grasping point of the workpiece to be grasped based on the first position deviation carried by the visual template of the target workpiece when the workpiece to be grasped is determined to be the target workpiece; Wherein, the visual template of the target workpiece is generated based on the visual template generation method as described in any one of claims 1 to 5; the first position deviation is used to describe the position deviation between the center point of the target workpiece and the grabbing point of the target workpiece.

7. The target detection method according to claim 6, characterized in that: After determining the position information of the gripping point of the workpiece to be gripped based on the first position deviation carried by the visual template of the target workpiece, the method further includes: Determining position information of the workpiece to be grasped relative to a material placement point based on a second position deviation carried by the visual template of the target workpiece; The second position deviation is used to describe the position deviation between the relative discharge point of the target workpiece and the grabbing point.

8. A visual template generating device, characterized in that: include: A drawing acquisition module is used to acquire a drawing of a target workpiece, wherein the drawing of the target workpiece has marked the material grabbing point of the target workpiece; a deviation determination module, configured to obtain a first position deviation between the material grabbing point and a center point of the target workpiece based on a drawing of the target workpiece; A template generation module is used to generate a visual template of the target workpiece based on the drawing of the target workpiece, and write the first position deviation into the visual template of the target workpiece so as to determine the position information of the grasping point of the target workpiece based on the first position deviation carried by the visual template of the target workpiece. The visual template is used to perform target detection on the target workpiece and determine the type of the target workpiece. When controlling the industrial robot to perform grasping operations, the type of the target workpiece, the placement direction and the position information of the grasping point of the target workpiece can be determined based on the visual template.

9. A target detection device, characterized in that: include: A template acquisition module is used to acquire the visual template of the target workpiece; A target detection module is used to perform target detection on the workpiece to be grasped based on the visual template of the target workpiece, and when it is determined that the workpiece to be grasped is the target workpiece, determine the position information of the grasping point of the workpiece to be grasped based on the first position deviation carried by the visual template of the target workpiece; Wherein, the visual template of the target workpiece is generated based on the visual template generation method as described in any one of claims 1 to 5; the first position deviation is used to describe the position deviation between the center point of the target workpiece and the grabbing point of the target workpiece.

10. A robot system, characterized in that: include: The target detection device, industrial robot and image sensor as claimed in claim 9.

11. The robot system according to claim 10, characterized in that: Also includes: The visual template generating device as claimed in claim 8.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the visual template generation method according to any one of claims 1 to 5 is implemented, or the steps of the target detection method according to claim 6 or 7 are implemented.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the visual template generation method according to any one of claims 1 to 5, or the steps of the target detection method according to claim 6 or 7.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the visual template generation method according to any one of claims 1 to 5, or the steps of the target detection method according to claim 6 or 7.