Identification, positioning and grasping device and method for robot grasping object operation

Abstract

The present invention discloses an identification, positioning and grasping device and method for robot grasping object operation, including a mark set on the object to be grasped, wherein the mark on the target object to be grasped is a target mark; further comprising: a first visual system; a first ranging motion system; a second visual system; a second ranging motion system; wherein the second visual system searches for and reads the target mark and the calibration mark in the second visual area, and when the calibration mark and the target mark coordinate data coincide, the second ranging motion system sends a grasping instruction, wherein the grasping instruction is used to grasp the target object. The present invention uses a two-dimensional code to assist the robot in identifying, positioning and storing grasping codes, and has the remarkable characteristics of being simple to use, technically feasible and cost-effective.

Description

Technical Field

[0001] The present invention relates to robot grasping objects, and in particular to an identification, positioning and grasping device and method for the robot grasping object operation. Background Art

[0002] Human-machine interaction is one of the key technologies of robots. Robots need to automatically identify different people and objects and accurately locate them in order to further realize interactive actions. From the current technical point of view, robots have relatively mature solutions for face recognition, but they cannot effectively recognize objects, especially products with highly consistent appearance produced through industrialization. Recognition here refers to the identification of objects including two objects with completely consistent physical parameters. On the other hand, the robot's grasping of objects is a basic function of robot-environment interaction. When the robot grasps an object, it needs to determine the three-dimensional structure and starting coordinate point of the object, and then determine the grasping method through the algorithm to implement effective interactive behavior. Industrial robots generally use manual assistance methods to determine the starting point of the coordinates, and then manually input the processing program. Humanoid robots are multi-task robots, and the starting coordinate points when performing different tasks need to be automatically determined. At present, when determining the starting point of the coordinates, humanoid robots mostly use stereoscopic vision systems including 2D or 3D imaging systems. First, real-time three-dimensional measurements of objects are performed within a certain accuracy range, and then the starting point of the coordinates is determined according to the algorithm. This method is very technically difficult and costly, which exceeds the cost-effective range of service robots for families. Summary of the invention

[0003] In order to solve the defects of the above-mentioned prior art, the present invention provides an identification, positioning and grasping device and method for robot grasping object operations, and proposes to use QR code to assist robot identification, positioning and grasping, which has the remarkable characteristics of simple use, technical feasibility and high cost performance.

[0004] To achieve the above technical objectives, the present invention adopts the following technical solutions: an identification, positioning and grasping device for robot grasping object operation, comprising a mark provided on the object, wherein the mark on the target object to be grasped is a target mark;

[0005] Also includes:

[0006] The first visual system is used to visually scan the area where the target is located to form a first visual area, and after finding the target in the first visual area, send a first ranging instruction to the first ranging motion system;

[0007] The first ranging motion system is used to receive the first ranging instruction, measure the distance c from the robot's visual origin to the target mark, and guide the robot to the front position of the target mark according to the distance, wherein the robot's hand is provided with a calibration mark;

[0008] The second visual system is used to visually scan the target mark at the front position and form a second visual area, and send a second ranging instruction to the second ranging motion system;

[0009] The second ranging motion system is used to receive the second ranging instruction, measure the horizontal angle ε and the vertical angle λ from the robot's visual origin to the target mark, and guide the robot's visual origin to align with the target mark;

[0010] Calculating the coordinate data of the target mark, acquiring the coordinate data of the calibration mark, and guiding the robot's hand movement to make the coordinate data of the calibration mark coincide with the coordinate data of the target mark;

[0011] When the coordinate data of the calibration mark and the target mark coincide, it indicates that the finger is located at the starting position for grasping, and the second ranging motion system sends a grasping instruction, wherein the grasping instruction is used to control the robot arm to grasp the target object.

[0012] Furthermore, the first ranging motion system is also used for:

[0013] Sending a first visual adjustment instruction to the first visual system, wherein the first visual system adjusts the focal length of the visual camera according to the first visual adjustment instruction;

[0014] And, get the coordinate data of the robot's visual origin .

[0015] Furthermore, the first visual system includes an image sensor, which is located at the central axis of the head and can rotate synchronously with the head;

[0016] The first visual system is also used for:

[0017] The head is rotated to drive the image sensor to find the target mark and read the target mark.

[0018] Furthermore, the second ranging motion system is also used for:

[0019] Sending a second visual adjustment instruction to the second visual system, wherein the second visual system adjusts the focal length of the visual camera according to the second visual adjustment instruction;

[0020] And, obtaining the horizontal angle ε and the vertical angle λ from the robot vision origin to the target mark, and obtaining the coordinate data of the calibration mark ,according to , , Calculate the coordinate data of the target marker ;

[0021] And, according to , , The distance that the calibration mark needs to move is calculated, and the calibration mark is guided to the target mark until the calibration mark coincides with the target mark.

[0022] Furthermore, a positioning line is provided on the target mark;

[0023] A positioning line is provided on the calibration mark;

[0024] The center of the image of the image sensor is provided with a positioning line;

[0025] The coincidence of the calibration mark and the target mark means that the positioning line of the target mark coincides with the positioning line of the calibration mark.

[0026] Furthermore, the target mark is directly engraved on the object, or indirectly provided on the object;

[0027] The target mark adopts a two-dimensional code, and the positioning line on the target mark adopts a cross line.

[0028] Further, it includes a mark set on the object, wherein the mark on the target object to be grasped is the target mark, and the center point of the crosshairs of the target mark is the starting point of the action coordinates of the robot grasping the object;

[0029] The method comprises the following steps:

[0030] A first visual detection step: visually scanning the area where the target is located to form a first visual area, and after finding the target in the first visual area, sending a first ranging instruction to the first ranging motion system;

[0031] The first ranging movement step: receiving the first ranging instruction, measuring the distance c from the robot vision origin to the target mark, and automatically planning the robot movement path according to the distance and position coordinates, wherein the front position refers to the front position where the horizontal distance from the robot vision origin to the target mark is less than the horizontal distance from the robot vision origin to the robot hand calibration mark, wherein the calibration mark is set on the robot hand;

[0032] A second visual detection step: visually scanning the target mark at the front position to form a second visual area, and sending a second ranging instruction to a second ranging motion system;

[0033] The second ranging movement step: receiving the second ranging instruction, and measuring the horizontal angle ε and the vertical angle λ from the robot's visual origin to the target mark;

[0034] Calculating the coordinate data of the target mark, acquiring the coordinate data of the calibration mark, and controlling the movement of the robot arm to guide the calibration mark to coincide with the coordinate data of the target mark;

[0035] When the coordinate data of the calibration mark and the target mark coincide, the second ranging motion system determines through the ranging data that the distance is consistent and then sends a grabbing instruction, wherein the grabbing instruction is used to grab the target object.

[0036] Furthermore, the first ranging movement step includes the following sub-steps:

[0037] Sending a first visual adjustment instruction to the first visual system, wherein the first visual system adjusts the focal length of the visual camera according to the first visual adjustment instruction;

[0038] And, get the coordinate data of the robot's visual origin .

[0039] Furthermore, the first visual system includes an image sensor and a laser distance measuring unit, the image sensor and the laser distance measuring unit are located on the same horizontal line and arranged in parallel at the central axis of the head and can rotate synchronously with the head;

[0040] The first visual inspection step includes the following sub-steps:

[0041] The head is rotated to drive the image sensor to search for the target mark. After matching the target object, the head is controlled to rotate so that the center of the image sensor is aligned with the center of the target mark to read the target mark.

[0042] Furthermore, the second ranging motion step includes the following sub-steps:

[0043] Sending a second visual adjustment instruction to the second visual system, wherein the second visual system adjusts the focal length of the visual camera according to the second visual adjustment instruction;

[0044] Furthermore, the angle sensor obtains the horizontal angle ε and the vertical angle λ from the robot vision origin to the target mark, and obtains the coordinate data of the calibration mark. ,according to , , Calculate the coordinate data of the target marker ;

[0045] And, according to , , Calculating the distance that the calibration mark needs to move, and guiding the calibration mark to the target mark until the calibration mark coincides with the target mark;

[0046] Wherein, a positioning line is provided on the target mark;

[0047] A positioning line is provided on the calibration mark;

[0048] The image center of the image sensor is the center positioning line of the visual origin;

[0049] The coincidence of the calibration mark and the target mark means that the positioning line of the target mark coincides with the positioning line of the calibration mark.

[0050] In summary, the present invention has achieved the following technical effects:

[0051] After the robot completes learning and establishes a basic task area, the robot will first determine the coordinates of the area where the object is located or directly determine the coordinates of the object after receiving the task instruction, approach the coordinates of the object with the cooperation of the robot motion system, and perform a visual scan on the target area. After reading the target QR code, the laser positioning and ranging are first completed, and the focal length of the visual camera is adjusted with this ranging information. After completing the precise positioning, the position coordinate information of the robot relative to the target can be obtained to guide the robot's hand to move to the target position. Since each servo link of the robot will cause cumulative errors of varying degrees, it is necessary to correct and guide the hand movement in real time to ensure that the robot's hand accurately reaches the point of action. This process is completed by the video image analysis program. The system defines that the center position of the crosshairs of the QR code is the coordinate starting point of the robot grabbing the object, and the physical center point on the robot's thumb (also marked with a crosshair) is the robot's action calibration point. When these two cross coordinate lines are within the video screen, the robot's visual processing software can generate accurate guidance instructions to control the thumb to reach the starting point of the action position. Point; the present invention includes a first robot vision system, a first ranging motion system, a robot-specific two-dimensional code, a second vision system, a second ranging motion system and a knowledge accumulation and storage system. The first vision system and the second vision system control the image sensor and the laser ranging unit to achieve search and positioning; the first ranging motion system and the second ranging motion system control the laser radar and the laser ranging unit to achieve real-time positioning, path planning and obstacle avoidance of the robot; the robot-specific two-dimensional code is composed of a two-dimensional code with a cross coordinate mark, which is used to assist the machine vision system to identify and locate the object; the knowledge accumulation and storage system is a storage device for instructions, objects, control processes, and necessary conditions after the robot completes each task; this patent can assist the robot to complete the identification and positioning of the object by using a robot-specific two-dimensional code, and uses the characteristic that the two-dimensional code can store information to pre-store the three-dimensional feature information code of the object in the two-dimensional code. After the robot reads the two-dimensional code, there is no need to establish a grasping strategy through three-dimensional scanning. The grasping program can be directly obtained through the two-dimensional code to complete the grasping action. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the robot searching for a target object;

[0053] Figure 2 It is a schematic diagram of the coordinates of the robot's visual origin and the target object at the initial time;

[0054] Figure 3 is a schematic diagram of the path the robot is looking for;

[0055] Figure 4 It is a schematic diagram of the robot arm coordinates;

[0056] Figure 5 It is a schematic diagram of the crosshairs of the visual center moving to the target object;

[0057] Figure 6 It is a schematic diagram of the overlap between the visual center crosshairs and the target object crosshairs;

[0058] Figure 7 It is a schematic diagram of the overlap of the visual center crosshairs, the target object crosshairs, and the finger crosshairs;

[0059] Figure 8 It is a schematic diagram of the robot's own coordinates. DETAILED DESCRIPTION

[0060] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0061] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

[0062] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0063] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0064] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0065] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0066] Example:

[0067] The identification, positioning and grasping device for robot grasping object operation, the robot described in the present invention is a humanoid robot, which is used in the service industry, such as helping the elderly to pick up cups at home, etc. It includes a mark set on the object, wherein a mark is set on each object as the target for the robot to grasp, and the mark on the target object to be grasped is the target mark, which is the target point for grasping. It includes an image sensor set on the head of the robot, the image sensor is used as a part of the visual system (described below), and the image center of the image sensor is the visual origin, that is, the origin of the cross positioning line of the image, which is used to accurately locate the target object during visual inspection. It includes a cross calibration mark set on the hand of the robot. In this embodiment, the calibration mark is set on the thumb of the hand. The calibration mark is used to calibrate the object. When the calibration mark and the target mark coincide or by learning a known starting position, it is the coordinate position of the robot hand at the starting point of the grasping action. The present invention proposes a device and method for locating a target object by hand.

[0068] Before introducing each system, let's first introduce the items that the robot grabs. One or more marks are set on each item. The marks are barcodes, two-dimensional codes, or any other marks that can be scanned and read. In this embodiment, a two-dimensional code is used, and a positioning line is set on each mark. The center of the positioning line is also the starting point of the coordinates of the robot acting on the item. In this embodiment, a crosshair is set at the center of the two-dimensional code as a positioning line. In addition, marks such as two-dimensional codes can be directly engraved on the items, such as directly engraved on cups and chairs, or prefabricated into specific special labels for use with existing items, such as making an elastic bottle sleeve with the name of the product, which can be put on an ordinary bottle with the same specifications for robot identification. In addition, the reading of marks such as two-dimensional codes, the calculation of the center point of the two-dimensional code pattern, the calculation of the effective distance of the two-dimensional code reading, the calculation and correction of the deflection angle of the two-dimensional code image, and the positioning of positioning lines such as crosshairs are all existing mature technologies. They are directly adopted in the present invention without making any improvements, and will not be repeated here.

[0069] At the same time, the QR code used in this embodiment is a robot-specific QR code, which is composed of a QR code with a cross coordinate mark. The QR code adopts a unified data format and contains information including the name code of the item, a three-dimensional feature data code and a robot grasping operation instruction code. The three-dimensional feature data is obtained according to design parameters or actual measurements of the product when designing robot-specific items. Therefore, it will be more accurate than any data obtained from secondary measurements. The robot grasping operation instructions are also pre-verified and optimized action instructions for the robot to perform grasping tasks. Not only can the design difficulty of the robot vision system be reduced, but also the computing power and storage space of the robot control system can be greatly reduced, thereby ensuring the accuracy and reliability of the grasping action.

[0070] In this embodiment, an example is given in which a robot grabs a cup, a QR code is on the cup, and a crosshair is on the QR code.

[0071] The present invention comprises:

[0072] (1) First visual system: used to visually scan the area where the target is located to form a first visual area, and after finding the target in the first visual area, send a first ranging instruction to the first ranging motion system.

[0073] The first visual system includes an image sensor and a laser distance measuring unit, which are used to achieve the search. The image sensor and the laser distance measuring unit are located on the same horizontal line and arranged in parallel at the central axis of the head and can rotate synchronously with the head. The first visual system is also used for:

[0074] Turning the head drives the image sensor to find the target mark and read the target mark.

[0075] In this embodiment, when the humanoid robot receives a task instruction, it can automatically analyze the search direction, and control the robot to move to the task area through the motion module, turn its head to drive the image sensor to search for the target object in the task area, and when the robot's main sensor matches the target object, it reads the QR code and then reads the information of the target object and the angle and force required to grasp the target object.

[0076] Applied to this embodiment, Figure 1 As shown, the robot uses the image sensor to scan the area where the cup is located, rotates its head so that the center of the crosshairs of the QR code on the cup coincides with the center of the image (the robot's visual origin), and reads the QR code to obtain the cup code. Through this code, the associated information stored in the robot control system can be retrieved, including the state, the items in the cup, the weight, shape, the force and procedure required for grasping, etc.

[0077] The coordinate analysis and positioning tracking of stationary objects through video images is an existing mature technology, which is directly adopted in the present invention without making any improvements thereto and will not be described in detail here.

[0078] (2) Also includes a first distance measurement motion system: used to receive a first distance measurement instruction, measure a straight-line distance c from the robot's visual origin to the target mark, and guide the robot to the front position of the target mark according to the distance, wherein the robot's hand is provided with a calibration mark;

[0079] The first ranging motion system controls the laser radar and laser ranging unit to work, and is used to:

[0080] Sending a first vision adjustment instruction to the first vision system, wherein the first vision system adjusts the focal length of the vision camera according to the first vision adjustment instruction to accurately locate the position of the target object, and after accurate positioning, the laser ranging unit starts the ranging operation;

[0081] And, get the coordinate data of the robot's visual origin , and use this coordinate as the coordinate origin of the entire system.

[0082] And, according to Several travel routes from the robot's visual origin to the target mark are calculated, and an optimal one is selected as the target route, wherein the target route is used to guide the robot to the front position.

[0083] In this embodiment, Figure 2 As shown in FIG. 1 , after knowing the coordinate data of the robot vision origin, the straight-line distance c from the robot vision origin to the cup is measured, and the positions of the two are combined, and then as shown in FIG. Figure 3As shown in the figure, several routes from the robot's visual origin to the cup are obtained. In the step of obtaining several routes, the laser radar completes the scanning of the indoor environment and establishes path planning. It can calibrate and store specific position points during the system learning stage to enable the robot to quickly and accurately execute motion instructions, while the laser rangefinder can assist the robot in achieving more accurate positioning. The system automatically matches the optimal route, and the first ranging motion system guides the robot to the front position.

[0084] In the present invention, the robot is guided to a front position. A method for determining the front position is also given in this embodiment, that is, the surface of the object with a QR code is the front side (symmetrical objects can also have multiple identical QR codes on different symmetrical surfaces), and the position on the front side that is a certain distance away from the object is the front position. Within this distance range, the robot's arm can reach the target object after moving.

[0085] (3) Also includes a second visual system: used for visually scanning the target mark at a front position and forming a second visual area, and sending a second ranging instruction to the second ranging motion system.

[0086] In this embodiment, there is a reasonable area at the front position, and any position within this area satisfies the robot arm's ability to reach the target object, i.e., the cup, after movement. Therefore, when actually guiding the movement, it is sufficient as long as the robot's visual origin is within this area. Figure 3 In the figure, a positive position point is selected in the area, which is the position of the triangle in the figure. At this position, the second vision system performs a second scan and places the target object in the second vision area. Figure 4 The figure shows a schematic diagram of the second visual area. At this time, the head turns so that the target object is in the second visual area. However, at this time, the robot's hand may be located in the second visual area or outside the second visual area. Figure 4 As shown, in short, there is a certain distance between the hand and the target object. Of course, the hand may just be located at the edge of the target object, and this situation is also included in the embodiment of the present invention.

[0087] The second ranging motion system is used to receive the second ranging instruction, measure the horizontal angle ε and vertical angle λ from the robot's visual origin to the target mark, and guide the robot's visual origin to align with the target mark; because the robot's visual area will change during the movement to the front position, the position of the crosshairs at the center of its image will not always coincide with the target position. Therefore, after the robot stops, the crosshairs whose visual origin is the center of the image are first aligned with the crosshairs of the target.

[0088] Then, the coordinate data of the target mark is calculated, the coordinate data of the calibration mark is obtained, and the robot's hand movement is guided to make the calibration mark coincide with the coordinate data of the target mark or move to the starting position point known through learning;

[0089] When the calibration mark reaches the starting position, the second ranging motion system sends a grabbing instruction, wherein the grabbing instruction is used to grab the target object;

[0090] The second distance measuring motion system includes a laser distance measuring unit and an angle sensor, which are located at the central axis of the head and can rotate synchronously with the head;

[0091] The second range-finding motion system is also used for:

[0092] Sending a second vision adjustment instruction to the second vision system, wherein the second vision system adjusts the focal length of the vision camera according to the second vision adjustment instruction;

[0093] In addition, the angle sensor obtains the horizontal angle ε and vertical angle λ from the robot's visual origin to the target mark, and obtains the coordinate data of the calibration mark ,according to , , Calculate the coordinate data of the target marker ;

[0094] And, according to , , Calculate the distance the calibration mark needs to move, and guide the calibration mark to the target mark until the calibration mark coincides with the target mark.

[0095] In this embodiment, if Figure 2 As shown in Figure 2, after knowing the horizontal angle ε and vertical angle λ and distance c from the robot vision origin to the target mark, the coordinates of the target mark can be calculated. , the robot guides the hand along Figure 2 The direction of the arrow is Movement to cup When the cross calibration mark on the contact surface of the thumb of the hand contacts and coincides with the cross line of the QR code on the cup or moves to the starting position known through learning, the positioning operation is completed, and then the robot controls the hand to grab the cup.

[0096] In this process, it can be simplified as follows: Figure 5 As shown in the second visual area, turn the head so that the visual center crosshairs move to the crosshairs of the target object. Figure 6As shown, the visual center crosshairs and the crosshairs of the target object coincide with each other, and then the crosshairs of the finger are moved to the crosshairs of the target object or to the starting position known through learning, such as Figure 7 As shown, after the movement, the crosshairs of the visual center, the crosshairs of the target object, and the crosshairs of the finger coincide with each other.

[0097] In addition, in the present invention, when guiding the robot hand, an arm motion system is also included: used to move the robot hand to a specified position, specifically, Figure 8 As shown, it is used to obtain arm shoulder joint coordinate data in real time , elbow joint coordinate data , wrist joint coordinate data , including the robot's hand movements. The control program for each joint of the robot arm is consistent with the control method of the industrial robot, which is an existing or achievable technology. The present invention can be directly adopted or completed after actual measurement in the design stage of the robot-specific item, and will not be repeated here.

[0098] (5) It also includes a knowledge accumulation and storage module: a module that remembers and saves instructions, objects, control processes, and necessary conditions after completing each task. After completing an instruction action, the robot automatically records and saves the final position coordinates of the object, as well as related information such as weight and temperature, as well as whether it is a one-handed or two-handed operation, and the force parameters during the operation (pressure sensor parameters and current detection values, etc.). The original data can be accumulated during the learning process, and can also be automatically accumulated and corrected during the later application process.

[0099] In another embodiment, a method for identifying, locating and grasping an object for a robot grasping operation is provided, comprising a mark provided on an object, wherein the mark on the target object to be grasped is a target mark;

[0100] The method comprises the following steps:

[0101] Reach the designated area through instructions.

[0102] S100: first visual detection step: visually scanning the area where the target is located to form a first visual area, and after finding the target in the first visual area, sending a first ranging instruction to the first ranging motion system;

[0103] S110 The first visual inspection step includes the following sub-steps:

[0104] Turn the head to drive the image sensor to search for the target mark. After matching the target object, control the head to rotate so that the center of the image sensor is aligned with the center of the target mark to read the target mark.

[0105] S200: first distance measurement movement step: receiving a first distance measurement instruction, measuring a distance c from the robot visual origin to the target mark, and guiding the robot to a front position of the target mark according to the distance, wherein the front position refers to a front position where a horizontal distance from the robot visual origin to the target mark is less than a horizontal distance from the robot visual origin to a calibration mark of the robot hand, wherein the calibration mark is set on the robot hand;

[0106] S200 The first ranging motion step includes the following sub-steps:

[0107] S210 sends a first vision adjustment instruction to the first vision system, wherein the first vision system adjusts the focal length of the vision camera according to the first vision adjustment instruction;

[0108] S220 obtains the coordinate data of the robot's visual origin ;

[0109] S230 calculates several travel routes for the robot's visual origin to reach the target mark according to c, and selects an optimal route as the target route, wherein the target route is used to guide the robot to the front position.

[0110] S300: second visual detection step: visually scanning the target mark at the front position to form a second visual area, and sending a second ranging instruction to the second ranging motion system;

[0111] S400: second distance measurement movement step: receiving a second distance measurement instruction, measuring a horizontal angle ε and a vertical angle λ from the robot's visual origin to the target mark, and guiding the robot's visual origin to align with the target mark;

[0112] Calculate the coordinate data of the target mark, obtain the coordinate data of the calibration mark, and guide the robot's hand movement so that the coordinate data of the calibration mark coincides with the coordinate data of the target mark or moves to the starting position known through learning;

[0113] When the calibration mark reaches the specified starting position, the second ranging motion system sends a grabbing instruction, wherein the grabbing instruction is used to grab the target object;

[0114] S400 The second ranging movement step includes the following sub-steps:

[0115] S410 sends a second visual adjustment instruction to the second visual system, wherein the second visual system adjusts the focal length of the visual camera according to the second visual adjustment instruction;

[0116] The S420 angle sensor obtains the horizontal angle ε and vertical angle λ from the robot's visual origin to the target mark, as well as the coordinate data of the calibration mark. ,according to , , Calculate the coordinate data of the target marker ;

[0117] S430 according to , , Calculate the distance the calibration mark needs to move, and guide the calibration mark to the target mark until the calibration mark coincides with the target mark; Figure 2 As shown, the finger position is , the target location is , move your finger to the target position along the direction of the arrow in the figure.

[0118] Among them, a positioning line is provided on the target mark;

[0119] A positioning line is provided on the calibration mark;

[0120] The center of the image of the image sensor is provided with a positioning line;

[0121] The coincidence of the calibration mark and the target mark means that the positioning line of the target mark coincides with the positioning line of the calibration mark.

[0122] The above are only preferred implementation modes of the present invention, and are not intended to limit the present invention in any form. Any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are within the scope of the technical solution of the present invention.

Claims

1. An identification, positioning and grasping device for robot grasping object operation, characterized in that: It includes a mark provided on the object, wherein the mark on the target object to be grasped is a target mark; Also includes: The first visual system is used to visually scan the area where the target is located to form a first visual area, and after finding the target in the first visual area, send a first ranging instruction to the first ranging motion system; The first ranging motion system is used to receive the first ranging instruction, measure the distance c from the robot's visual origin to the target mark, and guide the robot to the front position of the target mark according to the distance, wherein the robot's hand is provided with a calibration mark; The second visual system is used to visually scan the target mark at the front position and form a second visual area, and send a second ranging instruction to the second ranging motion system; The second ranging motion system is used to receive the second ranging instruction, measure the horizontal angle ε and the vertical angle λ from the robot's visual origin to the target mark, and guide the robot's visual origin to align with the target mark; As the robot moves to the front position, its visual area will change, and the position of the crosshairs at the center of its image will not always coincide with the target position. Therefore, after the robot stops, align the crosshairs whose visual origin is the center of the image with the crosshairs at the target. Calculating the coordinate data of the target mark, acquiring the coordinate data of the calibration mark, and guiding the robot's hand movement to make the coordinate data of the calibration mark coincide with the coordinate data of the target mark; When the coordinate data of the calibration mark and the target mark coincide, it indicates that the finger is located at the starting position for grasping, and the second ranging motion system sends a grasping instruction, wherein the grasping instruction is used to control the robot arm to grasp the target object; The first ranging motion system is also used for: Sending a first visual adjustment instruction to the first visual system, wherein the first visual system adjusts the focal length of the visual camera according to the first visual adjustment instruction; And, get the coordinate data of the robot's visual origin ; And, according to Calculate several routes from the robot's visual origin to the target mark, and select the best one as the target route, where the target route is used to guide the robot to the front position; After knowing the coordinate data of the robot's visual origin, the straight-line distance c from the robot's visual origin to the cup is measured, and the positions of the two are combined to obtain several routes from the robot's visual origin to the cup; in the step of obtaining several routes, the laser radar completes the scanning of the indoor environment and establishes path planning, which can realize the calibration and storage of specific position points in the system learning stage to enable the robot to quickly and accurately execute motion instructions, and the laser rangefinder can assist the robot to achieve more accurate positioning; the system automatically matches the optimal route, and the first ranging motion system guides the robot to the front position; a determination method is given for the front position, that is, the side of the object with a QR code is the front, and the position on the front that is a certain distance away from the object is the front position. Within this distance range, the robot's arm can reach the target object after movement; The first visual system includes an image sensor, which is located at the central axis of the head and can rotate synchronously with the head; The first visual system is also used for: Rotate the head to drive the image sensor to find the target mark and read the target mark; When the humanoid robot receives a task instruction, it can automatically analyze the search direction, and control the robot to move to the task area through the motion module, turn its head to drive the image sensor to find the target object in the task area, and when the robot's main sensor matches the target object, it reads the QR code, and then reads the information of the target object and the angle and force information required to grasp the target object; the robot uses the image sensor to scan the area where the cup is located, turns its head angle to make the crosshair center of the QR code on the cup coincide with the image center, and reads the QR code to obtain the cup code. Through this code, the associated information stored in the robot control system can be retrieved, including the state, the object in the cup, the weight, shape, the force and procedure required for grasping; The second distance measuring motion system includes a laser distance measuring unit and an angle sensor, which are located at the central axis of the head and can rotate synchronously with the head; The second range-finding motion system is also used for: Sending a second visual adjustment instruction to the second visual system, wherein the second visual system adjusts the focal length of the visual camera according to the second visual adjustment instruction; And, obtaining the horizontal angle ε and the vertical angle λ from the robot vision origin to the target mark, and obtaining the coordinate data of the calibration mark ,according to , , Calculate the coordinate data of the target marker ; And, according to , , Calculating the distance that the calibration mark needs to move, and guiding the calibration mark to the target mark until the calibration mark coincides with the target mark; After knowing the horizontal angle ε and vertical angle λ and distance c from the robot vision origin to the target mark, the coordinates of the target mark can be calculated. , the robot guides the hand by Movement to cup When the cross calibration mark on the contact surface of the thumb of the hand contacts and overlaps with the cross line of the QR code on the cup or moves to the starting position known through learning, the positioning operation is completed, and then the robot controls the hand to grab the cup; The target mark is provided with a positioning line; A positioning line is provided on the calibration mark; The center of the image of the image sensor is provided with a positioning line; The coincidence of the calibration mark and the target mark means that the positioning line of the target mark coincides with the positioning line of the calibration mark; The target mark is directly engraved on the object, or indirectly equipped on the object; The target mark is a two-dimensional code, and the positioning line on the target mark is a cross line; It also includes an arm motion system: used to move the robot hand to a specified position, specifically, to obtain arm shoulder joint coordinate data in real time , elbow joint coordinate data , wrist joint coordinate data , including the robot’s hand movements; It also includes a knowledge accumulation and storage module, which automatically records and saves the final location coordinates of the item, as well as weight, temperature-related information, whether it is a one-handed or two-handed operation, and the force parameters during the operation after completing a command action.

2. The identification, positioning and grasping method for robot grasping object operation of the identification, positioning and grasping device for robot grasping object operation according to claim 1, characterized in that: It includes a mark set on the object, wherein the mark on the target object to be grasped is the target mark, and the center point of the crosshairs of the target mark is the starting point of the action coordinates of the robot grasping the object; The method comprises the following steps: A first visual detection step: visually scanning the area where the target is located to form a first visual area, and after finding the target in the first visual area, sending a first ranging instruction to the first ranging motion system; The first ranging movement step: receiving the first ranging instruction, measuring the distance c from the robot vision origin to the target mark, and automatically planning the robot movement path according to the distance and position coordinates, wherein the front position refers to the front position where the horizontal distance from the robot vision origin to the target mark is less than the horizontal distance from the robot vision origin to the robot hand calibration mark, wherein the calibration mark is set on the robot hand; A second visual detection step: visually scanning the target mark at the front position to form a second visual area, and sending a second ranging instruction to a second ranging motion system; The second ranging movement step: receiving the second ranging instruction, and measuring the horizontal angle ε and the vertical angle λ from the robot's visual origin to the target mark; Calculating the coordinate data of the target mark, acquiring the coordinate data of the calibration mark, and controlling the movement of the robot arm to guide the calibration mark to coincide with the coordinate data of the target mark; When the coordinate data of the calibration mark and the target mark coincide, the second ranging motion system determines that the distance is consistent through the ranging data and then sends a grabbing instruction, wherein the grabbing instruction is used to grab the target object; The first ranging movement step includes the following sub-steps: Sending a first visual adjustment instruction to the first visual system, wherein the first visual system adjusts the focal length of the visual camera according to the first visual adjustment instruction; And, get the coordinate data of the robot's visual origin ; The first visual system includes an image sensor and a laser distance measuring unit, wherein the image sensor and the laser distance measuring unit are located on the same horizontal line and arranged in parallel at the central axis of the head and can rotate synchronously with the head; The first visual inspection step includes the following sub-steps: Turn the head to drive the image sensor to search for the target mark, and after matching the target object, control the head to rotate so that the center of the image sensor is aligned with the center of the target mark to read the target mark; The second ranging motion step includes the following sub-steps: Sending a second visual adjustment instruction to the second visual system, wherein the second visual system adjusts the focal length of the visual camera according to the second visual adjustment instruction; Furthermore, the angle sensor obtains the horizontal angle ε and the vertical angle λ from the robot vision origin to the target mark, and obtains the coordinate data of the calibration mark. ,according to , , Calculate the coordinate data of the target marker ; And, according to , , Calculating the distance that the calibration mark needs to move, and guiding the calibration mark to the target mark until the calibration mark coincides with the target mark; Wherein, a positioning line is provided on the target mark; A positioning line is provided on the calibration mark; The image center of the image sensor is the center positioning line of the visual origin; The coincidence of the calibration mark and the target mark means that the positioning line of the target mark coincides with the positioning line of the calibration mark.

Images