Object placement position calibration method, device, electronic device and storage medium

Abstract

This application discloses a method, apparatus, electronic device, and storage medium for calibrating the placement position of an object. In an image acquisition coordinate system, the reference acquisition position and reference placement position of a reference object, the target acquisition position of a target object, and a first angle and a second angle are acquired. Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system are determined according to the reference acquisition position and the target acquisition position, respectively. The reference calibration position obtained after calibrating the reference grasping position by angle and the target calibration position obtained after calibrating the target grasping position by angle are determined. Based on the reference calibration position and the target calibration position, the target placement position of the target object after angle calibration is determined. This application improves the accuracy of position calibration.

Description

Technical Field

[0001] This application relates to machine vision technology, and more particularly to a method, apparatus, electronic device, and storage medium for calibrating the placement position of an item. Background Technology

[0002] With the development of robotics technology, robots are being applied to more and more fields to achieve automated production. During the process of a robot picking up and placing a product, changes in the product's initial location may cause the robot to pick it up at a fixed position, but the final placement of the product may change.

[0003] In existing technologies, based on visual positioning technology, the relationship between the camera image coordinate system and the robot coordinate system is pre-calculated through camera calibration, and position calibration is performed based on the coordinate difference between the reference image at the placement position and the grasping position.

[0004] However, the robot also rotates during placement, and calibration based solely on the coordinate difference between the placement and grasping positions using a reference image results in low accuracy. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for calibrating the placement position of an item, in order to improve the accuracy of position calibration.

[0006] In a first aspect, embodiments of this application provide a method for calibrating the placement position of an item, the method comprising:

[0007] The reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target acquisition position of the target object in the image acquisition coordinate system, the reference acquisition position of the reference shooting point in the image acquisition coordinate system, the first angle between the reference acquisition position and the reference placement position, and the second angle between the reference acquisition position and the target acquisition position are obtained respectively.

[0008] Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system are determined according to the reference acquisition position and the target acquisition position, respectively.

[0009] Based on the reference acquisition position, the baseline grasping position, the target grasping position, the first angle, and the second angle, the baseline calibration position after angle calibration is obtained by calibrating the baseline grasping position in the coordinate system of the grasping device, and the target calibration position after angle calibration is obtained by calibrating the target grasping position in the coordinate system of the grasping device.

[0010] Based on the reference calibration position and the target calibration position, determine the target placement position after angle calibration of the target object, which is used to place the target object.

[0011] Secondly, embodiments of this application also provide an item placement position calibration device, which includes:

[0012] The image capture parameter determination module is used to obtain the reference capture position and reference placement position of the reference object in the image capture coordinate system, the target capture position of the target object in the image capture coordinate system, the reference capture position of the reference shooting point in the image capture coordinate system, and the first angle between the reference capture position and the reference placement position, and the second angle between the reference capture position and the target capture position.

[0013] The coordinate transformation module is used to determine the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system based on the coordinate transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, according to the reference acquisition position and the target acquisition position.

[0014] The angle calibration module is used to determine the angle-calibrated reference position obtained by calibrating the reference grasping position in the coordinate system of the grasping device, and the angle-calibrated target position obtained by calibrating the target grasping position in the coordinate system of the grasping device, based on the reference acquisition position, the reference grasping position, the target grasping position, the first angle, and the second angle.

[0015] The target placement calibration module is used to determine the target placement position after angle calibration based on the reference calibration position and the target calibration position, and is used to place the target object.

[0016] Thirdly, embodiments of this application also provide an electronic device, which includes:

[0017] One or more processors;

[0018] Storage device for storing one or more programs;

[0019] When one or more programs are executed by one or more processors, the one or more processors implement any of the item placement calibration methods provided in the embodiments of this application.

[0020] Fourthly, embodiments of this application also provide a storage medium including computer-executable instructions, which, when executed by a computer processor, are used to perform any of the item placement calibration methods provided in embodiments of this application.

[0021] This application obtains the reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target acquisition position of the target object in the image acquisition coordinate system, the reference acquisition position of the reference shooting point in the image acquisition coordinate system, and the first angle between the reference acquisition position and the reference placement position, and the second angle between the reference acquisition position and the target acquisition position; determines the first and second angles in the image acquisition coordinate system for subsequent angle calibration of the reference grasping position and the target grasping position; and determines the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, according to the reference acquisition position and the target acquisition position; and transforms the coordinates to the grasping device coordinate system. In this system, the target object's placement position is determined. Based on the reference acquisition position, the baseline grasping position, the target grasping position, the first angle, and the second angle, the reference calibrated position is obtained by calibrating the baseline grasping position in the grasping device coordinate system, and the target calibrated position is obtained by calibrating the target grasping position in the grasping device coordinate system. Angle calibration ensures that the imaging angles of the reference calibrated position and the target calibrated position are both at the reference acquisition position, unifying the imaging angles and improving the accuracy of the subsequently determined target placement position, compensating for errors caused by differences in imaging angles between the reference grasping position and the target grasping position. Based on the reference calibrated position and the target calibrated position, the target object's placement position after angle calibration is determined, and the target object is placed accordingly. Therefore, the technical solution of this application solves the problem of low accuracy in calibration based solely on the coordinate difference between the placement position and the grasping position using a reference image, achieving the effect of improving the accuracy of position calibration. Attached Figure Description

[0022] Figure 1 This is a flowchart of an article placement position calibration method according to Embodiment 1 of this application;

[0023] Figure 2a This is a flowchart of an article placement position calibration method according to Embodiment 2 of this application;

[0024] Figure 2b This is a schematic diagram of angle calibration of a reference gripping position according to Embodiment 2 of this application;

[0025] Figure 2c This is a schematic diagram of angle calibration of the target grasping position in Embodiment 2 of this application;

[0026] Figure 2d This is a schematic diagram of angle calibration for a target placement position according to Embodiment 2 of this application;

[0027] Figure 3 This is a schematic diagram of the structure of an item placement position calibration device according to Embodiment 3 of this application;

[0028] Figure 4 This is a schematic diagram of the structure of an electronic device according to Embodiment 4 of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first" and "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Example 1

[0032] Figure 1 This is a flowchart of an item placement position calibration method provided in Embodiment 1 of this application. This embodiment can be applied to the case of calibrating the placement position of a grasped item. The method can be executed by an item placement position calibration device, which can be implemented in software and / or hardware and specifically configured in an electronic device, such as a grasping robot.

[0033] See Figure 1 The method for calibrating the placement of items shown includes the following steps:

[0034] S110. Obtain the reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target acquisition position of the target object in the image acquisition coordinate system, the reference acquisition position of the reference shooting point in the image acquisition coordinate system, the first angle between the reference acquisition position and the reference placement position, and the second angle between the reference acquisition position and the target acquisition position.

[0035] The reference object can be a reference object used to obtain the reference acquisition position and reference placement position for subsequent target acquisition position calibration. The image acquisition coordinate system can be a coordinate system established with the camera's location as the origin, used to determine the coordinates of the captured image. The reference acquisition position can be a preset position in the image captured by the grasping device at the capture point, used to determine the position of the reference object when taking the picture. For example, the reference acquisition position can be the geometric center or other specific position in the image captured by the reference object, which is not specifically limited in this application. Specifically, the reference acquisition position can be marked in the form of coordinates. The reference placement position can be a preset position in the image captured by the grasping device at the placement point, used to determine the position of the reference object when it is placed. Specifically, the reference placement position can be marked in the form of coordinates.

[0036] The target object can be any product grasped by the gripping equipment in actual production. The target acquisition position can be a preset position in the image obtained by taking a picture of the target object at the capture point after it has been grasped by the gripping equipment, used to determine the position of the target object when taking the picture. The relative relationships of the target acquisition position, the reference acquisition position, and the reference placement position are the same in the entire image they are in. For example, the relative relationship can be the geometric center in the image, that is, the target acquisition position is the geometric center of the image taken by the target object at the grasping position; the reference acquisition position is the geometric center of the image taken by the reference object at the grasping position; and the reference placement position is the geometric center of the image taken by the reference object at the placement position.

[0037] The reference shooting point can be a preset shooting location point in the image acquisition coordinate system. Specifically, after the grasping device captures the reference object or target object, it stops moving at the reference shooting point and takes a picture. The reference acquisition location can be the position of the reference shooting point in the image acquisition coordinate system.

[0038] The first angle can be the angle between the reference acquisition position and the benchmark placement position, used to determine the difference in the imaging angle between the reference acquisition position and the benchmark placement position. The second angle can be the angle between the benchmark acquisition position and the target acquisition position, used to determine the difference in the imaging angle between the benchmark acquisition position and the target acquisition position.

[0039] S120. Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, determine the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system according to the reference acquisition position and the target acquisition position.

[0040] The coordinate system of the grasping device can be a coordinate system centered at a fixed point within the grasping device. For example, the coordinate system of the grasping device can be a coordinate system centered at the rotation center point of the grasping device. The coordinate transformation matrix can be a transformation matrix that converts coordinates in the image acquisition coordinate system to coordinates in the grasping device coordinate system. Specifically, multiplying the coordinates in the image acquisition coordinate system by the coordinate transformation matrix yields the corresponding coordinates in the grasping device coordinate system.

[0041] Move the end effector of the gripping device into the camera's field of view, and assign easily identifiable feature points. For example, feature points can be obtained by attaching printed circles to the fixture. Record the current coordinates of the gripping device Q0(x0,y0), identify the center image coordinates P0(u0,v0) of the corresponding circular feature point, and move the feature point nine times in a 3x3 grid within the camera's field of view using the gripping device. This yields nine sets of image coordinate points P0(u0,v0), P1(u1,v1), ..., P8(u8,v8) and their corresponding gripping device coordinates Q0(x0,y0), Q1(x1,y1), ..., Q8(x8,y8). At point P(u0,v0), the grasping device rotates the feature point within the camera's field of view by ±θ angles, obtaining image coordinate points P′1(u′1,v′1) and P′2(u′2,v′2). Using points P′1(u′1,v′1) and P′0(u′2,v′2) and the 2θ angle, the center coordinates P′0(u′0,v′0) are calculated, which are the rotation center coordinates corresponding to the image feature point. The rotation center coordinates of the grasping device corresponding to the 9 sets of image coordinate points are P... i (u i -(u′0-u0),v i -(v′0-v0)). Calculate P i (u i -(u′0-u0),v i -(v′0-v0)) and Q i (x i ,y i The transformation relationship A between P*A and Q is given by the following formula:

[0042]

[0043] The reference gripping position can be the corresponding position of the reference acquisition position in the gripping device's coordinate system. Specifically, the coordinates of the reference acquisition position are obtained by multiplying the coordinates of the reference acquisition position by the coordinate system transformation matrix. The coordinates of the reference object's reference gripping position in the gripping device's coordinate system can be determined using the following formula:

[0044]

[0045] Where [x'1,y'1] represents the coordinates of the reference capture position; [p'1,q'1,1] represents the coordinates of the reference acquisition position. Coordinate system transformation matrix.

[0046] The target grasping position can be the corresponding position of the target acquisition position in the coordinate system of the grasping device. Specifically, the coordinates of the target acquisition position are obtained by multiplying the coordinates of the target acquisition position by the coordinate system transformation matrix. The coordinates of the target object's position in the coordinate system of the grasping device can be determined according to the following formula:

[0047]

[0048] Where [x1,y1] represents the coordinates of the reference capture position; [p1,q1,1] represents the coordinates of the reference acquisition position. Coordinate system transformation matrix.

[0049] S130. Based on the reference acquisition position, the baseline grasping position, the target grasping position, the first angle, and the second angle, determine the baseline calibration position obtained after angle calibration of the baseline grasping position in the grasping device coordinate system, and the target calibration position obtained after angle calibration of the target grasping position in the grasping device coordinate system.

[0050] Based on the reference acquisition position, the reference grasping position, and the first angle, the reference calibrated position is obtained by calibrating the reference grasping position in the coordinate system of the grasping device. That is, the reference grasping position is rotated by the first angle around the reference acquisition position in the imaging coordinate system to obtain the reference calibrated position.

[0051] Based on the reference acquisition position, the first angle and the second angle of the target grasping position, the target grasping position is calibrated in the coordinate system of the grasping device to obtain the calibrated target position. That is, the target grasping position is rotated by the second angle around the reference acquisition position in the imaging coordinate system, and then rotated by the first angle to obtain the calibrated target position.

[0052] S140. Based on the reference calibration position and the target calibration position, determine the target placement position after the target object has been calibrated at an angle, and place the target object thereon.

[0053] The target placement location can be a preset position in the image obtained after the target object is grasped by the grasping device and photographed at the placement point. This position is used to determine the location of the target object when it is placed. Specifically, the target placement location can be marked in the form of coordinates.

[0054] The reference calibration position and the target calibration position, after angle calibration, can be considered to be at the same shooting angle, that is, at the angle of the reference acquisition position. Using the reference calibration position and the target calibration position, the translation amount required to move the reference calibration position to the target calibration position can be determined. Performing the same translation operation on the reference placement position yields the target placement position.

[0055] During the process of a gripping device grasping and placing a target object, changes in the object's incoming position can lead to inconsistent gripping positions when the device attempts to grasp it from a fixed location. This results in variations in the final placement position. For example, in a scenario where components are mounted on a circuit board, the target object can be a component. If the component's incoming position changes, placing it on the circuit board in its original position might cause misalignment or breakage of the component's pins, rendering the circuit board unusable. Therefore, after the gripping device grasps the target object, it needs to photograph it and calculate the changes in its position. Compensating for these positional variations during placement ensures consistency between the target object and the preset placement position.

[0056] In existing technologies, a coordinate transformation matrix A between the image acquisition coordinate system and the grasping device coordinate system is pre-calculated through camera calibration. The camera identifies the reference object's reference acquisition position, calculates the corresponding reference grasping position, and determines the reference placement position of the reference object. The coordinate difference between the two points is calculated, and other products calculate their placement point coordinates based on this difference. This method is simple and easy to implement, but it does not consider the angle of the shooting position, which limits its application. If the shooting position and the angle of the grasping device differ from the placement position, it will lead to a significant error in the placement position.

[0057] The technical solution of this embodiment obtains the reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target acquisition position of the target object in the image acquisition coordinate system, the reference acquisition position of the reference shooting point in the image acquisition coordinate system, and the first angle between the reference acquisition position and the reference placement position, and the second angle between the reference acquisition position and the target acquisition position; determines the first angle and the second angle in the image acquisition coordinate system for subsequent angle calibration of the reference grasping position and the target grasping position; and determines the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, according to the reference acquisition position and the target acquisition position; and transforms the coordinates to the grasping device coordinate system. In the coordinate system, the target object's placement position is determined subsequently. Based on the reference acquisition position, the baseline grasping position, the target grasping position, the first angle, and the second angle, the reference calibrated position is obtained by calibrating the baseline grasping position in the grasping device coordinate system, and the target calibrated position is obtained by calibrating the target grasping position in the grasping device coordinate system. Angle calibration ensures that the imaging angles of the reference calibrated position and the target calibrated position are both at the reference acquisition position, unifying the imaging angles and improving the accuracy of the subsequently determined target placement position, compensating for errors caused by differences in imaging angles between the reference grasping position and the target grasping position. Based on the reference calibrated position and the target calibrated position, the target object's placement position after angle calibration is determined, and the target object is placed accordingly. Therefore, the technical solution of this application solves the problem of low accuracy in calibration based solely on the coordinate difference between the placement position and the grasping position using a reference image, achieving the effect of improving the accuracy of position calibration.

[0058] Example 2

[0059] Figure 2a This is a flowchart of a method for calibrating the placement position of an item, provided in Embodiment 2 of this application. The technical solution of this embodiment is further refined based on the above technical solution.

[0060] Furthermore, the process of "determining the reference calibration position after angle calibration of the reference gripping position in the gripping device coordinate system, and the target calibration position after angle calibration of the target gripping position in the gripping device coordinate system, based on the reference acquisition position, the reference gripping position, the target gripping position, the first angle, and the second angle" is further refined into: "determining the reference angle transformation matrix based on the reference acquisition position and the first angle; obtaining the reference calibration position after angle calibration of the reference gripping position in the gripping device coordinate system based on the reference acquisition position, the reference gripping position, and the reference angle transformation matrix; determining the target angle transformation matrix based on the reference acquisition position, the first angle, and the second angle; obtaining the target calibration position after angle calibration of the target gripping position in the gripping device coordinate system based on the reference acquisition position, the target gripping position, and the target angle transformation matrix," in order to achieve angle calibration.

[0061] See Figure 2a The method for calibrating the placement of an item, as shown, includes:

[0062] S210. Obtain the reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target acquisition position of the target object in the image acquisition coordinate system, the reference acquisition position of the reference shooting point in the image acquisition coordinate system, and the first angle between the reference acquisition position and the reference placement position, and the second angle between the reference acquisition position and the target acquisition position.

[0063] S220. Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, determine the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system according to the reference acquisition position and the target acquisition position.

[0064] S230. Determine the reference angle transformation matrix based on the reference acquisition position and the first angle.

[0065] The reference angle transformation matrix can be used to calibrate the angle of the reference grasping position. Specifically, the reference acquisition position can be used as the rotation center, and the first angle can be used as the rotation angle to construct the reference angle transformation matrix.

[0066] In one optional embodiment, determining the reference angle transformation matrix based on the reference acquisition position and the first angle includes: taking the reference acquisition position as the rotation center position and the first angle as the rotation angle to determine the reference angle transformation matrix.

[0067] Using the reference acquisition position as the rotation center, the baseline grasping position is rotated around the reference acquisition position, unifying the image angle of the baseline grasping position to the image angle of the baseline placement position. The first angle is used as the rotation angle to compensate for the angle difference between the baseline grasping position and the baseline placement position. The baseline angle transformation matrix can be expressed as follows:

[0068]

[0069] Where (r'2-r0) is the first angle, (x0,y0,r0) is the reference acquisition position, and (x'2,y'2,r'2) is the reference placement position.

[0070] By using the reference acquisition position as the rotation center and the first angle as the rotation angle, and determining the reference angle transformation matrix, the reference acquisition position can be rotated by the first angle to calibrate the reference grasping position.

[0071] S240. Based on the reference acquisition position, the reference grasping position, and the reference angle transformation matrix, the reference grasping position is calibrated in the coordinate system of the grasping device to obtain the calibrated reference position.

[0072] The reference calibration position can be the position obtained by calibrating the reference grasping position according to the reference angle transformation matrix. The vector coordinates corresponding to the reference grasping position are determined by taking the reference acquisition position as the starting point and the reference grasping position as the ending point. The reference calibration position is obtained by calibrating this vector according to the reference angle transformation matrix.

[0073] In one optional embodiment, the reference acquisition position is calibrated in the coordinate system of the grasping device to obtain the calibrated reference position after angle calibration based on the reference acquisition position, the reference grasping position and the reference angle transformation matrix. This includes: taking the reference acquisition position as the origin in the coordinate system of the grasping device; and determining the calibrated reference position based on the product of the coordinates of the reference grasping position and the reference angle transformation matrix.

[0074] By using the reference acquisition position as the origin in the coordinate system of the grasping device, the coordinates of the baseline grasping position can be determined. Then, by multiplying the coordinates of the baseline grasping position by the baseline angle transformation matrix, the baseline calibration position after angle calibration can be determined. Specifically, the baseline calibration position can be determined according to the following formula:

[0075]

[0076] in, [x'1-x0,y'1-y0,1] represents the coordinates of the reference calibration position, and [x'1-x0,y'1-y0,1] represents the coordinates of the reference grasping position when the reference acquisition position is used as the origin in the coordinate system of the grasping device.

[0077] Figure 2b This is a schematic diagram of angle calibration for a reference gripping position.

[0078] By taking the reference acquisition position as the origin in the coordinate system of the grasping device, and determining the reference calibration position after angle calibration based on the product of the coordinates of the reference grasping position and the reference angle transformation matrix, the reference grasping position is rotated around the reference acquisition position by a first angle, so that the reference grasping position and the reference placement position are aligned at the same shooting angle.

[0079] S250. Based on the reference acquisition position, the first angle, and the second angle, determine the target angle transformation matrix.

[0080] The target angle transformation matrix can be a matrix used to correct the angle of the target grasping position. Specifically, the reference acquisition position can be used as the rotation center, and the sum of the first angle and the second angle can be used as the rotation angle to construct the target angle transformation matrix.

[0081] In one optional embodiment, determining the target angle transformation matrix based on the reference acquisition position, the first angle, and the second angle includes: taking the reference acquisition position as the rotation center position and taking the sum of the first angle and the second angle as the rotation angle to determine the target angle transformation matrix.

[0082] Using the reference acquisition position as the rotation center, the target grasping position is rotated around the reference acquisition position, unifying the image angle of the target grasping position to the image angle of the reference placement. The angle difference between the target grasping position and the reference grasping position is the second angle, and the angle difference between the reference placement position and the reference grasping position is the first angle. Therefore, the sum of the first and second angles is used as the rotation angle to compensate for the angle difference between the target grasping position and the reference placement position. The reference angle transformation matrix can be expressed as follows:

[0083]

[0084] Where (r1'-r1) is the second angle; (x1',y1',r1') is the reference grab position.

[0085] By using the reference acquisition position as the rotation center and the sum of the first and second angles as the rotation angle, the target angle transformation matrix is ​​determined. This allows for the calibration of the target grasping position angle by rotating the sum of the first and second angles with the reference acquisition position as the rotation center.

[0086] S260. Based on the reference acquisition position, target grasping position, and target angle transformation matrix, the target grasping position is calibrated in the coordinate system of the grasping device to obtain the calibrated target position after angle calibration.

[0087] The target calibration position can be the position after angular calibration of the target grasping position based on the target angle transformation matrix. The vector coordinates corresponding to the target grasping position are determined by taking the reference acquisition position as the starting point and the target grasping position as the ending point. The target calibration position is obtained by calibrating this vector according to the target angle transformation matrix.

[0088] In one optional embodiment, the target calibrated position is obtained by calibrating the target grasping position in the grasping device coordinate system based on the reference acquisition position, the target grasping position, and the target angle transformation matrix. This includes: using the reference acquisition position as the origin in the grasping device coordinate system; and determining the target calibrated position based on the product of the coordinates of the target grasping position and the target angle transformation matrix.

[0089] By using the reference acquisition position as the origin in the coordinate system of the grasping device, the coordinates of the target grasping position can be determined. Then, by multiplying the coordinates of the target grasping position by the target angle transformation matrix, the calibrated target position after angle calibration can be determined. Specifically, the target calibration position can be determined according to the following formula:

[0090]

[0091] in, [x1-x0, y1-y0, 1] represents the coordinates of the target calibration position, and [x1-x0, y1-y0, 1] represents the coordinates of the target grasping position when the reference acquisition position is used as the origin in the grasping device coordinate system.

[0092] Figure 2c A schematic diagram illustrating angle calibration for a target grasping position.

[0093] By using the reference acquisition position as the origin in the coordinate system of the grasping device, and based on the product of the coordinates of the target grasping position and the target angle transformation matrix, the target calibration position after angle calibration is determined, so that the target grasping position rotates around the reference acquisition position by the sum of the first and second angles, so that the target grasping position and the reference placement position are aligned at the same shooting angle.

[0094] S270. Based on the reference calibration position and the target calibration position, determine the target placement position after the target object has been calibrated at an angle, and place the target object thereon.

[0095] The technical solution of this embodiment determines a reference angle transformation matrix based on a reference acquisition position and a first angle; based on the reference acquisition position, the reference grasping position, and the reference angle transformation matrix, the reference grasping position is calibrated at an angle in the coordinate system of the grasping device to obtain a calibrated reference position; based on the reference acquisition position, the first angle, and the second angle, a target angle transformation matrix is ​​determined; based on the reference acquisition position, the target grasping position, and the target angle transformation matrix, the target grasping position is calibrated at an angle in the coordinate system of the grasping device to obtain a calibrated target position. Angle calibration is performed on both the reference grasping position and the target grasping position to ensure that the obtained reference calibration position and target calibration position are aligned at the same imaging angle, thereby improving the accuracy of position calibration.

[0096] In one optional embodiment, determining the target placement position after angle calibration of the target object based on the reference calibration position and the target calibration position, for placing the target object, includes: determining the target translation amount after angle calibration of the target object based on the reference calibration position and the target calibration position; and determining the target placement position based on the target translation amount and the reference placement position, for placing the target object.

[0097] The target translation amount is the translation between the reference calibration position and the target calibration position, and is used to determine the target placement position.

[0098] Based on the reference calibration position and the target calibration position, after angle calibration, their coordinates are at the same shooting angle. Therefore, based on the reference calibration position and the target calibration position, the target translation amount after angle calibration is determined. The reference placement position is then translated by the same amount of target translation until the target placement position is reached. Based on the target translation amount, the grasping device performs a corresponding translation to reach the target placement position and place the target object. Specifically, the target placement position can be determined using the following formula:

[0099]

[0100] Where (x2,y2,r2) is the target placement position.

[0101] Figure 2d A schematic diagram illustrating calibration for a target placement location.

[0102] By determining the target translation amount after angle calibration based on the reference calibration position and the target calibration position, the translation operation value of the gripping device can be determined. Based on the target translation amount and the reference placement position, the target placement position is determined for placing the target object. The angle-calibrated position can be determined to accurately place the target object.

[0103] Example 3

[0104] Figure 3 The diagram shown is a structural schematic of an item placement position calibration device provided in Embodiment 3 of this application. This embodiment is applicable to calibrating the placement position of a grasped item. The specific structure of the item placement position calibration device is as follows:

[0105] The image capture parameter determination module 310 is used to obtain the reference capture position and reference placement position of the reference object in the image capture coordinate system, the target capture position of the target object in the image capture coordinate system, the reference capture position of the reference shooting point in the image capture coordinate system, and the first angle between the reference capture position and the reference placement position, and the second angle between the reference capture position and the target capture position.

[0106] The coordinate transformation module 320 is used to determine the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system based on the coordinate transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, according to the reference acquisition position and the target acquisition position.

[0107] Angle calibration module 330 is used to determine the reference calibration position obtained by calibrating the reference gripping position in the coordinate system of the gripping device after angle calibration, and the target calibration position obtained by calibrating the target gripping position in the coordinate system of the gripping device after angle calibration, based on the reference acquisition position, the reference gripping position, the target gripping position, the first angle, and the second angle.

[0108] The target placement calibration module 340 is used to determine the target placement position after angle calibration of the target object based on the reference calibration position and the target calibration position, and is used to place the target object.

[0109] The technical solution of this embodiment obtains the reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target acquisition position of the target object in the image acquisition coordinate system, the reference acquisition position of the reference shooting point in the image acquisition coordinate system, and the first angle between the reference acquisition position and the reference placement position, and the second angle between the reference acquisition position and the target acquisition position; determines the first angle and the second angle in the image acquisition coordinate system for subsequent angle calibration of the reference grasping position and the target grasping position; and determines the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, according to the reference acquisition position and the target acquisition position; and transforms the coordinates to the grasping device coordinate system. In the coordinate system, the target object's placement position is determined subsequently. Based on the reference acquisition position, the baseline grasping position, the target grasping position, the first angle, and the second angle, the reference calibrated position is obtained by calibrating the baseline grasping position in the grasping device coordinate system, and the target calibrated position is obtained by calibrating the target grasping position in the grasping device coordinate system. Angle calibration ensures that the imaging angles of the reference calibrated position and the target calibrated position are both at the reference acquisition position, unifying the imaging angles and improving the accuracy of the subsequently determined target placement position, compensating for errors caused by differences in imaging angles between the reference grasping position and the target grasping position. Based on the reference calibrated position and the target calibrated position, the target object's placement position after angle calibration is determined, and the target object is placed accordingly. Therefore, the technical solution of this application solves the problem of low accuracy in calibration based solely on the coordinate difference between the placement position and the grasping position using a reference image, achieving the effect of improving the accuracy of position calibration.

[0110] Optional, the angle calibration module 330 includes:

[0111] The reference angle transformation matrix determination unit is used to determine the reference angle transformation matrix based on the reference acquisition position and the first angle.

[0112] The reference calibration position determination unit is used to perform angle calibration on the reference gripping position in the coordinate system of the gripping device based on the reference acquisition position, the reference gripping position and the reference angle transformation matrix to obtain the angle-calibrated reference calibration position.

[0113] The target angle transformation matrix determination unit is used to determine the target angle transformation matrix based on the reference acquisition position, the first angle, and the second angle.

[0114] The target calibration position determination unit is used to perform angle calibration on the target grasping position in the coordinate system of the grasping device based on the reference acquisition position, the target grasping position and the target angle transformation matrix to obtain the angle-calibrated target calibration position.

[0115] Optionally, the reference angle transformation matrix determination unit includes:

[0116] The reference angle transformation matrix construction sub-unit is used to determine the reference angle transformation matrix by taking the reference acquisition position as the rotation center position and the first angle as the rotation angle.

[0117] Optionally, the target angle transformation matrix determination unit includes:

[0118] The target angle transformation matrix construction sub-unit is used to determine the target angle transformation matrix by taking the reference acquisition position as the rotation center position and the sum of the first angle and the second angle as the rotation angle.

[0119] Optionally, the reference calibration position determination unit includes:

[0120] The origin determination sub-unit is used to use the reference acquisition position as the origin in the coordinate system of the grasping device.

[0121] The reference gripping position calibration subunit is used to determine the reference calibration position after angle calibration based on the product of the coordinates of the reference gripping position and the reference angle transformation matrix.

[0122] Optionally, the target calibration location determination unit includes:

[0123] The origin determination sub-unit is used to use the reference acquisition position as the origin in the coordinate system of the grasping device.

[0124] The target grasping position calibration subunit is used to determine the calibrated target position after angle calibration based on the product of the target grasping position coordinates and the target angle transformation matrix.

[0125] Optionally, the target placement calibration module 340 includes:

[0126] The target translation determination unit is used to determine the target translation amount after angle calibration of the target object based on the reference calibration position and the target calibration position.

[0127] The target object placement unit is used to determine the target placement position based on the target translation amount and the reference placement position, and is used to place the target object.

[0128] The item placement calibration device provided in this application embodiment can execute the item placement calibration method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the item placement calibration method.

[0129] Example 4

[0130] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application, as shown below. Figure 4 As shown, the electronic device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of processors 410 in the electronic device can be one or more. Figure 4 Taking a processor 410 as an example; the processor 410, memory 420, input device 430, and output device 440 in the electronic device can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0131] The memory 420, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the item placement position calibration method in this embodiment (e.g., the image parameter determination module 310, coordinate transformation module 320, angle calibration module 330, and target placement position calibration module 340). The processor 410 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 420, thereby implementing the aforementioned item placement position calibration method.

[0132] The memory 420 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 420 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 420 may further include memory remotely located relative to the processor 410, which can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0133] Input device 430 can be used to receive input character information and generate key signal inputs related to user settings and function control of the electronic device. Output device 440 may include display devices such as a display screen.

[0134] Example 5

[0135] Embodiment 5 of this application also provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform an object placement position calibration method. The method includes: acquiring, respectively, a reference acquisition position and a reference placement position of a reference object in an image acquisition coordinate system, a target acquisition position of a target object in the image acquisition coordinate system, a reference acquisition position of a reference shooting point in the image acquisition coordinate system, a first angle between the reference acquisition position and the reference placement position, and a second angle between the reference acquisition position and the target acquisition position; based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, according to the... The reference acquisition position and target acquisition position are used to determine the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system, respectively. Based on the reference acquisition position, the reference grasping position, the target grasping position, the first angle, and the second angle, the reference calibration position after angle calibration of the reference grasping position in the grasping device coordinate system and the target calibration position after angle calibration of the target grasping position in the grasping device coordinate system are determined. Based on the reference calibration position and the target calibration position, the target placement position after angle calibration of the target object is determined for placing the target object.

[0136] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the operation of the method described above, but can also perform related operations in the item placement position calibration method provided in any embodiment of this application.

[0137] Based on the above description of the implementation methods, those skilled in the art can clearly understand that this application can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0138] It is worth noting that in the embodiments of the search device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of this application.

[0139] Note that the above are only preferred embodiments of the present application and the technical principles employed. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present application. The scope of the present application is determined by the scope of the appended claims.

Claims

1. A method for calibrating the placement position of an item, characterized in that, include: The reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target acquisition position of the target object in the image acquisition coordinate system, the reference acquisition position of the reference shooting point in the image acquisition coordinate system, the first angle between the reference acquisition position and the reference placement position, and the second angle between the reference acquisition position and the target acquisition position are obtained respectively. Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system are determined according to the reference acquisition position and the target acquisition position, respectively. Based on the reference acquisition position, the reference grasping position, the target grasping position, the first angle, and the second angle, the reference calibration position after angle calibration is obtained by calibrating the reference grasping position in the grasping device coordinate system, and the target calibration position after angle calibration is obtained by calibrating the target grasping position in the grasping device coordinate system. Based on the reference calibration position and the target calibration position, a target placement position is determined after the target object has undergone angle calibration, for placing the target object.

2. The method according to claim 1, characterized in that, The step of determining, based on the reference acquisition position, the reference grasping position, the target grasping position, the first angle, and the second angle, the angle-calibrated reference position obtained by calibrating the reference grasping position in the grasping device coordinate system, and the angle-calibrated target position obtained by calibrating the target grasping position in the grasping device coordinate system, includes: Based on the reference acquisition position and the first angle, determine the reference angle transformation matrix; Based on the reference acquisition position, the reference grasping position, and the reference angle transformation matrix, the reference grasping position is calibrated in the coordinate system of the grasping device to obtain the calibrated reference position. Based on the reference acquisition position, the first angle, and the second angle, determine the target angle transformation matrix; Based on the reference acquisition position, the target grasping position, and the target angle transformation matrix, the target grasping position is calibrated in the coordinate system of the grasping device to obtain the calibrated target position.

3. The method according to claim 2, characterized in that, The step of determining the reference angle transformation matrix based on the reference acquisition position and the first angle includes: The reference acquisition position is used as the rotation center position, and the first angle is used as the rotation angle to determine the reference angle transformation matrix.

4. The method according to claim 2, characterized in that, The step of determining the target angle transformation matrix based on the reference acquisition position, the first angle, and the second angle includes: The target angle transformation matrix is ​​determined by taking the reference acquisition position as the rotation center position and the sum of the first angle and the second angle as the rotation angle.

5. The method according to claim 2, characterized in that, The step of obtaining the angle-calibrated reference position by calibrating the reference grasping position in the coordinate system of the grasping device based on the reference acquisition position, the reference grasping position, and the reference angle transformation matrix includes: The reference acquisition position is used as the origin in the coordinate system of the grasping device; The reference calibration position after angle calibration is determined by multiplying the coordinates of the reference grab position with the reference angle transformation matrix.

6. The method according to claim 2, characterized in that, The step of obtaining the calibrated target position by calibrating the target position in the coordinate system of the grasping device based on the reference acquisition position, the target grasping position, and the target angle transformation matrix includes: The reference acquisition position is used as the origin in the coordinate system of the grasping device; The target calibration position after angle calibration is determined by multiplying the coordinates of the target capture position with the target angle transformation matrix.

7. The method according to claim 1, wherein determining the target placement position after angle calibration of the target object based on the reference calibration position and the target calibration position, for placing the target object, comprises: Based on the reference calibration position and the target calibration position, determine the target translation amount after angle calibration of the target object; Based on the target translation amount and the reference placement position, the target placement position is determined for placing the target object.

8. A device for calibrating the placement position of an item, characterized in that, include: The image capture parameter determination module is used to obtain the reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target acquisition position of the target object in the image acquisition coordinate system, the reference acquisition position of the reference shooting point in the image acquisition coordinate system, the first angle between the reference acquisition position and the reference placement position, and the second angle between the reference acquisition position and the target acquisition position. The coordinate transformation module is used to determine the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system based on the coordinate transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, according to the reference acquisition position and the target acquisition position. An angle calibration module is used to determine, based on the reference acquisition position, the reference grasping position, the target grasping position, the first angle, and the second angle, the reference calibration position obtained after angle calibration of the reference grasping position in the grasping device coordinate system, and the target calibration position obtained after angle calibration of the target grasping position in the grasping device coordinate system. The target placement calibration module is used to determine the target placement position after angle calibration of the target object based on the reference calibration position and the target calibration position, and is used to place the target object.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the item placement calibration method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the item placement calibration method as described in any one of claims 1-7.

Images