Method and apparatus for calibrating a sports item position, electronic device and storage medium

Abstract

Embodiments of the present application disclose a kind of motion article position calibration method, device, electronic equipment and storage medium. Respectively, the reference photographing position, reference collection position and reference placement position of reference object are acquired, the target photographing position, target collection position of target object and first angle, second angle and third angle;Based on coordinate system conversion matrix, in the coordinate system of grabbing device, reference grabbing position and target grabbing position are determined;According to reference photographing position, reference grabbing position, target photographing position, target grabbing position, first angle, second angle and third angle, the reference calibration position after angle calibration to reference grabbing position is determined, and the target calibration position obtained after angle calibration to target grabbing position;According to reference calibration position and target calibration position, the target placement position after angle calibration to target object is determined.The accuracy and efficiency of placement position calibration are improved in the embodiments of the present application.

Description

Technical Field

[0001] This application relates to machine vision technology, and more particularly to a method, apparatus, electronic device, and storage medium for calibrating the position of a moving object. Background Technology

[0002] With the development of robotics technology, robots are being applied to more and more fields to achieve automated production. During the process of a robot picking up and placing a product, changes in the product's initial location may cause the robot to pick it up at a fixed position, but the final placement of the product may change.

[0003] In existing technologies, based on visual positioning technology, the relationship between the camera image coordinate system and the robot coordinate system is pre-calculated through camera calibration. Position calibration is performed based on the coordinate difference between the reference image at the placement position and the grasping position, and it is necessary to take pictures at a fixed position.

[0004] However, the robot will rotate during placement. Calibration based solely on the coordinate difference between the placement and grasping positions using a reference image is inaccurate and inefficient because it needs to stop at specific locations to take pictures. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for calibrating the position of sporting goods, thereby improving the accuracy and efficiency of position calibration.

[0006] In a first aspect, embodiments of this application provide a method for calibrating the position of a moving article, the method comprising:

[0007] The reference object is obtained from the reference image capture position, reference acquisition position and reference placement position in the image acquisition coordinate system; the target object is obtained from the target image capture position and target acquisition position in the image acquisition coordinate system; the first angle between the reference image capture position and the target image capture position; the second angle between the reference placement position and the reference image capture position; and the third angle between the reference acquisition position and the target acquisition position.

[0008] Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system are determined according to the reference acquisition position and the target acquisition position, respectively.

[0009] Based on the reference image position, reference grab position, target image position, target grab position, first angle, second angle, and third angle, determine the reference calibration position after calibrating the reference grab position in the grab device coordinate system, and the target calibration position after calibrating the target grab position in the grab device coordinate system.

[0010] Based on the reference calibration position and the target calibration position, determine the target placement position after angle calibration of the target object, which is used to place the target object.

[0011] Secondly, embodiments of this application also provide a calibration device for the position of a moving article, the calibration device for the position of a moving article comprising:

[0012] The image capture parameter determination module is used to obtain the reference image capture position, reference capture position and reference placement position of the reference object in the image capture coordinate system, the target image capture position and target capture position of the target object in the image capture coordinate system, as well as the first angle between the reference image capture position and the target image capture position, the second angle between the reference placement position and the reference image capture position, and the third angle between the reference capture position and the target capture position.

[0013] The coordinate transformation module is used to determine the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system based on the coordinate transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, according to the reference acquisition position and the target acquisition position.

[0014] The angle calibration module is used to determine the reference calibration position after angle calibration of the reference grasping position in the grasping device coordinate system, and the target calibration position after angle calibration of the target grasping position in the grasping device coordinate system, based on the reference image position, reference grasping position, target image position, target grasping position, first angle, second angle, and third angle.

[0015] The target placement calibration module is used to determine the target placement position after angle calibration based on the reference calibration position and the target calibration position, and is used to place the target object.

[0016] Thirdly, embodiments of this application also provide an electronic device, which includes:

[0017] One or more processors;

[0018] Storage device for storing one or more programs;

[0019] When one or more programs are executed by one or more processors, the one or more processors implement any of the methods for calibrating the position of a moving article as provided in the embodiments of this application.

[0020] Fourthly, embodiments of this application also provide a storage medium including computer-executable instructions, which, when executed by a computer processor, are used to perform any of the methods for calibrating the position of a moving object as provided in embodiments of this application.

[0021] This application obtains the reference image capture position, reference acquisition position, and reference placement position of the reference object in the image acquisition coordinate system; the target image capture position and target acquisition position of the target object in the image acquisition coordinate system; and the first angle between the reference image capture position and the target image capture position, the second angle between the reference placement position and the reference image capture position, and the third angle between the reference acquisition position and the target acquisition position. The first, second, and third angles are determined in the image acquisition coordinate system for subsequent angle calibration of the reference and target grasping positions. Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system are determined according to the reference acquisition position and the target acquisition position. The coordinates are then transformed to the grasping device coordinate system. This method facilitates the subsequent determination of the target object's placement position. By using the reference image position, reference gripping position, target image position, target gripping position, first angle, second angle, and third angle, the method determines the reference calibration position after angle calibration of the reference gripping position in the gripping device coordinate system, and the target calibration position after angle calibration of the target gripping position in the gripping device coordinate system. Angle calibration unifies the image angles of the reference calibration position and the target calibration position to the same angle, improving the accuracy of the subsequently determined target placement position and compensating for errors caused by differences between the reference image position and the target image position, as well as errors caused by differences in the image angles of the reference gripping position and the target gripping position. Based on the reference calibration position and the target calibration position, the target placement position after angle calibration is determined for placing the target object. Therefore, the technical solution of this application solves the problems of low accuracy and low calibration efficiency caused by calibrating solely based on the coordinate difference between the placement position and gripping position using only a reference image, thus achieving the effect of improving the accuracy and efficiency of position calibration. Attached Figure Description

[0022] Figure 1 This is a flowchart of a method for calibrating the position of a moving article according to Embodiment 1 of this application;

[0023] Figure 2a This is a flowchart of a method for calibrating the position of a moving article according to Embodiment 2 of this application;

[0024] Figure 2b This is a schematic diagram of angle calibration of a reference gripping position according to Embodiment 2 of this application;

[0025] Figure 2c This is a schematic diagram of angle calibration at a photographic calibration position according to Embodiment 2 of this application;

[0026] Figure 2d This is a schematic diagram of angle calibration of the target grasping position in Embodiment 2 of this application;

[0027] Figure 2e This is a schematic diagram of angle calibration for a target placement position according to Embodiment 2 of this application;

[0028] Figure 3 This is a schematic diagram of the structure of a calibration device for the position of a moving article according to Embodiment 3 of this application;

[0029] Figure 4 This is a schematic diagram of the structure of an electronic device according to Embodiment 4 of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first" and "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Example 1

[0033] Figure 1 This is a flowchart of a method for calibrating the position of a moving object according to Embodiment 1 of this application. This embodiment is applicable to situations where the placement position of a grasped object is calibrated without stopping the movement. The method can be executed by a device for calibrating the position of a moving object. This device can be implemented in software and / or hardware and is specifically configured in an electronic device, such as a robot.

[0034] See Figure 1The calibration method for the position of the moving object shown includes the following steps:

[0035] S110. Obtain the reference shooting position, reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target shooting position and target acquisition position of the target object in the image acquisition coordinate system, the first angle between the reference shooting position and the target shooting position, the second angle between the reference placement position and the reference shooting position, and the third angle between the reference acquisition position and the target acquisition position.

[0036] The reference object can be a reference object used to obtain the reference photographing position, reference acquisition position, and reference placement position for calibration of the subsequent target acquisition position. The image acquisition coordinate system can be a coordinate system established with the camera's location as the origin, used to determine the coordinates of the captured image. The reference photographing position can be the position of the reference object within the camera's field of view. The reference acquisition position can be the position of the reference object in the image captured at the reference photographing position after being grasped by the grasping device, used to determine the position of the reference object when taking the photograph. For example, the reference acquisition position can be the geometric center or other specific position in the image captured by the reference object, which is not specifically limited in this application. Specifically, the reference acquisition position can be marked in the form of coordinates. The reference placement position can be the position of the reference object in the image captured at the placement point after being grasped by the grasping device, used to determine the position of the reference object when it is placed. Specifically, the reference placement position can be marked in the form of coordinates.

[0037] The target object can be any product grasped by the gripping equipment in actual production. The target image capture position can be the position of the target object within the camera's field of view when it is captured. The target acquisition position can be the position of the target object in the image obtained after it has been grasped by the gripping equipment and captured at the target image capture position, used to determine the position of the target object when it is captured. The relative relationships of the target acquisition position, the reference acquisition position, and the reference placement position are the same within the entire image they are in. For example, the relative relationship can be the geometric center in the image; that is, the target acquisition position is the geometric center of the image captured by the target object at the target image capture position; the reference acquisition position is the geometric center of the image captured by the reference object at the reference image capture position; and the reference placement position is the geometric center of the image captured by the reference object at the placement position.

[0038] The first angle can be the angle between the reference shooting position and the target shooting position, used to determine the difference in shooting angles between the reference shooting position and the target shooting position. The second angle can be the angle between the reference placement position and the reference shooting position, used to determine the difference in shooting angles between the reference placement position and the reference shooting position.

[0039] S120. Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, determine the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system according to the reference acquisition position and the target acquisition position.

[0040] The coordinate system of the grasping device can be a coordinate system centered at a fixed point within the grasping device. For example, the coordinate system of the grasping device can be a coordinate system centered at the location of the rotation center point within the grasping device. The coordinate transformation matrix can be a transformation matrix that converts coordinates in the image acquisition coordinate system to coordinates in the grasping device coordinate system. Specifically, multiplying the coordinates in the image acquisition coordinate system by the coordinate transformation matrix yields the corresponding coordinates in the grasping device coordinate system.

[0041] Move the end effector of the gripping device into the camera's field of view, and assign easily identifiable feature points. For example, feature points can be obtained by attaching printed circles to the fixture. Record the current coordinates of the gripping device Q0(x0,y0), identify the center image coordinates P0(u0,v0) of the corresponding circular feature point, and move the feature point nine times in a 3x3 grid within the camera's field of view using the gripping device. This yields nine sets of image coordinate points P0(u0,v0), P1(u1,v1), ..., P8(u8,v8) and their corresponding gripping device coordinates Q0(x0,y0), Q1(x1,y1), ..., Q8(x8,y8). At point P(u0,v0), the grasping device rotates the feature point within the camera's field of view by ±θ angles, obtaining image coordinate points P1′(u′1,v′1) and P2′(u′2,v′2). Using P1′(u′1,v′1) and P2′(u′2,v′2) and the 2θ angle, the center coordinates P0′(u′0,v′0) are calculated, which are the rotation center coordinates corresponding to the image feature point. The rotation center coordinates of the grasping device corresponding to the 9 sets of image coordinate points are P. i (u i -(u′0-u0),v i -(v′0-v0)). Calculate P i (u i -(u′0-u0),v i -(v′0-v0)) and Q i (x i ,y i The transformation relationship A between P*A and Q is given by the following formula:

[0042]

[0043] The reference gripping position can be the corresponding position of the reference acquisition position in the gripping device's coordinate system. Specifically, the coordinates of the reference acquisition position are obtained by multiplying the coordinates of the reference acquisition position by the coordinate system transformation matrix. The coordinates of the reference object's reference gripping position in the gripping device's coordinate system can be determined using the following formula:

[0044]

[0045] Where [x'1,y'1] represents the coordinates of the reference capture position; [p'1,q'1,1] represents the coordinates of the reference acquisition position. Coordinate system transformation matrix.

[0046] The target grasping position can be the corresponding position of the target acquisition position in the coordinate system of the grasping device. Specifically, the coordinates of the target acquisition position are obtained by multiplying the coordinates of the target acquisition position by the coordinate system transformation matrix. The coordinates of the target object's position in the coordinate system of the grasping device can be determined according to the following formula:

[0047]

[0048] Where [x1,y1] represents the coordinates of the reference capture position; [p1,q1,1] represents the coordinates of the reference acquisition position. Coordinate system transformation matrix.

[0049] S130. Based on the reference photo position, reference grab position, target photo position, target grab position, first angle, second angle, and third angle, determine the reference calibration position after calibrating the reference grab position with angle in the grab device coordinate system, and the target calibration position after calibrating the target grab position with angle in the grab device coordinate system.

[0050] The reference image capture position, reference gripping position, and first angle are used to calibrate the image capture angle of the reference gripping position. Since the image capture positions of the reference object and the target object are different, the image capture angles are also different, requiring calibration. Based on the reference image capture position, the reference placement position, and the second angle, the reference calibration position is determined in the gripping device coordinate system after angle calibration. That is, the reference gripping position is rotated by the second angle around the reference image capture position in the image capture coordinate system to obtain the reference calibration position.

[0051] Based on the target's photographing position, the third angle of the target's grasping position, and the second angle, the target calibration position is determined in the coordinate system of the grasping device after angular calibration of the target's grasping position. That is, the target's grasping position is rotated by the second angle around the target's photographing position in the photographing coordinate system, and then rotated by the third angle to obtain the target calibration position.

[0052] S140. Based on the reference calibration position and the target calibration position, determine the target placement position after the target object has been calibrated at an angle, and place the target object thereon.

[0053] The target placement location can be a preset position in the image obtained after the target object is grasped by the grasping device and photographed at the placement point. This position is used to determine the location of the target object when it is placed. Specifically, the target placement location can be marked in the form of coordinates.

[0054] After angle calibration, the reference calibration position and the target calibration position can be considered to be at the same shooting angle, that is, at the same angle as the target shooting position. Using the reference calibration position and the target calibration position, the translation amount required to move the reference calibration position to the target calibration position can be determined. Performing the same translation operation on the reference placement position yields the target placement position.

[0055] During the process of a gripping device grasping and placing a target object, changes in the object's incoming position can lead to inconsistent gripping positions when the device attempts to grasp it from a fixed location. This results in variations in the final placement position. For example, in a scenario where components are mounted on a circuit board, the target object can be a component. If the component's incoming position changes, placing it on the circuit board in its original position might cause misalignment or breakage of the component's pins, rendering the circuit board unusable. Therefore, after the gripping device grasps the target object, it needs to photograph it and calculate the changes in its position. Compensating for these positional variations during placement ensures consistency between the target object and the preset placement position.

[0056] In existing technologies, a coordinate transformation matrix A between the image acquisition coordinate system and the grasping device coordinate system is pre-calculated through camera calibration. The camera identifies the reference object's reference acquisition position, calculates the corresponding reference grasping position, and determines the reference placement position of the reference object. The coordinate difference between the two points is calculated, and other products calculate their placement point coordinates based on this difference. This method is simple and easy to implement, but it does not consider the angle of the shooting position, which limits its use. If the angle of the grasping device differs between the shooting position and the placement position, it will lead to a significant error in the placement position. Furthermore, because the device needs to stop moving at a preset shooting point to take a picture, the working efficiency of the grasping device is reduced.

[0057] The technical solution of this embodiment obtains the reference image capture position, reference acquisition position, and reference placement position of the reference object in the image acquisition coordinate system; the target image capture position and target acquisition position of the target object in the image acquisition coordinate system; and the first angle between the reference image capture position and the target image capture position, the second angle between the reference placement position and the reference image capture position, and the third angle between the reference acquisition position and the target acquisition position. The first, second, and third angles are determined in the image acquisition coordinate system for subsequent angle calibration of the reference and target grasping positions. Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system are determined according to the reference acquisition position and the target acquisition position. The coordinates are then transformed to the grasping device coordinate system. In the coordinate system, to facilitate subsequent determination of the target object's placement position; by using the reference image position, reference grasping position, target image position, target grasping position, first angle, second angle, and third angle, the reference calibration position after angle calibration of the reference grasping position in the grasping device coordinate system, and the target calibration position after angle calibration of the target grasping position in the grasping device coordinate system are determined; angle calibration unifies the image angles of the reference calibration position and the target calibration position to the same image angle, unifying the shooting angle, improving the accuracy of the subsequently determined target placement position, and compensating for errors caused by differences between the reference image position and the target image position, as well as errors caused by differences in the image angles of the reference grasping position and the target grasping position; by using the reference calibration position and the target calibration position, the target placement position after angle calibration of the target object is determined, for placing the target object. Therefore, the technical solution of this application solves the problems of low accuracy and low calibration efficiency caused by calibrating only the coordinate difference between the placement position and the grasping position using a reference image, achieving the effect of improving the accuracy and efficiency of position calibration.

[0058] Example 2

[0059] Figure 2a This is a flowchart of a method for calibrating the position of a moving object according to Embodiment 2 of this application. The technical solution of this embodiment is further refined based on the above technical solution.

[0060] Furthermore, the phrase "based on the reference photo position, reference grab position, target photo position, target grab position, first angle, second angle, and third angle, determine the reference calibration position after calibrating the reference grab position in the coordinate system of the grabbing device" is refined to: "using the target photo position as the rotation center position and the second angle as the rotation angle, determine the reference angle transformation matrix."

[0061] Based on the target image position, the reference grasping position, and the target angle transformation matrix, the reference grasping position is calibrated in the coordinate system of the grasping device to obtain the "reference calibrated position after angle calibration", so as to calibrate the reference grasping position.

[0062] The original text, "Based on the reference shooting position, reference grasping position, target shooting position, target grasping position, first angle, second angle, and third angle, determine the reference calibration position after angle calibration of the reference grasping position in the grasping device coordinate system, and the target calibration position after angle calibration of the target grasping position in the grasping device coordinate system," is further refined into: "Based on the target shooting position and the first angle, determine the shooting angle transformation matrix; use the shooting angle transformation matrix to calibrate the shooting angle of the target grasping position to obtain the shooting calibration position, so that the shooting angle of the shooting calibration position is consistent with the shooting angle at the reference shooting position; based on the shooting calibration position, target grasping position, second angle, and third angle, determine the target angle transformation matrix; based on the target shooting position, target grasping position, and target angle transformation matrix, perform angle calibration of the target grasping position in the grasping device coordinate system to obtain the target calibration position," to perform angle calibration of the target grasping position.

[0063] See Figure 2a A method for calibrating the position of a moving object, as shown, includes:

[0064] S210. Obtain the reference shooting position, reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target shooting position and target acquisition position of the target object in the image acquisition coordinate system, the first angle between the reference shooting position and the target shooting position, the second angle between the reference placement position and the reference shooting position, and the third angle between the reference acquisition position and the target acquisition position.

[0065] S220. Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, determine the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system according to the reference acquisition position and the target acquisition position.

[0066] S230. Take the target image position as the rotation center position and the second angle as the rotation angle to determine the reference angle transformation matrix.

[0067] The reference angle transformation matrix can be used to calibrate the angle of the reference capture position. Specifically, the target image position can be used as the rotation center, and a second angle can be used as the rotation angle to construct the reference angle transformation matrix.

[0068] The target image position is used as the rotation center to rotate the reference grasping position around the target image position, unifying the image angle of the reference grasping position to the image angle of the reference placement position. A second angle is used as the rotation angle to compensate for the angle difference between the reference grasping position and the reference placement position. The reference angle transformation matrix can be expressed as follows:

[0069]

[0070] Where (r2'-r0) is the second angle, (x0,y0,r0) is the target image position, and (x'2,y'2,r2') is the reference placement position.

[0071] By using the target image location as the rotation center and the second angle as the rotation angle, a reference angle transformation matrix is ​​determined. This allows for the rotation of the second angle around the target image location to calibrate the reference capture position.

[0072] S240. Based on the target image position, the reference grasping position, and the target angle transformation matrix, the reference grasping position is calibrated in the coordinate system of the grasping device to obtain the reference calibration position after angle calibration.

[0073] The reference calibration position can be the position obtained by calibrating the reference grasping position according to the reference angle transformation matrix. Starting from the target image position and ending at the reference grasping position, the vector coordinates corresponding to the reference grasping position are determined. The reference calibration position is then obtained by calibrating this vector according to the reference angle transformation matrix.

[0074] In one optional embodiment, the reference grasping position is calibrated in the coordinate system of the grasping device based on the target image position, the reference grasping position, and the target angle transformation matrix to obtain the reference calibration position after angle calibration. This includes: taking the target image position as the origin in the coordinate system of the grasping device; and determining the reference calibration position after angle calibration based on the product of the coordinates of the reference grasping position and the reference angle transformation matrix.

[0075] By using the target image location as the origin in the coordinate system of the capturing device, the coordinates of the reference capturing position can be determined. Then, by multiplying the coordinates of the reference capturing position by the reference angle transformation matrix, the reference calibration position after angle calibration can be determined. Specifically, the reference calibration position can be determined using the following formula:

[0076]

[0077] in, [x'1-x0,y'1-y0,1] represents the coordinates of the reference calibration position, and [x'1-x0,y'1-y0,1] represents the coordinates of the reference grasping position when the target image position is taken as the origin in the coordinate system of the grasping device.

[0078] Figure 2b This is a schematic diagram of angle calibration for a reference gripping position.

[0079] By using the target photo position, the reference grab position, and the target angle transformation matrix, the reference grab position is calibrated in the coordinate system of the grab device to obtain the calibrated reference position after angle calibration. This allows the reference grab position to rotate around the target photo position by a second angle, so that the reference grab position and the reference placement position are aligned at the same photo angle.

[0080] S250. Determine the shooting angle transformation matrix based on the target shooting position and the first angle.

[0081] The image angle transformation matrix can be used to calibrate the angle of the target image position. Specifically, the target image position can be used as the rotation center, and a first angle can be used as the rotation angle to construct a reference angle transformation matrix.

[0082] In one optional embodiment, determining the photo angle transformation matrix based on the target photo position and the first angle includes: taking the target photo position as the rotation center position and the first angle as the rotation angle to determine the photo angle transformation matrix.

[0083] The target image position is used as the rotation center to rotate the target grasping position around it, unifying the image angle of the target grasping position to the image angle of the reference image position. The first angle is used as the rotation angle to compensate for the angle difference between the target image position and the reference image position. The image angle transformation matrix can be expressed as follows:

[0084]

[0085] Where (r0'-r0) is the first angle, (x0,y0,r0) is the target image position, and (x'0,y'0,r0') is the reference image position.

[0086] By using the target image location as the rotation center and the first angle as the rotation angle, a photo angle transformation matrix is ​​determined. This allows for the rotation of the first angle around the target image location to calibrate the photo angle at the target capture position.

[0087] S260. Use the shooting angle transformation matrix to calibrate the shooting angle of the target capturing position to obtain the shooting calibration position, so that the shooting angle of the shooting calibration position is consistent with the shooting angle of the reference shooting position.

[0088] The image calibration position can be the position after calibrating the reference grasping position based on the image angle transformation matrix. The image position of the target grasping position is different from that of the reference grasping position, and therefore the image angle is different. The image angle of the target grasping position is calibrated according to the image angle transformation matrix so that the image angle of the image calibration position is consistent with the image angle of the reference image position.

[0089] In one optional embodiment, a photo angle transformation matrix is ​​used to calibrate the photo angle of the target grasping position to obtain a photo calibration position, so that the shooting angle of the photo calibration position is consistent with the shooting angle of the reference photo position. This includes: taking the target photo position as the origin in the coordinate system of the grasping device; determining the angle calibration position after angle calibration based on the product of the coordinates of the target grasping position and the photo angle transformation matrix; determining the photo translation amount after angle calibration based on the target photo position and the reference photo position; and determining the photo calibration position based on the photo translation amount and the angle calibration position.

[0090] The angle calibration position is the position after calibrating the photographing angle of the target grasping position. The target photographing position is taken as the origin in the coordinate system of the grasping device; the angle calibration position is determined by multiplying the coordinates of the target grasping position by the photographing angle transformation matrix. Specifically, the angle calibration position can be determined according to the following formula:

[0091]

[0092] in, The position is for angle calibration, and [x1-x0,y1-y0,1] is the coordinate of the target grab position.

[0093] Based on the target image position and the reference image position, determine the image translation amount after angle calibration at the angle calibration position. Then, based on the image translation amount and the angle calibration position, determine the image calibration position. Specifically, the image calibration position can be determined using the following formula:

[0094]

[0095] in, To calibrate the position for taking a picture, To calibrate the horizontal coordinate in the location for taking the picture, To calibrate the ordinate in the position for taking a picture, r1 b To calibrate the angular coordinates of the location for taking a picture.

[0096] Combining the two formulas above, the image calibration position can be obtained using the following formula:

[0097]

[0098] Figure 2c This is a schematic diagram illustrating angle calibration for a photographic calibration position.

[0099] By using the target image capture position as the origin in the coordinate system of the grasping device, and determining the angle calibration position after angle calibration based on the product of the target grasping position coordinates and the image angle transformation matrix, the image translation amount after angle calibration is determined based on the target image capture position and the reference image capture position, and the image calibration position is determined based on the image translation amount and the angle calibration position. The image angle of the target grasping position is calibrated to compensate for the angle difference caused by different image capture positions, thereby improving the accuracy of the subsequent target placement position. At the same time, by compensating for the angle error of the image capture position, the grasping device can be made to grasp and capture images without stopping its movement, thereby improving the working efficiency of the grasping device.

[0100] S270. Based on the photo calibration position, target capture position, second angle, and third angle, determine the target angle transformation matrix.

[0101] The target angle transformation matrix can be a matrix used to correct the angle of the target's capture position. Specifically, the image calibration position can be used as the rotation center, and the sum of the second and third angles can be used as the rotation angle to construct the target angle transformation matrix.

[0102] In one optional embodiment, determining the target angle transformation matrix based on the image calibration position, the target grasping position, the second angle, and the third angle includes: taking the image calibration position as the rotation center position and taking the second angle and the third angle as the rotation angle to determine the target angle transformation matrix.

[0103] The image calibration position is used as the rotation center to rotate the target grasping position around the image calibration position, unifying the image angle of the target grasping position to the image angle of the reference placement. The third angle between the reference acquisition position and the target acquisition position is the angle difference between the reference grasping position and the target grasping position. The second angle is the angle difference between the reference placement position and the reference grasping position. Therefore, the sum of the second and third angles is used as the rotation angle to compensate for the angle difference between the target grasping position and the reference placement position. The target angle transformation matrix can be expressed as follows:

[0104]

[0105] Among them, (r1'-r1 b (r2'-r0') is the second angle; (r2'-r0') is the third angle; (x1',y1',r1') is the reference gripping position.

[0106] By using the photo calibration position as the rotation center and the sum of the second and third angles as the rotation angle, the target angle transformation matrix is ​​determined. This allows for the calibration of the target grasping position angle by rotating the sum of the second and third angles with the photo calibration position as the rotation center.

[0107] S280. Based on the target image position, target capture position, and target angle transformation matrix, the target capture position is calibrated in the coordinate system of the capture device to obtain the calibrated target position after angle calibration.

[0108] The target calibration position can be the position after calibrating the target's capture position based on the target angle transformation matrix. Starting from the target's image capture position and ending at the target's capture position, the corresponding vector coordinates are determined. This vector is then calibrated using the target angle transformation matrix to obtain the target calibration position.

[0109] In one optional embodiment, the target capturing position is calibrated in the target coordinate system based on the target capturing position, the target grasping position, and the target angle transformation matrix to obtain the calibrated target position after angle calibration. This includes: taking the target capturing position as the origin in the target coordinate system; and determining the calibrated target position after angle calibration based on the product of the coordinates of the target grasping position and the target angle transformation matrix.

[0110] By using the target's image capture location as the origin in the coordinate system of the capturing device, the coordinates of the target's capture location can be determined. Then, by multiplying the target's capture location coordinates by the target angle transformation matrix, the calibrated target position after angle calibration can be determined. Specifically, the target calibration position can be determined using the following formula:

[0111]

[0112] in, To calibrate the coordinates of the target's location, The coordinates of the target capture position when the target's image capture position is used as the origin in the capture device's coordinate system.

[0113] Figure 2d A schematic diagram illustrating angle calibration for a target grasping position.

[0114] By taking the target image position as the origin in the coordinate system of the grasping device, and by multiplying the coordinates of the target grasp position with the target angle transformation matrix, the calibrated target position after angle calibration is determined, so that the target grasp position rotates around the target image position by the sum of the second and third angles, so that the target grasp position and the reference placement position are aligned at the same image angle.

[0115] S290. Based on the reference calibration position and the target calibration position, determine the target placement position after the target object has been calibrated at an angle, and place the target object thereon.

[0116] The technical solution of this embodiment involves determining a photo angle transformation matrix based on the target photo position and a first angle; using the photo angle transformation matrix to calibrate the photo angle of the target grasping position to obtain a photo calibration position, ensuring that the photo angle of the photo calibration position is consistent with the photo angle of the reference photo position; determining a target angle transformation matrix based on the photo calibration position, the target grasping position, a second angle, and a third angle; and calibrating the angle of the target grasping position in the coordinate system of the grasping device based on the target photo position, the target grasping position, and the target angle transformation matrix to obtain the angle-calibrated target calibration position.

[0117] In one optional embodiment, determining the target placement position after angle calibration of the target object based on the reference calibration position and the target calibration position, for placing the target object, includes: determining the target translation amount after angle calibration of the target object based on the reference calibration position and the target calibration position; and determining the target placement position based on the target translation amount and the reference placement position, for placing the target object.

[0118] The target translation amount is the translation between the reference calibration position and the target calibration position, and is used to determine the target placement position.

[0119] Based on the reference calibration position and the target calibration position, after angle calibration, their coordinates are at the same shooting angle. Therefore, based on the reference calibration position and the target calibration position, the target translation amount after angle calibration is determined. The reference placement position is then translated by the same amount of target translation until the target placement position is reached. Based on the target translation amount, the grasping device performs a corresponding translation to reach the target placement position and place the target object. Specifically, the target placement position can be determined using the following formula:

[0120]

[0121] Where (x2,y2,r2) is the target placement position.

[0122] Figure 2e A schematic diagram illustrating calibration for a target placement location.

[0123] By determining the target translation amount after angle calibration based on the reference calibration position and the target calibration position, the translation operation value of the gripping device can be determined. Based on the target translation amount and the reference placement position, the target placement position is determined for placing the target object. The angle-calibrated position can be determined to accurately place the target object.

[0124] Example 3

[0125] Figure 3 The diagram shown is a structural schematic of a calibration device for the position of a moving object according to Embodiment 3 of this application. This embodiment is applicable to situations where the placement position of a grasped object needs to be calibrated without stopping the movement. The specific structure of the calibration device for the position of the moving object is as follows:

[0126] The image parameter determination module 310 is used to obtain the reference image position, reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target image position and target acquisition position of the target object in the image acquisition coordinate system, as well as the first angle between the reference image position and the target image position, the second angle between the reference placement position and the reference image position, and the third angle between the reference acquisition position and the target acquisition position.

[0127] The coordinate transformation module 320 is used to determine the reference grasping position of the reference object in the grasping device coordinate system, the target grasping position of the target object in the grasping device coordinate system, and the third angle between the reference grasping position and the target grasping position based on the coordinate transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, according to the reference acquisition position and the target acquisition position.

[0128] Angle calibration module 330 is used to determine the reference calibration position after angle calibration of the reference grasping position in the grasping device coordinate system, and the target calibration position after angle calibration of the target grasping position in the grasping device coordinate system, based on the reference photographing position, reference grasping position, target photographing position, target grasping position, first angle, second angle and third angle.

[0129] The target placement calibration module 340 is used to determine the target placement position after angle calibration of the target object based on the reference calibration position and the target calibration position, and is used to place the target object.

[0130] The technical solution of this embodiment obtains the reference image capture position, reference acquisition position, and reference placement position of the reference object in the image acquisition coordinate system; the target image capture position and target acquisition position of the target object in the image acquisition coordinate system; and the first angle between the reference image capture position and the target image capture position, the second angle between the reference placement position and the reference image capture position, and the third angle between the reference acquisition position and the target acquisition position. The first, second, and third angles are determined in the image acquisition coordinate system for subsequent angle calibration of the reference and target grasping positions. Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system are determined according to the reference acquisition position and the target acquisition position. The coordinates are then transformed to the grasping device coordinate system. In the coordinate system, to facilitate subsequent determination of the target object's placement position; by using the reference image position, reference grasping position, target image position, target grasping position, first angle, second angle, and third angle, the reference calibration position after angle calibration of the reference grasping position in the grasping device coordinate system, and the target calibration position after angle calibration of the target grasping position in the grasping device coordinate system are determined; angle calibration unifies the image angles of the reference calibration position and the target calibration position to the same image angle, unifying the shooting angle, improving the accuracy of the subsequently determined target placement position, and compensating for errors caused by differences between the reference image position and the target image position, as well as errors caused by differences in the image angles of the reference grasping position and the target grasping position; by using the reference calibration position and the target calibration position, the target placement position after angle calibration of the target object is determined, for placing the target object. Therefore, the technical solution of this application solves the problems of low accuracy and low calibration efficiency caused by calibrating only the coordinate difference between the placement position and the grasping position using a reference image, achieving the effect of improving the accuracy and efficiency of position calibration.

[0131] Optional, the angle calibration module 330 includes:

[0132] The camera angle transformation matrix determination unit is used to determine the camera angle transformation matrix based on the target camera position and the first angle.

[0133] The target image position calibration unit is used to calibrate the image angle of the target grasping position using an image angle transformation matrix to obtain the image calibration position, so that the image angle of the image calibration position is consistent with the image angle at the reference image position.

[0134] The target angle transformation matrix determination unit is used to determine the target angle transformation matrix based on the image calibration position, the target grasping position, the second angle, and the third angle.

[0135] The target grasping position calibration unit is used to calibrate the target grasping position in the coordinate system of the grasping device based on the target image position, the target grasping position, and the target angle transformation matrix, thus calibrating the target position.

[0136] Optional, the target image position calibration unit includes:

[0137] The origin determination subunit is used to use the target image position as the origin in the coordinate system of the grasping device.

[0138] The angle calibration position determination subunit is used to determine the angle calibration position after angle calibration based on the product of the coordinates of the target capture position and the image angle transformation matrix.

[0139] The image translation amount determination subunit is used to determine the image translation amount after angle calibration based on the target image position and the reference image position;

[0140] The image calibration position determination subunit is used to determine the image calibration position based on the image translation amount and angle calibration position.

[0141] Optionally, the camera angle transformation matrix determination unit includes:

[0142] The image angle transformation matrix generation sub-unit is used to determine the image angle transformation matrix by taking the target image position as the rotation center position and the first angle as the rotation angle.

[0143] Optionally, the target angle transformation matrix determination unit includes:

[0144] The target angle transformation matrix construction sub-unit is used to determine the target angle transformation matrix by taking the target image position as the rotation center position and taking the sum of the second angle and the third angle as the rotation angle.

[0145] Optional, the angle calibration module 330 includes:

[0146] A reference angle transformation matrix construction unit is used to determine the reference angle transformation matrix by taking the target image position as the rotation center position and the second angle as the rotation angle;

[0147] The reference grasping position calibration unit is used to perform angle calibration on the reference grasping position in the coordinate system of the grasping device based on the target image position, the reference grasping position and the target angle transformation matrix, so as to obtain the reference calibration position after angle calibration.

[0148] Optionally, the target placement calibration module 340 includes:

[0149] The target translation determination unit is used to determine the target translation amount after angle calibration of the target object based on the reference calibration position and the target calibration position.

[0150] The target object placement unit is used to determine the target placement position based on the target translation amount and the reference placement position, and to place the target object.

[0151] The calibration device for the position of a sporting item provided in this application can execute the calibration method for the position of a sporting item provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the calibration method for the position of a sporting item.

[0152] Example 4

[0153] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application, as shown below. Figure 4 As shown, the electronic device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of processors 410 in the electronic device can be one or more. Figure 4 Taking a processor 410 as an example; the processor 410, memory 420, input device 430, and output device 440 in the electronic device can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0154] The memory 420, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the calibration method for the position of a moving object in this embodiment of the application (e.g., the image parameter determination module 310, coordinate transformation module 320, angle calibration module 330, and target placement position calibration module 340). The processor 410 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 420, thereby implementing the above-described calibration method for the position of a moving object.

[0155] The memory 420 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 420 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 420 may further include memory remotely located relative to the processor 410, which can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0156] Input device 430 can be used to receive input character information and generate key signal inputs related to user settings and function control of the electronic device. Output device 440 may include display devices such as a display screen.

[0157] Example 5

[0158] Embodiment 5 of this application also provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform a method for calibrating the position of a moving object. The method includes: acquiring, respectively, a reference image capture position, a reference acquisition position, and a reference placement position of a reference object in an image acquisition coordinate system; a target image capture position and a target acquisition position of a target object in the image acquisition coordinate system; a first angle between the reference image capture position and the target image capture position; a second angle between the reference placement position and the reference image capture position; and a third angle between the reference acquisition position and the target acquisition position; based on the coordinates between the image acquisition coordinate system and the grasping device coordinate system... The system transformation matrix determines the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system based on the reference acquisition position and the target acquisition position. Based on the reference image position, the reference grasping position, the target image position, the target grasping position, the first angle, the second angle, and the third angle, the reference calibration position after angle calibration of the reference grasping position in the grasping device coordinate system and the target calibration position after angle calibration of the target grasping position in the grasping device coordinate system are determined. Based on the reference calibration position and the target calibration position, the target placement position after angle calibration of the target object is determined for placing the target object.

[0159] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the method operations described above, but can also perform related operations in the calibration method for the position of moving items provided in any embodiment of this application.

[0160] Based on the above description of the implementation methods, those skilled in the art can clearly understand that this application can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0161] It is worth noting that in the embodiments of the search device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of this application.

[0162] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for calibrating the position of a moving object, characterized in that, include: The reference object's reference shooting position, reference acquisition position, and reference placement position in the image acquisition coordinate system are obtained respectively; the target object's target shooting position and target acquisition position in the image acquisition coordinate system are obtained respectively; the first angle between the reference shooting position and the target shooting position is obtained; the second angle between the reference placement position and the reference shooting position is obtained; and the third angle between the reference acquisition position and the target acquisition position is obtained respectively. Based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system are determined according to the reference acquisition position and the target acquisition position, respectively. Based on the reference image position, the reference grasping position, the target image position, the target grasping position, the first angle, the second angle, and the third angle, determine the reference calibration position after angle calibration of the reference grasping position in the grasping device coordinate system, and the target calibration position after angle calibration of the target grasping position in the grasping device coordinate system. Based on the reference calibration position and the target calibration position, a target placement position is determined after the target object has undergone angle calibration, for placing the target object.

2. The method according to claim 1, characterized in that, The step of determining the target calibration position in the coordinate system of the grasping device after angle calibration of the target grasping position based on the reference image position, the reference grasping position, the target image position, the target grasping position, the first angle, the second angle, and the third angle includes: Based on the target shooting position and the first angle, a shooting angle transformation matrix is ​​determined; The shooting angle is calibrated at the target capture position using the shooting angle conversion matrix to obtain the shooting calibration position, so that the shooting angle at the shooting calibration position is consistent with the shooting angle at the reference shooting position. Based on the image calibration position, the target image position, the second angle, and the third angle, a target angle transformation matrix is ​​determined; Based on the target image location, the target capture location, and the target angle transformation matrix, the target capture location is calibrated in the coordinate system of the capture device to obtain the calibrated target location after angle calibration.

3. The method according to claim 2, characterized in that, The step of calibrating the target shooting position using the shooting angle transformation matrix to obtain a shooting calibration position, so that the shooting angle at the shooting calibration position is consistent with the shooting angle at the reference shooting position, includes: The target image location is taken as the origin in the coordinate system of the capturing device; The angle calibration position after angle calibration is determined by multiplying the coordinates of the target capture position with the image angle transformation matrix. Based on the target image position and the reference image position, determine the image translation amount after angle calibration of the angle calibration position; The image calibration position is determined based on the image translation amount and the angle calibration position.

4. The method according to claim 2, characterized in that, The step of determining the shooting angle transformation matrix based on the target shooting position and the first angle includes: The target shooting position is used as the rotation center position, and the first angle is used as the rotation angle to determine the shooting angle transformation matrix.

5. The method according to claim 2, characterized in that, The step of determining the target angle transformation matrix based on the image calibration position, the target image position, the second angle, and the third angle includes: The target image location is used as the rotation center, and the sum of the second angle and the third angle is used as the rotation angle to determine the target angle transformation matrix.

6. The method according to claim 1, characterized in that, The step of determining the reference calibration position in the coordinate system of the grasping device after angle calibration of the reference grasping position based on the reference image position, the reference grasping position, the target image position, the target grasping position, the first angle, the second angle, and the third angle includes: The target image location is used as the rotation center position, and the second angle is used as the rotation angle to determine the reference angle transformation matrix; Based on the target image position, the reference grasping position, and the reference angle transformation matrix, the reference grasping position is calibrated in the coordinate system of the grasping device to obtain the reference calibration position after angle calibration.

7. The method according to claim 1, wherein determining the target placement position after angle calibration of the target object based on the reference calibration position and the target calibration position, for placing the target object, comprises: Based on the reference calibration position and the target calibration position, determine the target translation amount after angle calibration of the target object; Based on the target translation amount and the reference placement position, the target placement position is determined for placing the target object.

8. A device for calibrating the position of a moving article, characterized in that, include: The image capture parameter determination module is used to obtain the reference image capture position, reference acquisition position and reference placement position of the reference object in the image acquisition coordinate system, the target image capture position and target acquisition position of the target object in the image acquisition coordinate system, the first angle between the reference image capture position and the target image capture position, and the second angle between the reference placement position and the reference image capture position. The coordinate transformation module is used to determine the reference grasping position of the reference object in the grasping device coordinate system and the target grasping position of the target object in the grasping device coordinate system based on the coordinate system transformation matrix between the image acquisition coordinate system and the grasping device coordinate system, according to the reference acquisition position and the target acquisition position. An angle calibration module is used to determine, based on the reference image position, the reference grasping position, the target image position, the target grasping position, the first angle, and the second angle, the reference calibration position after angle calibration of the reference grasping position in the grasping device coordinate system, and the target calibration position after angle calibration of the target grasping position in the grasping device coordinate system. The target placement calibration module is used to determine the target placement position after angle calibration of the target object based on the reference calibration position and the target calibration position, and is used to place the target object.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for calibrating the position of a moving object as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the method for calibrating the position of a moving article as described in any one of claims 1-7.

Images