Data calibration method, device, computer equipment and storage medium

Abstract

The present application relates to a data calibration method, device, computer equipment and storage medium, which obtains first image data corresponding to multiple points on a calibration plate, determines a first coordinate group based on the first image data; performs camera calibration on a first camera element based on the first coordinate group to obtain a first mapping matrix; obtains rotational image data corresponding to different angles of rotation of a material picking arm at a reference material picking position and second image data corresponding to multiple points on the calibration plate when the material picking arm is at the reference material picking position, and determines a second coordinate group; performs rotational calibration and motion system calibration on a second camera element based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine calibration data. The present application adopts a variety of calibration methods to obtain a calibrated turntable system, so that the calibrated turntable system is suitable for products of different models or sizes, thereby improving the versatility and accuracy of the calibrated turntable system.

Description

Technical Field

[0001] The present application relates to the technical field of machine vision systems, and in particular to a data calibration method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of automated production and intelligent manufacturing technology, precision mechanical equipment in industrial production has gradually become popular. In the field of high-precision positioning and detection, such as semiconductor manufacturing, precision parts processing, and organic light-emitting semiconductor (Organic Electroluminescence Display, OLED) processing and detection, products need to be accurately handled before processing and testing. In the process of handling products, it is necessary to combine the turntable with machine vision and manipulators to accurately calibrate the posture of the product and complete high-precision handling operations. The process of combining the turntable with machine vision and manipulators depends on an accurate calibration method, that is, the system combining the turntable with machine vision and manipulators is calibrated using a calibration method to obtain a calibrated turntable system.

[0003] In traditional technology, a single calibration method is used to calibrate the system that combines a turntable with machine vision and a robot to obtain a calibrated turntable system.

[0004] However, in the actual operation of products, products of different models and specifications are often involved, among which products of different models and specifications have significant differences in shape and size. The calibrated turntable system determined by a single calibration method is difficult to adapt to the change of products, resulting in low flexibility and adaptability of the calibrated turntable system. Summary of the invention

[0005] Based on this, it is necessary to provide a data calibration method, device, computer equipment and storage medium that can adapt to products of different models and specifications to address the above technical problems, thereby improving the flexibility and adaptability of the calibrated turntable system.

[0006] In a first aspect, the present application provides a data calibration method, which is applied to a turntable system, wherein the turntable system includes a material picking table and a corresponding first camera, a material picking arm and a corresponding second camera, and the method includes:

[0007] Acquire first image data corresponding to a plurality of points on the calibration plate, and determine a first coordinate group based on the first image data;

[0008] Performing camera calibration on the first camera element based on the first coordinate group to obtain a first mapping matrix;

[0009] Obtaining the rotation image data corresponding to different angles of rotation of the picking arm at the reference picking position and the second image data corresponding to multiple points of the calibration plate when the picking arm is at the reference picking position;

[0010] determining a second set of coordinates based on the rotated image data and the second image data;

[0011] The second camera element is subjected to rotation calibration and motion system calibration based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine calibration data.

[0012] In one embodiment, the second coordinate set includes an initial registration coordinate set; and determining the second coordinate set based on the rotated image data and the second image data includes:

[0013] According to the rotation image data, multiple sets of rotation point coordinates and calibration plate point coordinates corresponding to the target points on the calibration plate are determined; wherein the rotation point coordinates are the coordinates corresponding to the points of the calibration plate in the rotation image coordinate system, and the calibration plate point coordinates are the coordinates corresponding to the points of the calibration plate in the calibration plate coordinate system;

[0014] According to the multiple groups of rotation point coordinates corresponding to the target points on the calibration plate and the calibration plate point coordinates, an initial registration coordinate group is obtained.

[0015] In one embodiment, the second mapping matrix comprises an initial registration mapping matrix; and the method further comprises:

[0016] Performing rotation calibration on the second camera based on the initial registration coordinate group to obtain an initial registration mapping matrix;

[0017] According to the initial registration mapping matrix, the coordinates of the calibration plate points corresponding to the optical center of the second camera element when the material picking arm rotates at different angles at the reference material picking position are determined to obtain a set of optical center point coordinates;

[0018] According to the optical center point coordinate set, the calibration plate point coordinates corresponding to the rotation center of the picking arm when the picking arm is at the reference picking position are determined to obtain initial rotation calibration data.

[0019] In one embodiment, the second image data includes first motion image data and second motion image data, and the second coordinate group also includes first motion coordinate group and second motion coordinate group; determining the second coordinate group based on the rotation image data and the second image data further includes:

[0020] According to the first motion image data, the coordinates of the first material picking arm points and the calibration plate points corresponding to each point on the calibration plate are determined; according to the coordinates of the first material picking arm points and the calibration plate points corresponding to multiple points on the calibration plate, a first motion coordinate group is obtained; wherein the coordinates of the first material picking arm points are the coordinates corresponding to the points on the calibration plate in the first material picking arm coordinate system;

[0021] According to the second motion image data, the second material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate are determined; according to the second material picking arm point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate, the second motion coordinate group is obtained; wherein the second material picking arm point coordinates are the coordinates corresponding to the points on the calibration plate in the second material picking arm coordinate system.

[0022] In one embodiment, the second mapping matrix further includes a first motion mapping matrix and a second motion mapping matrix; the method further includes:

[0023] Performing a first motion calibration on the second camera element based on the first motion coordinate group to obtain a first motion mapping matrix; performing a second motion calibration on the second camera element based on the second motion coordinate group to obtain a second motion mapping matrix;

[0024] The calibration data of the rotation center of the picking arm when the picking arm is at any position is determined according to the first motion mapping matrix, the second motion mapping matrix and the initial rotation calibration data.

[0025] In one embodiment, the first image data includes static image data, and the first coordinate group includes a static coordinate group; acquiring the first image data corresponding to a plurality of points of the calibration plate, and determining the first coordinate group based on the first image data, comprises:

[0026] According to the static image data, the static point coordinates and the calibration plate point coordinates corresponding to the target point on the calibration plate are determined; wherein the static point coordinates are the coordinates of the point in the static image coordinate system;

[0027] According to the static point coordinates corresponding to the target points on the calibration plate and the point coordinates of the calibration plate, a static coordinate group is obtained.

[0028] In one embodiment, the first image data includes displacement image data, and the first coordinate group also includes a displacement coordinate group; acquiring the first image data corresponding to a plurality of points of the calibration plate, and determining the first coordinate group based on the first image data, further includes:

[0029] According to the displacement image data, the axis point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate are determined; wherein the axis point coordinates are the coordinates of the point on the calibration plate in the camera axis coordinate system;

[0030] According to the axis point coordinates corresponding to the multiple points on the calibration plate and the calibration plate point coordinates, a displacement coordinate group is obtained.

[0031] In a second aspect, the present application further provides a data calibration device, which is applied to a turntable system, wherein the turntable system includes a material picking table and a corresponding first camera, a material picking arm and a corresponding second camera, and the device includes:

[0032] A first coordinate determination module, used to obtain first image data corresponding to a plurality of points on the calibration plate, and determine a first coordinate group based on the first image data;

[0033] A first mapping matrix determination module performs camera calibration on the first camera element based on the first coordinate group to obtain a first mapping matrix;

[0034] A second coordinate determination module is used to obtain the rotation image data corresponding to the different angles of rotation of the material picking arm at the reference material picking position and the second image data corresponding to multiple points of the calibration plate when the material picking arm is at the reference material picking position; determine the second coordinate group based on the rotation image data and the second image data;

[0035] The second mapping matrix determination module is used to perform rotation calibration and motion system calibration on the second camera element based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine calibration data.

[0036] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0037] Acquire first image data corresponding to a plurality of points on the calibration plate, and determine a first coordinate group based on the first image data;

[0038] Performing camera calibration on the first camera element based on the first coordinate group to obtain a first mapping matrix;

[0039] Obtaining the rotation image data corresponding to different angles of rotation of the picking arm at the reference picking position and the second image data corresponding to multiple points of the calibration plate when the picking arm is at the reference picking position;

[0040] determining a second set of coordinates based on the rotated image data and the second image data;

[0041] The second camera element is subjected to rotation calibration and motion system calibration based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine calibration data.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0043] Acquire first image data corresponding to a plurality of points on the calibration plate, and determine a first coordinate group based on the first image data;

[0044] Performing camera calibration on the first camera element based on the first coordinate group to obtain a first mapping matrix;

[0045] Obtaining the rotation image data corresponding to different angles of rotation of the picking arm at the reference picking position and the second image data corresponding to multiple points of the calibration plate when the picking arm is at the reference picking position;

[0046] determining a second set of coordinates based on the rotated image data and the second image data;

[0047] The second camera element is subjected to rotation calibration and motion system calibration based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine calibration data.

[0048] The above-mentioned data calibration method, device, computer equipment and storage medium are applied to a turntable system, which includes a feeding table and a corresponding first camera, a feeding arm and a corresponding second camera, and the method includes: obtaining first image data corresponding to multiple points on the calibration plate, and determining a first coordinate group based on the first image data; performing camera calibration on the first camera based on the first coordinate group to obtain a first mapping matrix; obtaining rotational image data corresponding to different angles of rotation of the feeding arm at the reference feeding position and second image data corresponding to multiple points on the calibration plate when the feeding arm is at the reference feeding position; determining a second coordinate group based on the rotational image data and the second image data; performing rotational calibration and kinematic system calibration on the second camera based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine the calibration data. The present application adopts a variety of calibration methods to obtain a calibrated turntable system, and these multiple calibration methods can adapt to products of different models and specifications, thereby improving the versatility and accuracy of the calibrated turntable system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 An application environment diagram of a data calibration method in an embodiment;

[0051] Figure 2 A schematic diagram of a flow chart of a data calibration method in an embodiment;

[0052] Figure 3 A schematic diagram of the structure of a turntable system in one embodiment;

[0053] Figure 4 A schematic diagram of a coordinate system related to a turntable system according to an embodiment;

[0054] Figure 5 A schematic diagram of manually placing a "teaching sheet" in one embodiment;

[0055] Figure 6 A schematic diagram of a process for determining an initial registration mapping matrix in one embodiment;

[0056] Figure 7 is a structural schematic diagram of a data calibration device in one embodiment;

[0057] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] The data calibration method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the turntable system 104 through the network, and the turntable system 104 includes a material picking table and a corresponding first camera, a material picking arm and a corresponding second camera. The data storage system can store data that the turntable system 104 needs to process. The data storage system can be integrated on the turntable system 104, or it can be placed on the cloud or other network servers. The terminal 102 sends a data calibration request to the turntable system 104, and the turntable system 104 receives the data calibration request, obtains the first image data corresponding to the multiple points of the calibration plate, and determines the first coordinate group based on the first image data; performs camera calibration on the first camera based on the first coordinate group to obtain a first mapping matrix; obtains the rotation image data corresponding to the rotation of the material picking arm at different angles at the reference material picking position and the second image data corresponding to the multiple points of the calibration plate when the material picking arm is at the reference material picking position; determines the second coordinate group based on the rotation image data and the second image data; performs rotation calibration and kinematic system calibration on the second camera based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine the calibration data. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The turntable system 104 can include an independent server or a server cluster consisting of multiple servers.

[0060] In an exemplary embodiment, Figure 2 As shown, a data calibration method is provided, which is applied to Figure 1 The turntable system in the embodiment of the present invention is taken as an example, the turntable system includes a material taking table and a corresponding first camera, a material taking arm and a corresponding second camera, and the method includes the following steps 201 to 205. Among them:

[0061] Step 201: Acquire first image data corresponding to a plurality of points on a calibration plate, and determine a first coordinate group based on the first image data.

[0062] Step 202: calibrate the first camera element based on the first coordinate group to obtain a first mapping matrix.

[0063] Among them, the turntable system may include a turntable and a positioning device. The turntable system is used to rotate and position the target product, obtain corresponding calibration data, and enable the visual system to obtain the position data of the target product at different angles and positions according to the calibration data of the turntable system. Among them, the calibration data of the turntable system may include the initial rotation calibration data of the picking arm when it is at the reference picking position, the calibration data of the rotation center of the picking arm when the picking arm is at any position, and the calibration data of any point of the target product. The first camera corresponding to the picking table refers to a camera device installed on the picking table, which is used to take relevant images. The target product refers to the product that the picking arm needs to pick up or put out during the precision machining or inspection process.

[0064] Optionally, in the turntable system, the calibration plate is placed on the material-collecting table, the first camera is moved to a fixed position, and the first camera is used to obtain the camera data corresponding to the multiple points on the calibration plate to obtain the first image data. According to the first image data, the corresponding first coordinate group of the multiple points on the calibration plate is determined. And the first camera is calibrated according to the corresponding first coordinate group of the multiple points on the calibration plate to obtain a first mapping matrix. Among them, the multiple points on the calibration plate may include a target point, which is set according to the actual situation and may be the center point of the calibration plate.

[0065] Step 203, obtaining the rotation image data corresponding to different rotation angles of the picking arm at the reference picking position and the second image data corresponding to multiple points of the calibration plate when the picking arm is at the reference picking position.

[0066] Step 204: Determine a second coordinate set based on the rotated image data and the second image data.

[0067] Step 205 , performing rotation calibration and motion system calibration on the second camera element based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine calibration data.

[0068] The second camera element refers to a camera device installed on the material picking arm, which is used to capture relevant images.

[0069] Optionally, in the turntable system, the second camera is moved to a fixed position, and the second camera is used to obtain the camera data corresponding to multiple points on the calibration plate when the material picking arm rotates at different angles at the reference material picking position, so as to obtain the second image data. According to the second image data, the second coordinate group corresponding to the multiple points on the calibration plate is determined. And the second camera is subjected to rotation calibration and kinematic system calibration according to the second coordinate group corresponding to the multiple points on the calibration plate, so as to obtain the second mapping matrix.

[0070] The above-mentioned data calibration method is applied to a turntable system, which includes a feeding table and a corresponding first camera, a feeding arm and a corresponding second camera. The method includes: obtaining first image data corresponding to multiple points on a calibration plate, and determining a first coordinate group based on the first image data; performing camera calibration on the first camera based on the first coordinate group to obtain a first mapping matrix; obtaining rotational image data corresponding to different angles of rotation of the feeding arm at a reference feeding position and second image data corresponding to multiple points on the calibration plate when the feeding arm is at the reference feeding position; determining a second coordinate group based on the rotational image data and the second image data; performing rotation calibration and motion system calibration on the second camera based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine the calibration data. The present application adopts a variety of calibration methods for the turntable system, which can make the calibrated turntable system suitable for target products of different models or sizes, thereby enhancing the flexibility and versatility of product posture calibration; at the same time, the posture of the target product can be calibrated according to the calibrated turntable system, so that the picking arm (manipulator) can pick up or put down the product, which can improve the accuracy and reliability of the picking or putting down operation.

[0071] In an exemplary embodiment, Figure 3 The figure is a schematic diagram of the structure of the turntable system, which includes a first turntable, a second turntable, a material picking platform, a camera axis, a first camera element corresponding to the material picking platform, a material picking device, a second camera element corresponding to the material picking arm, and a suction cup.

[0072] Optionally, the first turntable is used to carry and position the target product, so that the target product is within the operable range of the picking arm (manipulator) during the picking process, and the picking arm can accurately pick up the target product. During the picking and placing process, the picking arm transfers the target product from the first turntable to the second turntable to perform processing and inspection tasks on the target product. The camera axis is located above the picking table, and one of the axes of the camera axis is coaxially provided with two first camera elements along its axial direction. The camera axis can be used to carry the first camera element, and the position of the first camera element can also be adjusted so that the first camera element can collect camera data at different points. Optionally, the two first camera elements can move along one of the axes of the camera axis according to the parameters of the target product. The picking device includes a picking arm and a picking axis. The picking device is used to grab and place the target product, and the picking axis can adjust the position of the picking arm. The suction cup is located at the end of the picking arm, and can absorb the product by vacuum or clamping, and remove the target product from the first turntable when picking up the material, and place the target product on the second turntable when unloading the material. Optionally, the relative posture of the suction cup and the second camera is fixed to ensure that the posture of the target product will not change during the picking and placing process.

[0073] Exemplarily, during the calibration of the turntable system, the first camera captures the image of the target product placed on the feeding table of the first turntable. Specifically, after the first turntable is controlled to move the target product placed on the feeding table to the reference feeding position of the feeding arm, the first camera captures multiple points of the target product to obtain the corresponding first image data, and then obtain the first mapping matrix. The feeding arm is controlled to rotate at different angles at the reference feeding position, and the second camera obtains the corresponding rotation image data when the feeding arm rotates at different angles at the reference feeding position and the second image data corresponding to multiple points of the calibration plate when the feeding arm is at the reference feeding position, and then obtains the second mapping matrix. The first mapping matrix and the second mapping matrix are used to determine the calibration data.

[0074] Exemplarily, the calibration data of any point of the target product can be determined according to the first mapping matrix, and then the first posture data of the target product can be determined; the calibration data of the rotation center of the picking arm when the picking arm is at any position can be determined according to the second mapping matrix, and then the second posture data of the picking arm can be obtained. According to the first posture data, the second posture data and the standard posture of the target product, the posture compensation value of the picking arm corresponding to the initial posture of the target product is calculated, so that the picking arm can correctly pick up and put down the target product according to the posture compensation value of the picking arm. Among them, the standard posture of the target product is set according to the actual situation.

[0075] In an exemplary embodiment, Figure 4As shown, it is a schematic diagram of the coordinate system related to the turntable system. Optionally, multiple coordinate systems are established according to the structural schematic diagram of the turntable system, including the calibration plate coordinate system {C}, the static image coordinate system {I}, the camera axis coordinate system {Z}, the rotating image coordinate system {I}, the first material picking arm coordinate system {B1} and the second material picking arm coordinate system {B2}.

[0076] Optionally, the calibration plate coordinate system {C} is a coordinate system established based on the calibration plate, such as taking a specific point on the calibration plate as the origin, so that the x-axis and y-axis corresponding to the calibration plate coordinate system {C} are used to describe the position information on the calibration plate. The static image coordinate system {I} is a coordinate system used in the camera data captured by the first camera, such as taking the upper left corner of the image captured by the first camera as the origin, so that the x-axis and y-axis corresponding to the static image coordinate system {I} point to the horizontal and vertical directions of the image captured by the first camera, respectively. The camera axis coordinate system {Z} is aligned with the camera axis of the first camera, taking the center of the camera axis of the first camera as the origin, so that the x-axis and y-axis corresponding to the camera axis coordinate system {Z} point to the longitudinal axis and the transverse axis of the camera axis, respectively, and the z-axis points to the opposite direction of the shooting direction of the first camera. The camera axis coordinate system {Z} can be used to describe the spatial position and posture of the first camera. The rotating image coordinate system {i} is a coordinate system used in the camera data captured by the second camera when the material picking arm rotates at different angles at the reference material picking position. For example, the upper left corner of the image captured by the second camera is taken as the origin, so that the x-axis and y-axis corresponding to the rotating image coordinate system {i} point to the horizontal and vertical directions of the image captured by the second camera, respectively. The first material picking arm coordinate system {B1} can be a coordinate system established based on the material picking arm, and the second material picking arm coordinate system {B2} can be a coordinate system established based on the material picking axis.

[0077] In this embodiment, multiple coordinate systems corresponding to the turntable system are determined, and multiple calibration methods are adopted for the turntable system using these multiple coordinate systems, thereby calibrating the turntable system. This allows target products of different specifications and types to automatically adapt to these multiple coordinate systems, thereby improving the flexibility and adaptability of the calibrated turntable system.

[0078] In an exemplary embodiment, the reference position is determined according to a pre-built turntable system, and a "teaching sheet" needs to be placed manually for teaching, such as Figure 5 The figure shows the schematic diagram of manually placing the "teaching film". The placement of the "teaching film" includes: evenly placing two target products on the feeding table, and keeping the two target products parallel to the edge of the feeding table; making one end of the two target products in the same direction exceed the feeding table by a certain distance; making the distance between the edge of the two target products and the edge of the feeding table remain at D, , where W is the width of the material taking table, g is the distance between two target products, and w is the width of the target product.

[0079] In this embodiment, the reference material picking position and the reference camera position can be determined through the teaching film, which can provide a basis for the subsequent turntable system calibration process and provide reference position data for the material picking arm, thereby improving the accuracy of the material picking arm during the material picking and placing process.

[0080] In an exemplary embodiment, the first image data includes static image data, and the first coordinate group includes a static coordinate group; the first image data corresponding to multiple points on the calibration plate is obtained, and the first coordinate group is determined based on the first image data, including: determining the static point coordinates and the calibration plate point coordinates corresponding to the target point on the calibration plate according to the static image data; wherein the static point coordinates are the coordinates of the point in the static image coordinate system; and obtaining the static coordinate group according to the static point coordinates and the calibration plate point coordinates corresponding to the target point on the calibration plate.

[0081] Exemplarily, after the first camera is fixed at a preset camera position, the position of the calibration plate pre-placed on the material-collecting platform is adjusted so that the calibration plate can be within the camera field of view of the first camera, and the calibration plate can remain parallel to the camera field of view of the first camera. Wherein, the calibration plate is provided with a target point and a plurality of calibration points. After the position of the calibration plate is adjusted, the first camera is used to collect image data of the target point on the calibration plate to obtain static image data. According to the static image data and the static image coordinate system {I}, the static point coordinates corresponding to the target point on the calibration plate are determined, and the calibration plate point coordinates corresponding to the target point are determined according to the calibration plate coordinate system {C}. The static point coordinates corresponding to the target point on the calibration plate and the calibration plate point coordinates are recorded as a static coordinate group.

[0082] In the previous exemplary embodiment, the first camera element is statically calibrated according to the static coordinate group to obtain a static mapping matrix .

[0083] For example, the image point coordinates corresponding to the target point I0 on the calibration plate are marked as , the coordinates of the calibration plate point corresponding to the target point I0 on the calibration plate are marked as According to the static point coordinates corresponding to the target point on the fixed plate and the point coordinates on the calibration plate, i.e., the static coordinate group, the first camera is statically calibrated to obtain the static mapping matrix corresponding to the first camera after static calibration , The following relationship is satisfied:

[0084]

[0085] in, That is the view mapping matrix , A is a 2*2 non-singular square matrix, and A is obtained by a composite transformation, where the composite transformation can include isometric transformation, scaling transformation and push transformation. T is a translation vector used to represent the translation transformation.

[0086] In this embodiment, the first camera is used to collect image data of the target point on the calibration plate to obtain static image data. According to the static image data, the static image coordinate system {I} and the calibration plate coordinate system {C}, the static point coordinates and the calibration plate point coordinates corresponding to the target point in the static image data are determined, and the mapping relationship between the static image coordinate system {I} and the calibration plate coordinate system {C} is established to obtain the static mapping matrix , which can ensure the accuracy of the calibrated turntable system, and then accurately measure the position and posture data of the target product according to the camera data corresponding to the first camera element of the target product, thereby improving the accuracy of the material picking by the picking arm.

[0087] In an exemplary embodiment, the first image data includes displacement image data, and the first coordinate group includes a displacement coordinate group; obtaining the first image data corresponding to multiple points on the calibration plate, and determining the first coordinate group based on the first image data, further comprising: determining the axial point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate according to the displacement image data; wherein the axial point coordinates are the coordinates of the points on the calibration plate in the camera axis coordinate system; and obtaining the displacement coordinate group according to the axial point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate.

[0088] Exemplarily, the camera axis is controlled to move, and the first camera is used to collect image data of multiple points on the calibration plate to obtain multiple displacement image data, and the center of the image corresponding to each displacement image data is infinitely close to the center of one of the points. For each displacement image data, the axis point coordinates corresponding to the point corresponding to the displacement image data are determined according to the displacement image data and the camera axis coordinate system {Z}, and the calibration plate point coordinates corresponding to the point corresponding to the displacement image data are determined according to the displacement image data and the calibration plate coordinate system {C}. According to the axis point coordinates and calibration plate point coordinates corresponding to the points corresponding to each displacement image data, multiple groups of displacement coordinate groups are obtained.

[0089] In the previous exemplary embodiment, the first camera element is displacement calibrated according to the multiple sets of displacement coordinate sets to obtain a displacement mapping matrix .

[0090] In this embodiment, according to each displacement image data, the axis point coordinates and calibration plate point coordinates corresponding to each point on the calibration plate are determined to obtain multiple sets of displacement coordinate groups, and the mapping relationship between the camera axis coordinate system {Z} and the calibration plate coordinate system {C} is established to obtain the displacement mapping matrix , so that the calibrated first camera element can adapt to dynamic changes, thereby improving the universality of the calibrated turntable system.

[0091] In the previous exemplary embodiment, according to the static mapping matrix and the displacement mapping matrix Determine the calibration data, that is, according to the static mapping matrix and the displacement mapping matrix The calibration data of any point of the target product can be determined, and then the pose data of any point of the target product on the calibration board can be determined. The pose data of any point of the target product on the calibration board refers to the coordinates of any point of the image data acquired by the first camera in the calibration board coordinate system {C} when the camera axis is at any camera position.

[0092] Exemplarily, in the image data acquired by the camera device (such as the first camera and the second camera), two specific points undergo the same mapping matrix transformation, and the relative distance between the two points in the calibration plate coordinate system {C} remains unchanged. Suppose any point on the image data acquired by the first camera The corresponding coordinates in the calibration plate coordinate system {C} are , let the image center of the image data be The corresponding coordinates in the calibration plate coordinate system {C} are , then we get:

[0093]

[0094] According to the above formula , we can find the imaging axis corresponding to the first imaging element at any imaging position (where the axis coordinate of the imaging axis is marked as ), any point of the image data obtained by the first camera Coordinates in the calibration plate coordinate system {C} for:

[0095] (3)

[0096] in, It represents the coordinates of the optical center of the first imaging element (the image center of the image data acquired by the first imaging element) in the calibration plate coordinate system {C} when the imaging axis is at any imaging position. That is, the calibration data of any point of the target product, according to The position and posture data of any point of the target product on the calibration plate can be determined.

[0097] In this embodiment, according to the static mapping matrix and the displacement mapping matrix Determining the calibration data of any point of the target product, and then determining the posture data of any point of the target product on the calibration plate, can ensure the accuracy of the posture data of the target product, and then ensure the accuracy of the picking or placing operation of the picking arm.

[0098] In an exemplary embodiment, the second coordinate group includes an initial registration coordinate group; based on the rotated image data and the second image data, the second coordinate group is determined, including: according to the rotated image data, determining multiple groups of rotation point coordinates and calibration plate point coordinates corresponding to the target points on the calibration plate; wherein the rotation point coordinates are the coordinates corresponding to the points on the calibration plate in the rotated image coordinate system, and the calibration plate point coordinates are the coordinates corresponding to the points on the calibration plate in the calibration plate coordinate system; according to the multiple groups of rotation point coordinates and calibration plate point coordinates corresponding to the target points on the calibration plate, an initial registration coordinate group is obtained.

[0099] Fix the calibration plate on the turntable (first turntable), and control the first turntable to rotate so that the calibration plate can be within the camera field of view of the second camera, and move the material picking arm to the reference material picking position. After the material picking arm is moved to the reference material picking position, control the material picking arm to rotate multiple times in sequence, and maintain the rotation angle of the material picking arm each time , and the number of rotations of the material taking arm . After each rotation of the picking arm at the reference picking position, the second camera is used to collect image data of the target point on the calibration plate to obtain corresponding rotation image data. After controlling the picking arm to rotate multiple times in sequence, the second camera obtains the rotation image data corresponding to the picking arm at multiple rotation angles at the reference picking position. Among them, the target point on the calibration plate can correspond to the image center of the rotation image data.

[0100] Optionally, each rotational image data corresponds to a rotation angle of a material picking arm at a reference material picking position. For each rotational image data, the rotational point coordinates corresponding to the image center corresponding to the rotational image data are determined according to the rotational image data and the rotational image coordinate system {i}, and the calibration plate point coordinates corresponding to the image center corresponding to the rotational image data are determined according to the rotational image data and the calibration plate coordinate system {C}. According to the rotational point coordinates corresponding to the target point on the calibration plate and the calibration plate point coordinates at each rotation angle of the material picking arm at the reference material picking position, multiple groups of initial registration coordinate groups are obtained.

[0101] In this embodiment, obtaining the image data corresponding to the material picking arm when it rotates at different angles in the reference material picking position helps to reduce errors and enhance the accuracy of the turntable system calibration process, thereby ensuring the reliability of the mapping relationship between the rotating image coordinate system {i} and the calibration plate coordinate system {C}; at the same time, calibrating the material picking arm at multiple angles in the reference position can enable the turntable system to adapt to products of different models and specifications, thereby improving the universality and flexibility of product posture calibration.

[0102] In an exemplary embodiment, the second mapping matrix includes an initial registration mapping matrix; Figure 6 As shown, determining the initial registration mapping matrix includes steps 602 to 606. Among them:

[0103] Step 602: rotationally calibrate the second camera element based on the initial registration coordinate group to obtain an initial registration mapping matrix.

[0104] Step 604, according to the initial registration mapping matrix, determine the calibration plate point coordinates corresponding to the optical center of the second camera element when the picking arm rotates at different angles at the reference picking position, and obtain the optical center point coordinate set.

[0105] Step 606, determining the calibration plate point coordinates corresponding to the rotation center of the picking arm when the picking arm is at the reference picking position according to the optical center point coordinate set, and obtaining initial rotation calibration data.

[0106] The optical center of the second camera element refers to the center of the image corresponding to the image data acquired by the second camera element.

[0107] For example, for each rotation angle of the material picking arm at the reference material picking position, the initial registration mapping matrix , find the coordinates of the calibration plate coordinate system {C} corresponding to the optical center of the second camera, and obtain an optical center point coordinate. The optical center point coordinates corresponding to the optical center of the second camera under multiple rotation angles of the material picking arm at the reference material picking position are marked as the optical center point coordinate set, and the optical center point coordinate set is expressed as .in, It is the coordinate in the calibration plate coordinate system {C} corresponding to the optical center of the second camera element at the i-th rotation angle of the material picking arm in the reference position.

[0108] The least squares method is used to operate the optical center point coordinate set P, and a circle is fitted to obtain a fitting circle, and the sum of the squares of the distances from each point coordinate in the optical center point coordinate set P to the circumference of the fitting circle is minimized. The calculation formula is as follows:

[0109]

[0110] in, It can be used to represent the sum of the squares of the distances from each point coordinate in the optical center point coordinate set P to the circumference. is the radius of the fitted circle. The coordinates of the center of the fitted circle are marked as , That is, the coordinate of the calibration plate point corresponding to the rotation center of the material picking arm when the material picking arm is in the reference material picking position. The rotation center of the material picking arm is the rotation center of the suction cup. Recorded as the initial rotation calibration data.

[0111] In this embodiment, according to the initial registration mapping matrix , determine the calibration plate point coordinates corresponding to the optical center of the second camera element when the picking arm rotates at different angles in the reference picking position, obtain the optical center point coordinate set, and then obtain the initial rotation calibration data, which can make the initial rotation calibration data applicable to products of various models and specifications, so that the picking arm can quickly and accurately perform picking operations.

[0112] In an exemplary embodiment, the second image data includes first motion image data and second motion image data, and the second coordinate group also includes a first motion coordinate group and a second motion coordinate group; based on the rotation image data and the second image data, the second coordinate group is determined, and it also includes: determining the first material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate according to the first motion image data; obtaining the first motion coordinate group according to the first material picking arm point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate; wherein the first material picking arm point coordinates are the coordinates corresponding to the points on the calibration plate in the first material picking arm coordinate system; determining the second material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate according to the second motion image data; obtaining the second motion coordinate group according to the second material picking arm point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate; wherein the second material picking arm point coordinates are the coordinates corresponding to the points on the calibration plate in the second material picking arm coordinate system.

[0113] Optionally, the first axial direction of the material picking arm is fixed to the reference material picking position, wherein the first axial direction of the material picking arm can be the Y axis of the material picking axis. By controlling the angle of the suction cup at the end of the material picking arm so that the camera field of view of the second camera element is parallel to the calibration plate, the x-axis and y-axis of the material picking arm are controlled to move multiple times, and the second camera element is used to collect image data of multiple points on the calibration plate to obtain multiple first motion image data. Among them, the image center corresponding to each first motion image data is infinitely close to the center of one of the points on the calibration plate. For each first motion image data, the first material picking arm point coordinates corresponding to the corresponding point of the first motion image data are determined according to the first motion image data and the first material picking arm coordinate system {B1}, and the calibration plate point coordinates corresponding to the corresponding point of the first motion image data are determined according to the first motion image data and the calibration plate coordinate system {C}. According to the first material picking arm point coordinates and calibration plate point coordinates corresponding to the corresponding points of each first motion image data, multiple groups of first motion coordinate groups are obtained.

[0114] The second axial direction of the material picking arm is moved to the reference material picking position, wherein the second axial direction of the material picking arm can be the x-axis of the material picking arm. By controlling the angle of the suction cup at the end of the material picking arm so that the camera field of view of the second camera element is kept parallel to the calibration plate, the x-axis of the material picking arm and the Y-axis of the material picking axis are controlled to move multiple times, and the second camera element is used to collect image data of multiple points on the calibration plate to obtain multiple second motion image data. Among them, the image center corresponding to each second motion image data is infinitely close to the center of one of the points on the calibration plate. For each second motion image data, the second material picking arm point coordinates corresponding to the corresponding point of the second motion image data are determined according to the second motion image data and the second material picking arm coordinate system {B2}, and the calibration plate point coordinates corresponding to the corresponding point of the second motion image data are determined according to the second motion image data and the calibration plate coordinate system {C}. According to the second material picking arm point coordinates and calibration plate point coordinates corresponding to the corresponding points of each second motion image data, multiple sets of second motion coordinate groups are obtained.

[0115] In the previous exemplary embodiment, the method also includes: performing a first motion calibration on the second camera element based on the first motion coordinate group to obtain a first motion mapping matrix; performing a second motion calibration on the second camera element based on the second motion coordinate group to obtain a second motion mapping matrix; and determining the calibration data of the rotation center of the material picking arm when the material picking arm is in any position according to the first motion mapping matrix, the second motion mapping matrix and the initial rotation calibration data.

[0116] Optionally, according to the plurality of first motion coordinate groups, the second camera element is calibrated for the first motion to obtain a first motion mapping matrix According to the multiple sets of second motion coordinate groups, the second camera element is calibrated for the second motion to obtain the second motion mapping matrix .

[0117] According to the first motion mapping matrix and the second motion mapping matrix , obtain the calibration plate point coordinates corresponding to the rotation center of the picking arm when the xyY axis of the picking device is at any position (x, y, Y), that is, when the picking arm is at any position. The rotation center of the picking arm refers to the rotation center of the suction cup at the end of the picking arm of the picking device. Optionally, the xy axis is the axial direction corresponding to the picking arm, and the Y axis is the axial direction corresponding to the picking arm. When the xyY axis of the picking device is at any position (x, y, Y), the calibration plate point coordinates corresponding to the rotation center of the picking arm, that is, the calibration data of the rotation center of the picking arm Determined by the following formula:

[0118]

[0119] In the above formula, is the initial rotation calibration data, is the calibration plate point coordinate corresponding to the optical center of the second camera when the material picking arm is at the reference material picking position, is the coordinate of the calibration plate point corresponding to the optical center of the second camera when the material picking arm is at the position (x, y, Y0), is the coordinate of the calibration plate point corresponding to the optical center of the second camera when the material picking axis is at the position (x, y, Y0), It is the coordinate of the point on the calibration plate corresponding to the optical center of the second camera when the material picking axis is at the axis position (x, y, Y).

[0120] In this embodiment, the data of the picking arm in the xy-axis coordinate system and the xY-axis coordinate system (i.e., the first picking arm coordinate system {B1} and the second picking arm coordinate system {B2}) are calibrated respectively to obtain the first motion mapping matrix and the second motion mapping matrix, and the motions of different axes of the picking device are associated through the first motion mapping matrix and the second motion mapping matrix, so that the data of the picking arm in different motion states can be accurately calibrated, thereby improving the flexibility of the picking arm in operation under complex scenarios; at the same time, according to the first motion mapping matrix, the second motion mapping matrix and the initial rotation calibration data, the calibration data of the rotation center of the picking arm when the picking arm is in any position is determined, so that the calibrated turntable system can quickly obtain the corresponding posture data of the rotation center of the picking arm on the calibration plate at any position and posture of the picking arm, thereby improving the efficiency and accuracy of the picking or placing operation of the picking arm.

[0121] In an exemplary embodiment, a standard posture teaching is performed on the target product to obtain the standard posture of the target product. For example, through the standard posture teaching, the connection line between two key points (MARK points) of the target product is obtained, and the angle and midpoint data of the connection line between the two MARK points are recorded as the standard posture of the target product.

[0122] Exemplarily, two MARK points are pre-set on the target product; after the target product is placed at the reference material collection position, a camera device is used to photograph the two MARK points of the target product to obtain corresponding image data. The camera device may include a first camera element and a second camera element. The angle of the line connecting the two MARK points is calculated based on the image data. Data and midpoint data, and the angle Data and midpoint The data is recorded as the standard posture of the target product. Optionally, the angle of the line connecting the two MARK points It can be the angle between the two MARK points and a certain reference direction. For example, the angle between the two MARK points and the positive direction of the X-axis of the calibration plate coordinate system {C} can be calculated using the inverse tangent function to obtain the angle between the two MARK points. .

[0123] In the process of picking up the target product, the first camera and the second camera are controlled to respectively pick up the target product to obtain corresponding image data, and the calibration data of any point of the target product is determined according to the image data and the first mapping matrix, and then the posture data of any point of the target product on the calibration plate is determined, and the posture data is recorded as the first posture data. According to the image data and the second mapping matrix, the calibration data of the rotation center of the picking arm when the picking arm is at any position is determined, and then the posture data corresponding to the rotation center of the picking arm on the calibration plate when the picking arm is at any position is determined, and the posture data is recorded as the second posture data.

[0124] Optionally, determine the midpoint of the line connecting the two MARK points of the target product based on the first pose data , and determine the angle of the line connecting the two MARK points of the target product based on multiple coordinate systems . Optional, the angle of the line connecting the two MARK points of the target product The angle between the two MARK points and a certain reference direction can be set as the midpoint of the line connecting the two MARK points of the target product. and angle Denote as the initial posture of the target product.

[0125] The compensation value of the material picking arm when performing the material picking operation is calculated according to the standard posture and the initial posture of the target product, and the material picking arm performs the material picking operation on the target product according to the compensation value, wherein the compensation value includes an angle compensation value.

[0126] Exemplarily, the first camera and the second camera are controlled to respectively capture Mark 1 and Mark 2 on the target product to be picked up, and then the initial posture of the target product is determined to be ( , ). According to the standard posture of the target product ( , ) and the initial posture of the target product ( , ) Calculate the angle difference , get the angle compensation value during the material picking operation .

[0127] Determine the calibration data of the rotation center of the material picking arm when the material picking arm is in any position according to the second motion mapping matrix , and then determine the second posture data corresponding to the rotation center of the picking arm when the picking arm is at any position . When determining the second pose data Then, the initial posture of the target product corresponds to the midpoint of Mark1-2 Around Rotation , get the midpoint of the rotated Mark1-2 .in, It can be expressed as:

[0128]

[0129] In the above formula, and is the coordinate of the rotation center in the calibration plate coordinate system {C}, is the coordinate of the optical center of the second camera element in the calibration plate coordinate system {C} when the material picking arm is in the reference material picking position, for . The angle compensation value when obtaining the material picking operation Then, the initial posture of the target product is adjusted according to the compensation value. Calibrate and after calibrating the initial posture of the target product, control the picking arm to pick up the target product.

[0130] In this embodiment, the target product is taught with a standard posture to obtain the standard posture of the target product, and the initial posture of the target product is calibrated according to the standard posture, which can ensure that the material picking arm grabs the target product with a consistent posture during the material picking or material discharging process, thereby effectively improving the accuracy and stability of material picking. Using multiple calibration methods to obtain the first mapping matrix and the second mapping matrix, and determining the initial posture of the target product based on multiple coordinate systems and the first mapping matrix can improve the versatility of the target product posture determination; calibrating the initial posture of the target product according to the standard posture of the target product and the second mapping matrix can effectively improve the accuracy and stability of material picking or material discharging to adapt to the material picking or material discharging operation requirements of products in different environments.

[0131] In an exemplary embodiment, when the target product is quickly changed, that is, when another product (new product) on the turntable system is picked up or placed, the camera position of the camera device needs to be determined to ensure that the camera device can accurately capture the mark points on the new product for positioning and identification.

[0132] For example, the target camera position of each Mark point on the new product is determined based on the spacing between the Mark points on the new product and the distance from each Mark point on the new product to the edge of the product in the Y direction. The Y direction is the Y direction of the calibration plate coordinate system {C}, and the new product may include two Mark points. Assume that the coordinates of the reference camera position of the camera device in the calibration plate coordinate system {C} are The axis coordinates of the target camera positions of Mark1 and Mark2 on the new product are and , then determine the reference camera position and and The calculation relationship between them is as follows:

[0133]

[0134] Where g is the spacing between new products, The distance between the mark points on the new product. This is the preset distance that the new product exceeds the material reclaiming table. The distance is set according to the actual situation. It is the distance from the mark point on the new product to the edge of the new product.

[0135] In this embodiment, multiple coordinate systems are used to calibrate the turntable system using multiple calibration methods to obtain a calibrated turntable system. The calibrated turntable system is used to determine the spacing between the Mark points on the new product, as well as the distance from each Mark point on the new product to the edge of the product in the Y direction, and then the target camera position of each Mark point on the new product is determined. This can improve the accuracy and efficiency of changing the model of the target product. Furthermore, using the calibrated turntable system for model changing does not require manual teaching, which can reduce the risks brought by manual operation.

[0136] In another embodiment, a data calibration method is provided, the method comprising:

[0137] Acquire first image data corresponding to multiple points on the calibration plate, and determine a first coordinate group based on the first image data; the first image data includes static image data, and the first coordinate group includes a static coordinate group; determine the static point coordinates corresponding to the target point on the calibration plate and the calibration plate point coordinates according to the static image data; wherein the static point coordinates are the coordinates of the point in the static image coordinate system; obtain the static coordinate group according to the static point coordinates corresponding to the target point on the calibration plate and the calibration plate point coordinates.

[0138] The first image data also includes displacement image data, and the first coordinate group also includes a displacement coordinate group; according to the displacement image data, the axial point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate are determined; wherein the axial point coordinates are the coordinates of the points on the calibration plate in the camera axis coordinate system; according to the axial point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate, the displacement coordinate group is obtained.

[0139] The first camera element is calibrated based on the first coordinate group to obtain a first mapping matrix, wherein the first mapping matrix is ​​used to determine calibration data.

[0140] Obtain the rotational image data corresponding to when the picking arm rotates at different angles at the reference picking position, and determine multiple sets of rotational point coordinates and calibration plate point coordinates corresponding to the target points on the calibration plate based on the rotational image data; wherein the rotational point coordinates are the coordinates corresponding to the points on the calibration plate in the rotational image coordinate system, and the calibration plate point coordinates are the coordinates corresponding to the points on the calibration plate in the calibration plate coordinate system; obtain the initial alignment coordinate set based on the multiple sets of rotational point coordinates and calibration plate point coordinates corresponding to the target points on the calibration plate.

[0141] Based on the initial registration coordinate group, the second camera element is rotationally calibrated to obtain an initial registration mapping matrix; according to the initial registration mapping matrix, the calibration plate point coordinates corresponding to the optical center of the second camera element when the material picking arm is rotated by different angles at the reference material picking position are determined to obtain an optical center point coordinate set; according to the optical center point coordinate set, the calibration plate point coordinates corresponding to the rotation center of the material picking arm when the material picking arm is at the reference material picking position are determined to obtain initial rotation calibration data.

[0142] According to the first motion image data, the first material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate are determined; according to the first material picking arm point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate, the first motion coordinate group is obtained; wherein the first material picking arm point coordinates are the coordinates corresponding to the points on the calibration plate in the first material picking arm coordinate system.

[0143] According to the second motion image data, the second material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate are determined; according to the second material picking arm point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate, the second motion coordinate group is obtained; wherein the second material picking arm point coordinates are the coordinates corresponding to the points on the calibration plate in the second material picking arm coordinate system.

[0144] Based on the first motion coordinate group, the second camera element is calibrated for the first motion to obtain a first motion mapping matrix; based on the second motion coordinate group, the second motion calibration is performed on the second camera element to obtain a second motion mapping matrix; according to the first motion mapping matrix, the second motion mapping matrix and the initial rotation calibration data, the calibration data of the rotation center of the material picking arm when the material picking arm is in any position is determined.

[0145] In this embodiment, the static point coordinates corresponding to the target point on the calibration plate and the point coordinates of the calibration plate are determined, and the first camera is statically calibrated to obtain a static mapping matrix , can establish the mapping relationship between the static image coordinate system {I} and the calibration plate coordinate system {C}; according to the static mapping matrix , which can ensure the accuracy of the target product posture data. Determine the axis point coordinates and calibration plate point coordinates corresponding to multiple points on the calibration plate, perform displacement calibration on the first camera, and obtain the displacement mapping matrix , establish the mapping relationship between the camera axis coordinate system {Z} and the calibration plate coordinate system {C}, so that the calibrated turntable system can adapt to dynamic changes. According to the static mapping matrix and the displacement mapping matrix Determining the calibration data of any point of the target product can obtain more accurate position and posture data of the target product, thereby improving the accuracy of the picking or placing operation of the picking arm. Obtaining the corresponding image data when the picking arm rotates at different angles in the reference picking position helps to reduce errors and enhance the accuracy of the turntable system calibration process, thereby ensuring the reliability of the mapping relationship between the rotating image coordinate system {i} and the calibration plate coordinate system {C}; at the same time, calibrating the picking arm at multiple angles in the reference position can enable the turntable system to adapt to products of different models and specifications, thereby improving the universality and flexibility of product posture calibration. According to the initial registration mapping matrix , determine the calibration plate point coordinates corresponding to the optical center of the second camera when the material picking arm rotates at different angles at the reference material picking position, obtain the optical center point coordinate set, and then obtain the initial rotation calibration data, which can make the initial rotation calibration data applicable to products of various models and specifications. By associating the movements of different axes of the material picking device through the first motion mapping matrix and the second motion mapping matrix, the data of the material picking arm in different motion states can be accurately calibrated, improving the flexibility of the material picking arm in complex scenes; at the same time, according to the first motion mapping matrix, the second motion mapping matrix and the initial rotation calibration data, determine the calibration data of the rotation center of the material picking arm when the material picking arm is in any position, so that the calibrated turntable system can quickly determine the posture data corresponding to the rotation center of the material picking arm on the calibration plate at any position and posture of the material picking arm, thereby improving the efficiency and accuracy of the material picking or placing operation of the material picking arm. The present application uses a variety of calibration methods to calibrate the turntable system, which can improve the versatility of the calibrated turntable system. By calibrating the initial posture of the target product according to the calibrated turntable system, the accuracy and stability of the picking or placing operations of the picking arm can be effectively improved, so that the calibrated turntable system can adapt to the picking or placing operations of products in different environments.

[0146] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0147] Based on the same inventive concept, the embodiment of the present application also provides a data calibration device for implementing the data calibration method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more data calibration device embodiments provided below can refer to the limitations on the data calibration method above, and will not be repeated here.

[0148] In an exemplary embodiment, Figure 7As shown, a data calibration device is provided, which is applied to a turntable system. The turntable system includes a material picking table and a corresponding first camera, a material picking arm and a corresponding second camera. The device includes: a first coordinate determination module 702, a first mapping matrix determination module 704, a second coordinate determination module 706, and a second mapping matrix determination module 708, wherein:

[0149] A first coordinate determination module 702 is used to obtain first image data corresponding to a plurality of points on the calibration plate, and determine a first coordinate group based on the first image data;

[0150] A first mapping matrix determination module 704 performs camera calibration on the first camera element based on the first coordinate group to obtain a first mapping matrix;

[0151] The second coordinate determination module 706 is used to obtain the rotation image data corresponding to the different angles of rotation of the material picking arm at the reference material picking position and the second image data corresponding to multiple points on the calibration plate when the material picking arm is at the reference material picking position; based on the rotation image data and the second image data, determine the second coordinate group.

[0152] The second mapping matrix determination module 708 is used to perform rotation calibration and motion system calibration on the second camera element based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine calibration data.

[0153] In an exemplary embodiment, the second coordinate group includes an initial registration coordinate group; the second coordinate determination module 706 is also used to determine multiple groups of rotation point coordinates and calibration plate point coordinates corresponding to the target points on the calibration plate based on the rotated image data; wherein the rotation point coordinates are the coordinates corresponding to the points on the calibration plate in the rotated image coordinate system, and the calibration plate point coordinates are the coordinates corresponding to the points on the calibration plate in the calibration plate coordinate system; the initial registration coordinate group is obtained based on the multiple groups of rotation point coordinates and calibration plate point coordinates corresponding to the target points on the calibration plate.

[0154] In an exemplary embodiment, the second mapping matrix includes an initial registration mapping matrix; and the data calibration further includes:

[0155] The rotation calibration module is used to rotationally calibrate the second camera element based on the initial alignment coordinate group to obtain an initial alignment mapping matrix; according to the initial alignment mapping matrix, the calibration plate point coordinates corresponding to the optical center of the second camera element when the material picking arm rotates at different angles at the reference material picking position are determined to obtain an optical center point coordinate set; according to the optical center point coordinate set, the calibration plate point coordinates corresponding to the rotation center of the material picking arm when the material picking arm is at the reference material picking position are determined to obtain initial rotation calibration data.

[0156] In an exemplary embodiment, the second image data includes first motion image data and second motion image data, and the second coordinate group also includes a first motion coordinate group and a second motion coordinate group; the second coordinate determination module 706 is also used to determine the first material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate according to the first motion image data; obtain the first motion coordinate group according to the first material picking arm point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate; wherein the first material picking arm point coordinates are the coordinates corresponding to the points on the calibration plate in the first material picking arm coordinate system; determine the second material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate according to the second motion image data; obtain the second motion coordinate group according to the second material picking arm point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate; wherein the second material picking arm point coordinates are the coordinates corresponding to the points on the calibration plate in the second material picking arm coordinate system.

[0157] In an exemplary embodiment, the second mapping matrix further includes a first motion mapping matrix and a second motion mapping matrix; and the data calibration further includes:

[0158] The material picking arm rotation center calibration module is used to perform a first motion calibration on the second camera element based on a first motion coordinate group to obtain a first motion mapping matrix; perform a second motion calibration on the second camera element based on a second motion coordinate group to obtain a second motion mapping matrix; and determine the calibration data of the rotation center of the material picking arm when the material picking arm is in any position based on the first motion mapping matrix, the second motion mapping matrix and the initial rotation calibration data.

[0159] In an exemplary embodiment, the first image data includes static image data, and the first coordinate group includes a static coordinate group; the first coordinate determination module 702 is also used to determine the static point coordinates and the calibration plate point coordinates corresponding to the target point on the calibration plate according to the static image data; wherein the static point coordinates are the coordinates of the point in the static image coordinate system; and the static coordinate group is obtained according to the static point coordinates and the calibration plate point coordinates corresponding to the target point on the calibration plate.

[0160] In an exemplary embodiment, the first image data also includes displacement image data, and the first coordinate group also includes a displacement coordinate group; the first coordinate determination module 702 is also used to determine the axial point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate according to the displacement image data; wherein the axial point coordinates are the coordinates of the point on the calibration plate in the camera axis coordinate system; the displacement coordinate group is obtained according to the axial point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate.

[0161] Each module in the above data calibration device can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0162] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as the first image data, the first mapping matrix, and the second coordinate group. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data calibration method is implemented.

[0163] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0164] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0165] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0166] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0167] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0169] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0170] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data calibration method, characterized in that: Applied to a turntable system, the turntable system includes a material taking table and a corresponding first camera, a material taking arm and a corresponding second camera, and the method includes: Acquire first image data corresponding to a plurality of points on a calibration plate, and determine a first coordinate group based on the first image data; Performing camera calibration on the first camera element based on the first coordinate group to obtain a first mapping matrix; Obtaining the rotation image data corresponding to the different rotation angles of the picking arm at the reference picking position and the second image data corresponding to multiple points on the calibration plate when the picking arm is at the reference picking position; wherein the second image data includes the first motion image data and the second motion image data; Based on the rotation image data and the second image data, a second coordinate group is determined, including: determining the first material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate according to the first motion image data; obtaining a first motion coordinate group according to the first material picking arm point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate; wherein the first material picking arm point coordinates are the coordinates corresponding to the points on the calibration plate in the first material picking arm coordinate system; determining the second material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate according to the second motion image data; obtaining a second motion coordinate group according to the second material picking arm point coordinates and the calibration plate point coordinates corresponding to multiple points on the calibration plate; wherein the second material picking arm point coordinates are the coordinates corresponding to the points on the calibration plate in the second material picking arm coordinate system, and the second coordinate group includes the first motion coordinate group and the second motion coordinate group; The second camera element is subjected to rotation calibration and motion system calibration based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine calibration data.

2. The method according to claim 1, characterized in that The second coordinate group also includes an initial registration coordinate group; and the determining the second coordinate group based on the rotated image data and the second image data further includes: Determine multiple sets of rotation point coordinates and calibration plate point coordinates corresponding to target points on the calibration plate according to the rotation image data; wherein the rotation point coordinates are coordinates corresponding to the points of the calibration plate in the rotation image coordinate system, and the calibration plate point coordinates are coordinates corresponding to the points of the calibration plate in the calibration plate coordinate system; An initial registration coordinate group is obtained according to the multiple groups of rotation point coordinates and calibration plate point coordinates corresponding to the target points on the calibration plate.

3. The method according to claim 2, characterized in that The second mapping matrix comprises an initial registration mapping matrix; the method further comprises: Performing rotation calibration on the second camera based on the initial registration coordinate group to obtain an initial registration mapping matrix; According to the initial registration mapping matrix, the calibration plate point coordinates corresponding to the optical center of the second camera element when the material picking arm rotates at different angles at the reference material picking position are determined to obtain an optical center point coordinate set; According to the optical center point coordinate set, the calibration plate point coordinates corresponding to the rotation center of the material picking arm when the material picking arm is at the reference material picking position are determined to obtain initial rotation calibration data.

4. The method according to claim 3, characterized in that The second mapping matrix also includes a first motion mapping matrix and a second motion mapping matrix; the method further includes: Performing a first motion calibration on the second camera element based on the first motion coordinate group to obtain a first motion mapping matrix; performing a second motion calibration on the second camera element based on the second motion coordinate group to obtain a second motion mapping matrix; The calibration data of the rotation center of the material picking arm when the material picking arm is at any position is determined according to the first motion mapping matrix, the second motion mapping matrix and the initial rotation calibration data.

5. The method according to claim 1, characterized in that: The first image data includes static image data, and the first coordinate group includes a static coordinate group; the acquiring the first image data corresponding to the plurality of points of the calibration plate and determining the first coordinate group based on the first image data includes: According to the static image data, determine the static point coordinates and the calibration plate point coordinates corresponding to the target point on the calibration plate; wherein the static point coordinates are the coordinates of the point in the static image coordinate system; The static coordinate group is obtained according to the static point coordinates corresponding to the target point on the calibration plate and the calibration plate point coordinates.

6. The method according to claim 1, characterized in that The first image data further includes displacement image data, and the first coordinate group further includes a displacement coordinate group; the acquiring the first image data corresponding to the plurality of points of the calibration plate and determining the first coordinate group based on the first image data further includes: According to the displacement image data, the axis point coordinates and the calibration plate point coordinates corresponding to each of the points on the calibration plate are determined; wherein the axis point coordinates are the coordinates of the points on the calibration plate in the camera axis coordinate system; A displacement coordinate group is obtained according to the axis point coordinates corresponding to the plurality of points on the calibration plate and the calibration plate point coordinates.

7. A data calibration device, characterized in that: Applied to a turntable system, the turntable system includes a material taking table and a corresponding first camera, a material taking arm and a corresponding second camera, and the device includes: A first coordinate determination module, used to obtain first image data corresponding to a plurality of points on a calibration plate, and determine a first coordinate group based on the first image data; A first mapping matrix determination module is configured to perform camera calibration on the first camera element based on the first coordinate group to obtain a first mapping matrix; The second coordinate determination module is used to obtain the rotation image data corresponding to the material picking arm rotating at different angles at the reference material picking position and the second image data corresponding to multiple points on the calibration plate when the material picking arm is at the reference material picking position; wherein the second image data includes the first motion image data and the second motion image data; based on the rotation image data and the second image data, the second coordinate group is determined, including: determining the first material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate according to the first motion image data; determining the first material picking arm point coordinates and the calibration plate point coordinates corresponding to the multiple points on the calibration plate according to the first motion image data; determining the second coordinate group based on the rotation image data and the second motion image data; determining the second coordinate group based on the rotation image data and the second motion image data; determining the first material picking arm point coordinates and the calibration plate point coordinates corresponding to each point on the calibration plate according to the rotation image data; determining the first material picking arm point coordinates and the calibration plate point coordinates corresponding to the multiple points on the calibration plate according to the rotation image data; determining the second coordinate group based on the rotation image data and the second motion ... The coordinates of the points on the calibration plate are determined to obtain a first motion coordinate group; wherein the coordinates of the first material picking arm points are the coordinates corresponding to the points on the calibration plate in the first material picking arm coordinate system; according to the second motion image data, the coordinates of the second material picking arm points and the coordinates of the calibration plate points corresponding to each of the points on the calibration plate are determined; according to the coordinates of the second material picking arm points and the coordinates of the calibration plate points corresponding to the multiple points on the calibration plate, a second motion coordinate group is obtained; wherein the coordinates of the second material picking arm points are the coordinates corresponding to the points on the calibration plate in the second material picking arm coordinate system, and the second coordinate group includes the first motion coordinate group and the second motion coordinate group; The second mapping matrix determination module is used to perform rotation calibration and motion system calibration on the second camera element based on the second coordinate group to obtain a second mapping matrix; wherein the first mapping matrix and the second mapping matrix are used to determine calibration data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images