Calibration mapping method, device, equipment and storage medium

Abstract

The present disclosure relates to a calibration mapping method, apparatus, device, and storage medium. The method includes: obtaining a first image of a calibration plate at a first set position in a mechanical coordinate system; determining first feature information of multiple calibration points on the calibration plate based on the first image, and obtaining a second image of the calibration plate at a second set position in the mechanical coordinate system; identifying multiple calibration points from the second image based on the first feature information; determining a mapping relationship between a first image coordinate system of the first set position and a second image coordinate system of the second set position based on the first image coordinates of the multiple calibration points in the first image and the second image coordinates of the multiple calibration points in the second image; and mapping the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence between the first image coordinate system and the mechanical coordinate system. The present disclosure can improve calibration mapping efficiency, reduce the probability of calibration point errors, and improve calibration mapping accuracy.

Description

Technical Field

[0001] The present disclosure relates to the field of machine vision technology, and in particular to a calibration mapping method, apparatus, device, and storage medium. Background Art

[0002] Hand-eye calibration is a critical and fundamental issue in machine vision applications. Simply put, its purpose is to establish the correspondence between the mechanical coordinate system and the camera image coordinate system, and ultimately convert the visual recognition results back to the mechanical coordinate system. The accuracy and precision of the calibration results directly determine the proper functioning of the vision system. The calibration plate is the foundation of this process and fundamentally determines its effectiveness.

[0003] The nine-point method is a commonly used calibration mapping method, which requires the camera and the marker (such as a calibration plate, etc.) to move relative to each other at 9 points. In multi-station vision application scenarios, it is often encountered that the camera and the marker cannot move relative to each other at individual stations; there are also cases where customers require only one hand-eye calibration for the sake of improving efficiency and ease of operation. In order to solve the problems encountered in the above two situations, the calibration mapping method is usually used to solve them. However, the calibration mapping method in the related art requires multiple manual attempts to find the same calibration points in the image coordinate system of one station as in the image coordinate system of another station. This is not only labor-intensive and inefficient, but also cannot guarantee the accuracy of the calibration mapping. Summary of the Invention

[0004] To overcome the problems existing in the related art, the embodiments of the present disclosure provide a calibration mapping method, apparatus, device and storage medium to solve the defects in the related art.

[0005] According to a first aspect of an embodiment of the present disclosure, a calibration mapping method is provided, the method comprising:

[0006] Acquire a first image of the calibration plate at a first set position in the mechanical coordinate system;

[0007] determining first feature information of a plurality of calibration points on the calibration plate based on the first image, where the first feature information is determined by set identification information of each of the plurality of calibration points;

[0008] Acquire a second image of the calibration plate at a second set position in the mechanical coordinate system;

[0009] identifying the plurality of calibration points from the second image based on the first feature information;

[0010] determining a mapping relationship between a first image coordinate system of the first set position and a second image coordinate system of the second set position based on the first image coordinates of the plurality of calibration points in the first image and the second image coordinates of the plurality of calibration points in the second image;

[0011] Based on the mapping relationship and a predetermined correspondence between the first image coordinate system and the mechanical coordinate system, the coordinates of the second image coordinate system are mapped to the mechanical coordinate system.

[0012] In one embodiment, identifying the plurality of calibration points from the second image based on the first feature information includes:

[0013] Identifying setting identification information of each calibration point in the second image;

[0014] The setting identification information of each calibration point is matched with the first feature information to obtain a plurality of calibration points in the second image.

[0015] In one embodiment, the setting identification information includes any one or a combination of the following:

[0016] Letters; numbers; graphics; a graphic matrix consisting of two or more letters or two or more numbers or two or more graphics; set color levels.

[0017] In one embodiment, before acquiring the first image of the calibration plate at the first set position in the mechanical coordinate system, the method further includes:

[0018] In response to controlling the mechanical component to place the calibration plate at the first set position based on the vacuum suction device, the vacuum state of the vacuum suction device is broken and the mechanical component is removed.

[0019] In one embodiment, the method further comprises:

[0020] In response to controlling the mechanical component to move the calibration plate from the first set position to the second set position based on the vacuum suction device, the vacuum state of the vacuum suction device is broken and the mechanical component is removed.

[0021] In one embodiment, the method further comprises:

[0022] In response to moving the calibration plate from the first set position to the second set position, the focal length and / or shooting angle of the image acquisition device are adjusted based on the received adjustment instruction so that the image acquisition device includes the multiple calibration points in the image acquisition range of the second set position.

[0023] In one embodiment, mapping the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence relationship between the first image coordinate system and the mechanical coordinate system includes:

[0024] Mapping the target image coordinates of the second image coordinate system to the first image coordinate system based on the mapping relationship to obtain second target image coordinates;

[0025] Based on the corresponding relationship, the second target image coordinates are mapped to the mechanical coordinate system to obtain target mechanical coordinates.

[0026] According to a second aspect of an embodiment of the present disclosure, a calibration mapping device is provided, the device comprising:

[0027] A first image acquisition module is used to acquire a first image of the calibration plate at a first set position in the mechanical coordinate system;

[0028] a first feature determination module, configured to determine first feature information of a plurality of calibration points on the calibration plate based on the first image, wherein the first feature information is determined by set identification information of each of the plurality of calibration points;

[0029] A second image acquisition module, configured to acquire a second image of the calibration plate at a second set position in the mechanical coordinate system;

[0030] a calibration point recognition module, configured to recognize the plurality of calibration points from the second image based on the first feature information;

[0031] a mapping relationship determining module, configured to determine a mapping relationship between a first image coordinate system of the first set position and a second image coordinate system of the second set position based on the first image coordinates of the plurality of calibration points in the first image and the second image coordinates of the plurality of calibration points in the second image;

[0032] An image coordinate mapping module is configured to map the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence between the first image coordinate system and the mechanical coordinate system.

[0033] In one embodiment, the calibration point recognition module includes:

[0034] an identification recognition unit, configured to recognize set identification information of each calibration point in the second image;

[0035] A calibration point recognition unit is used to match the set identification information of each calibration point with the first feature information to obtain multiple calibration points in the second image.

[0036] In one embodiment, the setting identification information includes any one or a combination of the following:

[0037] Letters; numbers; graphics; a graphic matrix consisting of two or more letters or two or more numbers or two or more graphics; set color levels.

[0038] In one embodiment, the apparatus further comprises:

[0039] The first control module is configured to, in response to controlling the mechanical component to place the calibration plate at the first set position based on the vacuum suction device, destroy the vacuum state of the vacuum suction device and remove the mechanical component.

[0040] In one embodiment, the apparatus further comprises:

[0041] The second control module is configured to, in response to controlling the mechanical component to move the calibration plate from the first set position to the second set position based on the vacuum suction device, destroy the vacuum state of the vacuum suction device and remove the mechanical component.

[0042] In one embodiment, the apparatus further comprises:

[0043] an image acquisition device adjustment module, configured to adjust the focal length and / or shooting angle of the image acquisition device based on a received adjustment instruction in response to moving the calibration plate from the first set position to the second set position, so that the image acquisition device includes the multiple calibration points within the image acquisition range of the second set position.

[0044] In one embodiment, the image coordinate mapping module includes:

[0045] A second coordinate acquisition unit, configured to map the target image coordinates of the second image coordinate system to the first image coordinate system based on the mapping relationship to obtain second target image coordinates;

[0046] The mechanical coordinate acquisition unit is configured to map the second target image coordinates to the mechanical coordinate system based on the corresponding relationship to obtain target mechanical coordinates.

[0047] According to a third aspect of an embodiment of the present disclosure, a calibration plate is provided, which has a plurality of calibration points of a set pattern, wherein the calibration points are distributed on the calibration plate in an array form, and setting identification information uniquely corresponding to each calibration point is provided at a set position around each calibration point, wherein the calibration points are used for coordinate calibration, and the setting identification information is used to identify the characteristics of the calibration points to which they belong.

[0048] In one embodiment, the set pattern includes but is not limited to a solid circle pattern.

[0049] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, the device comprising:

[0050] a processor and a memory for storing instructions executable by the processor;

[0051] Wherein, the processor is configured to:

[0052] Acquire a first image of the calibration plate at a first set position in the mechanical coordinate system;

[0053] determining first feature information of a plurality of calibration points on the calibration plate based on the first image, where the first feature information is determined by set identification information of each of the plurality of calibration points;

[0054] Acquire a second image of the calibration plate at a second set position in the mechanical coordinate system;

[0055] identifying the plurality of calibration points from the second image based on the first feature information;

[0056] determining a mapping relationship between a first image coordinate system of the first set position and a second image coordinate system of the second set position based on the first image coordinates of the plurality of calibration points in the first image and the second image coordinates of the plurality of calibration points in the second image;

[0057] Based on the mapping relationship and a predetermined correspondence between the first image coordinate system and the mechanical coordinate system, the coordinates of the second image coordinate system are mapped to the mechanical coordinate system.

[0058] According to a fifth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the following is achieved:

[0059] Acquire a first image of the calibration plate at a first set position in the mechanical coordinate system;

[0060] determining first feature information of a plurality of calibration points on the calibration plate based on the first image, where the first feature information is determined by set identification information of each of the plurality of calibration points;

[0061] Acquire a second image of the calibration plate at a second set position in the mechanical coordinate system;

[0062] identifying the plurality of calibration points from the second image based on the first feature information;

[0063] determining a mapping relationship between a first image coordinate system of the first set position and a second image coordinate system of the second set position based on the first image coordinates of the plurality of calibration points in the first image and the second image coordinates of the plurality of calibration points in the second image;

[0064] Based on the mapping relationship and a predetermined correspondence between the first image coordinate system and the mechanical coordinate system, the coordinates of the second image coordinate system are mapped to the mechanical coordinate system.

[0065] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0066] The present disclosure obtains a first image of a calibration plate at a first set position in a mechanical coordinate system, determines first feature information of multiple calibration points on the calibration plate based on the first image, wherein the first feature information is determined by the set identification information of each of the multiple calibration points, and obtains a second image of the calibration plate at a second set position in the mechanical coordinate system, identifies the multiple calibration points from the second image based on the first feature information, and then determines a mapping relationship between the first image coordinate system of the first set position and the second image coordinate system of the second set position based on the first image coordinates of the multiple calibration points in the first image and the second image coordinates of the multiple calibration points in the second image. Then, based on the mapping relationship and a predetermined correspondence between the first image coordinate system and the mechanical coordinate system, the coordinates of the second image coordinate system can be mapped to the mechanical coordinate system. Since each calibration point in the calibration plate has set identification information, it is possible to determine the first feature information based on the set identification information of each of the multiple calibration points, and then automatically determine the calibration points corresponding to the multiple calibration points in the first image from the second image based on the first feature information, thereby avoiding determining the multiple calibration points in the second image through manual observation, thereby improving the efficiency of the calibration mapping process, reducing the problem of incorrect calibration point determination due to manual observation, and improving the accuracy of the calibration mapping.

[0067] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0069] Figure 1A is a flowchart showing a calibration mapping method according to an exemplary embodiment;

[0070] Figure 1Bis a schematic diagram of a calibration plate according to an exemplary embodiment;

[0071] Figure 2 is a flowchart illustrating how to identify the plurality of calibration points from the second image based on the first feature information according to an exemplary embodiment;

[0072] Figure 3 is a flowchart of a calibration mapping method according to another exemplary embodiment;

[0073] Figure 4 is a flow chart showing how to map the coordinates of the second image coordinate system to the mechanical coordinate system according to an exemplary embodiment;

[0074] Figure 5 is a block diagram of a hand-eye calibration device according to an exemplary embodiment;

[0075] Figure 6 is a block diagram of a hand-eye calibration device according to another exemplary embodiment;

[0076] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0077] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0078] Hand-eye calibration is a critical and fundamental issue in machine vision applications. Simply put, its purpose is to establish the correspondence between the mechanical coordinate system and the camera image coordinate system, and ultimately convert the visual recognition results back to the mechanical coordinate system. The accuracy and precision of the calibration results directly determine the proper functioning of the vision system. The calibration plate is the foundation of this process and fundamentally determines its effectiveness.

[0079] The nine-point method is a commonly used calibration mapping method, which requires the camera and the marker (such as a calibration plate, etc.) to move relative to each other at 9 points. In multi-station visual application scenarios, it is often encountered that the camera and the marker cannot move relative to each other at individual stations; there are also considerations for improving efficiency and simplifying operation to reduce the number of hand-eye calibrations. In order to solve the problems encountered in the above two situations, the present disclosure adopts a calibration mapping method in the embodiment, that is, firstly perform hand-eye calibration on one station to obtain a calibration matrix, that is, establish a correspondence between the image coordinate system and the mechanical coordinate system; the other stations perform calibration mapping operations with the station, and the image coordinates of the other stations can be mapped to the image coordinate system of the station, and then converted into mechanical coordinates through the calibration matrix. However, the inventors found in the process of implementing the embodiments of the present application that the current calibration mapping method requires multiple manual methods to find the same multiple calibration points in the image coordinate system of one station as in the image coordinate system of another station, which is not only labor-intensive and inefficient, but also cannot guarantee the accuracy of the calibration mapping.

[0080] In view of this, the embodiments of the present disclosure provide the following calibration mapping method, apparatus, device and storage medium to address the deficiencies in the related art.

[0081] Figure 1A It is a flowchart of a calibration mapping method according to an exemplary embodiment; the method of this embodiment can be applied to a server with data processing capabilities (such as a server cluster consisting of one server or multiple servers), or can be applied to a terminal device with data processing capabilities (such as a tablet computer, laptop computer, etc.).

[0082] like Figure 1A As shown, the method includes the following steps S101-S106:

[0083] In step S101 , a first image of a calibration plate at a first set position in a mechanical coordinate system is acquired.

[0084] In this embodiment, after the calibration plate is placed at a first set position in the mechanical coordinate system by using a mechanical component, a first image of the calibration plate can be acquired by using an image acquisition device.

[0085] The first set position may correspond to a first common working position of the mechanical component.

[0086] In this embodiment, the above-mentioned image acquisition device may include a camera, and the position of the camera may be set based on actual needs, such as being set on a mechanical device fixed in the environment, or being set on the above-mentioned mechanical component, which is not limited in this embodiment.

[0087] For example, the mechanical component may include a manipulator of a robot, which is not limited in this embodiment.

[0088] In one embodiment, the calibration plate may have a plurality of calibration points with a set pattern, each calibration point may be distributed on the calibration plate in an array, and each calibration point may be provided with unique setting identification information corresponding to each calibration point at a set position around the calibration point, wherein the calibration point is used for coordinate calibration, and the setting identification information is used to identify the characteristics of the calibration point. For example, Figure 1B FIG. 1 is a schematic diagram of a calibration plate according to an exemplary embodiment. Figure 1B As shown, the above-mentioned set pattern may include but is not limited to a solid circle pattern.

[0089] The above-mentioned setting identification information includes any one or more of the following: letters; numbers; graphics; a graphic matrix consisting of two or more letters or two or more numbers or two or more graphics; set color levels. For example, the setting identification information in digital form, such as Figure 1B As shown, each solid circle pattern in the calibration plate can represent a calibration point, and each calibration point has unique setting identification information in digital form, and the setting identification information can be used to distinguish different calibration points.

[0090] In step S102, first feature information of a plurality of calibration points on the calibration plate is determined based on the first image.

[0091] In this embodiment, after obtaining the first image of the calibration plate at the first set position in the mechanical coordinate system, multiple calibration points located on the calibration plate in the first image can be determined based on the user's selection instructions, and then the first feature information of the multiple calibration points can be determined based on the set identification information of each calibration point in the multiple calibration points.

[0092] In one embodiment, the first characteristic information of the plurality of calibration points may be a combination of setting identification information of each of the plurality of calibration points.

[0093] In step S103 , a second image of the calibration plate at a second set position in the mechanical coordinate system is acquired.

[0094] In this embodiment, when the first characteristic information of multiple calibration points on the calibration plate is determined based on the first image, and the first characteristic information is determined by the set identification information of each of the multiple calibration points, the calibration plate can be moved from the first set position in the mechanical coordinate system to the second set position in the mechanical coordinate system using mechanical components, and the second image of the calibration plate can be obtained using an image acquisition device.

[0095] The second set position may correspond to a second commonly used workstation of the mechanical component, wherein the second commonly used workstation is different from the first commonly used workstation.

[0096] In step S104, the plurality of calibration points are identified from the second image based on the first feature information.

[0097] In this embodiment, after obtaining the second image of the calibration plate at the second set position in the mechanical coordinate system, since each calibration point in the calibration plate has its own set identification information, the first feature information can be determined based on the set identification information of the multiple calibration points determined in the above-mentioned first image, and then the multiple calibration points selected by the user in the first image can be identified from the second image based on the first feature information.

[0098] For example, Figure 2 FIG. 1 is a flow chart showing how to identify the plurality of calibration points from the second image based on the first feature information according to an exemplary embodiment. Figure 2 As shown, the above step S104 may further specifically include the following steps S1041-S1042:

[0099] In step S1041, setting identification information of each calibration point in the second image is identified.

[0100] In this embodiment, after obtaining a second image of the calibration plate at the second set position in the mechanical coordinate system, the set identification information of each calibration point in the second image can be identified based on a set image recognition algorithm, such as the digital identification information, graphic identification information, identification information combining numbers and graphics, or color scale information of each identification point.

[0101] The set image recognition algorithm may be selected from relevant technologies based on actual needs, and this embodiment does not limit this.

[0102] In step S1042, the setting identification information of each calibration point is matched with the first feature information to obtain multiple calibration points in the second image.

[0103] In this embodiment, after identifying the setting identification information of each calibration point in the second image, the setting identification information of each calibration point may be matched with the first feature information to obtain a plurality of calibration points in the second image.

[0104] For example, when the set identification information of four calibration points selected from the first image is "A", "B", "C" and "D" respectively, the first feature information of the four calibration points can be determined to be "ABCD". Then, after obtaining the second image information, the set identification information of each calibration point in the second image can be identified, such as obtaining the identification information of each calibration point "A", "F", "B", "C" and "D", etc., and then the identification information of each identified calibration point can be matched with the first feature information "ABCD", so that the four calibration points with the set identification information of "A", "B", "C" and "D" can be identified from the second image.

[0105] In step S105, based on the first image coordinates of the multiple calibration points in the first image and the second image coordinates of the multiple calibration points in the second image, a mapping relationship between the first image coordinate system of the first set position and the second image coordinate system of the second set position is determined.

[0106] In this embodiment, after obtaining a first image of the calibration plate at a first set position in the mechanical coordinate system, first image coordinates of a plurality of calibration points on the calibration plate may be determined based on the first image.

[0107] In one embodiment, after acquiring a first image of the calibration plate at a first set position in a mechanical coordinate system, multiple calibration points may be selected from the first image, and then, based on the predetermined first image coordinate system of the first set position, the first image coordinates of each of the selected multiple calibration points may be determined. For example, after acquiring the first image, four calibration points may be selected within the camera's field of view, and the image coordinates PA1, PA2, PA3, and PA4 of each calibration point may be recorded, denoted as a set {PA}, which serves as the first image coordinates of the four calibration points.

[0108] In addition, after acquiring a second image of the calibration plate at a second set position in the mechanical coordinate system and identifying multiple calibration points selected from the first image from the second image, the second image coordinates of each of the multiple calibration points can be determined based on the predetermined second image coordinate system at the second set position. For example, after identifying multiple calibration points selected from the first image from the second image, four calibration points corresponding one-to-one to the four calibration points selected in the first image can be selected in the camera field of view, and the image coordinates PB1, PB2, PB3, and PB4 of each calibration point can be recorded, denoted as a set {PB}, and used as the second image coordinates of the four calibration points.

[0109] On this basis, when the first image coordinates of the multiple calibration points in the first image and the second image coordinates of the multiple calibration points in the second image are determined, the mapping relationship between the above-mentioned first image coordinate system and the second image coordinate system can be determined based on the first image coordinates and the second image coordinates of the multiple calibration points.

[0110] It is worth noting that the algorithm for determining the mapping relationship between two different coordinate systems based on the coordinates of the same multiple calibration points in two different coordinate systems can be found in the contents recorded in the relevant technology. For example, a mapping relationship matrix between two different coordinate systems can be constructed based on the algorithm in the relevant technology, and then the parameters of the mapping relationship matrix can be solved based on the coordinates of the above-mentioned multiple calibration points in the two different coordinate systems to obtain the mapping relationship matrix between the first image coordinate system and the second image coordinate system, etc. This embodiment does not limit this.

[0111] In step S106 , based on the mapping relationship and a predetermined correspondence relationship between the first image coordinate system and the mechanical coordinate system, the coordinates of the second image coordinate system are mapped to the mechanical coordinate system.

[0112] In this embodiment, after determining the mapping relationship between the first image coordinate system and the second image coordinate system, the corresponding relationship between the second image coordinate system and the mechanical coordinate system can be deduced based on the mapping relationship and the predetermined corresponding relationship between the first image coordinate system and the mechanical coordinate system, and then the coordinates of the points in the second image coordinate system can be mapped to the above-mentioned mechanical coordinate system, that is, the calibration mapping of the second image coordinate system to the mechanical coordinate system is realized.

[0113] In one embodiment, the correspondence between the first image coordinate system and the mechanical coordinate system may be determined based on a hand-eye calibration method in related art, which is not limited in this embodiment.

[0114] In another embodiment, the specific implementation of the above step S106 can also refer to the following Figure 4 The illustrated embodiment will not be described in detail here.

[0115] From the above description, it can be seen that the method of this embodiment can determine the first feature information based on the set identification information of each of the multiple calibration points because the various calibration points in the calibration plate have set identification information, and then automatically determine the calibration points corresponding to the multiple calibration points in the first image from the second image based on the first feature information, thereby avoiding the need to determine the multiple calibration points in the second image through manual observation, thereby improving the efficiency of the calibration mapping process, reducing the problem of incorrect determination of calibration points due to manual observation, and improving the accuracy of the calibration mapping.

[0116] Figure 3This is a flow chart of a calibration mapping method according to another exemplary embodiment. The method of this embodiment can be applied to a server with data processing capabilities (such as a server cluster consisting of one server or multiple servers), or can be applied to a terminal device with data processing capabilities (such as a tablet computer, laptop computer, etc.). Figure 3 The method includes the following steps S301-S308: In step S301, in response to controlling the mechanical component to place the calibration plate in the first set position based on the vacuum suction device, the vacuum state of the vacuum suction device is destroyed and the mechanical component is removed.

[0117] In this embodiment, a mechanical component (e.g., a manipulator of a robot or other mechanical device) can be controlled to use a vacuum suction device to suck the calibration plate and place the calibration plate at a first predetermined position within the mechanical coordinate system. After the placement operation is completed, the vacuum state of the vacuum suction device can be broken, separating the vacuum suction device of the mechanical component from the calibration plate, and then the mechanical component can be removed.

[0118] In step S302 , a first image of the calibration plate at a first set position in the mechanical coordinate system is acquired.

[0119] In step S303, first feature information of a plurality of calibration points on the calibration plate is determined based on the first image, wherein the first feature information is determined by setting identification information of each of the plurality of calibration points.

[0120] In step S304 , in response to controlling the mechanical component to move the calibration plate from the first set position to the second set position based on the vacuum suction device, the vacuum state of the vacuum suction device is destroyed and the mechanical component is removed.

[0121] In this embodiment, after acquiring a first image of the calibration plate, the mechanical component can be controlled to move the calibration plate from the first set position to a second set position in the mechanical coordinate system using a vacuum suction device. After the placement operation is completed, the vacuum state of the vacuum suction device can be broken, separating the vacuum suction device of the mechanical component from the calibration plate, and then removing the mechanical component.

[0122] In one embodiment, after the calibration plate is moved from the first set position to the second set position, the focal length and / or shooting angle of the image acquisition device can also be adjusted based on the received adjustment instruction, so that the image acquisition device includes multiple calibration points in the image acquisition range of the second set position, that is, it is ensured that the acquired second image can include multiple calibration points selected in the first image.

[0123] It is understandable that if the mechanical components touch or shake the already placed calibration plate during removal, the image coordinates of the selected calibration points on the calibration plate will change, thus affecting the accuracy of subsequent coordinate system mapping based on the image coordinates. Therefore, in this embodiment, a vacuum suction device is used to absorb and move the calibration plate to ensure that the calibration plate is not touched or shaken during the removal of the mechanical components after placement, thereby ensuring the accuracy of subsequent coordinate system mapping.

[0124] In step S305 , a second image of the calibration plate at a second set position in the mechanical coordinate system is acquired.

[0125] In step S306, the plurality of calibration points are identified from the second image based on the first feature information.

[0126] In step S307, based on the first image coordinates of the multiple calibration points in the first image and the second image coordinates of the multiple calibration points in the second image, a mapping relationship between the first image coordinate system of the first set position and the second image coordinate system of the second set position is determined.

[0127] In step S308 , based on the mapping relationship and a predetermined correspondence relationship between the first image coordinate system and the mechanical coordinate system, the coordinates of the second image coordinate system are mapped to the mechanical coordinate system.

[0128] The explanations and instructions for steps S302-S303 and S305-S308 can be found in the above Figure 1A Steps S101 to S106 in the illustrated embodiment are not described in detail here.

[0129] From the above description, it can be seen that this embodiment improves the accuracy of moving the calibration plate by controlling the mechanical parts to move the calibration plate to the specified position based on the vacuum suction device, and by destroying the vacuum state of the vacuum suction device and removing the mechanical parts after the calibration plate is placed. It ensures that the calibration plate is not touched or shaken during the process of removing the mechanical parts, thereby avoiding the problem of affecting the accuracy of the calibration mapping due to accidental touching of the calibration plate, and can improve the accuracy of the calibration mapping.

[0130] Figure 4 This is a flow chart showing how to map the coordinates of the second image coordinate system to the mechanical coordinate system according to an exemplary embodiment. This embodiment is based on the above embodiment and illustrates how to map the coordinates of the second image coordinate system to the mechanical coordinate system. Figure 4 As shown, the mapping of the coordinates of the second image coordinate system to the mechanical coordinate system in the above step S106 or step S308 may include the following steps S401-S402:

[0131] In step S401 , the target image coordinates of the second image coordinate system are mapped to the first image coordinate system based on the mapping relationship to obtain second target image coordinates.

[0132] In this embodiment, after determining the mapping relationship between the first image coordinate system of the first set position and the second image coordinate system of the second set position, the target image coordinates of a point in the second image coordinate system can be mapped to the first image coordinate system to obtain the coordinates of the point in the second image coordinate system, that is, the above-mentioned second target image coordinates.

[0133] In step S402 , based on the corresponding relationship, the second target image coordinates are mapped to the mechanical coordinate system to obtain target mechanical coordinates.

[0134] In this embodiment, after obtaining the second target image coordinates of the point in the first image coordinate system, the second target image coordinates can be mapped to the mechanical coordinate system based on the correspondence between the first image coordinate system and the mechanical coordinate system, thereby obtaining the coordinates of the point in the mechanical coordinate system, that is, the above-mentioned target mechanical coordinates.

[0135] From the above description, it can be seen that this embodiment maps the target image coordinates of the second image coordinate system to the first image coordinate system based on the mapping relationship to obtain the second target image coordinates, and then maps the second target image coordinates to the mechanical coordinate system based on the corresponding relationship to obtain the target mechanical coordinates. This can achieve the mapping of the coordinates of the second image coordinate system to the mechanical coordinate system, and ensure the accuracy of the coordinate mapping.

[0136] Figure 5 This is a block diagram of a hand-eye calibration device according to an exemplary embodiment; the device of this embodiment can be applied to a server with data processing capabilities (such as a server cluster consisting of one server or multiple servers), or can be applied to a terminal device with data processing capabilities (such as a tablet computer, laptop computer, etc.). Figure 5 As shown, the apparatus may include: a first image acquisition module 110, a first feature determination module 120, a second image acquisition module 130, a calibration point recognition module 140, a mapping relationship determination module 150, and an image coordinate mapping module 160, wherein:

[0137] A first image acquisition module 110 is configured to acquire a first image of the calibration plate at a first set position in the mechanical coordinate system;

[0138] A first feature determination module 120 is configured to determine first feature information of a plurality of calibration points on the calibration plate based on the first image, wherein the first feature information is determined by set identification information of each of the plurality of calibration points;

[0139] A second image acquisition module 130 is configured to acquire a second image of the calibration plate at a second set position in the mechanical coordinate system;

[0140] a calibration point recognition module 140, configured to recognize the plurality of calibration points from the second image based on the first feature information;

[0141] a mapping relationship determining module 150, configured to determine a mapping relationship between a first image coordinate system of the first set position and a second image coordinate system of the second set position based on the first image coordinates of the plurality of calibration points in the first image and the second image coordinates of the plurality of calibration points in the second image;

[0142] The image coordinate mapping module 160 is configured to map the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence relationship between the first image coordinate system and the mechanical coordinate system.

[0143] From the above description, it can be seen that since each calibration point in the calibration plate of the device of this embodiment has set identification information, it is possible to determine the first feature information based on the set identification information of each of the multiple calibration points, and then automatically determine the calibration points corresponding to the multiple calibration points in the first image from the second image based on the first feature information, thereby avoiding the need to determine the multiple calibration points in the second image through manual observation, thereby improving the efficiency of the calibration mapping process, reducing the problem of incorrect determination of calibration points due to manual observation, and improving the accuracy of the calibration mapping.

[0144] Figure 6 This is a block diagram of a hand-eye calibration device according to another exemplary embodiment; the device of this embodiment can be applied to a server with data processing capabilities (e.g., a server cluster consisting of one server or multiple servers), or can be applied to a terminal device with data processing capabilities (e.g., a tablet computer, a laptop computer, etc.). Among them, the first image acquisition module 210, the first feature determination module 220, the second image acquisition module 230, the calibration point recognition module 240, the mapping relationship determination module 250, and the image coordinate mapping module 260 are the same as those described above. Figure 5 The functions of the first image acquisition module 110, the first feature determination module 120, the second image acquisition module 130, the calibration point recognition module 140, the second coordinate determination module 150, the mapping relationship determination module 150 and the image coordinate mapping module 160 of the embodiment shown are the same and are not described in detail here. Figure 6 Show,

[0145] The calibration point identification module 240 may include:

[0146] The identification unit 241 is used to identify the set identification information of each calibration point in the second image;

[0147] The calibration point recognition unit 242 is configured to match the set identification information of each calibration point with the first feature information to obtain a plurality of calibration points in the second image.

[0148] In one embodiment, the above-mentioned setting identification information may include any one or a combination of the following:

[0149] Letters; numbers; graphics; a graphic matrix consisting of two or more letters or two or more numbers or two or more graphics; set color levels.

[0150] In one embodiment, the above apparatus may further include:

[0151] The first control module 270 is configured to, in response to controlling the mechanical component to place the calibration plate at the first set position based on the vacuum suction device, destroy the vacuum state of the vacuum suction device and remove the mechanical component.

[0152] In one embodiment, the above-mentioned apparatus further includes:

[0153] The second control module 280 is used for controlling the mechanical component to move the calibration plate from the first set position to the second set position based on the vacuum suction device in response to breaking the vacuum state of the vacuum suction device and removing the mechanical component.

[0154] In one embodiment, the above apparatus may further include:

[0155] The image acquisition device adjustment module 290 is used to adjust the focal length and / or shooting angle of the image acquisition device based on the received adjustment instruction in response to moving the calibration plate from the first set position to the second set position, so that the image acquisition device includes the multiple calibration points in the image acquisition range of the second set position.

[0156] In one embodiment, the image coordinate mapping module 260 may further include:

[0157] A second coordinate acquisition unit 261 is configured to map the target image coordinates of the second image coordinate system to the first image coordinate system based on the mapping relationship to obtain second target image coordinates;

[0158] The mechanical coordinate acquiring unit 262 is configured to map the second target image coordinates to the mechanical coordinate system based on the corresponding relationship to obtain target mechanical coordinates.

[0159] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0160] Figure 7 9 is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, device 900 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. In this embodiment, the electronic device may include a normally-on image acquisition device for acquiring image information.

[0161] Reference Figure 7 , the device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0162] The processing component 902 generally controls the overall operation of the device 900, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 902 may include one or more modules to facilitate interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate interaction between the multimedia component 908 and the processing component 902.

[0163] The memory 904 is configured to store various types of data to support the operations of the device 900. Examples of such data include instructions for any application or method operating on the device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0164] The power component 906 provides power to the various components of the device 900. The power component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 900.

[0165] The multimedia component 908 includes a screen that provides an output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0166] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC), which is configured to receive external audio signals when the device 900 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0167] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0168] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the device 900. For example, the sensor assembly 914 can detect the open / closed state of the device 900, the relative positioning of components, such as the display and keypad of the device 900. The sensor assembly 914 can also detect changes in the position of the device 900 or a component of the device 900, the presence or absence of user contact with the device 900, the orientation or acceleration / deceleration of the device 900, and temperature changes of the device 900. The sensor assembly 914 can also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0169] The communication component 916 is configured to facilitate wired or wireless communication between the device 900 and other devices. The device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G or 5G or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0170] In an exemplary embodiment, the apparatus 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0171] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the instructions can be executed by the processor 920 of the apparatus 900 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0172] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0173] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A calibration mapping method, characterized in that: The method comprises: Acquire a first image of the calibration plate at a first set position in the mechanical coordinate system; determining first feature information of a plurality of calibration points on the calibration plate based on the first image, where the first feature information is determined by set identification information of each of the plurality of calibration points; Acquire a second image of the calibration plate at a second set position in the mechanical coordinate system; identifying the plurality of calibration points from the second image based on the first feature information; determining a mapping relationship between a first image coordinate system of the first set position and a second image coordinate system of the second set position based on the first image coordinates of the plurality of calibration points in the first image and the second image coordinates of the plurality of calibration points in the second image; Mapping the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence relationship between the first image coordinate system and the mechanical coordinate system; Mapping the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence relationship between the first image coordinate system and the mechanical coordinate system includes: Mapping the target image coordinates of the second image coordinate system to the first image coordinate system based on the mapping relationship to obtain second target image coordinates; Based on the corresponding relationship, the second target image coordinates are mapped to the mechanical coordinate system to obtain target mechanical coordinates.

2. The method according to claim 1, characterized in that The identifying the plurality of calibration points from the second image based on the first feature information includes: Identifying setting identification information of each calibration point in the second image; The setting identification information of each calibration point is matched with the first feature information to obtain a plurality of calibration points in the second image.

3. The method according to claim 1, characterized in that The setting identification information includes any one or more of the following: Letters; numbers; graphics; a graphic matrix consisting of two or more letters or two or more numbers or two or more graphics; set color levels.

4. The method according to claim 1, wherein Before acquiring the first image of the calibration plate at the first set position in the mechanical coordinate system, the method further includes: In response to controlling the mechanical component to place the calibration plate at the first set position based on the vacuum suction device, the vacuum state of the vacuum suction device is broken and the mechanical component is removed.

5. The method according to claim 1, wherein The method further comprises: In response to controlling the mechanical component to move the calibration plate from the first set position to the second set position based on the vacuum suction device, the vacuum state of the vacuum suction device is broken and the mechanical component is removed.

6. The method according to claim 1, characterized in that The method further comprises: In response to moving the calibration plate from the first set position to the second set position, the focal length and / or shooting angle of the image acquisition device are adjusted based on the received adjustment instruction so that the image acquisition device includes the multiple calibration points in the image acquisition range of the second set position.

7. A calibration mapping device, characterized in that: The device comprises: A first image acquisition module is used to acquire a first image of the calibration plate at a first set position in the mechanical coordinate system; a first feature determination module, configured to determine first feature information of a plurality of calibration points on the calibration plate based on the first image, wherein the first feature information is determined by set identification information of each of the plurality of calibration points; A second image acquisition module, configured to acquire a second image of the calibration plate at a second set position in the mechanical coordinate system; a calibration point recognition module, configured to recognize the plurality of calibration points from the second image based on the first feature information; a mapping relationship determining module, configured to determine a mapping relationship between a first image coordinate system of the first set position and a second image coordinate system of the second set position based on the first image coordinates of the plurality of calibration points in the first image and the second image coordinates of the plurality of calibration points in the second image; an image coordinate mapping module, configured to map the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence between the first image coordinate system and the mechanical coordinate system; The image coordinate mapping module includes: A second coordinate acquisition unit, configured to map the target image coordinates of the second image coordinate system to the first image coordinate system based on the mapping relationship to obtain second target image coordinates; The mechanical coordinate acquisition unit is configured to map the second target image coordinates to the mechanical coordinate system based on the corresponding relationship to obtain target mechanical coordinates.

8. The device according to claim 7, characterized in that The calibration point recognition module includes: an identification recognition unit, configured to recognize set identification information of each calibration point in the second image; A calibration point recognition unit is used to match the set identification information of each calibration point with the first feature information to obtain multiple calibration points in the second image.

9. The device according to claim 7, characterized in that The setting identification information includes any one or more of the following: Letters; numbers; graphics; a graphic matrix consisting of two or more letters or two or more numbers or two or more graphics; set color levels.

10. The device according to claim 7, characterized in that The device further comprises: The first control module is configured to, in response to controlling the mechanical component to place the calibration plate at the first set position based on the vacuum suction device, destroy the vacuum state of the vacuum suction device and remove the mechanical component.

11. The device according to claim 7, characterized in that The device further comprises: The second control module is configured to, in response to controlling the mechanical component to move the calibration plate from the first set position to the second set position based on the vacuum suction device, destroy the vacuum state of the vacuum suction device and remove the mechanical component.

12. The device according to claim 7, characterized in that The device further comprises: an image acquisition device adjustment module, configured to adjust the focal length and / or shooting angle of the image acquisition device based on a received adjustment instruction in response to moving the calibration plate from the first set position to the second set position, so that the image acquisition device includes the multiple calibration points within the image acquisition range of the second set position.

13. An electronic device, characterized in that: The device comprises: a processor and a memory for storing instructions executable by the processor; Wherein, the processor is configured to: Acquire a first image of the calibration plate at a first set position in the mechanical coordinate system; determining first feature information of a plurality of calibration points on the calibration plate based on the first image, where the first feature information is determined by set identification information of each of the plurality of calibration points; Acquire a second image of the calibration plate at a second set position in the mechanical coordinate system; identifying the plurality of calibration points from the second image based on the first feature information; determining a mapping relationship between a first image coordinate system of the first set position and a second image coordinate system of the second set position based on the first image coordinates of the plurality of calibration points in the first image and the second image coordinates of the plurality of calibration points in the second image; Mapping the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence relationship between the first image coordinate system and the mechanical coordinate system; Mapping the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence relationship between the first image coordinate system and the mechanical coordinate system includes: Mapping the target image coordinates of the second image coordinate system to the first image coordinate system based on the mapping relationship to obtain second target image coordinates; Based on the corresponding relationship, the second target image coordinates are mapped to the mechanical coordinate system to obtain target mechanical coordinates.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it achieves: Acquire a first image of the calibration plate at a first set position in the mechanical coordinate system; determining first feature information of a plurality of calibration points on the calibration plate based on the first image, where the first feature information is determined by set identification information of each of the plurality of calibration points; Acquire a second image of the calibration plate at a second set position in the mechanical coordinate system; identifying the plurality of calibration points from the second image based on the first feature information; determining a mapping relationship between a first image coordinate system of the first set position and a second image coordinate system of the second set position based on the first image coordinates of the plurality of calibration points in the first image and the second image coordinates of the plurality of calibration points in the second image; Mapping the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence relationship between the first image coordinate system and the mechanical coordinate system; Mapping the coordinates of the second image coordinate system to the mechanical coordinate system based on the mapping relationship and a predetermined correspondence relationship between the first image coordinate system and the mechanical coordinate system includes: Mapping the target image coordinates of the second image coordinate system to the first image coordinate system based on the mapping relationship to obtain second target image coordinates; Based on the corresponding relationship, the second target image coordinates are mapped to the mechanical coordinate system to obtain target mechanical coordinates.

Images