Object pose calculation method and device based on multi-camera multi-Aruco code

Abstract

The present application proposes a method and device for calculating the pose of an object based on multiple cameras and multiple Aruco codes, including the following steps: setting multiple cameras and an object to be measured in a target scene; obtaining a target scene image taken by each camera, wherein the target scene image includes at least one Aruco code, and using an angle selection algorithm to select the optimal Aruco code for each camera; calculating the area of ​​each optimal Aruco code in the target scene image to obtain the optimal camera, and parsing the optimal Aruco code corresponding to the optimal camera to obtain the pose of the object to be measured. This solution covers the entire target scene through multiple cameras and multiple Aruco codes, fundamentally solving the problem of limited viewing angle, and minimizes errors by selecting the optimal Aruco code and the optimal camera in turn, so that the obtained results are more accurate.

Description

Technical Field

[0001] The present application relates to the field of robotics, and in particular to a method and device for calculating the pose of an object based on multiple cameras and multiple Aruco codes. Background Art

[0002] As an advanced marking system based on two-dimensional patterns, Aruco code plays a critical role in today's cutting-edge fields such as computer vision, robot positioning and augmented reality. It relies on the aruco module in the OpenCV library to run effectively, presenting a unique two-dimensional code form similar to a chessboard. Although it is related to traditional two-dimensional codes, it has its own distinct characteristics. In terms of functional nature, Aruco code mainly exists as a powerful single-target tracking and pose estimation tool. Its core principle is that it can accurately determine the specific position and orientation information of an object in three-dimensional space based on the specific feature information detected in the image. This feature provides solid technical support for applications in many related fields.

[0003] The advantages of Aruco code are significant. Its high-precision positioning capability enables it to be used in many scenarios with strict requirements on position accuracy, such as providing reliable data basis in tasks such as precision assembly and precise navigation. The characteristics of rapid recognition and encoding greatly improve the efficiency of information processing, and can complete the recognition and information extraction of a large number of markers in a short period of time, effectively improving the overall operation speed of the system. In addition, its relatively low requirements for processing power also make it widely applicable. Even in some devices or environments with limited computing resources, it can still run stably and will not be restricted by excessive performance requirements. It is precisely based on these outstanding advantages that Aruco code has been particularly widely used in the field of robotics and has become an important part of many robot-related technical solutions.

[0004] However, with the continuous in-depth expansion of Aruco code in actual application scenarios, a series of shortcomings of Aruco code have gradually been exposed and become the key factors restricting its further development and application effect. First, Aruco code is highly dependent on the visual system, which means that in poor visual environment, such as low light conditions, the clarity and contrast of the image will be significantly reduced, making it difficult for Aruco code to be accurately recognized and detected. At the same time, in complex scenes with other interference objects, the interference objects may confuse or block the pattern of Aruco code, seriously affecting its recognition effect, thereby reducing the reliability and stability of the entire system. Secondly, due to the principle characteristics of Aruco code itself, its viewing angle limitation problem is more prominent. When the viewing angle of Aruco code changes significantly, the recognition of its pattern will drop sharply, which directly leads to a significant reduction in the accuracy of the returned results and a large error. In some tasks where robots need to move flexibly and operate at multiple angles, this error caused by the change of viewing angle may lead to serious consequences, such as robot positioning deviation, operation errors, etc., which poses a serious threat to the smooth completion of the entire task. Therefore, how to effectively overcome these shortcomings of Aruco code in practical applications has become one of the important directions of current research and technological improvement in related fields. Summary of the invention

[0005] The embodiments of the present application provide a method and device for calculating the object pose based on multiple cameras and multiple Aruco codes. Multiple cameras are used to cover the entire target scene, and multiple Aruco codes are set on the object to be measured to fundamentally solve the problem of limited viewing angle. The error is minimized to the greatest extent by selecting the optimal Aruco code and the optimal camera in turn, so that the obtained result is more accurate.

[0006] In a first aspect, an embodiment of the present application provides a method for calculating an object pose based on multi-camera multi-Aruco codes, the method comprising:

[0007] Multiple cameras and an object to be measured are set in the target scene, and a cube with Aruco codes drawn on six sides is rigidly connected to the object to be measured;

[0008] Obtain a target scene image taken by each camera, wherein the target scene image includes at least one Aruco code, and use an angle selection algorithm to select the optimal Aruco code for each camera, wherein the angle selection algorithm is: calculating the angle between the first direction vector and the second direction vector of each Aruco code in the target scene image in the corresponding camera coordinate system, and selecting the Aruco code with the smallest angle as the optimal Aruco code, wherein the first direction vector is the Z-axis orientation of the Aruco code in the corresponding camera coordinate system, and the second direction vector is the direction vector from the origin of the Aruco code to the origin of the corresponding camera coordinate system;

[0009] The area of ​​each optimal Aruco code in the target scene image is calculated, the camera corresponding to the optimal Aruco code with the largest area is selected as the optimal camera, and the optimal Aruco code corresponding to the optimal camera is analyzed to obtain the pose of the object to be measured.

[0010] In a second aspect, an embodiment of the present application provides an object pose calculation device based on multi-camera multi-Aruco codes, including:

[0011] A setting module is used to set multiple cameras and an object to be tested in a target scene, and use a cube with Aruco codes drawn on six sides to rigidly connect with the object to be tested;

[0012] An acquisition module is used to acquire a target scene image taken by each camera, wherein the target scene image includes at least one Aruco code, and an angle selection algorithm is used to select an optimal Aruco code for each camera. The angle selection algorithm is as follows: calculating an angle between a first direction vector and a second direction vector of each Aruco code in the target scene image in a corresponding camera coordinate system, and selecting an Aruco code with a minimum angle as the optimal Aruco code, wherein the first direction vector is a Z-axis orientation of the Aruco code in the corresponding camera coordinate system, and the second direction vector is a direction vector from an origin of the Aruco code to an origin of the corresponding camera coordinate system;

[0013] The calculation module is used to calculate the area of ​​each optimal Aruco code in the target scene image, select the camera corresponding to the optimal Aruco code with the largest area as the optimal camera, and analyze the optimal Aruco code corresponding to the optimal camera to obtain the posture of the object to be measured.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a method for calculating an object pose based on multi-camera multi-Aruco codes.

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes a method for calculating an object pose based on multi-camera multi-Aruco codes.

[0016] The main contributions and innovations of the present invention are as follows:

[0017] The embodiment of the present application ensures that the Aruco code can be normally collected no matter where it is by setting multiple cameras in the target scene, thereby avoiding the problem of being limited by a single viewing angle; in order to ensure that the Aruco code can be normally collected, the present solution uses a cube with Aruco codes drawn on six sides to be rigidly connected to the object to be measured, thereby ensuring that the Aruco code of the object to be measured can be captured by multiple cameras under any working state; the present solution first selects the optimal Aruco code in the target scene image acquired by each camera, and then selects the camera with the largest area of ​​the optimal Aruco code in the target scene image as the optimal camera, ensuring that the best Aruco code can be accurately selected for analysis under multiple cameras and multiple Aruco codes, making the posture analysis of the object to be measured more accurate.

[0018] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 is a flow chart of a method for calculating an object pose based on multi-camera multi-Aruco codes according to an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of a cube with an Aruco code drawn thereon and rigidly connected to the object to be tested according to an embodiment of the present application;

[0022] Figure 3 It is a structural block diagram of an object pose calculation device based on multi-camera multi-Aruco code according to an embodiment of the present application;

[0023] Figure 4 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0025] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0026] Embodiment 1

[0027] The embodiment of the present application provides a method for calculating the position and posture of an object based on multiple cameras and multiple Aruco codes. Multiple cameras are used to cover the entire target scene, and multiple Aruco codes are set on the object to be measured to fundamentally solve the problem of limited viewing angle. The error is minimized to the greatest extent by selecting the optimal Aruco code and the optimal camera in turn, so that the result obtained is more accurate. Specifically, refer to Figure 1 , the method comprising:

[0028] Multiple cameras and an object to be measured are set in the target scene, and a cube with Aruco codes drawn on six sides is rigidly connected to the object to be measured;

[0029] Obtain a target scene image taken by each camera, wherein the target scene image includes at least one Aruco code, and use an angle selection algorithm to select the optimal Aruco code for each camera, wherein the angle selection algorithm is: calculating the angle between the first direction vector and the second direction vector of each Aruco code in the target scene image in the corresponding camera coordinate system, and selecting the Aruco code with the smallest angle as the optimal Aruco code, wherein the first direction vector is the Z-axis orientation of the Aruco code in the corresponding camera coordinate system, and the second direction vector is the direction vector from the origin of the Aruco code to the origin of the corresponding camera coordinate system;

[0030] The area of ​​each optimal Aruco code in the target scene image is calculated, the camera corresponding to the optimal Aruco code with the largest area is selected as the optimal camera, and the optimal Aruco code corresponding to the optimal camera is analyzed to obtain the pose of the object to be measured.

[0031] In some specific embodiments, each camera is installed on a fixed bracket in the target scene, and the camera cannot be moved after installation. The position of each camera can be set according to the target scene to ensure that the target scene can be monitored without blind spots.

[0032] In some specific embodiments, a schematic diagram of a cube with an Aruco code and a rigid connection with the object to be tested is shown in FIG. Figure 2 As shown, in this solution, the object to be measured is a mechanical gripper, the Aruco code labels on the six faces of the cube are different, and the Aruco code on each face is associated with the object to be measured.

[0033] Specifically, after a cube with six different Aruco code labels is rigidly connected to the object to be measured, no matter which side of the cube's Aruco code is photographed, the Aruco code detection program can be used to obtain the position and posture of the object to be measured.

[0034] In some embodiments, after multiple cameras and objects to be measured are set up in the target scene, each camera is calibrated. First, the respective IDs corresponding to the cameras in the OpenCV environment are obtained for OpenCV to capture camera data. Then, a camera calibration program is used in the OpenCV environment to calibrate each camera based on the GP290-20-12x9 checkerboard to obtain the intrinsic parameters and distortion coefficients of each camera. Then, an Aruco code detection program is prepared. The Aruco code detection program verifies the calibration results of the camera intrinsic parameters and distortion coefficients by detecting the position and posture of the Aruco code captured by each camera relative to the camera, and compares the position and posture of the Aruco code relative to the camera in the real world with the position and posture of the Aruco code relative to the camera obtained by the Aruco code detection program to complete the camera calibration.

[0035] In some embodiments, an Aruco code detection program is used to detect each Aruco code captured by the camera to obtain an rvec vector and a tvec vector corresponding to each Aruco code. The rvec vector is used to align the coordinate system of the corresponding Aruco code with the camera coordinate system. The tvec vector represents the position of the origin of the coordinate system of the Aruco code in the camera coordinate system. The first direction vector of the corresponding Aruco code is obtained based on the rvec vector, and the second direction vector of the corresponding Aruco code is obtained based on the tvec vector.

[0036] Furthermore, a rotation matrix is ​​obtained according to the rvec vector, the last column of which is the first direction vector corresponding to the Aruco code, and the second direction vector corresponding to the Aruco code is obtained by subtracting the tvec vector from the origin coordinates of the camera coordinate system.

[0037] Specifically, the rvec vector is a Vector, use Rodrigues formula to convert rvec vector to common The rotation matrix of is expressed as follows:

[0038]

[0039] In which, assuming that rvec vector , its size Unit Vector , is the identity matrix, The antisymmetric matrix constructed for the rotation axis,

[0040]

[0041] Specifically, since the rvec vector in this solution is used to align the coordinate system of the corresponding Aruco code with the camera coordinate system, the last column of the rotation matrix obtained according to the rvec vector is the Z-axis orientation of the Aruco code in the corresponding camera coordinate system.

[0042] Specifically, since the tvec vector represents the position of the origin of the coordinate system of the corresponding Aruco code in the corresponding camera coordinate system, it can be concluded that the tvec vector is exactly the movement distance of the Aruco code on the three axes of XYZ. , so use the camera origin minus The second direction vector corresponding to the Aruco code can be obtained.

[0043] In some specific embodiments, the angle between the first direction vector and the second direction vector is calculated by dot product calculation, and the formula is as follows:

[0044]

[0045]

[0046] in, is the angle, u is the first direction vector, and v is the second direction vector.

[0047] Finally, among the calculation results of each camera and all the Aruco codes captured, the Aruco code with the smallest angle is selected as the optimal Aruco code for the corresponding camera.

[0048] Specifically, the smaller the angle between the first direction vector and the second direction vector is, the more perpendicular the Aruco code is to the camera, and the better the photographing effect is.

[0049] In some other embodiments, a distance judgment method is used to select the optimal Aruco code for each camera. The distance judgment method is: calculating the distance between each camera and each Aruco code in the corresponding target scene image, and selecting the Aruco code with the closest distance as the optimal Aruco code.

[0050] Specifically, when the distance between the Aruco code and the corresponding camera is closer, the effect of the Aruco code captured by the camera is better, so the distance between the Aruco code and the corresponding camera can also be used as a judgment basis to judge the optimal Aruco code.

[0051] Furthermore, in some other embodiments, the angle selection algorithm and the distance selection algorithm may be used simultaneously in a weighted sum method to obtain the optimal Aruco code.

[0052] In some specific embodiments, the detectMarkers function is used to detect the optimal Aruco code to obtain the coordinates of the four corner points of the optimal Aruco code, and the area of ​​the optimal Aruco code in the target scene image is calculated based on the coordinates of the four corner points of the optimal Aruco code, wherein the detectMarkers function is a standard function in the aruco module in the image processing library OpenCV.

[0053] Specifically, since the resolution of each target scene image is the same, the larger the area of ​​the optimal Aruco code in the target scene image, the better the recognition effect of the Aruco code. Therefore, this scheme selects the camera corresponding to the optimal Aruco code with the largest area as the optimal camera.

[0054] Specifically, this solution first selects the optimal Aruco code in the target scene image obtained by each camera, and then compares the area of ​​each Aruco code in the corresponding target scene image to select the optimal camera, so as to select the best Aruco code for recognition in a multi-camera and multi-Aruco code scene, so that the Aruco code with the best recognition effect can be accurately found regardless of low light conditions or occlusion, perspective offset, etc.

[0055] In the prior art, one camera is usually used to calculate an Aruco code, so the coordinate system conversion when parsing the Aruco code to obtain the position and posture of the Aruco code in the real world is very simple. However, in this solution, since multiple cameras are used, it is not certain which camera is used to obtain the Aruco code for posture recognition, so it is necessary to additionally design a coordinate system conversion method for coordinate system conversion.

[0056] In some specific embodiments, any camera in the target scene is selected as a world camera, and the coordinate system of the world camera is the world coordinate system. A calibrated Aruco code that can be seen by all cameras is placed in the target scene. The conversion relationship between any camera and the world camera is obtained based on the rotation matrix and translation matrix of the calibrated Aruco code in each camera including the world camera. If the optimal camera is a world camera, the optimal Aruco code of the optimal camera is directly parsed to obtain the pose of the object to be measured. If the optimal camera is not a world camera, the optimal Aruco code corresponding to the optimal camera is converted to the world coordinate system according to the conversion relationship between the optimal camera and the world camera, and then parsed to obtain the pose of the object to be measured.

[0057] Specifically, the rotation matrix and translation matrix of the calibrated Aruco code in each camera including the world camera are obtained respectively, and the conversion relationship between any camera and the world camera is obtained based on the rotation matrix and translation matrix of the calibrated Aruco code in any camera and the rotation matrix and translation matrix of the calibrated Aruco code in the world camera. The formula is as follows:

[0058]

[0059]

[0060] Among them, 0 is the world camera, 1 is any camera except the world camera, To calibrate the Aruco code, is the rotation matrix, is the translation matrix, Represents the rotation matrix of camera 1 in the world coordinate system of world camera 0, Represents the translation matrix of camera 1 in the world coordinate system of world camera 0.

[0061] Specifically, the rotation matrix and translation matrix are obtained with the camera as the origin, according to the following formula:

[0062]

[0063] It can be seen that the rotation matrix is ​​transposed and the translation matrix is ​​inverted to obtain the rotation and translation matrices of the camera in the Aruco code coordinate system.

[0064] In some specific embodiments, when the optimal camera is a world camera, the Aruco code detection program is used to obtain the rvec vector and tvec vector of the optimal Aruco code corresponding to the optimal camera as the final rvec vector and the final tvec vector, the rvec vector is a rotation matrix, and the tvec vector is a translation matrix; when the optimal camera is not a world camera, the conversion relationship between the optimal camera coordinate system and the world camera coordinate system is calculated, and then based on the formula

[0065]

[0066]

[0067] Get the final rvec vector and the final tvec vector.

[0068] Furthermore, the final rvec vector and the final tvec vector are analyzed to obtain the position and posture of the object to be tested.

[0069] In some specific embodiments, although the camera in the target scene is fixed, it may still move due to some external factors. The calibration of the camera in the target scene by the calibration Aruco code is often one-time, so it is unlikely to put the calibration Aruco code in the same position for recalibration. Therefore, in order to ensure that the conversion relationship between any camera and the world camera can still be known after the camera moves, when the optimal camera is the world camera, the optimal Aruco code of the optimal camera is used as the calibration Aruco code to re-acquire the conversion relationship between any camera in the target scene and the world camera.

[0070] Embodiment 2

[0071] Based on the same idea, refer to Figure 3 , the present application also proposes an object pose calculation device based on multi-camera multi-Aruco code, comprising:

[0072] A setting module is used to set multiple cameras and an object to be tested in a target scene, and use a cube with Aruco codes drawn on six sides to rigidly connect with the object to be tested;

[0073] An acquisition module is used to acquire a target scene image taken by each camera, wherein the target scene image includes at least one Aruco code, and an angle selection algorithm is used to select an optimal Aruco code for each camera. The angle selection algorithm is as follows: calculating an angle between a first direction vector and a second direction vector of each Aruco code in the target scene image in a corresponding camera coordinate system, and selecting an Aruco code with a minimum angle as the optimal Aruco code, wherein the first direction vector is a Z-axis orientation of the Aruco code in the corresponding camera coordinate system, and the second direction vector is a direction vector from an origin of the Aruco code to an origin of the corresponding camera coordinate system;

[0074] The calculation module is used to calculate the area of ​​each optimal Aruco code in the target scene image, select the camera corresponding to the optimal Aruco code with the largest area as the optimal camera, and analyze the optimal Aruco code corresponding to the optimal camera to obtain the posture of the object to be measured.

[0075] Embodiment 3

[0076] This embodiment also provides an electronic device, referring to Figure 4 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0077] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0078] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0079] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0080] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the object pose calculation methods based on multi-camera multi-Aruco codes in the above embodiments.

[0081] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0082] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0083] The input and output device 408 is used to input or output information. In this embodiment, the input information may be the Aruco code, the area of ​​each Aruco code in the target scene image, etc., and the output information may be the position and posture of the object to be measured, etc.

[0084] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program:

[0085] Multiple cameras and an object to be measured are set in the target scene, and a cube with Aruco codes drawn on six sides is rigidly connected to the object to be measured;

[0086] Obtain a target scene image taken by each camera, wherein the target scene image includes at least one Aruco code, and use an angle selection algorithm to select the optimal Aruco code for each camera, wherein the angle selection algorithm is: calculating the angle between the first direction vector and the second direction vector of each Aruco code in the target scene image in the corresponding camera coordinate system, and selecting the Aruco code with the smallest angle as the optimal Aruco code, wherein the first direction vector is the Z-axis orientation of the Aruco code in the corresponding camera coordinate system, and the second direction vector is the direction vector from the origin of the Aruco code to the origin of the corresponding camera coordinate system;

[0087] The area of ​​each optimal Aruco code in the target scene image is calculated, the camera corresponding to the optimal Aruco code with the largest area is selected as the optimal camera, and the optimal Aruco code corresponding to the optimal camera is analyzed to obtain the pose of the object to be measured.

[0088] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0089] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0090] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 4 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0091] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for calculating object pose based on multi-camera multi-Aruco code, characterized in that: The following steps are involved: Multiple cameras and an object to be measured are set in the target scene, and a cube with Aruco codes drawn on six sides is rigidly connected to the object to be measured; Obtain a target scene image captured by each camera, wherein the target scene image includes at least one Aruco code, and use an angle selection algorithm to select the optimal Aruco code for each camera. The angle selection algorithm is as follows: calculate the angle between the first direction vector and the second direction vector of each Aruco code in the target scene image in the corresponding camera coordinate system, and select the Aruco code with the smallest angle as the optimal Aruco code, wherein an Aruco code detection program is used to detect each Aruco code captured by the camera to obtain an rvec vector and a tvec vector corresponding to each Aruco code, rvec The c vector is used to align the coordinate system of the corresponding Aruco code with the camera coordinate system. The tvec vector represents the position of the origin of the coordinate system of the Aruco code in the camera coordinate system. The rotation matrix is ​​obtained according to the rvec vector. The last column of the rotation matrix is ​​the first direction vector of the corresponding Aruco code. The second direction vector of the corresponding Aruco code is obtained by subtracting the tvec vector from the origin coordinate of the camera coordinate system. The first direction vector is the Z-axis orientation of the Aruco code in the corresponding camera coordinate system, and the second direction vector is the direction vector from the origin of the Aruco code to the origin of the corresponding camera coordinate system. The area of ​​each optimal Aruco code in the target scene image is calculated, the camera corresponding to the optimal Aruco code with the largest area is selected as the optimal camera, and the optimal Aruco code corresponding to the optimal camera is analyzed to obtain the pose of the object to be measured.

2. The object pose calculation method based on multi-camera multi-Aruco code according to claim 1 is characterized in that: The Aruco code labels on the six faces of the cube are different, and the Aruco code on each face is associated with the object to be detected.

3. The object pose calculation method based on multi-camera multi-Aruco code according to claim 1 is characterized in that: The detectMarkers function is used to detect the optimal Aruco code to obtain the coordinates of the four corner points of the optimal Aruco code, and the area of ​​the optimal Aruco code in the target scene image is calculated based on the coordinates of the four corner points of the optimal Aruco code.

4. The object pose calculation method based on multi-camera multi-Aruco code according to claim 1 is characterized in that: Any camera in the target scene is selected as the world camera, and the coordinate system of the world camera is the world coordinate system. A calibration Aruco code that can be seen by all cameras is placed in the target scene. The conversion relationship between any camera and the world camera is obtained based on the rotation matrix and translation matrix of the calibration Aruco code in each camera including the world camera. If the optimal camera is the world camera, the optimal Aruco code of the optimal camera is directly parsed to obtain the pose of the object to be measured. If the optimal camera is not the world camera, the optimal Aruco code corresponding to the optimal camera is converted to the world coordinate system according to the conversion relationship between the optimal camera and the world camera, and then parsed to obtain the pose of the object to be measured.

5. The object pose calculation method based on multi-camera multi-Aruco code according to claim 4 is characterized in that: When the optimal camera is a world camera, the optimal Aruco code of the optimal camera is used as the calibration Aruco code to re-acquire the conversion relationship between any camera in the target scene and the world camera.

6. An object pose calculation device based on multi-camera multi-Aruco code, characterized in that: include: A setting module is used to set multiple cameras and an object to be tested in a target scene, and use a cube with Aruco codes drawn on six sides to rigidly connect with the object to be tested; An acquisition module is used to acquire a target scene image captured by each camera, wherein the target scene image includes at least one Aruco code, and an angle selection algorithm is used to select an optimal Aruco code for each camera. The angle selection algorithm is as follows: calculating an angle between a first direction vector and a second direction vector of each Aruco code in the target scene image in a corresponding camera coordinate system, and selecting an Aruco code with a minimum angle as the optimal Aruco code, wherein an Aruco code detection program is used to detect each Aruco code captured by the camera to obtain an rvec vector and a tvec vector corresponding to each Aruco code. The rvec vector is used to align the coordinate system of the corresponding Aruco code with the camera coordinate system. The tvec vector represents the position of the origin of the coordinate system of the Aruco code in the camera coordinate system. The rotation matrix is ​​obtained according to the rvec vector. The last column of the rotation matrix is ​​the first direction vector of the corresponding Aruco code. The second direction vector of the corresponding Aruco code is obtained by subtracting the tvec vector from the origin coordinate of the camera coordinate system. The first direction vector is the Z-axis orientation of the Aruco code in the corresponding camera coordinate system, and the second direction vector is the direction vector from the origin of the Aruco code to the origin of the corresponding camera coordinate system. The calculation module is used to calculate the area of ​​each optimal Aruco code in the target scene image, select the camera corresponding to the optimal Aruco code with the largest area as the optimal camera, and analyze the optimal Aruco code corresponding to the optimal camera to obtain the posture of the object to be measured.

7. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the object pose calculation method based on multi-camera multi-Aruco code as described in any one of claims 1-5.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, and the process includes an object pose calculation method based on multi-camera multi-Aruco codes according to any one of claims 1-5.

Images