Data processing method, data processing device and calibration system

Abstract

The application relates to a data processing method, a data processing device and a calibration system, wherein the data processing method comprises the following steps: acquiring a plurality of groups of first reference coordinate points of a calibration board in a first camera coordinate system corresponding to a plurality of groups of poses, and a plurality of groups of second reference coordinate points of the calibration board in a second camera coordinate system corresponding to a plurality of groups of poses; acquiring position information of a to-be-calibrated object in the second camera coordinate system corresponding to a plurality of groups of different poses of a second camera; determining a conversion relationship between the first camera coordinate system and the second camera coordinate system according to the first reference coordinate points and the second reference coordinate points; and obtaining position information of the to-be-calibrated object in the first camera coordinate system according to the conversion relationship and the position information, so that the accurate solution of the position coordinates of an object outside the camera field of view is realized by using a plurality of cameras, and the calibration of the spatial position of the object outside the camera field of view is realized, and the accuracy of the calibration result is improved.

Description

Data processing methods, data processing devices, and calibration systems Technical Field

[0001] This application relates to the field of spatial position calibration of objects outside the camera's field of view, and in particular to data processing methods, data processing devices, and calibration systems. Background Technology

[0002] During camera scanning, calibrating objects outside the camera's field of view plays a crucial role in the 3D reconstruction of the scanned object. For example, in eye-tracking technology, it is necessary to calibrate the spatial position of an LED (light-emitting diode) light source, rigidly connected to an infrared camera and located outside the camera's field of view, within the infrared camera's coordinate system, in order to establish a 3D model for solving the human eye's line of sight. Currently, calibrating the spatial position of objects outside the camera's field of view often involves introducing a plane mirror, using the camera to photograph a target plate with the plane mirror, and performing calibration based on the principle of mirror reflection. However, this method suffers from magnification errors due to the plane mirror, resulting in low accuracy of the calibration results.

[0003] There is currently no effective solution to the problem of low accuracy in spatial positioning of objects outside the camera's field of view in related technologies. Summary of the Invention

[0004] This embodiment provides a data processing method, a data processing device, and a calibration system to solve the problem of low accuracy in spatial position calibration of objects outside the camera's field of view in related technologies.

[0005] Firstly, this embodiment provides a data processing method for spatial position calibration of an object to be calibrated outside the camera's field of view, including:

[0006] Obtain the first reference coordinate points corresponding to multiple poses of the target plate in the first camera coordinate system, and the second reference points corresponding to multiple poses of the target plate in the second camera coordinate system.

[0007] The position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera is obtained; wherein the object to be calibrated is within the field of view of the second camera and outside the field of view of the first camera, and the relative position between the object to be calibrated and the first camera is fixed;

[0008] Based on the first reference coordinate point and the second reference coordinate point, determine the transformation relationship between the first camera coordinate system and the second camera coordinate system;

[0009] Based on the transformation relationship and the position information, the position information of the object to be calibrated in the first camera coordinate system is obtained.

[0010] In some embodiments, obtaining the first reference coordinate points corresponding to multiple poses of the target plate in the first camera coordinate system and the second reference coordinate points corresponding to multiple poses of the target plate in the second camera coordinate system includes:

[0011] Acquire a first image corresponding to multiple poses of the target plate obtained by the first camera, and a second image corresponding to multiple poses of the target plate obtained by the second camera;

[0012] The first reference coordinate point is obtained based on the first image and the camera intrinsic parameters of the first camera;

[0013] The second reference coordinate point is obtained based on the second image and the camera intrinsic parameters of the second camera.

[0014] In some embodiments, obtaining the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera includes:

[0015] A third image corresponding to multiple different poses of the object to be calibrated and the second camera is acquired by adjusting the pose of the object itself, and the reference point of the object to be calibrated is extracted from the third image.

[0016] Based on the camera intrinsic parameters of the second camera, the pixel coordinates of the reference point are converted into ray information of the object to be calibrated in the second camera coordinate system, thereby obtaining the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera.

[0017] In some embodiments, determining the transformation relationship between the first camera coordinate system and the second camera coordinate system based on the first reference coordinate point and the second reference coordinate point includes:

[0018] Obtain the third reference coordinate point of the target plate in the world coordinate system;

[0019] Based on the third reference coordinate point and the first reference coordinate point, the first transformation relationship between the first camera and the world coordinate system is obtained;

[0020] Based on the third reference coordinate point and the second reference coordinate point, a second transformation relationship between the second camera and the world coordinate system is obtained;

[0021] Based on the first transformation relationship and the second transformation relationship, calculate the transformation relationship between the first camera coordinate system and the second camera coordinate system.

[0022] In some embodiments, obtaining the position information of the object to be calibrated in the first camera coordinate system based on the transformation relationship and the position information includes:

[0023] Based on the ray information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera, and the transformation relationship, multiple sets of ray information of the object to be calibrated in the first camera coordinate system are obtained.

[0024] Based on multiple sets of ray information of the object to be calibrated in the first camera coordinate system, the coordinates of the reference point of the object to be calibrated in the first camera coordinate system are calculated, and the position information of the object to be calibrated in the first camera coordinate system is obtained based on the reference point coordinates.

[0025] In some embodiments, calculating the reference point coordinates of the object to be calibrated in the first camera coordinate system based on multiple sets of ray information of the object to be calibrated in the first camera coordinate system includes:

[0026] Based on multiple sets of ray information of the object to be calibrated in the first camera coordinate system, the point with the minimum distance between each ray of the multiple sets of ray information in the first camera coordinate system is calculated to obtain the reference point coordinates of the object to be calibrated in the first camera coordinate system.

[0027] Secondly, this embodiment provides a data processing device for spatial position calibration of an object to be calibrated outside the camera's field of view, comprising: a first acquisition module, a second acquisition module, a conversion module, and a calibration module; wherein:

[0028] The first acquisition module is used to acquire first reference coordinate points corresponding to multiple poses of the target plate in the first camera coordinate system, and second reference coordinate points corresponding to multiple poses of the target plate in the second camera coordinate system.

[0029] The second acquisition module is used to acquire the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera; wherein the object to be calibrated is within the field of view of the second camera and outside the field of view of the first camera, and the relative position between the object to be calibrated and the first camera is fixed;

[0030] The conversion module is used to determine the conversion relationship between the first camera coordinate system and the second camera coordinate system based on the first reference coordinate point and the second reference coordinate point;

[0031] The calibration module is used to obtain the position information of the object to be calibrated in the first camera coordinate system based on the transformation relationship and the position information.

[0032] Thirdly, this embodiment provides a calibration system for spatial position calibration of an object to be calibrated outside the camera's field of view, including: a first camera, a second camera, a calibration board, and a processor; wherein: the first camera and the second camera are both communicatively connected to the processor;

[0033] The first camera is used to acquire first images of the target plate in multiple poses and transmit the first images to the processor;

[0034] The second camera is used to acquire second images of the target plate in multiple poses and a third image of the object to be calibrated that is outside the field of view of the first camera, and to transmit the second image and the third image to the processor;

[0035] The processor is configured to execute the data processing method described in the first aspect above.

[0036] In some embodiments, the calibration plate is a double-sided calibration plate, which is disposed between the first camera and the second camera; the first camera is used to acquire a calibration image of the first side of the double-sided calibration plate to obtain a first image; the second camera is used to acquire a calibration image of the second side of the double-sided calibration plate to obtain a second image.

[0037] In some embodiments, the calibration system is used to simultaneously calibrate the spatial position of multiple objects to be calibrated that are not within the camera's field of view.

[0038] This application provides a data processing method, a data processing device, and a calibration system. The data processing method provided in this embodiment acquires first reference coordinate points corresponding to multiple poses of a first camera coordinate system sub-tablet, and second reference coordinate points corresponding to multiple poses of a second camera coordinate system sub-tablet; it acquires position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera; wherein the object to be calibrated is within the field of view of the second camera and outside the field of view of the first camera, and the relative position between the object to be calibrated and the first camera is fixed; based on the first and second reference coordinate points, it determines the transformation relationship between the first and second camera coordinate systems; based on the transformation relationship and the position information, it obtains the position information of the object to be calibrated in the first camera coordinate system. By utilizing multiple cameras, it accurately solves for the position coordinates of objects outside the camera's field of view, thereby achieving the calibration of the spatial position of objects outside the camera's field of view and improving the accuracy of the calibration results.

[0039] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0040] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0041] Figure 1 is a hardware structure block diagram of the terminal of the data processing method of the related technology;

[0042] Figure 2 is a flowchart of the data processing method of this embodiment;

[0043] Figure 3 is a schematic diagram of a second camera pose adjustment according to this embodiment;

[0044] Figure 4 is a flowchart of the data processing method of this preferred embodiment;

[0045] Figure 5 is a structural block diagram of the data processing device in this embodiment;

[0046] Figure 6 is a schematic diagram of the calibration system in this embodiment. Detailed Implementation

[0047] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0048] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0049] The method embodiments provided in this example can be executed in a terminal, computer, or similar computing device. For example, running on a terminal, FIG1 is a hardware structure block diagram of the terminal of the data processing method of this embodiment. As shown in FIG1, the terminal may include one or more (only one is shown in FIG1) processors 102 and a memory 104 for storing data, wherein the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that the structure shown in FIG1 is only illustrative and does not limit the structure of the terminal. For example, the terminal may also include more or fewer components than shown in FIG1, or have a different configuration than shown in FIG1.

[0050] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the data processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0051] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0052] This embodiment provides a data processing method. Figure 2 is a flowchart of the data processing method of this embodiment. As shown in Figure 2, the process includes the following steps:

[0053] Step S210: Obtain the first reference coordinate points corresponding to multiple poses of the first camera coordinate system subscript plate, and the second reference coordinate points corresponding to multiple poses of the second camera coordinate system subscript plate.

[0054] The calibration plate can be any planar object used for camera calibration and equipped with calibration images. For example, in this embodiment, the calibration plate can be a double-sided calibration plate with two calibration surfaces, made using double-sided exposure and magnetron sputtering. The first camera can be a camera used for calibrating the position of the object to be calibrated outside the camera's field of view, i.e., the object to be calibrated is located outside the field of view of the first camera. The second camera is introduced to assist the first camera in calibrating the spatial position of the object to be calibrated outside the camera's field of view. The second camera and the first camera can be cameras with the same structure or cameras with different structures. The calibration plate can be set within the common field of view of the first and second cameras, so that the first and second cameras simultaneously acquire images of the calibration plate. For example, the second camera can be set opposite to the first camera, and the double-sided calibration plate can be set between the first and second cameras, so that the first camera acquires an image of one side of the double-sided calibration plate, and the second camera acquires an image of the other side of the double-sided calibration plate. Further, the calibration plate can be provided with several marker points or a checkerboard pattern. The method for setting the calibration image can be customized according to the actual application scenario, and no specific limitations are made here.

[0055] It is understandable that the first reference coordinate point can specifically be the coordinate point extracted from the images of the target plate in different poses captured by the first camera after adjusting the target plate to multiple sets of different poses. The second reference coordinate point can specifically be the coordinate point extracted from the images of the target plate in different poses captured by the second camera after adjusting the target plate to multiple sets of different poses. During the pose adjustment of the target plate, the first and second cameras need to maintain a fixed relative position and simultaneously capture images of the target plate. Therefore, for each pose of the target plate, there are corresponding reference coordinate points in both the first and second camera coordinate systems. Furthermore, the number of poses of the target plate can be set according to actual needs.

[0056] Step S220: Obtain the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera; wherein, the object to be calibrated is within the field of view of the second camera and outside the field of view of the first camera, and the relative position between the object to be calibrated and the first camera is fixed.

[0057] When setting the position of the second camera, the target object must be located within the field of view of the second camera while both the first and second cameras simultaneously capture images of the target. During the image acquisition process of the second camera on the target object, multiple sets of position information of the target object can be obtained by changing the pose of the second camera, thereby improving the accuracy of spatial position calibration of the target object. The number of poses of the second camera adjusted can be adjusted according to the actual situation.

[0058] Preferably, under each pose of the second camera, multiple sets of poses of the target plate can be adjusted to obtain the first and second reference coordinate points corresponding to the multiple sets of poses. Then, while keeping the object to be calibrated, the first camera, and the target plate fixed, the pose of the second camera is adjusted further, and the first and second reference coordinate points of the target plate under the new pose of the second camera, as well as the image of the object to be calibrated under the new pose of the second camera, are obtained. For example, 20 different poses can be set for the second camera, and images of the target plate in 5 different poses can be acquired under each of the 20 different poses of the second camera. The 5 pairs of images of the target plate under each second camera pose under the first and second cameras are set as a group, until 20 groups of images are obtained. The pose of the second camera can be adjusted by an automated robotic arm or manually, and is not specifically limited here.

[0059] Figure 3 is a schematic diagram of the second camera pose adjustment according to this embodiment. As shown in Figure 3, the first camera 302 and the object to be calibrated 301 are connected by a rigid structure, wherein the object to be calibrated 301 is located outside the field of view of the first camera 302 and within the field of view of the second camera 304. The black box represents the second camera 304 in different poses. It can be seen that during the process of the second camera 304 adjusting its own pose, the object to be calibrated 301 is always located within the field of view of the second camera 304, and the target plate 303 is located between the first camera 302 and the second camera 304. It is understood that the schematic diagram in Figure 3 is only an illustrative example of the positional relationship between the first camera, the second camera, the object to be calibrated, and the target plate, and does not limit the specific settings of the first camera, the second camera, and the target plate in this embodiment.

[0060] Step S230: Determine the transformation relationship between the first camera coordinate system and the second camera coordinate system based on the first reference coordinate point and the second reference coordinate point.

[0061] Furthermore, based on the first reference coordinate point, a first transformation relationship between the first camera coordinate system and the world coordinate system can be determined, and based on the second reference coordinate point, a second transformation relationship between the second camera coordinate system and the world coordinate system can be determined, thereby obtaining the transformation relationship between the first camera coordinate system and the second camera coordinate system. Specifically, in each pose of the second camera, there are multiple sets of first and second reference coordinate points corresponding to different poses of the target, and the relative positions between the first and second cameras are fixed. Therefore, based on the multiple sets of first and second reference coordinate points in each pose of the second camera, the following collinearity constraint equations can be solved simultaneously to obtain the transformation relationship between the first camera coordinate system and the second camera coordinate system in each pose of the second camera.

[0062] RT w2B =RTC2B *RT w2C (1)

[0063] Among them, RT w2B To illustrate the transformation relationship between the first camera coordinate system and the world coordinate system, RT w2C For the transformation relationship between the second camera coordinate system and the world coordinate system, RT C2B This defines the transformation relationship between the first camera coordinate system and the second camera coordinate system. Based on the above equations, the transformation between the RT coordinate system and the second camera coordinate system can be achieved. C2B Solving this equation allows us to determine the external parameters between the first and second cameras, that is, the transformation relationship between the first camera coordinate system and the second camera coordinate system.

[0064] Step S240: Based on the transformation relationship and position information, obtain the position information of the object to be calibrated in the first camera coordinate system.

[0065] The position information of the object to be calibrated in the second camera coordinate system can be obtained based on a reference point in the image of the object acquired by the second camera. Further, this reference point can be the centroid of the object. Specifically, based on the camera intrinsic parameters of the second camera, the pixel coordinates of the reference point in the image of the object to be calibrated can be converted into ray information pointing from the origin of the second camera coordinate system to the reference point of the object to be calibrated in the second camera coordinate system. This ray information pointing from the origin of the second camera coordinate system to the reference point of the object to be calibrated is the position information in the second camera coordinate system.

[0066] Understandably, after obtaining the transformation relationship between the first camera coordinate system and the second camera coordinate system based on the above step S230, the position information of the object to be calibrated in the second camera coordinate system can be transformed to the first camera coordinate system according to this transformation relationship. For each pose of the second camera, a ray pointing from the origin of the second camera coordinate system to the reference point of the object to be calibrated can be obtained in the first camera coordinate system based on the above transformation relationship. Therefore, by adjusting various poses of the second camera, multiple ray information pointing from the origin of the second camera coordinate system at different positions to the reference point of the object to be calibrated can be obtained in the first camera coordinate system. In practical applications, due to the influence of errors, the direction of the rays may be deviated, resulting in deviations in the points pointed to by different rays. Therefore, the point with the minimum sum of distances between all rays in the first camera coordinate system can be used as the reference point of the object to be calibrated in the first camera coordinate system. The specific solution process is shown below.

[0067]

[0068] Where p is the coordinate of the reference point of the object to be calibrated in the first camera coordinate system, i is the number of rays, and n iLet o be the direction vector of the i-th ray. i Let I be the starting point of the i-th ray, which is also the origin of the second camera coordinate system corresponding to the i-th pose of the second camera. Let I be the identity matrix and T be the transpose operation. By solving for the coordinates of the reference point of the object to be calibrated in the first camera coordinate system, the position information of the object to be calibrated in the first camera coordinate system can be determined. Furthermore, before calibrating the object to be calibrated, the intrinsic parameters of the first and second cameras can be calibrated separately. Specifically, the intrinsic parameters can be calibrated using Zhang Zhengyou calibration.

[0069] Compared to related technologies that use plane mirrors to calibrate objects outside the camera's field of view, resulting in larger errors and lower calibration accuracy, this embodiment introduces a second camera and a calibration plate. Based on the positional relationship between the first and second cameras, the spatial position of the object to be calibrated is determined, thereby improving the accuracy of the calibration results. Furthermore, this embodiment's calibration method avoids the limitations of using a calibration plate with a plane mirror, which restricts the adjustment of the calibration plate's pose and results in high difficulty and cost in calibration plate manufacturing. This improves calibration flexibility and reduces the cost of calibration plate production. Additionally, this embodiment's calibration method can simultaneously calibrate multiple objects, thus improving calibration efficiency.

[0070] Steps S210 to S240 above involve obtaining first reference coordinate points corresponding to multiple poses of the first camera coordinate system subscript plate and second reference coordinate points corresponding to multiple poses of the second camera coordinate system subscript plate; obtaining position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera; wherein the object to be calibrated is within the field of view of the second camera and outside the field of view of the first camera, and the relative position between the object to be calibrated and the first camera is fixed; determining the transformation relationship between the first camera coordinate system and the second camera coordinate system based on the first reference coordinate points and the second reference coordinate points; obtaining the position information of the object to be calibrated in the first camera coordinate system based on the transformation relationship and the position information; and using multiple cameras to accurately solve the position coordinates of the object outside the field of view, thereby realizing the calibration of the spatial position of the object outside the field of view and improving the accuracy of the calibration results.

[0071] Furthermore, in one embodiment, based on the above step S210, obtaining the first reference coordinate points corresponding to multiple poses of the first camera coordinate system subscript plate and the second reference coordinate points corresponding to multiple poses of the second camera coordinate system subscript plate specifically includes the following steps:

[0072] Step S211: Obtain a first image corresponding to multiple poses of the target plate acquired by the first camera, and a second image corresponding to multiple poses of the target plate acquired by the second camera. The first and second images contain the calibration pattern of the target plate, such as a checkerboard pattern or a marker pattern.

[0073] Step S212: Obtain a first reference coordinate point based on the first image and the camera intrinsic parameters of the first camera. Specifically, the first reference coordinate point can be the coordinates of several points contained in the first image, such as corner coordinates.

[0074] Step S213: Obtain the second reference coordinate point based on the second image and the camera intrinsic parameters of the second camera.

[0075] By obtaining the reference coordinate points in the first camera coordinate system and the second camera coordinate system under multiple poses of the target plate, the transformation relationship between the first camera coordinate system and the second camera coordinate system can be accurately solved.

[0076] In another embodiment, based on the above step S220, obtaining the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple sets of different poses of the second camera specifically includes the following steps:

[0077] Step S221: Obtain a third image of the object to be calibrated and multiple different poses of the second camera, obtained by adjusting the pose of the object itself, and extract the reference point of the object to be calibrated from the third image.

[0078] Step S222: Based on the camera intrinsic parameters of the second camera, the pixel coordinates of the reference point are converted into ray information of the object to be calibrated in the second camera coordinate system, thereby obtaining the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera.

[0079] By acquiring the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera, the accuracy of subsequent calculation of the position information of the object to be calibrated in the first camera coordinate system can be improved.

[0080] In another embodiment, based on step S230 above, the transformation relationship between the first camera coordinate system and the second camera coordinate system is determined according to the first reference coordinate point and the second reference coordinate point, specifically including the following steps:

[0081] Step S231: Obtain the third reference coordinate point of the target plate in the world coordinate system.

[0082] The third reference coordinate point is the coordinate of the point contained in the calibration pattern on the target plate in the world coordinate system. When the target plate is a double-sided target plate and the calibration pattern is a checkerboard, the third reference coordinate point can be the coordinate of each corner point of the double-sided target plate in the world coordinate system, determined according to the relative positional relationship between the two sides.

[0083] Step S232: Based on the third reference coordinate point and the first reference coordinate point, obtain the first transformation relationship between the first camera and the world coordinate system.

[0084] Step S233: Based on the third reference coordinate point and the second reference coordinate point, obtain the second transformation relationship between the second camera and the world coordinate system.

[0085] Step S234: Calculate the transformation relationship between the first camera coordinate system and the second camera coordinate system based on the first transformation relationship and the second transformation relationship.

[0086] In another embodiment, based on the above step S240, the position information of the object to be calibrated in the first camera coordinate system is obtained according to the transformation relationship and position information. This may specifically include the following steps:

[0087] Step S241: Based on the ray information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the object in the second camera, and the transformation relationship, obtain multiple sets of ray information of the object to be calibrated in the first camera coordinate system.

[0088] Step S242: Based on multiple sets of ray information of the object to be calibrated in the first camera coordinate system, calculate the reference point coordinates of the object to be calibrated in the first camera coordinate system, and obtain the position information of the object to be calibrated in the first camera coordinate system based on the reference point coordinates.

[0089] Furthermore, in one embodiment, based on the above step S242, the coordinates of the reference point of the object to be calibrated in the first camera coordinate system are calculated according to multiple sets of ray information of the object to be calibrated in the first camera coordinate system. This may specifically include the following steps:

[0090] Step S2421: Based on the multiple sets of ray information of the object to be calibrated in the first camera coordinate system, solve for the minimum distance between each ray in the multiple sets of ray information in the first camera coordinate system, and obtain the reference point coordinates of the object to be calibrated in the first camera coordinate system.

[0091] By finding the point with the minimum sum of distances between rays in multiple sets of ray information in the first camera coordinate system, the coordinates of the reference point of the object to be calibrated in the first camera coordinate system can be accurately calculated, reducing the impact of the above calculation errors on the calibration results, thereby improving the accuracy of spatial position calibration of the object to be calibrated.

[0092] The present embodiment will now be described and illustrated through preferred embodiments.

[0093] Figure 4 is a flowchart of the data processing method of this preferred embodiment. As shown in Figure 4, the data processing method includes the following steps:

[0094] Step S410: Perform the intrinsic parameter calibration of the first camera and the second camera respectively;

[0095] Step S420: Set the first camera and the second camera on both sides of the target plate;

[0096] In step S430, the first camera and the second camera simultaneously take pictures;

[0097] Step S440: Under the current pose of the second camera, keep the first camera and the second camera relatively fixed, and determine whether the image acquisition of the target board under 5 different poses has been completed. If yes, proceed to step S450; otherwise, proceed to step S470.

[0098] Step S450: Determine whether the image acquisition of the object to be calibrated under 20 different poses of the second camera has been completed. If yes, proceed to step S460; otherwise, proceed to step S480.

[0099] Step S460: Based on multiple sets of target images acquired by the first camera and the second camera, and multiple sets of images of the object to be calibrated acquired by the second camera, solve for the position information of the object to be calibrated in the coordinate system of the first camera.

[0100] Step S470: Adjust the pose of the target plate;

[0101] Step S480: Adjust the pose of the second camera.

[0102] This embodiment also provides a data processing apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0103] Figure 5 is a structural block diagram of the data processing device 50 of this embodiment. As shown in Figure 5, the data processing device 50 includes: a first acquisition module 52, a second acquisition module 54, a conversion module 56, and a calibration module 58; wherein:

[0104] The first acquisition module 52 is used to acquire the first reference coordinate points corresponding to multiple poses of the first camera coordinate system subscript plate, and the second reference coordinate points corresponding to multiple poses of the second camera coordinate system subscript plate.

[0105] The second acquisition module 54 is used to acquire the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera; wherein the object to be calibrated is within the field of view of the second camera and outside the field of view of the first camera, and the relative position between the object to be calibrated and the first camera is fixed.

[0106] The transformation module 56 is used to determine the transformation relationship between the first camera coordinate system and the second camera coordinate system based on the first reference coordinate point and the second reference coordinate point;

[0107] The calibration module 58 is used to obtain the position information of the object to be calibrated in the first camera coordinate system based on the transformation relationship and position information.

[0108] The aforementioned data processing device 50 acquires first reference coordinate points corresponding to multiple poses of the first camera coordinate system sub-tablet and second reference coordinate points corresponding to multiple poses of the second camera coordinate system sub-tablet; acquires position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera; wherein the object to be calibrated is within the field of view of the second camera and outside the field of view of the first camera, and the relative position between the object to be calibrated and the first camera is fixed; determines the transformation relationship between the first camera coordinate system and the second camera coordinate system based on the first reference coordinate points and the second reference coordinate points; and obtains the position information of the object to be calibrated in the first camera coordinate system based on the transformation relationship and the position information. By using multiple cameras, it accurately solves the position coordinates of objects outside the field of view, thereby realizing the calibration of the spatial position of objects outside the field of view and improving the accuracy of the calibration results.

[0109] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0110] This embodiment also provides a calibration system for spatial positioning of an object to be calibrated outside the camera's field of view. Figure 6 is a schematic diagram of the calibration system 60 of this embodiment. As shown in Figure 6, the calibration system 60 includes: a first camera 62, a second camera 64, a target plate 66, and a processor 68; wherein: the first camera 62 and the second camera 64 are both communicatively connected to the processor 68; the first camera 62 is used to acquire first images of the target plate 66 in multiple poses and transmit the first images to the processor 68; the second camera 64 is used to acquire second images of the target plate 66 in multiple poses and a third image of the object to be calibrated located outside the field of view of the first camera 62, and transmit the second and third images to the processor 68; the processor 68 is used to execute the data processing method of any of the above embodiments.

[0111] The aforementioned calibration system 60 acquires images of the object to be calibrated located outside the field of view of the first camera 62 by introducing a second camera 64 and a calibration plate 66, and uses a processor 68 to implement the aforementioned data processing method, thereby improving the accuracy of solving the position coordinates of the object outside the field of view and thus realizing the calibration of the spatial position of the object outside the field of view, and improving the accuracy of the calibration results.

[0112] Furthermore, in one embodiment, the calibration plate 66 is a double-sided calibration plate, which is disposed between the first camera 62 and the second camera 64; the first camera 62 is used to acquire a calibration image of the first side of the double-sided calibration plate to obtain a first image; the second camera 64 is used to acquire a calibration image of the second side of the double-sided calibration plate to obtain a second image.

[0113] In another embodiment, the calibration system 60 is used to simultaneously calibrate the spatial positions of multiple objects to be calibrated that are not within the camera's field of view.

[0114] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0115] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0116] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0117] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A data processing method for spatial position calibration of an object to be calibrated outside the camera's field of view, characterized in that, include: Obtain the first reference coordinate points corresponding to multiple poses of the target plate in the first camera coordinate system, and the second reference points corresponding to multiple poses of the target plate in the second camera coordinate system. Obtaining the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera specifically includes: acquiring a third image of the object to be calibrated and multiple different poses of the second camera, obtained by adjusting the pose of the second camera; extracting the reference point of the object to be calibrated from the third image; converting the pixel coordinates of the reference point into ray information of the object to be calibrated in the second camera coordinate system according to the camera intrinsic parameters of the second camera, thereby obtaining the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera; wherein, the object to be calibrated is within the field of view of the second camera and outside the field of view of the first camera, and the relative position between the object to be calibrated and the first camera is fixed; Based on the first reference coordinate point and the second reference coordinate point, determine the transformation relationship between the first camera coordinate system and the second camera coordinate system; based on the transformation relationship and the position information, obtain the position information of the object to be calibrated in the first camera coordinate system, specifically including: based on the ray information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera, and the transformation relationship, obtain multiple sets of ray information of the object to be calibrated in the first camera coordinate system; based on the multiple sets of ray information of the object to be calibrated in the first camera coordinate system, calculate the reference point coordinates of the object to be calibrated in the first camera coordinate system, and obtain the position information of the object to be calibrated in the first camera coordinate system based on the reference point coordinates.

2. The data processing method according to claim 1, characterized in that, The step of obtaining the first reference coordinate points corresponding to multiple poses of the target plate in the first camera coordinate system and the second reference coordinate points corresponding to multiple poses of the target plate in the second camera coordinate system includes: obtaining a first image corresponding to multiple poses of the target plate acquired by the first camera and a second image corresponding to multiple poses of the target plate acquired by the second camera; obtaining the first reference coordinate points based on the first image and the camera intrinsic parameters of the first camera; and obtaining the second reference coordinate points based on the second image and the camera intrinsic parameters of the second camera.

3. The data processing method according to claim 1, characterized in that, The step of determining the transformation relationship between the first camera coordinate system and the second camera coordinate system based on the first reference coordinate point and the second reference coordinate point includes: obtaining a third reference coordinate point of the target in the world coordinate system; obtaining a first transformation relationship between the first camera and the world coordinate system based on the third reference coordinate point and the first reference coordinate point; obtaining a second transformation relationship between the second camera and the world coordinate system based on the third reference coordinate point and the second reference coordinate point; and calculating the transformation relationship between the first camera coordinate system and the second camera coordinate system based on the first transformation relationship and the second transformation relationship.

4. The data processing method according to claim 1, characterized in that, The step of calculating the reference point coordinates of the object to be calibrated in the first camera coordinate system based on multiple sets of ray information of the object to be calibrated in the first camera coordinate system includes: based on multiple sets of ray information of the object to be calibrated in the first camera coordinate system, solving for the point with the minimum distance between each ray of the multiple sets of ray information in the first camera coordinate system, and obtaining the reference point coordinates of the object to be calibrated in the first camera coordinate system.

5. A data processing apparatus, characterized in that, The system is used for spatial position calibration of an object to be calibrated outside the camera's field of view, comprising: a first acquisition module, a second acquisition module, a conversion module, and a calibration module; wherein: the first acquisition module is used to acquire first reference coordinate points corresponding to multiple poses of a target plate in a first camera coordinate system, and second reference coordinate points corresponding to multiple poses of the target plate in a second camera coordinate system; the second acquisition module is used to acquire position information of the object to be calibrated in a second camera coordinate system corresponding to multiple different poses of the second camera, specifically including: acquiring a third image of the object to be calibrated and multiple different poses of the second camera, obtained by adjusting the pose of the second camera, and extracting a reference point of the object to be calibrated from the third image; converting the pixel coordinates of the reference point into ray information of the object to be calibrated in the second camera coordinate system according to the camera intrinsic parameters of the second camera, thereby obtaining the position information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera; In this system, the object to be calibrated is within the field of view of the second camera and outside the field of view of the first camera, and the relative position between the object to be calibrated and the first camera is fixed. The conversion module is used to determine the conversion relationship between the first camera coordinate system and the second camera coordinate system based on the first reference coordinate point and the second reference coordinate point. The calibration module is used to obtain the position information of the object to be calibrated in the first camera coordinate system based on the conversion relationship and the position information, specifically including: obtaining multiple sets of ray information of the object to be calibrated in the first camera coordinate system based on the ray information of the object to be calibrated in the second camera coordinate system corresponding to multiple different poses of the second camera and the conversion relationship; calculating the reference point coordinates of the object to be calibrated in the first camera coordinate system based on the multiple sets of ray information of the object to be calibrated in the first camera coordinate system; and obtaining the position information of the object to be calibrated in the first camera coordinate system based on the reference point coordinates.

6. A calibration system for calibrating the spatial position of an object to be calibrated outside the camera's field of view, characterized in that, include: The system comprises a first camera, a second camera, a target plate, and a processor; wherein: the first camera and the second camera are both communicatively connected to the processor; the first camera is used to acquire first images of the target plate in multiple poses and transmit the first images to the processor; the second camera is used to acquire second images of the target plate in multiple poses and a third image of the object to be calibrated located outside the field of view of the first camera, and transmit the second images and the third images to the processor; The processor is configured to execute the data processing method according to any one of claims 1 to 4.

7. The calibration system according to claim 6, characterized in that, The calibration plate is a double-sided calibration plate, which is positioned between the first camera and the second camera. The first camera is used to acquire a calibration image of the first side of the double-sided calibration plate to obtain a first image. The second camera is used to acquire a calibration image of the second side of the double-sided calibration plate to obtain a second image.

8. The calibration system according to claim 6, characterized in that, The calibration system is used to simultaneously calibrate the spatial position of multiple objects to be calibrated that are not within the camera's field of view.

Images