A method, device and computer device for adjusting calibration parameters of binocular device

By performing initial calibration and parallax mean adjustment of binocular equipment, the convergence distance consistency problem is solved, and the convergence distance parameter solidification in different usage scenarios is achieved, ensuring the accuracy of parallax information of binocular equipment at the preset distance.

CN118710735BActive Publication Date: 2024-11-12ZHEJIANG HEALNOC TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411204832.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-12
Estimated Expiration
2044-08-30

AI Technical Summary

Technical Problem

During the convergence distance parameter curing process of existing binocular equipment, due to the limitations of the lens or patch process, the optical path consistency and sensor rotation angle offset are inaccurate, which affects image correction and parallax comfort feedback, and cannot achieve convergence in different usage scenarios. Distance parameter curing.

Method used

By performing initial calibration of the binocular device, shooting with a preset calibration plate, calculating the parallax mean, adjusting the calibration parameters of the camera until the parallax mean is less than the threshold, ensuring consistency of the convergence distance.

Benefits of technology

Acquisition of accurate parallax information at the preset convergence distance is realized, ensuring the consistency of convergence distance of binocular equipment, solving the problem of poor convergence distance consistency in the prior art, and the convergence distance parameter solidification of different usage scenarios can be carried out.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118710735B_ABST
    Figure CN118710735B_ABST
Patent Text Reader

Abstract

The present application relates to a calibration parameter adjustment method, device and computer equipment for a binocular device. The method includes: performing initial calibration to obtain calibration parameters; obtaining a left image and a right image under the premise that the distance between the camera and a preset calibration plate is a preset convergence distance; correcting the left image to obtain a first corrected image; correcting the right image to obtain a second corrected image; calculating the parallax mean result between the corrected images; when the absolute value of the parallax mean result is greater than or equal to a preset parallax mean threshold, adjusting the principal point position of the calibration parameters of the binocular device to obtain a second left calibration parameter and a second right calibration parameter; repeating the step of calculating the parallax mean until the absolute value of the obtained parallax mean result is less than the preset parallax mean threshold. The use of this method can solve the problem in the prior art that the convergence distance parameters cannot be solidified due to the poor consistency of the convergence distance of the binocular device.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of binocular devices, and in particular to a method, device and computer equipment for adjusting calibration parameters of binocular devices. Background Art

[0002] With the development of science and technology, binocular devices such as binocular endoscopes are gradually being widely used. With the widespread application of binocular endoscopes, people's requirements for binocular endoscopes are getting higher and higher.

[0003] In some specific application scenarios, doctors have different requirements for observation distances. At different observation distances, different convergence distances need to be adjusted to ensure better 3D comfort. In order to improve the performance in this specific application scenario, the existing technology can be implemented by solidifying different convergence distance parameters of binocular endoscopes in different usage scenarios.

[0004] However, in the process of fixing the convergence distance parameters of the binocular endoscope, it is necessary to collect the user's feedback information on the binocular parallax comfort, and fix the convergence distance parameters based on the feedback information of the binocular parallax comfort. Therefore, it is particularly important to obtain effective feedback information on the binocular parallax comfort.

[0005] In the application process of binocular endoscopes, due to the limitations of lens or patch technology, the consistency of the optical path between different lenses and the consistency of the relative rotation angle or offset of the sensor cannot be well guaranteed. As a result, after the binocular endoscope image is corrected, the image cropping and magnification cannot be consistent. This causes the parallel device to calculate the translation distance inaccurately when performing the principal point position translation calculation, affecting the consistency of the convergence distance of the binocular camera. When the convergence distance consistency of the binocular device is poor, the adjustment of the parallax level cannot collect accurate parallax information, and it is impossible to obtain effective feedback information on the binocular parallax comfort, resulting in the inability to solidify the convergence distance parameters.

[0006] Currently, no effective solution has been proposed to solve the problem that the convergence distance parameters of binocular devices are not consistent enough and accurate disparity information cannot be collected even when the disparity level is adjusted, which makes it impossible to solidify the convergence distance parameters for different usage scenarios. Summary of the invention

[0007] Based on this, it is necessary to provide a calibration parameter adjustment method, device and computer equipment for a binocular device to address the above technical problems.

[0008] In a first aspect, the present application provides a method for adjusting calibration parameters of a binocular device. The binocular device includes two cameras, a left camera and a right camera; the method includes the following steps:

[0009] Performing an initial calibration on the binocular device to obtain a first left calibration parameter of the left camera and a first right calibration parameter of the right camera;

[0010] Under the premise that two optical paths of the camera of the binocular device are perpendicular to the preset calibration plate, and the distance between the camera of the binocular device and the preset calibration plate is a preset convergence distance, the preset calibration plate is photographed by the binocular device to obtain a left image photographed by the left camera and a right image photographed by the right camera;

[0011] Correcting the left image using the first left calibration parameter to obtain a first corrected image; correcting the right image using the first right calibration parameter to obtain a second corrected image;

[0012] Calculating a disparity mean between the first corrected image and the second corrected image to obtain a disparity mean result;

[0013] When the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, based on the disparity mean result, adjusting the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device;

[0014] The second left calibration parameter is used as a new first left calibration parameter, and the second right calibration parameter is used as a new first right calibration parameter, and the step of calculating the disparity mean is repeatedly performed until the absolute value of the obtained disparity mean result is less than the preset disparity mean threshold.

[0015] In one embodiment, calculating the mean disparity between the first corrected image and the second corrected image to obtain the mean disparity result comprises the following steps:

[0016] Based on each corner point in the first corrected image and each corner point in the second corrected image, a disparity mean between the first corrected image and the second corrected image is calculated to obtain the disparity mean result.

[0017] In one embodiment, the calculating the disparity mean between the first corrected image and the second corrected image based on each corner point in the first corrected image and each corner point in the second corrected image to obtain the disparity mean result comprises the following steps:

[0018] Determining corner points in the first corrected image and corner points in the second corrected image;

[0019] determining a pair of corner points between the first corrected image and the second corrected image based on the corner points in the first corrected image and the corner points in the second corrected image;

[0020] Calculating the disparity of each of the corner point pairs between the first corrected image and the second corrected image based on the corner point pairs between the first corrected image and the second corrected image;

[0021] Determining a mean value of the disparity of all the corner point pairs between the first corrected image and the second corrected image based on the disparity of each of the corner point pairs between the first corrected image and the second corrected image;

[0022] The disparity mean of all the corner point pairs between the first corrected image and the second corrected image is determined as the disparity mean between the first corrected image and the second corrected image, to obtain the disparity mean result.

[0023] In one of the embodiments, determining the mean disparity of all corner point pairs between the first corrected image and the second corrected image based on the disparity of each corner point pair between the first corrected image and the second corrected image comprises the following steps:

[0024] Averaging the disparities of all the corner point pairs between the first corrected image and the second corrected image to obtain a disparity mean value of all the corner point pairs between the first corrected image and the second corrected image;

[0025] Or, based on the position of each of the corner points, determine the weight corresponding to the disparity of each of the corner point pairs; based on the weight corresponding to the disparity of each of the corner point pairs, perform weighted summation on the disparity of each of the corner point pairs to obtain the mean disparity of all the corner point pairs between the first corrected image and the second corrected image.

[0026] In one of the embodiments, when the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, based on the disparity mean result, adjusting the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device comprises the following steps:

[0027] When the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, based on the positive and negative signs of the disparity mean result, the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device are adjusted to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device.

[0028] In one of the embodiments, when the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, based on the positive and negative signs of the disparity mean result, adjusting the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device comprises the following steps:

[0029] When the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, if the disparity mean result is a positive number, the first principal point position of the first left calibration parameter is moved rightward by a preset adjustment distance to obtain a new first principal point position; the preset adjustment distance is a distance value that is proportional to the absolute value of the disparity mean result;

[0030] Based on the new first principal point position, obtaining the second left calibration parameter of the left camera of the binocular device;

[0031] Moving the second principal point position of the first right calibration parameter to the left by the preset adjustment distance to obtain a new second principal point position;

[0032] Based on the new second principal point position, the second right calibration parameter of the right camera of the binocular device is obtained.

[0033] In one of the embodiments, when the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, based on the positive and negative signs of the disparity mean result, the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device are adjusted to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device, further comprising the following steps:

[0034] When the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, if the disparity mean result is a negative number, the first principal point position of the first left calibration parameter is moved to the left by the preset adjustment distance to obtain a third principal point position;

[0035] Based on the third principal point position, obtaining the second left calibration parameter of the left camera of the binocular device;

[0036] The second principal point position in the first right calibration parameter is moved rightward by the preset adjustment distance to obtain a fourth principal point position;

[0037] Based on the fourth principal point position, the second right calibration parameter of the right camera of the binocular device is obtained.

[0038] In one embodiment, the method further comprises the following steps:

[0039] The first left calibration parameter and the first right calibration parameter in the last step of calculating the parallax mean are determined as the convergence distance solidification parameters corresponding to the preset convergence distance.

[0040] In a second aspect, the present application further provides a calibration parameter adjustment device for a binocular device. The binocular device includes two cameras, a left camera and a right camera; the device includes:

[0041] A parameter acquisition module, used to perform initial calibration on the binocular device to obtain a first left calibration parameter of the left camera and a first right calibration parameter of the right camera;

[0042] An image acquisition module, configured to use the binocular device to shoot the preset calibration plate to obtain a left image captured by the left camera and a right image captured by the right camera, under the premise that both optical paths of the camera of the binocular device are perpendicular to the preset calibration plate and the distance between the camera of the binocular device and the preset calibration plate is a preset convergence distance;

[0043] A calibration module, configured to calibrate the left image using the first left calibration parameter to obtain a first calibrated image; and calibrate the right image using the first right calibration parameter to obtain a second calibrated image;

[0044] A calculation module, used for calculating the mean disparity between the first corrected image and the second corrected image to obtain a mean disparity result;

[0045] an adjustment module, configured to adjust the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device based on the disparity mean result when the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, so as to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device;

[0046] And an updating module, used for taking the second left calibration parameter as a new first left calibration parameter and the second right calibration parameter as a new first right calibration parameter, and repeatedly executing the step of calculating the disparity mean until the absolute value of the obtained disparity mean result is less than the preset disparity mean threshold.

[0047] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the calibration parameter adjustment method of the binocular device described in the first aspect is implemented.

[0048] The calibration parameter adjustment method, device and computer device of the binocular device perform initial calibration on the binocular device to obtain the parameters after initial calibration, and adjust the principal point position of the calibration parameters of the binocular device according to the parameters after initial calibration until the absolute value of the parallax mean result of the image captured under the premise that the distance between the camera of the binocular device and the preset calibration plate is the preset convergence distance satisfies the preset condition. When the absolute value of the parallax mean is less than the preset parallax threshold, it is ensured that the binocular device obtains a very small parallax when capturing a target at the preset convergence distance, and can ensure the consistency of the convergence distance of the binocular device at the preset convergence distance, thereby solving the problem in the prior art that the convergence distance consistency of the binocular device is poor and accurate parallax information cannot be collected even when the parallax level is adjusted, resulting in the inability to solidify the convergence distance parameters for different usage scenarios.

[0049] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0051] Figure 1 A hardware structure block diagram of a terminal of a calibration parameter adjustment method for a binocular device provided in an embodiment of the present application;

[0052] Figure 2 A flowchart of a calibration parameter adjustment method for a binocular device provided in one embodiment of the present application;

[0053] Figure 3 A flowchart of a method for adjusting calibration parameters of a binocular device provided in a preferred embodiment of the present application;

[0054] Figure 4 This is a structural block diagram of a calibration parameter adjustment device for a binocular device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0056] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0057] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of a terminal of the calibration parameter adjustment method of a binocular device of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0058] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the calibration parameter adjustment method of the binocular device in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0059] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0060] In this embodiment, a calibration parameter adjustment method for a binocular device is provided. The binocular device includes two cameras, namely a left camera and a right camera. Figure 2 is a flow chart of the calibration parameter adjustment method of the binocular device of this embodiment. Figure 2 As shown, the process includes the following steps:

[0061] Step S210, performing an initial calibration on the binocular device to obtain a first left calibration parameter of the left camera and a first right calibration parameter of the right camera.

[0062] In this step, the binocular device may be one of a binocular endoscope, a binocular microscope, a binocular camera, a binocular MR (Mixed Reality) device, and the like. Specifically, in some application scenarios, because the observation distance is inconsistent, in order to ensure better 3D (three-dimensional) comfort requirements, the convergence distance information solidification parameter setting is performed according to different observation distances to achieve the switching of the convergence distance parameters and obtain a device that can meet the viewing comfort requirements in different scenarios. The initial calibration of the binocular device may be to use a known calibration plate to perform internal parameter calibration and external parameter calibration on the binocular device. The calibration plate may be a calibration plate for realizing the calibration of the binocular device, and may specifically be a calibration plate with a clear geometric structure, for example, a checkerboard calibration plate, a circular array calibration plate, a random dot matrix calibration plate, a stripe calibration plate, an Archimedean spiral calibration plate, a color or texture calibration plate, or other custom calibration plates. The internal parameter calibration may be to determine the internal parameter matrix and distortion coefficient of each camera of the binocular device. The above-mentioned intrinsic parameter matrix may be a matrix containing information such as the focal length of the camera, the coordinates of the principal point, etc. The above-mentioned extrinsic parameter calibration may be to determine the relative position and rotation relationship between the two cameras of the binocular device or to determine the position and posture of each camera relative to the world coordinate system.

[0063] Specifically, the binocular device is calibrated with internal and external parameters using a known calibration plate. After the calibration plate is fixed, the calibration plate can be photographed from multiple angles and distances using the two cameras of the binocular device to obtain the captured images of the two cameras, and then the binocular device is calibrated based on the captured images of the two cameras to obtain the first left calibration parameter of the left camera and the first right calibration parameter of the right camera. The first left calibration parameter may include the external parameter of the left camera and the internal parameter of the left camera. The internal parameter of the left camera is the internal parameter matrix and distortion coefficient of the left camera. The external parameter of the left camera is the rotation matrix of the left camera relative to the right camera and the translation vector of the left camera relative to the right camera. The first right calibration parameter may include the external parameter of the right camera and the internal parameter of the right camera. The internal parameter of the right camera is the internal parameter matrix and distortion coefficient of the right camera. The external parameters of the right camera are the rotation matrix of the right camera relative to the left camera and the translation vector of the right camera relative to the left camera. In this step, the first left calibration parameter of the left camera and the first right calibration parameter of the right camera are obtained by initially calibrating the binocular device. By obtaining the initial calibration parameters of the two cameras, it is convenient to use the initial calibration parameters of the two cameras to calibrate the image taken at a preset convergence distance to obtain the calibration result, so as to adjust the principal point position of the calibration parameters of the binocular device through the calibration result, so as to adjust the calibration parameters of the binocular device and improve the convergence distance consistency of the binocular device.

[0064] Step S220, under the premise that the two optical paths of the camera of the binocular device are perpendicular to the preset calibration plate, and the distance between the camera of the binocular device and the preset calibration plate is a preset convergence distance, the preset calibration plate is photographed by the binocular device to obtain a left image photographed by the left camera and a right image photographed by the right camera.

[0065] The preset calibration plate can be a card with a preset size and type and corner points for realizing binocular device calibration. In order to ensure that after the calibration parameters of the binocular device are adjusted, when the distance between the camera of the binocular device and the target is the preset convergence distance, the captured image has good convergence distance consistency, the binocular device needs to shoot the preset calibration plate under the premise that the distance between the camera and the preset calibration plate is the preset convergence distance, and then use the captured image to feedback and adjust the calibration parameters of the binocular device.

[0066] In order to ensure the accuracy of the calibration parameter adjustment of the binocular device, the two optical paths of the binocular device's camera can be used to shoot under the premise that both of the two optical paths of the binocular device's camera are perpendicular to the preset calibration plate. By shooting under the premise that both of the two optical paths of the binocular device's camera are perpendicular to the preset calibration plate, it can be ensured that there is no parallax introduced between the preset calibration plate images captured by the binocular device due to the inconsistent distances between the various lenses of the binocular device and the preset calibration plate. In this way, the accuracy of the calibration parameter adjustment of the binocular device can be improved.

[0067] In this step, the two optical paths of the camera of the binocular device are perpendicular to the preset calibration plate, and the distance between the camera of the binocular device and the preset calibration plate is the preset convergence distance, and the binocular device is used to shoot the preset calibration plate, so that the calibration parameters of the binocular device are adjusted more accurately. In addition, the calibration parameters of the binocular device are adjusted by using the image captured under the premise that the distance between the camera of the binocular device and the preset calibration plate is the preset convergence distance, which can ensure that the adjusted parameters are suitable for the preset convergence distance, so that the binocular device after the final adjustment, when the distance between the camera and the target is the preset convergence distance, the image captured has good convergence distance consistency.

[0068] Step S230, correcting the left image using the first left calibration parameter to obtain a first corrected image; and correcting the right image using the first right calibration parameter to obtain a second corrected image.

[0069] In order to correct the left image and the right image, it is necessary to correct the left image using the first left calibration parameter, and to correct the right image using the first right calibration parameter. Specifically, the correction of the left image using the first left calibration parameter may be to correct the left image using the intrinsic reference matrix and distortion coefficient of the left camera in the first left calibration parameter. Among them, the specific method of correcting the left image using the intrinsic reference matrix, distortion coefficient and external reference of the left camera in the first left calibration parameter is not specifically limited in this embodiment. As long as the left image can be corrected according to the intrinsic reference matrix and distortion coefficient of the left camera in the first left calibration parameter. For example, the cv2.undistort() function of OpenCV may be used to correct the left image according to the intrinsic reference matrix, distortion coefficient and external reference of the left camera in the first left calibration parameter. The correction of the right image using the first right calibration parameter may be to correct the right image using the intrinsic reference matrix and distortion coefficient of the right camera in the first right calibration parameter. Among them, the specific method of correcting the right image by using the intrinsic matrix, distortion coefficient and external parameter of the right camera in the first right calibration parameter is not specifically limited in this embodiment. As long as the right image can be corrected according to the intrinsic matrix and distortion coefficient of the right camera in the first right calibration parameter. For example, the right image can be corrected using the cv2.undistort() function of OpenCV according to the intrinsic matrix, distortion coefficient and external parameter of the right camera in the first right calibration parameter. It should be noted that there is only horizontal parallax but no vertical parallax between the first corrected image and the second corrected image obtained after correction. Further, in the process of correcting the left image using the first left calibration parameter and correcting the right image using the first right calibration parameter, in order to simplify the subsequent calculation process, the left image and the right image can be mapped in the same coordinate system, so that the first corrected image and the second corrected image are in the same coordinate system. This step facilitates the subsequent adjustment of the calibration parameters of the binocular device according to the parallax between the first corrected image and the second corrected image by acquiring the first corrected image and the second corrected image.

[0070] Step S240, calculating the mean disparity between the first corrected image and the second corrected image to obtain a mean disparity result.

[0071] The above-mentioned calculation of the mean disparity between the first corrected image and the second corrected image to obtain the mean disparity result can be: using a preset feature point recognition method to respectively determine the feature points of the first corrected image and the feature points of the second corrected image; according to the feature points of the first corrected image and the feature points of the second corrected image, using a preset matching algorithm, matching the feature points of the first corrected image with the feature points of the second corrected image to obtain each feature point pair in the first corrected image and the second corrected image; calculating the disparity between two feature points in each feature point pair to obtain the disparity of each feature point pair; averaging the disparity of each feature point pair to obtain the average disparity of all feature point pairs; and taking the average disparity of all feature point pairs as the mean disparity result.

[0072] Step S250, when the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, based on the disparity mean result, adjust the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device.

[0073] Among them, the above-mentioned preset disparity mean threshold can be specifically set according to the specific situation, as long as it can be guaranteed that, when the absolute value of the disparity mean result of the binocular device is less than the calibration parameter corresponding to the preset disparity mean threshold, the convergence distance consistency of the image at the preset convergence distance captured by the binocular device meets the requirements.

[0074] Step S260: using the second left calibration parameter as a new first left calibration parameter and the second right calibration parameter as a new first right calibration parameter, and repeatedly executing the step of calculating the disparity mean until the absolute value of the obtained disparity mean result is less than a preset disparity mean threshold.

[0075] In this step, the above-mentioned step of calculating the mean disparity may be step S230 to step S240. The above-mentioned taking the second left calibration parameter as the new first left calibration parameter and the second right calibration parameter as the new first right calibration parameter may be taking the second left calibration parameter as the first left calibration parameter of the next step of calculating the mean disparity and taking the second right calibration parameter as the first right calibration parameter of the next step of calculating the mean disparity.

[0076] In the above steps S210 to S260, the binocular device is initially calibrated to obtain the parameters after initial calibration, and the principal point position of the calibration parameters of the binocular device is adjusted according to the parameters after initial calibration, until the absolute value of the parallax mean result of the image captured under the premise that the distance between the camera of the binocular device and the preset calibration plate is the preset convergence distance meets the preset condition. When the absolute value of the parallax mean is less than the preset parallax threshold, it is ensured that when the binocular device shoots a target at the preset convergence distance after calibration, the parallax obtained is very small, and the convergence distance consistency of the binocular device at the preset convergence distance can be ensured, which solves the problem in the prior art that the convergence distance parameters cannot be solidified due to the poor convergence distance consistency of the binocular device.

[0077] In one embodiment, based on step S240, calculating the mean disparity between the first corrected image and the second corrected image to obtain the mean disparity result includes the following steps:

[0078] Step S242, based on each corner point in the first corrected image and each corner point in the second corrected image, calculate the mean disparity between the first corrected image and the second corrected image to obtain a mean disparity result.

[0079] Further, in one embodiment, step S242, based on each corner point in the first corrected image and each corner point in the second corrected image, calculates the disparity mean between the first corrected image and the second corrected image to obtain the disparity mean result, includes the following steps:

[0080] Step S2421, determining the corner points in the first corrected image and the corner points in the second corrected image.

[0081] The above-mentioned determination of the corner points in the first corrected image and the corner points in the second corrected image may be performed by using a preset corner point recognition method to determine the corner points in the first corrected image and the corner points in the second corrected image. The above-mentioned preset corner point recognition method may be one or more of the methods such as SIFT (Scale-Invariant Feature Transform, a feature detection method based on scale space), SURF (Speeded Up Robust Features, a feature detection method using integral images), and Harris corner point detection algorithm. The specific corner point recognition method is not specifically limited in this embodiment, as long as the corner points in the first corrected image and the corner points in the second corrected image can be determined by the corner point recognition method.

[0082] Step S2422: determining corner point pairs between the first corrected image and the second corrected image based on the corner points in the first corrected image and the corner points in the second corrected image.

[0083] In this step, the above-mentioned determination of the corner point pairs between the first corrected image and the second corrected image based on the corner points in the first corrected image and the corner points in the second corrected image may be performed according to a preset corner point matching algorithm to match the corner points in the first corrected image and the corner points in the second corrected image to obtain the corner point pairs between the first corrected image and the second corrected image. The above-mentioned preset corner point matching algorithm may be one or more of a BFM (Brute-Force Matching) matching algorithm, a FLANN (Fast Library for Approximate Nearest Neighbors) matching algorithm, a RANSAC (Random Sample Consensus) matching algorithm, and the like. It should be noted that the specific preset corner point matching algorithm is not specifically limited in this embodiment, as long as the corner points in the first corrected image and the corner points in the second corrected image can be matched by the preset corner point matching algorithm to obtain the corner point pairs between the first corrected image and the second corrected image.

[0084] Step S2423: Calculate the disparity of each corner point pair between the first corrected image and the second corrected image based on the corner point pairs between the first corrected image and the second corrected image.

[0085] The above-mentioned calculation of the disparity of each corner point pair between the first corrected image and the second corrected image based on the corner point pair between the first corrected image and the second corrected image may be based on the coordinates of each corner point in the corner point pair between the first corrected image and the second corrected image, to determine each corner point pair between the first corrected image and the second corrected image, and then, based on each corner point pair between the first corrected image and the second corrected image, to calculate the disparity of each corner point pair between the first corrected image and the second corrected image.

[0086] Step S2424: determining the average disparity of all corner point pairs between the first corrected image and the second corrected image based on the disparity of each corner point pair between the first corrected image and the second corrected image.

[0087] Step S2425: Determine the disparity mean of all corner point pairs between the first corrected image and the second corrected image as the disparity mean between the first corrected image and the second corrected image, and obtain a disparity mean result.

[0088] In the above steps S2421 to S2425, the corner point pairs between the first corrected image and the second corrected image are determined by determining the corner points in the first corrected image and the corner points in the second corrected image. Then, based on the corner point pairs between the first corrected image and the second corrected image, the disparity mean of each corner point pair between the first corrected image and the second corrected image is calculated. Then, according to the disparity mean of each corner point pair between the first corrected image and the second corrected image, the disparity mean result is obtained. By calculating the disparity mean result, it is convenient to judge whether the calibration parameters of the binocular device meet the requirements of convergence distance consistency according to the disparity mean result.

[0089] Specifically, in one embodiment, the above step S2423, based on the disparity of each corner point pair between the first corrected image and the second corrected image, determines the disparity mean of all corner point pairs between the first corrected image and the second corrected image, including the following steps:

[0090] Step S24232, averaging the disparities of all corner point pairs between the first corrected image and the second corrected image to obtain the disparity mean of all corner point pairs between the first corrected image and the second corrected image.

[0091] The following is an example of averaging the disparities of all corner point pairs between the first corrected image and the second corrected image to obtain the disparity mean of all corner point pairs between the first corrected image and the second corrected image:

[0092] For example, if there are n pairs of corner points on the first corrected image and the second corrected image, the calculation process of the disparity mean result avgx is:

[0093]

[0094] Among them, xll is the coordinate of the point in the first corrected image of the first diagonal point pair, xrl is the coordinate of the point in the second corrected image of the first diagonal point pair, xrl-xll is the disparity value of the first diagonal point pair, xl2 is the coordinate of the point in the first corrected image of the second diagonal point pair, xr2 is the coordinate of the point in the second corrected image of the second diagonal point pair, xr2-xl2 is the disparity value of the second diagonal point pair, xln is the coordinate of the point in the first corrected image of the nth diagonal point pair, xrn is the coordinate of the point in the second corrected image of the nth diagonal point pair, and xrn-xln is the disparity value of the nth diagonal point pair.

[0095] Or, step S24234, based on the position of each corner point, determine the weight corresponding to the disparity of each corner point pair; based on the weight corresponding to the disparity pair of each corner point, perform weighted summation on the disparity of each corner point pair to obtain the mean disparity of all corner point pairs between the first corrected image and the second corrected image.

[0096] In some cases, such as when the convergence distance is close, the test plate will be slightly larger, and there will be inconsistent parallax of the corner points in the peripheral area and the middle area of ​​the left and right images captured by the binocular device. In this case, different weights can be assigned to the corner point pairs in the peripheral area and the middle area of ​​the left and right images captured by the binocular device to obtain a more accurate disparity mean result.

[0097] The weight corresponding to the disparity of each corner point pair is determined based on the position of each corner point. The weight function is determined first, and then the weight corresponding to the disparity pair of each corner point is determined according to the position of each corner point and the weight function. The weight function can be a two-dimensional Gaussian function. For example, the weight function can be a Gauss weight function. The weight corresponding to the disparity of each corner point pair is weighted and summed to obtain the mean disparity of all corner point pairs between the first corrected image and the second corrected image. The disparity of each corner point pair is multiplied by its corresponding weight to obtain the disparity weight result of each corner point pair, and the disparity weight result of each corner point pair is weighted and summed to obtain the mean disparity of all corner point pairs between the first corrected image and the second corrected image. The sum of the weights corresponding to the disparities of all intersection points is 1.

[0098] The process of calculating the disparity mean avgx of all corner point pairs between the first corrected image and the second corrected image using the above calculation method is as follows:

[0099]

[0100] Among them, ratio1 is the weight corresponding to the disparity of the first pair of diagonal points, ratio2 is the weight corresponding to the disparity of the second pair of diagonal points, and ration is the weight corresponding to the disparity of the nth pair of diagonal points.

[0101] In the above steps S24232 to S24234, the disparity mean of all corner point pairs between the first corrected image and the second corrected image is calculated through the disparity of each corner point pair, and then the disparity mean result is obtained. By calculating the disparity mean result, it is convenient to judge whether the calibration parameters of the binocular device meet the requirements of convergence distance consistency based on the disparity mean result.

[0102] In addition, in one embodiment, based on step S250, when the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, based on the disparity mean result, adjusting the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device includes the following steps:

[0103] Step S252, when the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, based on the positive and negative signs of the disparity mean result, adjust the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device.

[0104] In the process of calculating the disparity mean result, the disparity of the current corner point pair can be calculated by calculating the coordinates of the points in the second corrected image in the same corner point pair and subtracting the difference of the coordinates of the points in the first corrected image. Therefore, the positive and negative signs of the disparity mean result can be used to characterize the positive and negative signs of the disparity at the convergence distance. Specifically, when the disparity mean result is a positive number, the disparity at the convergence distance is a positive disparity. When the disparity mean result is a negative number, the disparity at the convergence distance is a negative disparity. Furthermore, according to the positive and negative signs of the disparity at the convergence distance, the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device are adjusted accordingly to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device, so that the disparity at the convergence distance gradually approaches 0.

[0105] In one embodiment, in the above step S252, when the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, based on the positive and negative signs of the disparity mean result, adjusting the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device includes the following steps:

[0106] Step S2521, when the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, if the disparity mean result is a positive number, the first principal point position of the first left calibration parameter is moved rightward by a preset adjustment distance to obtain a new first principal point position; the preset adjustment distance is a distance value that is proportional to the absolute value of the disparity mean result;

[0107] Step S2522, based on the new first principal point position, obtain a second left calibration parameter of the left camera of the binocular device;

[0108] Step S2523, moving the second principal point position of the first right calibration parameter to the left by a preset adjustment distance to obtain a new second principal point position;

[0109] Step S2524: Based on the new second principal point position, obtain the second right calibration parameters of the right camera of the binocular device.

[0110] When a positive parallax is obtained at the convergence distance, it means that the point in the left image is further to the left in the horizontal direction relative to the corresponding point in the right image. In this case, in order to restore the parallax to zero, it is necessary to adjust the principal point position of the calibration parameters of the two cameras of the binocular device. The principal point position of the left camera can be moved to the right by a certain distance, and the principal point position of the right camera can be moved to the left by the same distance. Specifically, the first principal point position in the first left calibration parameter can be moved to the right by a preset adjustment distance to obtain a new first principal point position, and the second left calibration parameter can be obtained based on the new first principal point position. It can also be that the second principal point position in the first right calibration parameter is moved to the left by a preset adjustment distance to obtain a new second principal point position, and the second right calibration parameter is obtained based on the new second principal point position. The above-mentioned second left coordinate parameter is the left coordinate parameter after the first left coordinate parameter is updated, and the above-mentioned second right coordinate parameter is the right coordinate parameter after the first right coordinate parameter is updated.

[0111] The calculation process of the preset adjustment distance c is as follows:

[0112]

[0113] Among them, ratio is an empirical value, and ratio is less than 1.

[0114] It should be noted that the above preset adjustment distance may be a distance length in pixels. For example, if the main point position of the left camera is moved to the right by a preset adjustment distance, and the preset adjustment distance is a distance length of 1 pixel, the main point position of the left camera may be moved to the right by 1 pixel.

[0115] The second left calibration parameter of the left camera of the binocular device can be obtained by determining the internal reference matrix in the second left calibration parameter according to the parameter value of the first principal point position in the second left calibration parameter of the left camera of the binocular device, and then determining the second left calibration parameter of the left camera of the binocular device. The second right calibration parameter of the right camera of the binocular device can be obtained by determining the internal reference matrix in the second right calibration parameter according to the parameter value of the second principal point position in the second right calibration parameter of the right camera of the binocular device, and then determining the second right calibration parameter of the right camera of the binocular device.

[0116] In one embodiment, in the above step S252, when the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, based on the positive and negative signs of the disparity mean result, the first principal point position of the first left calibration parameter of the binocular device and the second principal point position of the first right calibration parameter are adjusted to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device, and the following steps are also included:

[0117] Step S2525, when the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, if the disparity mean result is a negative number, the principal point position in the first left calibration parameter is moved to the left by a preset adjustment distance to obtain a third principal point position;

[0118] Step S2526, based on the third principal point position, obtaining a second left calibration parameter of the left camera of the binocular device;

[0119] Step S2527, moving the principal point position in the first right calibration parameter to the right by a preset adjustment distance to obtain a fourth principal point position;

[0120] Step S2528: Based on the fourth principal point position, obtain the second right calibration parameter of the right camera of the binocular device.

[0121] When a negative parallax is obtained at the convergence distance, it means that the point in the left image is further to the right in the horizontal direction relative to the corresponding point in the right image. In this case, in order to restore the parallax to zero, it is necessary to adjust the principal point position of the calibration parameters of the two cameras of the binocular device. The principal point position of the right camera can be moved to the right by a preset adjustment distance, and the principal point position of the left camera can be moved to the left by the same distance. Specifically, the principal point position in the first left calibration parameter can be moved to the left by a preset adjustment distance to obtain a third principal point position, and then based on the third principal point position, the second left calibration parameter is obtained, and the principal point position in the first right calibration parameter is moved to the right by a preset adjustment distance to obtain a fourth principal point position, and then based on the fourth principal point position, the second right calibration parameter is obtained.

[0122] The second left calibration parameter of the left camera of the binocular device can be obtained by determining the internal reference matrix in the second left calibration parameter according to the parameter value of the third principal point position in the second left calibration parameter of the left camera of the binocular device, and then determining the second left calibration parameter of the left camera of the binocular device. The second right calibration parameter of the right camera of the binocular device can be obtained by determining the internal reference matrix in the second right calibration parameter according to the parameter value of the fourth principal point position in the second right calibration parameter of the right camera of the binocular device, and then determining the second right calibration parameter of the right camera of the binocular device.

[0123] In the above steps S2525 to S2528, when the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, the principal point position of the calibration parameters of the binocular device is adjusted based on the positive and negative signs of the disparity mean result, so as to obtain the second left calibration parameters of the left camera and the second right calibration parameters of the right camera of the binocular device, so that the disparity at the convergence distance gradually approaches 0.

[0124] In addition, in one embodiment, the calibration parameter adjustment method of the binocular device further includes the following steps:

[0125] Step S270: determining the first left calibration parameter and the first right calibration parameter in the last step of calculating the parallax mean as convergence distance solidified parameters corresponding to the preset convergence distance.

[0126] Because the absolute value of the disparity mean result obtained in the last step of calculating the disparity mean is less than the preset disparity mean threshold, the disparity value at this time meets the consistency requirement of the convergence distance. Therefore, the first left calibration parameter and the first right calibration parameter in the last step of calculating the disparity mean can be determined as the convergence distance solidified parameters corresponding to the preset convergence distance.

[0127] Preferably, different convergence distances can be set according to different needs, different parallax adjustment levels of the binocular device can be set according to different convergence distances, and corresponding convergence distance fixed parameters can be matched for each convergence distance to ensure the consistency of the convergence distance of the binocular device at the convergence distances corresponding to each level.

[0128] The present embodiment is described and illustrated below through preferred embodiments.

[0129] Figure 3 1 is a flow chart of a calibration parameter adjustment method for a binocular device provided in a preferred embodiment of the present application. The binocular device includes two cameras, a left camera and a right camera; Figure 3 As shown, the calibration parameter adjustment method of the binocular device includes the following steps:

[0130] Step S310, performing initial calibration on the binocular device to obtain a first left calibration parameter of the left camera and a first right calibration parameter of the right camera;

[0131] Step S320, under the premise that the two optical paths of the camera of the binocular device are perpendicular to the preset calibration plate, and the distance between the camera of the binocular device and the preset calibration plate is a preset convergence distance, the preset calibration plate is photographed by the binocular device to obtain a left image photographed by the left camera and a right image photographed by the right camera;

[0132] Step S330, correcting the left image using the first left calibration parameter to obtain a first corrected image; and correcting the right image using the first right calibration parameter to obtain a second corrected image;

[0133] Step S340, calculating the disparity mean between each corner point in the first corrected image and each corner point in the second corrected image to obtain a disparity mean result;

[0134] Step S350, when the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, based on the positive and negative signs of the disparity mean result, adjusting the principal point position of the first left calibration parameter and the principal point position of the first right calibration parameter of the binocular device to obtain a second left calibration parameter of the left camera and a second right calibration parameter of the right camera of the binocular device;

[0135] Step S360, using the second left calibration parameter as a new first left calibration parameter, and the second right calibration parameter as a new first right calibration parameter, and repeatedly executing the step of calculating the disparity mean until the absolute value of the obtained disparity mean result is less than a preset disparity mean threshold;

[0136] Step S370: Determine the first left calibration parameter and the first right calibration parameter in the last step of calculating the parallax mean as convergence distance solidified parameters corresponding to the preset convergence distance.

[0137] In the above steps S310 to S370, the binocular device is initially calibrated to obtain the parameters after initial calibration, and the principal point position of the calibration parameters of the binocular device is adjusted according to the parameters after initial calibration, until the absolute value of the parallax mean result of the image captured under the premise that the distance between the camera of the binocular device and the preset calibration plate is the preset convergence distance meets the preset condition. When the absolute value of the parallax mean is less than the preset parallax threshold, it is ensured that the binocular device obtains a very small parallax when shooting a target at the preset convergence distance, and the convergence distance consistency of the binocular device at the preset convergence distance can be ensured, which solves the problem in the prior art that the convergence distance parameters cannot be solidified due to the poor convergence distance consistency of the binocular device.

[0138] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0139] Based on the same inventive concept, a calibration parameter adjustment device for a binocular device is also provided in this embodiment, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0140] In one embodiment, Figure 4 is a structural block diagram of a calibration parameter adjustment device for a binocular device provided in an embodiment of the present application, wherein the binocular device includes two cameras, namely a left camera and a right camera; Figure 4 As shown, the calibration parameter adjustment device of the binocular device includes:

[0141] The parameter acquisition module 41 is used to perform initial calibration on the binocular device to obtain a first left calibration parameter of the left camera and a first right calibration parameter of the right camera;

[0142] The image acquisition module 42 is used to use the binocular device to shoot the preset calibration plate under the premise that the two optical paths of the camera of the binocular device are perpendicular to the preset calibration plate and the distance between the camera of the binocular device and the preset calibration plate is a preset convergence distance, so as to obtain a left image captured by the left camera and a right image captured by the right camera;

[0143] The calibration module 43 is used to calibrate the left image using the first left calibration parameter to obtain a first calibrated image; and to calibrate the right image using the first right calibration parameter to obtain a second calibrated image;

[0144] A calculation module 44, configured to calculate a disparity mean between the first corrected image and the second corrected image to obtain a disparity mean result;

[0145] An adjustment module 45 is used to adjust the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device based on the disparity mean result when the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, so as to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device;

[0146] And an updating module 46 is used to use the second left calibration parameter as a new first left calibration parameter and the second right calibration parameter as a new first right calibration parameter, and repeatedly perform the step of calculating the disparity mean until the absolute value of the obtained disparity mean result is less than a preset disparity mean threshold.

[0147] The calibration parameter adjustment device of the binocular device performs an initial calibration on the binocular device to obtain the parameters after the initial calibration, and adjusts the principal point position of the calibration parameters of the binocular device according to the parameters after the initial calibration, until the absolute value of the parallax mean result of the image captured under the premise that the distance between the camera of the binocular device and the preset calibration plate is the preset convergence distance meets the preset condition. When the absolute value of the parallax mean is less than the preset parallax threshold, it is ensured that the binocular device obtains a very small parallax when capturing a target at the preset convergence distance, and can ensure the consistency of the convergence distance of the binocular device at the preset convergence distance, thereby solving the problem in the prior art that the convergence distance parameters cannot be solidified due to the poor consistency of the convergence distance of the binocular device.

[0148] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, a calibration parameter adjustment method for a binocular device in any one of the above embodiments is implemented.

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0151] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A calibration parameter adjustment method for a binocular device, It is characterized in that The binocular device includes two cameras, namely a left camera and a right camera; the method includes: Performing an initial calibration on the binocular device to obtain a first left calibration parameter of the left camera and a first right calibration parameter of the right camera; Under the premise that two optical paths of the camera of the binocular device are perpendicular to the preset calibration plate, and the distance between the camera of the binocular device and the preset calibration plate is a preset convergence distance, the preset calibration plate is photographed by the binocular device to obtain a left image photographed by the left camera and a right image photographed by the right camera; Correcting the left image using the first left calibration parameter to obtain a first corrected image; correcting the right image using the first right calibration parameter to obtain a second corrected image; Calculating a disparity mean between the first corrected image and the second corrected image to obtain a disparity mean result; When the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, based on the disparity mean result, adjusting the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device; The second left calibration parameter is used as a new first left calibration parameter, and the second right calibration parameter is used as a new first right calibration parameter, and the step of calculating the disparity mean is repeatedly performed until the absolute value of the obtained disparity mean result is less than the preset disparity mean threshold.

2. The calibration parameter adjustment method of the binocular device according to claim 1, It is characterized in that The calculating the disparity mean between the first corrected image and the second corrected image to obtain the disparity mean result includes: Based on each corner point in the first corrected image and each corner point in the second corrected image, a disparity mean between the first corrected image and the second corrected image is calculated to obtain the disparity mean result.

3. The calibration parameter adjustment method of the binocular device according to claim 2, It is characterized in that The step of calculating the disparity mean between the first corrected image point and the second corrected image point based on each corner point in the first corrected image and each corner point in the second corrected image to obtain the disparity mean result includes: Determining corner points in the first corrected image and corner points in the second corrected image; determining a pair of corner points between the first corrected image and the second corrected image based on the corner points in the first corrected image and the corner points in the second corrected image; Calculating the disparity of each of the corner point pairs between the first corrected image and the second corrected image based on the corner point pairs between the first corrected image and the second corrected image; Determining a mean value of the disparity of all the corner point pairs between the first corrected image and the second corrected image based on the disparity of each of the corner point pairs between the first corrected image and the second corrected image; The disparity mean of all the corner point pairs between the first corrected image and the second corrected image is determined as the disparity mean between the first corrected image and the second corrected image, to obtain the disparity mean result.

4. The calibration parameter adjustment method of the binocular device according to claim 3, It is characterized in that The step of determining the disparity mean of all corner point pairs between the first corrected image and the second corrected image based on the disparity of each corner point pair between the first corrected image and the second corrected image comprises: Averaging the disparities of all the corner point pairs between the first corrected image and the second corrected image to obtain a disparity mean value of all the corner point pairs between the first corrected image and the second corrected image; Or, based on the position of each of the corner points, determine the weight corresponding to the disparity of each of the corner point pairs; based on the weight corresponding to the disparity of each of the corner point pairs, perform weighted summation on the disparity of each of the corner point pairs to obtain the mean disparity of all of the corner point pairs between the first corrected image and the second corrected image.

5. The calibration parameter adjustment method of the binocular device according to claim 1, It is characterized in that When the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, based on the disparity mean result, adjusting the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device, including: When the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, based on the positive and negative signs of the disparity mean result, the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device are adjusted to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device.

6. The calibration parameter adjustment method of the binocular device according to claim 5, It is characterized in that When the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, based on the positive and negative signs of the disparity mean result, adjusting the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device, including: When the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, if the disparity mean result is a positive number, the first principal point position of the first left calibration parameter is moved rightward by a preset adjustment distance to obtain a new first principal point position; the preset adjustment distance is a distance value that is proportional to the absolute value of the disparity mean result; Based on the new first principal point position, obtaining the second left calibration parameter of the left camera of the binocular device; Moving the second principal point position of the first right calibration parameter to the left by the preset adjustment distance to obtain a new second principal point position; Based on the new second principal point position, the second right calibration parameter of the right camera of the binocular device is obtained.

7. The calibration parameter adjustment method of the binocular device according to claim 5, It is characterized in that When the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, based on the positive and negative signs of the disparity mean result, adjusting the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device, further comprising: When the absolute value of the disparity mean result is greater than or equal to the preset disparity mean threshold, if the disparity mean result is a negative number, the first principal point position of the first left calibration parameter is moved to the left by the preset adjustment distance to obtain a third principal point position; Based on the third principal point position, obtaining the second left calibration parameter of the left camera of the binocular device; The second principal point position of the first right calibration parameter is moved rightward by the preset adjustment distance to obtain a fourth principal point position; Based on the fourth principal point position, the second right calibration parameter of the right camera of the binocular device is obtained.

8. The calibration parameter adjustment method of the binocular device according to claim 1, It is characterized in that The method further comprises: The first left calibration parameter and the first right calibration parameter in the last step of calculating the parallax mean are determined as the convergence distance solidification parameters corresponding to the preset convergence distance.

9. A calibration parameter adjustment device for a binocular device, It is characterized in that The binocular device includes two cameras, namely a left camera and a right camera; the device includes: A parameter acquisition module, used to perform initial calibration on the binocular device to obtain a first left calibration parameter of the left camera and a first right calibration parameter of the right camera; An image acquisition module, configured to use the binocular device to shoot the preset calibration plate to obtain a left image captured by the left camera and a right image captured by the right camera, under the premise that both optical paths of the camera of the binocular device are perpendicular to the preset calibration plate and the distance between the camera of the binocular device and the preset calibration plate is a preset convergence distance; A calibration module, configured to calibrate the left image using the first left calibration parameter to obtain a first calibrated image; and calibrate the right image using the first right calibration parameter to obtain a second calibrated image; A calculation module, used for calculating the mean disparity between the first corrected image and the second corrected image to obtain a mean disparity result; an adjustment module, configured to adjust the first principal point position of the first left calibration parameter and the second principal point position of the first right calibration parameter of the binocular device based on the disparity mean result when the absolute value of the disparity mean result is greater than or equal to a preset disparity mean threshold, so as to obtain the second left calibration parameter of the left camera and the second right calibration parameter of the right camera of the binocular device; And an updating module, used for taking the second left calibration parameter as a new first left calibration parameter and the second right calibration parameter as a new first right calibration parameter, and repeatedly executing the step of calculating the disparity mean until the absolute value of the obtained disparity mean result is less than the preset disparity mean threshold.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the calibration parameter adjustment method of the binocular device described in any one of claims 1 to 8 are implemented.