Calibration Method, Monocular Structured Light Module, Electronic Device, and Storage Medium

Abstract

Embodiments of the present application relate to the field of machine vision technology, and disclose a calibration method, a monocular structured light module, an electronic device, and a storage medium, which are applied to a monocular structured light module provided with a single-point rangefinder. The method includes: obtaining a single-point image captured by the monocular structured light module of a target scene, and obtaining the true distance corresponding to the single-point image; wherein the single-point image is the projection point of the light beam emitted by the single-point rangefinder on the target scene; obtaining a scene image captured by the monocular structured light module of the target scene, and determining a target point with the same name as the projection point in the scene image; determining the disparity value corresponding to the target point according to the scene image and a preset reference image; calibrating the absolute accuracy of the monocular structured light module according to the disparity value corresponding to the target point, the calibration distance, and the true distance, realizing the automatic calibration of the absolute accuracy of the monocular structured light module, greatly improving the stability and robustness of the monocular structured light module, reducing the maintenance cost, and enhancing the user experience.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of machine vision, and in particular, to a calibration method, a monocular structured light module, an electronic device, and a storage medium. Background Art

[0002] The monocular structured light technology has become increasingly mature. Regarding new applications of the monocular structured light technology, new scenarios for the application of the monocular structured light technology have emerged continuously. For example, three-dimensional modeling, security monitoring, smart home robots, mobile payment, augmented reality (AR), virtual reality (VR), etc. have become an indispensable part of people's production and life.

[0003] The core of the monocular structured light module consists of a structured light projector and a camera. The structured light projector emits a structured light pattern, giving image features to the target scene, and the camera can capture these features as image information, that is, a scene map. The monocular structured light module has completed reference map and parameter calibration at the factory. Therefore, the monocular structured light module can perform homonymous point matching on the scene map and the reference map, thereby calculating the disparity information, and then converting the disparity information into depth information to achieve 3D calculation.

[0004] However, the inventors of the present application have found that the entire 3D calculation has very high requirements for the absolute accuracy of the parameter calibration of the monocular structured light module. When users use the monocular structured light module, it is inevitable that there will be bumps, drops, scratches, etc., and the scenarios in which users use the monocular structured light module are constantly changing. Some scenarios have too low or too high temperatures, too dry or too wet humidity, which will cause changes in the relative positions between the structured light projector and the camera. The absolute accuracy of the parameter calibration of the monocular structured light module will have a large error. In the industry, only by designing the hardware structure method for buffering and shock absorption, it is difficult to ensure that the absolute accuracy does not generate errors for a long time. When the absolute accuracy generates errors, users can only return the monocular structured light module to the factory for re-calibration, which is too costly and time-consuming. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a calibration method, a monocular structured light module, an electronic device, and a storage medium, which can realize the automatic calibration of the absolute accuracy of the monocular structured light module, greatly improve the stability and robustness of the monocular structured light module, reduce the maintenance costs of users and manufacturers, and improve the user experience.

[0006] To solve the above technical problems, an embodiment of the present application provides a calibration method, which is applied to a monocular structured light module. The monocular structured light module is provided with a single-point rangefinder, and the method includes the following steps: obtaining a single-point image captured by the monocular structured light module of a target scene, and obtaining the true distance corresponding to the single-point image; wherein, the single-point image is the projection point of the light beam emitted by the single-point rangefinder on the target scene, and the true distance is the distance between the monocular structured light module and the projection point measured by the single-point rangefinder; obtaining a scene image captured by the monocular structured light module of the target scene, and determining a target point in the scene image that has the same name as the projection point; determining the disparity value corresponding to the target point according to the scene image and a preset reference image; and calibrating the absolute accuracy of the monocular structured light module according to the disparity value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance.

[0007] An embodiment of the present application further provides a monocular structured light module, which includes: a structured light projector, a single-point rangefinder, a camera, and a calibration module. The structured light projector, the single-point rangefinder, and the camera are all disposed on the same surface of the monocular structured light module; the single-point rangefinder is configured to emit a light beam to a target scene to form a projection point, and measure the true distance between the monocular structured light module and the projection point; the camera is configured to capture the projection point to obtain a single-point image; the structured light projector is configured to emit a structured light pattern to the target scene to form a structured light projection; the camera is further configured to capture the structured light projection to obtain a scene image; and the calibration module is configured to determine a target point in the scene image that has the same name as the projection point, determine the disparity value corresponding to the target point according to the scene image and a preset reference image, and calibrate the absolute accuracy of the monocular structured light module according to the disparity value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance.

[0008] An embodiment of the present application further provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the above calibration method.

[0009] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above calibration method is implemented.

[0010] The calibration method, monocular structured light module, electronic device, and storage medium provided by the embodiments of the present application first obtain a single-point image captured by the monocular structured light module of a target scene and the corresponding true distance when calibrating. The single-point image is the projection point of the light beam emitted by the single-point rangefinder on the target scene, and the true distance is the distance between the monocular structured light module and the projection point measured by the single-point rangefinder. Subsequently, the monocular structured light module captures a scene image of the same target scene, and a target point with the same name as the projection point is determined in the scene image. According to the scene image and a preset reference image, the parallax value corresponding to the target point can be determined. Finally, according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance between the monocular structured light module and the projection point, the absolute accuracy of the monocular structured light module is calibrated. Considering that a change in the relative position between the structured light projector and the camera in the monocular structured light module will cause a large error in the absolute accuracy, after the absolute accuracy has an error, the user can only return the monocular structured light module to the factory for recalibration, which is too costly and time-consuming. The embodiments of the present application add a single-point rangefinder to the monocular structured light module. The single-point rangefinder is relatively stable, and its ranging performance will not easily change, and the measured distance will not have a large error. Based on the distance measured by the single-point rangefinder and the depth value calculated by the 3D solution of the monocular structured light module, the absolute accuracy is automatically calibrated, which can greatly improve the stability and robustness of the monocular structured light module, while reducing the maintenance costs of users and manufacturers and enhancing the user experience.

[0011] In addition, the calibrating the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance includes: determining the depth value corresponding to the target point according to the parallax value corresponding to the target point; calculating the absolute value of the difference between the depth value corresponding to the target point and the true distance, and determining whether the absolute value is greater than a preset trigger threshold; and when the absolute value is greater than the preset trigger threshold, calibrating the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance. After a large error occurs in the absolute accuracy of the monocular structured light module, the most intuitive manifestation is that the depth value calculated by the 3D solution is very inaccurate, while the true distance measured by the single-point rangefinder is very accurate. Therefore, the present application compares the depth value calculated by the 3D solution with the true distance, and when the difference between the two is too large and unacceptable, the calibration of the absolute accuracy is triggered, which can save calibration resources while ensuring that the monocular structured light module always works in a better state.

[0012] In addition, the single-point images include a plurality of single-point images captured by the monocular structured light module at different positions of the target scene, the true distances include the true distances corresponding to the respective single-point images, the scene images include a plurality of scene images captured by the monocular structured light module at the different positions of the target scene, and there are a plurality of target points; the method for correcting the absolute accuracy of the monocular structured light module according to the parallax values corresponding to the target points, the calibration distance between the reference image and the monocular structured light module, and the true distances includes: constructing a system of equations according to the parallax values corresponding to the respective target points, the calibration distance between the reference image and the monocular structured light module, and the respective true distances; wherein, the system of equations includes an unknown first calibration parameter and an unknown second calibration parameter, the first calibration parameter is the product of the true baseline length and the true focal length of the monocular structured light module, and the second calibration parameter is the overall offset error in the parallax direction; solving the system of equations to obtain the solved first calibration parameter and the solved second calibration parameter; replacing the product of the calibrated baseline length and the calibrated focal length of the monocular structured light module with the solved first calibration parameter, and replacing the calibrated parallax of the monocular structured light module with the corrected parallax; wherein, the corrected parallax is the sum of the calibrated parallax of the monocular structured light module and the second calibration parameter. Considering that the factors affecting the absolute accuracy of the monocular structured light module are the calibration distance, the calibrated baseline length, the calibrated focal length, and the calibrated parallax, and during daily use, the calibration distance does not change, but the calibrated baseline length, the calibrated focal length, and the calibrated parallax will change. Considering that the calibrated baseline length and the calibrated focal length play a role together in the form of a product in the parallax calculation, the present application sets the first calibration parameter and the second calibration parameter, and solving these two parameters can achieve the correction of the absolute accuracy of the monocular structured light module. At the same time, the present application solves these two calibration parameters by constructing a system of equations, which can further improve the scientificity, accuracy, and reliability of the correction.

[0013] In addition, before obtaining the single-point image captured by the monocular structured light module for the target scene and obtaining the true distance corresponding to the single-point image, the following steps are included: setting the time-division projection of the single-point rangefinder and the structured light projector of the monocular structured light module, where the structured light projector does not work during the working pulse time of the single-point rangefinder, and the single-point rangefinder does not work during the working pulse time of the structured light projector; wherein, the frame rate of the single-point rangefinder is slower than that of the structured light projector. Considering that both the structured light projector and the single-point rangefinder work by emitting specific light beams to the target scene, they will interfere with and affect each other. In the embodiments of the present application, the time-division projection of the single-point rangefinder and the structured light projector is set before using the monocular structured light module, which can prevent the single-point rangefinder and the structured light projector from interfering with and affecting each other while realizing the real-time automatic correction of the barrage structured light module. Setting the frame rate of the single-point rangefinder to be slower than that of the structured light projector can give priority to ensuring the 3D calculation function of the monocular structured light module.

[0014] In addition, before obtaining the single-point image captured by the monocular structured light module for the target scene and obtaining the true distance corresponding to the single-point image, the following steps are included: turning off the structured light projector of the monocular structured light module and turning on the single-point rangefinder; after obtaining the true distance corresponding to the single-point image and before obtaining the scene image captured by the monocular structured light module for the target scene, the following steps are further included: turning off the single-point rangefinder and turning on the structured light projector. Considering that both the structured light projector and the single-point rangefinder work by emitting specific light beams to the target scene, they will interfere with and affect each other. The embodiments of the present application allow the user to manually turn off the single-point rangefinder and the structured light projector, so as to ensure that only one of them is in the working state at the same time and prevent them from interfering with and affecting each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] One or more embodiments are illustrated by way of example in the accompanying drawings, and these exemplary illustrations do not limit the embodiments.

[0016] Figure 1 is the flowchart of the calibration method according to an embodiment of the present application Figure 1 ;

[0017] Figure 2 is the schematic appearance diagram of a monocular structured light module provided in an embodiment of the present application;

[0018] Figure 3 is the schematic diagram of the 3D calculation process provided in an embodiment of the present application;

[0019] Figure 4In one embodiment of the present application, it is a flowchart for calibrating the absolute accuracy of a monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the actual distance between the monocular structured light module and the projection point;

[0020] Figure 5 It is the flow of the calibration method according to another embodiment of the present application Figure 2 ;

[0021] Figure 6 It is the flow of the calibration method according to another embodiment of the present application Figure 3 ;

[0022] Figure 7 It is a schematic diagram of a monocular structured light module according to another embodiment of the present application;

[0023] Figure 8 It is a schematic structural diagram of an electronic device according to another embodiment of the present application. Detailed implementation manners

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the embodiments of the present application will be elaborated in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that in the embodiments of the present application, many technical details are presented for the readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions required to be protected by the present application can still be implemented. The following division of each embodiment is for convenience of description and should not constitute any limitation on the specific implementation manner of the present application. Each embodiment can be combined and cross-referenced with each other on the premise of not being contradictory.

[0025] One embodiment of the present application relates to a calibration method applied to an electronic device in a monocular structured light module. Among them, the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is taken as an example of a server for illustration. The implementation details of the calibration method in this embodiment will be specifically described below. The following content is only implementation details provided for convenience of understanding and is not necessary for implementing this solution.

[0026] The specific process of the calibration method in this embodiment can be as Figure 1 shown and includes:

[0027] Step 101, obtain a single-point image captured by the monocular structured light module for the target scene, and obtain the actual distance corresponding to the single-point image. The single-point image is the projection point of the light beam emitted by the single-point rangefinder on the target scene, and the actual distance is the distance between the monocular structured light module and the projection point measured by the single-point rangefinder.

[0028] Specifically, the monocular structured light module is provided with a single-point rangefinder. The single-point rangefinder can emit a single-point light beam towards the target scene, and measure the distance from the monocular structured light module to the target scene according to the propagation time and speed of the light beam. The structured light projector, the single-point rangefinder, and the camera are all arranged on the same surface of the monocular structured light module. The camera can capture the projection point formed by the single-point light beam emitted by the single-point rangefinder on the target scene, and can also capture the structured light projection formed by the structured light pattern emitted by the structured light projector on the target scene.

[0029] In one example, the schematic appearance diagram of the monocular structured light module can be as Figure 2 shown. The single-point rangefinder is arranged between the structured light projector and the camera. The distance between the center of the structured light projector and the optical center of the camera is the baseline length L of the monocular structured light module.

[0030] In specific implementation, when the server corrects the absolute accuracy of the monocular structured light module, the single-point rangefinder emits a single-point light beam towards the target scene, forming a projection point on the target scene. At this time, the camera can capture the projection point on the target scene. The server obtains the single-point image captured by the monocular structured light module of the target scene from the camera. At the same time, the single-point rangefinder completes the ranging task. The server obtains the true distance between the monocular structured light module and the projection point measured by the single-point rangefinder from the single-point rangefinder. The single-point rangefinder has low cost, good stability, and is not easily affected by the outside world.

[0031] It can be understood that after the server obtains the true distance between the monocular structured light module and the projection point measured by the single-point rangefinder, it can record the coordinates of the projection point as (Xp, Yp), and record this true distance as d p and use d p as the calibration benchmark for the absolute accuracy of the monocular structured light module.

[0032] Step 102: Obtain the scene image captured by the monocular structured light module of the target scene, and determine the target point with the same name as the projection point in the scene image.

[0033] Specifically, after the server obtains the single-point image captured by the monocular structured light module of the target scene and the corresponding true distance of the single-point image, it can continue to obtain the scene image captured by the monocular structured light module of the target scene, and determine the target point with the same name as the projection point in the scene image, that is, find the point with the same coordinates as the projection point in the scene image, and use the point with the same coordinates as the projection point in the scene image as the target point with the same name as the projection point.

[0034] In specific implementation, after the single-point ranging is completed, the structured light projector of the monocular structured light module emits a structured light pattern towards the same target scene at the same position, forming a structured light projection on the target scene. The camera captures this structured light projection to obtain the scene image.

[0035] It is understandable that the point in the scene graph with the same position as the projection point, that is, the target point with the same name as the projection point, and the coordinates of the projection point are denoted as (Xp, Yp), then the coordinates of the target point are also (Xp, Yp).

[0036] Step 103: Determine the disparity value corresponding to the target point according to the scene graph and a preset reference graph.

[0037] Specifically, according to the 3D calculation principle, when data such as the calibrated baseline length and calibrated focal length are known, the server can perform 3D calculation on the scene graph according to the scene graph and the preset reference graph, and calculate the disparity values corresponding to each pixel point in the scene graph, that is, determine the disparity value corresponding to the target point. Among them, the preset reference graph can be set by those skilled in the art according to actual needs, and the embodiments of the present application do not make specific limitations on this.

[0038] To more clearly elaborate on how the server determines the disparity value corresponding to the target point according to the scene graph and the preset reference graph, the 3D calculation after imaging of the monocular structured light module is introduced below. The calculation process is as Figure 3 shown. The monocular structured light module has completed the acquisition of the reference image and the calibration of relevant parameters during factory production, such as the calibration distance d between the reference graph and the monocular structured light module 0 , the calibrated baseline length L of the monocular structured light module, the calibrated focal length F of the monocular structured light module, etc. When the user actually uses it, the three-dimensional coordinate point Q in the target scene is imaged as the pixel point q on the camera. The server finds the matching point p based on the preset matching algorithm. The three-dimensional coordinate point P corresponding to the point p on the reference plane position. According to the pixel point q and the pixel point p, the disparity value e corresponding to the point q can be calculated, and the depth value corresponding to the three-dimensional coordinate point Q can also be calculated according to the following formula based on the calibration distance d 0 , the calibrated baseline length L, the calibrated focal length F, the disparity value e, etc.: d 1 = 1 / [(1 / d0) + (e / FL)].

[0039] Step 104: Calibrate the absolute accuracy of the monocular structured light module according to the disparity value corresponding to the target point, the calibration distance between the reference graph and the monocular structured light module, and the actual distance between the monocular structured light module and the projection point.

[0040] In specific implementation, after the server determines the disparity value corresponding to the target point, it can calibrate the absolute accuracy of the monocular structured light module according to the disparity value corresponding to the target point, the calibration distance between the reference graph and the monocular structured light module, and the actual distance between the monocular structured light module and the projection point.

[0041] In one example, the server can calculate the depth value corresponding to the target point based on the parallax value corresponding to the target point and the calibration distance between the reference image and the monocular structured light module, and correct and calibrate the absolute accuracy of the monocular structured light module according to the depth value corresponding to the target point and the actual distance between the monocular structured light module and the projection point.

[0042] In this embodiment, the monocular structured light module is provided with a single-point rangefinder. When calibrating, the server first obtains a single-point image captured by the monocular structured light module of the target scene and the actual distance corresponding to the single-point image. The single-point image is the projection point of the light beam emitted by the single-point rangefinder on the target scene, and the actual distance is the distance between the monocular structured light module and the projection point measured by the single-point rangefinder. Subsequently, the server obtains a scene image captured by the monocular structured light module of the same target scene, determines the target point with the same name as the projection point in the scene image, and can determine the parallax value corresponding to the target point according to the scene image and the preset reference image. Finally, according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the actual distance between the monocular structured light module and the projection point, the absolute accuracy of the monocular structured light module is calibrated. Considering that a change in the relative position between the structured light projector and the camera in the monocular structured light module will cause a large error in the absolute accuracy, after the absolute accuracy has an error, the user can only return the monocular structured light module to the factory for recalibration, which is costly and time-consuming. In the embodiment of the present application, a single-point rangefinder is added to the monocular structured light module. The single-point rangefinder is relatively stable, and its ranging performance will not change easily, and the measured distance will not have a large error. Based on the distance measured by the single-point rangefinder and the depth value calculated by the 3D of the monocular structured light module, the automatic calibration of the absolute accuracy can greatly improve the stability and robustness of the monocular structured light module, while reducing the maintenance costs of users and manufacturers and improving the user experience.

[0043] In one embodiment, the server corrects the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the actual distance between the monocular structured light module and the projection point, which can be achieved through the following steps: Figure 4 as shown below:

[0044] Step 201, determine the depth value corresponding to the target point according to the parallax value corresponding to the target point.

[0045] Specifically, after the server determines the parallax value corresponding to the target point according to the scene image and the preset reference image, it can perform 3D calculation according to the parallax value corresponding to the target point to determine the depth value corresponding to the target point.

[0046] Step 202: Calculate the absolute value of the difference between the depth value corresponding to the target point and the true distance, and determine whether the absolute value is greater than a preset trigger threshold. If so, execute Step 203; otherwise, execute Step 204.

[0047] Step 203: Calibrate the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance between the monocular structured light module and the projection point.

[0048] Step 204: Exit the calibration.

[0049] In a specific implementation, after a large error occurs in the absolute accuracy of the monocular structured light module, the most intuitive manifestation is that the depth value calculated by 3D is very inaccurate, while the true distance measured by the single-point rangefinder between the monocular structured light module and the projection point is real and accurate. Therefore, by comparing the depth value corresponding to the target point with the true distance, it can be determined whether a large error has occurred in the structured light module. The server calculates the difference between the depth value corresponding to the target point and the true distance and takes the absolute value of the difference, and determines whether the absolute value is greater than a preset trigger threshold. If the absolute value is greater than the preset trigger threshold, it indicates that a large error has occurred in the absolute accuracy of the monocular structured light module and calibration is required. The server then calibrates the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance between the monocular structured light module and the projection point. If the absolute value is less than or equal to the preset trigger threshold, it indicates that no large error has occurred in the absolute accuracy of the monocular structured light module, which does not affect normal use and no calibration is required. Therefore, the server exits the calibration process.

[0050] In this embodiment, considering that after a large error occurs in the absolute accuracy of the monocular structured light module, the most intuitive manifestation is that the depth value calculated by 3D is very inaccurate, while the true distance measured by the single-point rangefinder is very accurate. Therefore, this application compares the depth value calculated by 3D with the true distance. When the difference between the two is large and unacceptable, the calibration of the absolute accuracy is triggered, which can save calibration resources while ensuring that the monocular structured light module always works in a better state.

[0051] Another embodiment of this application relates to a calibration method. The implementation details of the calibration method in this embodiment will be specifically described below. The following content is only provided for convenience of understanding and is not necessary for implementing this solution. The specific process of the calibration method in this embodiment can be as Figure 5 shown and includes:

[0052] Step 301: Obtain a plurality of single-point images of the target scene captured by the monocular structured light module at different positions, and obtain the true distance corresponding to each single-point image.

[0053] Step 302: Obtain a plurality of scene images captured by the monocular structured light module for the target scene at the different positions, and determine target points that are homologous to the projection points in each scene image.

[0054] In a specific implementation, the single-point images obtained by the server include a plurality of single-point images captured by the monocular structured light module for the target scene at different positions, and the true distances measured by the single-point rangefinder include the true distances corresponding to the respective single-point images, that is, the true distances between the monocular structured light module and the projection points measured by the single-point rangefinder at different positions. Similarly, the scene images obtained by the server include a plurality of scene images captured by the monocular structured light module for the target scene at the above-mentioned different positions, and the server can respectively determine the target points that are homologous to the projection points in each scene image.

[0055] Step 303: Determine the parallax values corresponding to the respective target points according to each scene image and a preset reference image.

[0056] Specifically, after the server obtains a plurality of scene images captured by the monocular structured light module for the target scene at different positions and determines the target points that are homologous to the projection points in each scene image, it can determine the parallax values corresponding to the respective target points according to each scene image and the preset reference image. Among them, the preset reference image can be set by those skilled in the art according to actual needs when the monocular structured light module leaves the factory, and the embodiments of the present application do not make specific limitations thereto.

[0057] Step 304: Construct an equation set according to the parallax values corresponding to the respective target points, the calibration distance between the reference image and the monocular structured light module, and the respective true distances.

[0058] Step 305: Solve the equation set to obtain the solved first calibration parameter and the solved second calibration parameter.

[0059] Specifically, after the server determines the parallax values corresponding to the respective target points, it can construct an equation set according to the parallax values corresponding to the respective target points, the calibration distance between the reference image and the monocular structured light module, and the respective true distances. The equation set includes an unknown first calibration parameter and an unknown second calibration parameter. The first calibration parameter is the product of the true baseline length and the true focal length of the monocular structured light module, and the second calibration parameter is the overall offset error in the parallax direction. The server performs least squares solution on the constructed equation set to obtain the solved first calibration parameter and the solved second calibration parameter.

[0060] In a specific implementation, considering that the calibration distance d between the reference image and the monocular structured light module 0 will not change, bumps, drops, and random rubbing during daily use will not affect the calibration distance d 0, but these phenomena can easily cause changes in the relative positions between the structured light projector and the camera, that is, they can cause changes in the calibration baseline length L and the calibration focal length F of the monocular structured light module. At the same time, due to the change in the baseline length, the parallax values calculated by the server are also inaccurate. Referring to the 3D calculation formula d 1 = 1 / [(1 / d0)+(e / FL)], it can be seen that the calibration baseline length L and the calibration focal length F play a role together in the form of a product. Therefore, in this embodiment, a first correction parameter and a second correction parameter are set to respectively correct the product of the calibration baseline length L and the calibration focal length F, and the parallax values calculated based on inaccurate parameters.

[0061] In one example, when the server constructs an equation set according to the parallax values corresponding to each target point, the calibration distance between the reference image and the monocular structured light module, and each true distance, it can traverse each target point, construct an equation corresponding to the current target point according to the parallax value corresponding to the current target point, the calibration distance between the reference image and the monocular structured light module, and the true distance corresponding to the current target point, and finally form an equation set according to the equations corresponding to each target point. The number of equations is at least 2.

[0062] In another example, the number of equations constructed by the server is at least 3, that is, the server constructs an overdetermined equation set and solves the overdetermined equation set based on the least squares method to solve for the first correction parameter and the second correction parameter. There are two unknown parameters to be solved in this application, namely the first correction parameter and the second correction parameter. In order to solve the first correction parameter and the second correction parameter more precisely and scientifically, this application chooses to construct an overdetermined equation set, and the number of equations required is at least 3, that is, the number of target points is at least 3, which means that at least 3 single-point images need to be taken at 3 different positions. Each single-point image corresponds to an equation in the overdetermined equation set, so as to solve for the first correction parameter and the second correction parameter.

[0063] In one example, the server constructs an equation corresponding to the current target point according to the following formula based on the parallax value corresponding to the current target point, the calibration distance between the reference image and the monocular structured light module, and the true distance corresponding to the current target point: d p = 1 / {1 / d 0 + [e (Xp,Yp) + e 0 / K}, where d p is the true distance corresponding to the current target point, d 0 is the calibration distance between the reference image and the monocular structured light module, e (Xp,Yp) is the parallax value corresponding to the current target point, K is the first correction parameter, and e 0 is the second correction parameter.

[0064] In one example, for the convenience of solving, the server can simplify the equation, and the simplified equation can be as follows: [(1 / d p )-(1 / d 0 )]K - e 0 = e (Xp,Yp) .

[0065] Step 306: Replace the product of the calibration baseline length and the calibration focal length of the monocular structured light module with the solved first correction parameter, and replace the calibration parallax of the monocular structured light module with the corrected parallax.

[0066] In a specific implementation, since the baseline length L and the focal length F play a role together in the form of a product, this application does not distinguish between the baseline length L and the focal length F in detail, but directly replaces the product L*F of the calibration baseline length and the calibration focal length of the monocular structured light module with the first correction parameter K, and then calculates the sum of the calibration parallax e of the monocular structured light module and the second correction parameter e 0 (e + e 0 ), and uses (e + e 0 ) as the corrected parallax to replace the calibration parallax of the monocular structured light module, thereby completing the correction of the baseline length, focal length, and parallax of the monocular structured light module, that is, completing the correction of the absolute accuracy of the monocular structured light module.

[0067] In this embodiment, considering that the factors affecting the absolute accuracy of the monocular structured light module are the calibration distance, calibration baseline length, calibration focal length, and calibration parallax, and in daily use, the calibration distance will not change, but the calibration baseline length, calibration focal length, and calibration parallax will change. Considering that the calibration baseline length and the calibration focal length play a role together in the form of a product in parallax calculation, this application sets the first correction parameter and the second correction parameter. Solving these two parameters can achieve the correction of the absolute accuracy of the monocular structured light module. At the same time, this application solves these two correction parameters by constructing a system of equations, which can further improve the scientificity, accuracy, and reliability of the correction.

[0068] In one embodiment, before obtaining a single-point image taken by a monocular structured light module of the target scene and obtaining the real distance corresponding to the single-point image, the server may also set a single-point rangefinder and a structured light projector of the monocular structured light module to perform time-sharing projection. The structured light projector does not work during the working pulse time of the single-point rangefinder, and the single-point rangefinder does not work during the working pulse time of the structured light projector. The server may also set the frame rate of the single-point rangefinder to be slower than the frame rate of the structured light projector. Considering that both the structured light projector and the single-point rangefinder work by emitting a specific light beam to the target scene, the two may interfere with and affect each other. In an embodiment of the present application, the single-point rangefinder and the structured light projector are set to perform time-sharing projection before using the monocular structured light module. While realizing real-time automatic correction of the barrage structured light module, the single-point rangefinder and the structured light projector are prevented from interfering with and affecting each other. Setting the frame rate of the single-point rangefinder to be slower than the frame rate of the structured light projector can prioritize the 3D solution function of the monocular structured light module.

[0069] Another embodiment of the present application relates to a correction method. The implementation details of the correction method of this embodiment are described in detail below. The following content is only for the convenience of understanding the implementation details, and is not necessary for the implementation of this solution. The specific process of the correction method of this embodiment can be as follows: Figure 6 As shown, including:

[0070] Step 401, turn off the structured light projector and turn on the single-point rangefinder.

[0071] Step 402, obtain a single-point image of the target scene captured by the monocular structured light module, and obtain the real distance corresponding to the single-point image. The single-point image is the projection point of the light beam emitted by the single-point rangefinder on the target scene, and the real distance is the distance between the monocular structured light module and the projection point measured by the single-point rangefinder.

[0072] Among them, step 402 is substantially the same as step 101 and will not be described again here.

[0073] Step 403, turn off the single-point rangefinder and turn on the structured light projector.

[0074] Step 404: obtain a scene graph of the target scene captured by the monocular structured light module, and determine a target point with the same name as the projection point in the scene graph.

[0075] Step 405: Determine the disparity value corresponding to the target point according to the scene graph and the preset reference graph.

[0076] Step 406 , calibrating the absolute accuracy of the monocular structured light module according to the disparity value corresponding to the target point, the calibrated distance between the reference image and the monocular structured light module, and the actual distance between the monocular structured light module and the projection point.

[0077] Among them, steps 404 to 406 are substantially the same as steps 102 to 104, and will not be elaborated here.

[0078] In a specific implementation, considering that both the structured light projector and the single-point rangefinder work by emitting specific light beams to the target scene, they will interfere with and affect each other. Embodiments of the present application allow users to manually turn off the single-point rangefinder and the structured light projector, so as to ensure that only one of them is in the working state at the same time, preventing them from interfering with and affecting each other.

[0079] The step division of the above various methods is only for clear description. During implementation, they can be combined into one step or some steps can be split into multiple steps. As long as the same logical relationship is included, they are all within the protection scope of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs, but without changing the core design of the algorithm and process, are all within the protection scope of this patent.

[0080] Another embodiment of the present application relates to a monocular structured light module. The implementation details of the monocular structured light module in this embodiment will be specifically described below. The following content is only the implementation details provided for convenient understanding and is not necessary for implementing this solution. The schematic diagram of the monocular structured light module in this embodiment can be as Figure 7 shown. The monocular structured light module includes a structured light projector 501, a single-point rangefinder 502, a camera 503, and a calibration module 504. The external view schematic diagram of the monocular structured light module in this embodiment can be as Figure 2 shown. The structured light projector 501, the single-point rangefinder 502, and the camera 503 are all arranged on the same surface of the monocular structured light module.

[0081] The single-point rangefinder 502 is used to emit a light beam to the target scene to form a projection point, so as to measure the real distance between the monocular structured light module and the projection point.

[0082] The camera 503 is used to photograph the projection point to obtain a single-point image.

[0083] The structured light projector 501 is used to emit a structured light pattern to the target scene to form a structured light projection.

[0084] The camera 503 is also used to photograph the structured light projection to obtain a scene image.

[0085] The calibration module 504 is used to determine the target point with the same name as the projection point in the scene image, determine the parallax value corresponding to the target point according to the scene image and a preset reference image, and calibrate the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the real distance between the monocular structured light module and the projection point.

[0086] It is worth mentioning that each module involved in this embodiment is a logical module. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or implemented as a combination of multiple physical units. In addition, to highlight the innovative part of this application, units not closely related to solving the technical problems proposed in this application are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.

[0087] Another embodiment of this application relates to an electronic device, as Figure 8 shown, including: at least one processor 601; and a memory 602 communicatively connected to the at least one processor 601; wherein, the memory 602 stores instructions executable by the at least one processor 601, and the instructions are executed by the at least one processor 601 to enable the at least one processor 601 to execute the calibration methods in the above embodiments.

[0088] Among them, the memory and the processor are connected by a bus. The bus can include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art, so they will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be an element or multiple elements, such as multiple receivers and transmitters, and provides a unit for communicating with various other devices on the transmission medium. The data processed by the processor is transmitted over the wireless medium through the antenna. Further, the antenna also receives data and transmits the data to the processor.

[0089] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. The memory can be used to store the data used by the processor when executing operations.

[0090] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method embodiments described above are implemented.

[0091] That is, those skilled in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs.

[0092] Those of ordinary skill in the art can understand that the above embodiments are specific embodiments for implementing the present application. In actual applications, various changes can be made to them in form and details without departing from the spirit and scope of the present application.

Claims

1. A calibration method, characterized in that, it is applied to a monocular structured light module, and the monocular structured light module is provided with a single-point rangefinder. The method includes: Obtaining a single-point image captured by the monocular structured light module of a target scene, and obtaining the true distance corresponding to the single-point image; wherein, the single-point image is the projection point of the light beam emitted by the single-point rangefinder on the target scene, and the true distance is the distance between the monocular structured light module and the projection point measured by the single-point rangefinder; Obtaining a scene image captured by the monocular structured light module of the target scene, and determining a target point in the scene image that has the same name as the projection point; Determining the parallax value corresponding to the target point according to the scene image and a preset reference image; Calibrating the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance; The single-point image includes a plurality of single-point images captured by the monocular structured light module at different positions of the target scene, the true distance includes the true distances corresponding to each single-point image, the scene image includes a plurality of scene images captured by the monocular structured light module at the different positions of the target scene, and the number of target points is plural; The calibrating the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance includes: Constructing a system of equations according to the parallax values corresponding to each target point, the calibration distance between the reference image and the monocular structured light module, and each true distance; wherein, the system of equations includes an unknown first calibration parameter and an unknown second calibration parameter, the first calibration parameter is the product of the true baseline length and the true focal length of the monocular structured light module, and the second calibration parameter is the overall offset error in the parallax direction; Solving the system of equations to obtain the solved first calibration parameter and the solved second calibration parameter; Replacing the product of the calibrated baseline length and the calibrated focal length of the monocular structured light module with the solved first calibration parameter, and replacing the calibrated parallax of the monocular structured light module with a calibrated parallax; wherein, the calibrated parallax is the sum of the calibrated parallax of the monocular structured light module and the second calibration parameter.

2. The calibration method according to claim 1, characterized in that, the calibrating the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance includes: Determining the depth value corresponding to the target point according to the parallax value corresponding to the target point; Calculating the absolute value of the difference between the depth value corresponding to the target point and the true distance, and determining whether the absolute value is greater than a preset trigger threshold; In the case where the absolute value is greater than the preset trigger threshold, calibrating the absolute accuracy according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance.

3. The calibration method according to claim 1, wherein, constructing an equation set based on the parallax values corresponding to the respective target points, the calibration distance between the reference image and the monocular structured light module, and the respective true distances, includes: traversing each of the target points, and constructing an equation corresponding to the current target point according to the parallax value corresponding to the current target point, the calibration distance between the reference image and the monocular structured light module, and the true distance corresponding to the current target point; forming an equation set according to the equations corresponding to the respective target points; wherein, the number of the equations is at least two.

4. The calibration method according to claim 3, wherein, by the following formula, constructing an equation corresponding to the current target point according to the parallax value corresponding to the current target point, the calibration distance between the reference image and the monocular structured light module, and the true distance corresponding to the current target point: d p = 1 / {1 / d 0 + [e (Xp,Yp) + e 0 / K} Among them, d p is the true distance corresponding to the current target point, d 0 is the calibration distance between the reference image and the monocular structured light module, e (Xp,Yp) is the parallax value corresponding to the current target point, K is the first correction parameter, e 0 is the second correction parameter.

5. The calibration method according to any one of claims 1 to 4, wherein, before obtaining the single-point image captured by the monocular structured light module for the target scene and obtaining the true distance corresponding to the single-point image, it includes: setting the single-point range finder and the structured light projector of the monocular structured light module to project at different times, the structured light projector does not work during the working pulse time of the single-point range finder, and the single-point range finder does not work during the working pulse time of the structured light projector; wherein, the frame rate of the single-point range finder is slower than the frame rate of the structured light projector.

6. The calibration method according to any one of claims 1 to 4, wherein, before obtaining the single-point image captured by the monocular structured light module for the target scene and obtaining the true distance corresponding to the single-point image, it includes: turning off the structured light projector of the monocular structured light module and turning on the single-point range finder; after obtaining the true distance corresponding to the single-point image and before obtaining the scene image captured by the monocular structured light module for the target scene, it further includes: turning off the single-point range finder and turning on the structured light projector.

7. A monocular structured light module, wherein, the module includes: a structured light projector, a single-point range finder, a camera, and a calibration module, and the structured light projector, the single-point range finder, and the camera are all arranged on the same surface of the monocular structured light module; the single-point range finder is used to emit a light beam to the target scene to form a projection point, and measure the true distance between the monocular structured light module and the projection point; the camera is used to capture the projection point to obtain a single-point image; the structured light projector is used to emit a structured light pattern to the target scene to form a structured light projection; the camera is further used to capture the structured light projection to obtain a scene image; the calibration module is used to determine a target point in the scene image that is the same name as the projection point, determine the parallax value corresponding to the target point according to the scene image and a preset reference image, and calibrate the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance. The single-point images include a plurality of single-point images captured by the monocular structured light module at different positions, the true distances include the true distances corresponding to the respective single-point images, the scene images include a plurality of scene images captured by the monocular structured light module at the different positions, and there are a plurality of target points; Correcting the absolute accuracy of the monocular structured light module according to the parallax value corresponding to the target point, the calibration distance between the reference image and the monocular structured light module, and the true distance includes: Constructing a system of equations according to the parallax values corresponding to the respective target points, the calibration distance between the reference image and the monocular structured light module, and the respective true distances; wherein, the system of equations includes an unknown first correction parameter and an unknown second correction parameter, the first correction parameter is the product of the true baseline length and the true focal length of the monocular structured light module, and the second correction parameter is the overall offset error in the parallax direction; Solving the system of equations to obtain the solved first correction parameter and the solved second correction parameter; Replacing the product of the calibrated baseline length and the calibrated focal length of the monocular structured light module with the solved first correction parameter, and replacing the calibrated parallax of the monocular structured light module with a corrected parallax; wherein, the corrected parallax is the sum of the calibrated parallax of the monocular structured light module and the second correction parameter.

8. An electronic device, characterized in that, it includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the correction method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the correction method according to any one of claims 1 to 6.

Images