A binocular speckle stereo camera system

Abstract

The embodiment of the present application provides a binocular speckle stereo camera system, comprising: a binocular camera and a speckle projection device, wherein the binocular camera comprises a first imaging unit and a second imaging unit, the relative position of the speckle projection device and the binocular camera is adjustable; the speckle projection device is used for projecting speckles to the surface of a to-be-measured object; the binocular camera is used for collecting first image data of the to-be-measured object including the surface including speckles by using the first imaging unit, and collecting second image data of the to-be-measured object including the surface including speckles by using the second imaging unit; and the depth information of the to-be-measured object is calculated based on the speckles in the first image data and the speckles in the second image data, wherein the speckles exist in the common field of view of the first imaging unit and the second imaging unit. The embodiment of the present application realizes the improvement of the quality of the obtained depth image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional camera, in particular to a binocular speckle stereo camera system. BACKGROUND

[0002] The binocular stereo camera is a kind of depth camera, which uses two cameras to simulate human eyes to capture images of a three-dimensional target in a three-dimensional space, and obtains a depth image with depth information by feature matching of the two cameras and data processing of the captured images, wherein the depth information includes the depth distance of the target in the three-dimensional space, etc. Through the depth information combined with the two-dimensional coordinate information of the image, the real scene of the target in the three-dimensional space can be restored.

[0003] In the prior art, the binocular stereo camera is often equipped with a structured light projection device to project structured light onto a three-dimensional target. Since targets with different depth information will exhibit different projection effects, the depth information of the target can be obtained by analyzing the parallax of the projection effect. However, the existing binocular stereo camera has certain problems. The quality of the depth image obtained by the binocular stereo camera is easily affected by the projected structured light and the reflection of the target itself, resulting in a low quality of the depth image. For example, the energy and density of the projected structured light decrease sharply with the increase of the image acquisition distance, resulting in a blurred image. Or when the target has high reflectivity, the quality of the obtained depth image cannot be guaranteed. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a binocular speckle stereo camera system to improve the quality of the obtained depth image. The specific technical solutions are as follows:

[0005] The binocular speckle stereo camera system provided by the embodiments of the present application comprises:

[0006] A binocular camera and a speckle projection device, wherein the binocular camera comprises a first imaging unit and a second imaging unit, and the relative position of the speckle projection device and the binocular camera is adjustable.

[0007] The speckle projection device is configured to project speckles onto the surface of a to-be-measured object.

[0008] The binocular camera is configured to acquire first image data of the to-be-measured object including speckles on the surface by using the first imaging unit, and acquire second image data of the to-be-measured object including speckles on the surface by using the second imaging unit; and calculate the depth information of the to-be-measured object based on the speckles in the first image data and the speckles in the second image data, wherein the speckles exist in the common field of view of the first imaging unit and the second imaging unit.

[0009] In a possible implementation, the binocular camera is further configured to: acquire first test image data of the object to be measured including the surface including speckles by using the first imaging unit, and acquire second test image data of the object to be measured including the surface including speckles by using the second imaging unit; analyze the speckles in the first test image data and the speckles in the second test image data to obtain adjustment parameters of the speckle projection device, wherein the adjustment parameters include position adjustment parameters; and display the adjustment parameters, so that a user adjusts the position of the speckle projection device according to the position adjustment parameters in the adjustment parameters.

[0010] In a possible implementation, the speckle projection device includes a plurality of projectors, and a plurality of speckles exist in the common field of view of the first imaging unit and the second imaging unit, and the adjustment parameters further include energy adjustment parameters.

[0011] The binocular camera is specifically configured to: analyze the speckles in the first test image data and the speckles in the second test image data to obtain speckle density parameters and speckle energy parameters of the speckles in each unit area region in the common field of view; determine a first projector to be adjusted in position and position adjustment parameters of the first projector according to the speckle density parameters, and determine a second projector to be adjusted in brightness and energy adjustment parameters of the second projector according to the speckle energy parameters; and display the adjustment parameters, so that a user adjusts the position of the first projector according to the position adjustment parameters and adjusts the projection light brightness of the second projector according to the energy adjustment parameters.

[0012] In a possible implementation, the binocular camera is further configured to: in a case where the speckle density parameters indicate that the densities of the speckles in each unit area region are all within a preset density range, and the speckle energy parameters indicate that the average brightnesses of the speckles in each unit area region are all within a preset brightness range, determine that the adjustment of the speckle projection device is completed.

[0013] In a possible implementation, the binocular camera is further configured to: acquire speckle density parameters and speckle energy parameters of the speckles in each unit area region in the common field of view; for each unit area region, calculate a reliability score of the unit area region according to the speckle density parameters and the speckle energy parameters of the speckles in the unit area region; and display the reliability scores of the unit area regions.

[0014] In a possible implementation, the binocular camera is further configured to: in a case where a ratio of a first quantity to a second quantity is greater than a preset percentage threshold, determine that the speckle projection device needs to be adjusted, wherein the first quantity is a quantity of unit area regions whose reliability scores are lower than a preset score threshold, and the second quantity is a total quantity of unit area regions in the common field of view.

[0015] In a possible implementation, the system further comprises a host computer connected with the binocular camera and the speckle projection device respectively.

[0016] The binocular camera is further configured to: acquire first test image data of the object to be measured including the speckle on the surface by using the first imaging unit, and acquire second test image data of the object to be measured including the speckle on the surface by using the second imaging unit; and send the first test image data and the second test image data to the host computer.

[0017] The host computer is configured to analyze the speckle in the first test image data and the speckle in the second test image data, to obtain an adjustment parameter of the speckle projection device, wherein the adjustment parameter comprises a position adjustment parameter; and display the adjustment parameter, so that a user adjusts the position of the speckle projection device according to the position adjustment parameter in the adjustment parameter.

[0018] In a possible implementation, the speckle projection device comprises a plurality of projectors, and a plurality of speckles exist in a common field of view of the first imaging unit and the second imaging unit, and the adjustment parameter further comprises an energy adjustment parameter.

[0019] The host computer is specifically configured to: analyze the speckle in the first test image data and the speckle in the second test image data, to obtain a speckle density parameter and a speckle energy parameter of the speckle in each unit area region in the common field of view; determine a first projector to be adjusted in position and a position adjustment parameter of the first projector according to the speckle density parameter, and determine a second projector to be adjusted in brightness and an energy adjustment parameter of the second projector according to the speckle energy parameter; and display the adjustment parameter, so that the user adjusts the position of the first projector according to the position adjustment parameter, and adjusts the projection light brightness of the second projector according to the energy adjustment parameter.

[0020] In a possible implementation, the host computer is further configured to: in a case where the speckle density parameter indicates that the density of the speckle in each unit area region is within a preset density interval, and the speckle energy parameter indicates that the average brightness of the speckle in each unit area region is within a preset brightness interval, determine that the adjustment of the speckle projection device is completed.

[0021] In a possible implementation, the host computer is further configured to: acquire the speckle density parameter and the speckle energy parameter of the speckle in each unit area region in the common field of view; for each unit area region, calculate a reliability score of the unit area region according to the speckle density parameter and the speckle energy parameter of the speckle in the unit area region; and display the reliability score of each unit area region.

[0022] In a possible implementation, the host computer is further configured to determine that the speckle projection device needs to be adjusted when the ratio of the first quantity to the second quantity is greater than a preset percentage threshold, where the first quantity is a quantity of unit-area regions with a reliability score lower than a preset score threshold, and the second quantity is a total quantity of unit-area regions in the common field of view.

[0023] In a possible implementation, the host computer is further configured to, when the image data acquisition instruction is acquired, control the speckle projection device to start projecting speckles; after a first preset time delay from the time when the image data acquisition instruction is acquired, control the binocular camera to collect image data; and after a second preset time delay from the time when the image data acquisition instruction is acquired, control the speckle projection device to be turned off, where the first preset time delay is greater than a lighting duration of the speckle projection device, and a difference between the second preset time delay and the first preset time delay is greater than a shooting duration of the binocular camera.

[0024] In a possible implementation, the speckle projection device includes one projector; and the binocular camera is specifically configured to: synthesize a plurality of first image data of the to-be-measured object with a surface including one speckle to obtain first synthesized image data including a plurality of speckles; synthesize a plurality of second image data of the to-be-measured object with a surface including one speckle to obtain second synthesized image data including a plurality of speckles; match the speckles in the first synthesized image data and the second synthesized image data to obtain a speckle matching result; and obtain depth information of the to-be-measured object according to the speckle matching result and a pre-labeled extrinsic parameter of the binocular camera.

[0025] Embodiments of the present application have the following beneficial effects:

[0026] The binocular speckle stereo camera system provided by the embodiments of the present application projects speckles to the surface of the object to be measured by using a speckle projection device, so that the speckles exist in the common field of view of the first imaging unit and the second imaging unit included in the binocular camera, then the first imaging unit is used to collect the first image data of the object to be measured including the speckles on the surface, and the second imaging unit is used to collect the second image data of the object to be measured including the speckles on the surface, so as to calculate the depth information of the object to be measured based on the speckles in the first image data and the speckles in the second image data, thereby obtaining the depth image of the object to be measured. Since the relative position of the speckle projection device and the binocular camera is adjustable, the adjustable speckle projection device makes the speckle projection angle and the projection distance not fixed, which can be adjusted according to the requirements, so as to reduce the uncontrollable quality of the obtained depth image caused by the reflection of the object to be measured itself, and then the depth image of the object to be measured is obtained based on the best imaging angle as much as possible, thereby effectively improving the quality of the obtained depth image.

[0027] Of course, implementing any product or method of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other embodiments can also be obtained by those skilled in the art based on these drawings.

[0029] Figure 1 The structure schematic diagram of the first binocular speckle stereo camera system provided by the embodiments of the present application is shown in the figure.

[0030] Figure 2 The structure schematic diagram of the second binocular speckle stereo camera system provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art based on the present application belong to the scope of protection of the present application.

[0032] Since the quality of the depth image obtained in the prior art is uncontrollable, the depth image is blurred. In order to solve this problem, the embodiments of the present application provide a binocular speckle stereo camera system.

[0033] The binocular speckle stereo camera system provided by the embodiments of the present application is described in detail below through specific examples.

[0034] Referring to Figure 1 , Figure 1 A structural schematic diagram of the first binocular speckle stereo camera system in the embodiments of the present application is provided, which includes a binocular camera and a speckle projection device.

[0035] The relative position of the speckle projection device and the binocular camera is adjustable.

[0036] The speckle projection device is configured to project speckles to the surface of the object to be measured.

[0037] The binocular camera is configured to acquire first image data of the object to be measured with the surface including speckles by using the first imaging unit, and acquire second image data of the object to be measured with the surface including speckles by using the second imaging unit; and calculate the depth information of the object to be measured based on the speckles in the first image data and the speckles in the second image data.

[0038] The first imaging unit and the second imaging unit have a common field of view in which speckles exist. The speckles can cover part or all of the common field of view.

[0039] The object to be measured can be any object in a three-dimensional space that needs to acquire a depth image. The speckles are laser spots projected by the speckle projection device. The binocular camera acquires an image of the object with the surface including speckles, and the speckles on the surface of the object in the image are registered and analyzed, so as to obtain the depth information of the object.

[0040] When measuring the depth of the surface of the object to be measured, the speckle projection device first projects speckles to the surface of the object to be measured, so that the surface of the object to be measured presents the projection points of the speckles, and speckles also exist in the common field of view of the first imaging unit and the second imaging unit of the binocular camera. Because the textures of different parts of the surface of the object to be measured are different, and the distances between different positions on the surface of the object to be measured and the binocular camera can also be different, the speckle conditions of different parts of the surface of the object to be measured are different, so that the speckles of the object to be measured in the common field of view are also different. The first imaging unit and the second imaging unit acquire images of the object to be measured with the surface including speckles, and obtain first image data and second image data of the object to be measured, respectively.

[0041] Since the positions and angles at which the first imaging unit and the second imaging unit acquire images are different, the acquisition angles of the first image data and the second image data of the to-be-measured object including the speckle on the surface are different, and the speckles included in the first image data and the second image data are matched and analyzed and calculated, so that the depth information of the to-be-measured object can be determined, and then the depth image of the to-be-measured object is obtained. The algorithm for calculating the depth of an object based on a speckle and a binocular camera can be referred to the prior art, and is not specifically limited in the present application.

[0042] The speckle projection device can include one or more projectors, and when the speckle projection device projects the speckle, one projector can be turned on to project the speckle alone, or multiple projectors can be turned on to project the speckle together, which are all within the protection scope of the present application.

[0043] In an embodiment of the present application, the speckle projection device includes multiple projectors, and multiple speckles exist in the common field of view of the first imaging unit and the second imaging unit.

[0044] When multiple projectors are turned on to project the speckle together, multiple speckles exist in the common field of view of the first imaging unit and the second imaging unit. The number of image data acquired by the first imaging unit and the second imaging unit can be determined according to the number of projectors, or can be determined according to the needs of a user.

[0045] In an embodiment of the present application, the speckle projection device includes one projector, and the binocular camera is specifically configured to: synthesize multiple pieces of first image data of the to-be-measured object including one speckle on the surface to obtain first synthesized image data including multiple speckles; synthesize multiple pieces of second image data of the to-be-measured object including one speckle on the surface to obtain second synthesized image data including multiple speckles; match the speckles in the first synthesized image data and the second synthesized image data to obtain a speckle matching result; and obtain the depth information of the to-be-measured object according to the speckle matching result and the pre-calibrated extrinsic parameters of the binocular camera.

[0046] If the speckle projection device only includes one projector, the surface of the object to be measured only includes one speckle. At this time, the binocular camera can collect multiple image data of the object to be measured including one speckle on the surface, that is, the first imaging unit can collect multiple first image data of the object to be measured including one speckle on the surface, and the second imaging unit can collect multiple second image data of the object to be measured including one speckle on the surface; the multiple first image data and the multiple second image data are synthesized respectively to obtain first synthesized image data and second synthesized image data respectively including multiple speckles, and the multiple speckles respectively included in the two are matched to obtain a speckle matching result, which represents the information of the surface speckle of the object to be measured. According to the obtained speckle matching result and the pre-calibrated external parameters of the binocular camera, the depth information of the object to be measured can be obtained.

[0047] In one example, the binocular camera and the speckle projection device can be independently installed or connected by an external data line. When they are independently installed, the speckle projection device can be turned on in a constant light mode so that there is always a projected speckle in the common field of view of the binocular camera. When they are connected by an external data line, the speckle projection device and the binocular camera can be turned on synchronously so that there is always a projected speckle in the common field of view of the binocular camera only when the binocular camera collects image data.

[0048] As can be seen from the above, the binocular speckle stereo camera system provided by the embodiments of the present application uses the speckle projection device to project speckles onto the surface of the object to be measured, so that there are speckles in the common field of view of the first imaging unit and the second imaging unit of the binocular camera. Then, the first imaging unit collects first image data of the object to be measured including speckles on the surface, and the second imaging unit collects second image data of the object to be measured including speckles on the surface. Thus, based on the speckles in the first image data and the speckles in the second image data, the depth information of the object to be measured is calculated to obtain a depth image of the object to be measured. Since the relative position of the speckle projection device and the binocular camera is adjustable, the adjustable speckle projection device makes the projection angle and the projection distance of the speckles not fixed, which can be adjusted according to the requirements, so as to reduce the uncontrollable quality of the obtained depth image caused by the reflection of the object to be measured itself, and further obtain the depth image of the object to be measured based on the best imaging angle, thereby effectively improving the quality of the obtained depth image.

[0049] In one embodiment of the present application, the binocular camera is further configured to: acquire first test image data of the object to be measured with the first imaging unit, and acquire second test image data of the object to be measured with the second imaging unit; analyze the speckles in the first test image data and the second test image data to obtain adjustment parameters of the speckle projection device. The adjustment parameters include position adjustment parameters, and the adjustment parameters are displayed to allow a user to adjust the position of the speckle projection device according to the position adjustment parameters.

[0050] After the first imaging unit acquires the first test image data of the object to be measured with the first imaging unit, and the second imaging unit acquires the second test image data of the object to be measured with the second imaging unit, the speckles in the first test image data and the second test image data are obtained. Analyzing the speckles can obtain adjustment parameters of the speckle projection device. The adjustment parameters can be used to indicate the adjustment of the speckle projection device, and the speckle projection device can be adjusted.

[0051] As mentioned above, the relative position of the speckle projection device and the binocular camera is adjustable, and the position adjustment parameters included in the adjustment parameters are used to indicate whether the position of the speckle projection device is reasonable. The user can determine whether the position of the speckle projection device needs to be adjusted and how to adjust the position of the speckle projection device relative to the binocular camera or the object to be measured according to the position adjustment parameters.

[0052] In one example, the projected speckles need to be uniformly distributed. If the position of the speckle projection device is not appropriate, the projected speckles will not be uniformly distributed. Therefore, after analyzing the speckles in the first test image data and the second test image data, the user can adjust the position of the speckle projection device according to the obtained position adjustment parameters, so that the projected speckles can meet the requirements.

[0053] For example, when the overall density of the speckles on the surface of the object to be measured is less than expected, the speckle projection device can be placed close to the object to be measured. When the overall density of the speckles on the surface of the object to be measured is greater than expected, the speckle projection device can be placed away from the object to be measured. When the density of the speckles in the unit area on the left side of the surface of the object to be measured is less than expected, and the density of the speckles in the unit area on the right side of the surface of the object to be measured is greater than expected, the speckle projection device can be rotated to the left. When the density of the speckles in the unit area on the left side of the surface of the object to be measured is greater than expected, and the density of the speckles in the unit area on the right side of the surface of the object to be measured is less than expected, the speckle projection device can be rotated to the right, and so on.

[0054] It can be seen from the above that the binocular speckle stereo camera system provided by the embodiments of the present application can analyze the position adjustment parameters of the speckle projection device according to the speckles in the first test image data and the second test image data collected by the first imaging unit and the second imaging unit, and since the relative positions between the speckle projection device and the binocular camera are adjustable, the speckles projected on the surface of the object to be measured can be adjusted to meet the requirements, and thus a depth image with more stable quality can be obtained.

[0055] In an embodiment of the present application, the speckle projection device comprises a plurality of projectors, and there are a plurality of speckles in the common field of view of the first imaging unit and the second imaging unit, and the adjustment parameters further comprise energy adjustment parameters; the binocular camera is specifically configured to: analyze the speckles in the first test image data and the speckles in the second test image data to obtain speckle density parameters and speckle energy parameters of the speckles in each unit area region in the common field of view; determine the first projector to be adjusted in position and the position adjustment parameters of the first projector according to the speckle density parameters, and determine the second projector to be adjusted in brightness and the energy adjustment parameters of the second projector according to the speckle energy parameters; and display the adjustment parameters, so that a user adjusts the position of the first projector according to the position adjustment parameters and adjusts the projection light brightness of the second projector according to the energy adjustment parameters.

[0056] It is mentioned above that there are speckles in the common field of view, and after the first imaging unit collects the first test image data of the object to be measured including speckles on the surface and the second imaging unit collects the second test image data of the object to be measured including speckles on the surface, the speckles in the first test image data and the second test image data are analyzed, and speckle density parameters and speckle energy parameters of the speckles in each unit area region in the common field of view of the first imaging unit and the second imaging unit can also be obtained.

[0057] The division of the unit area region can be self-defined according to actual conditions. In one example, the common field of view can be divided into n*m square regions with equal areas, each of which is a unit area region, where n and m are preset integers. The speckle density parameter is used to represent the density information of the projected speckles. It can be understood that when the position of the speckle projection device changes, the density of the speckles projected on the surface of the object to be measured will also change. For example, if the distance between the position of the speckle projection device and the object to be measured becomes farther, the density of the speckles on the surface of the object to be measured will become smaller. For example, when the angle between the projection angle of the speckle projection device and the surface of the object to be measured changes, the density of the speckles in different regions on the surface of the object to be measured will change. Therefore, based on the determined speckle density parameter, the first projector to be adjusted in position can be determined, and then the position adjustment parameter of the first projector is determined, and then the position of the speckle projection device is adjusted based on the position adjustment parameter, for example, the distance between the speckle projection device and the object to be measured is changed or the projection angle is changed.

[0058] In the process of determining the depth information, it is expected that the density of the projected speckles is uniform. Therefore, the purpose of adjusting the position of the speckle projection device according to the position adjustment parameter of the speckle projection device is to make the density of the projected speckles meet the preset density requirement. In one example, the final purpose of the position adjustment of the speckle projection device is to make the density of the speckles in each unit area region on the surface of the object to be measured within the preset density range.

[0059] For example, for any unit area region, when the speckle density parameter of the unit area region does not meet the preset density condition, the projector corresponding to the unit area region is determined as the first projector. The preset density condition can be self-defined according to actual conditions. For example, the density of the speckles in the unit area region is not within the preset density range.

[0060] For example, for any unit area region, when the density of the speckles on the surface of the object to be measured in the unit area region is less than the expectation, the projector corresponding to the unit area region can be prompted to be close to the object to be measured. When the density of the speckles on the surface of the object to be measured in the unit area region is greater than the expectation, the projector corresponding to the unit area region can be prompted to be away from the object to be measured. For example, for two adjacent first unit area region and second unit area region, when the density of the speckles in the first unit area region is less than the expectation and the density of the speckles in the second unit area region is greater than the expectation, the projection angle of the projector corresponding to the first unit area region can be prompted to be adjusted to be close to the first unit area region, and the projection angle of the projector corresponding to the second unit area region can be prompted to be adjusted to be close to the first unit area region.

[0061] For example, for any unit area region, when the density of speckle in the left unit area region of the surface of the object to be measured is less than expected and the density of speckle in the right unit area region of the surface of the object to be measured is greater than expected, it can be prompted to rotate the corresponding projector in the unit area region to the left; when the density of speckle in the left unit area region of the surface of the object to be measured is greater than expected and the density of speckle in the right unit area region of the surface of the object to be measured is less than expected, it can be prompted to rotate the corresponding projector in the unit area region to the right, and the like.

[0062] The speckle energy parameter is used to represent the energy information of the projected speckle, that is, the light intensity of the speckle on the surface of the object to be measured. When the light intensity of the speckle on the surface of the object to be measured is very low, it can cause errors in the depth information due to missed detection of the speckle; when the light intensity of the speckle on the surface of the object to be measured is very high, it can cause errors in the depth information due to overexposure of the speckle. Therefore, it is desirable that the light intensity of the speckle on the surface of the object to be measured be within a suitable range, that is, the average brightness of each speckle represented by the speckle energy parameter be within a preset brightness interval.

[0063] The brightness of the speckle projected by the speckle projection device in the image can be controlled by the exposure time, and therefore, in one example, the purpose of the brightness adjustment by the projection device is to cause the errors in the brightness of the speckle projected by each projector to be within a preset brightness error range, that is, to cause the brightness of the speckle projected by each projector to be substantially the same.

[0064] In determining the second projector to be subjected to brightness adjustment according to the speckle energy parameter, in one example, the mean value of the brightness of the speckle projected by each projector can be calculated, and the projector whose speckle brightness has a difference from the mean value greater than a preset brightness threshold is taken as the second projector. The energy adjustment parameter of the second projector indicates that the brightness of the speckle projected by the second projector is adjusted to the mean value.

[0065] In one example, when the image is captured by the binocular camera, the exposure parameter of the binocular camera can also be adjusted according to the mean value to cause the brightness of the speckle in the captured image in the binocular camera to be within expectation.

[0066] In the embodiments of the present application, the preset brightness interval and the preset density interval can be empirical values or experimental values, which need to be determined according to the actual performance of the binocular camera and the speckle projection device.

[0067] As can be seen from the above, the binocular speckle stereo camera system provided by the embodiment of the present application can analyze the position adjustment parameter and the energy adjustment parameter of the speckle projection device according to the speckles in the first test image data and the second test image data collected by the first imaging unit and the second imaging unit, and adjust the position and the energy of the speckle projection device according to the adjustment parameter. Compared with the prior art, when the binocular camera and the speckle projection device are at the same working distance or have the same height to the measured surface, the energy density remains unchanged after the model of the speckle projection device is determined. When the energy of the interference light is relatively strong, the speckles are easily covered by the interference light because the energy is too low. There is always a situation that some reflective surfaces cannot be normally imaged due to the fixed spatial position relationship. The speckle projection device provided by the embodiment of the present application can reduce the distance between the measured object and the projector, improve the energy density, and ensure imaging. Moreover, the projection angle of the speckles is not fixed, and the optimal imaging angle can be freely adjusted according to the reflection of the measured object, so that the density and the brightness of the projected speckles on the surface of the measured object are more in line with the requirements, and a depth image with higher quality is obtained.

[0068] In an embodiment of the present application, the binocular camera is further configured to determine that the adjustment of the speckle projection device is completed when the speckle density parameter indicates that the density of the speckles in each unit area region is within a preset density interval, and the speckle energy parameter indicates that the average brightness of the speckles in each unit area region is within a preset brightness interval.

[0069] It is mentioned above that the position and the energy of the speckle projection device are adjusted according to the determined density adjustment parameter and the energy adjustment parameter, so that the density and the brightness of the projected speckles meet the requirements. In the embodiment, the position adjustment of the speckle projection device is determined to be completed when the speckle density parameter indicates that the density of the speckles in each unit area region is within a preset density interval, and the speckle energy parameter indicates that the average brightness of the speckles in each unit area region is within a preset brightness interval. After the density adjustment parameter and the energy adjustment parameter are determined, the user adjusts the position and the energy of the speckle projection device until the density of the speckles projected by the speckle projection device meets the requirement that the density of the speckles in each unit area region is within a preset density interval, and the energy of the speckles meets the requirement that the average brightness of the speckles in each unit area region is within a preset brightness interval, and then the adjustment of the speckle projection device is completed.

[0070] The preset density interval is a preset interval value, which indicates that when the value of the speckle density is within the interval, the user analyzes the speckles projected on the surface of the measured object, and can obtain accurate depth information of the measured object.

[0071] The preset brightness interval is also a preset interval value, and indicates that when the average brightness value of the speckle is in the interval, the user can obtain accurate depth information of the to-be-measured object after analyzing the speckle projected on the surface of the to-be-measured object.

[0072] In an embodiment of the present application, the binocular camera is further configured to: acquire the speckle density parameter and the speckle energy parameter of the speckle in each unit area region in the common field of view; for each unit area region, calculate a reliability score of the unit area region according to the speckle density parameter and the speckle energy parameter of the speckle in the unit area region; and display the reliability scores of the unit area regions.

[0073] After the speckle density parameter and the speckle energy parameter of the speckle in each unit area region in the common field of view are acquired, for each unit area region, the reliability score of the unit area region can be calculated according to the speckle density parameter and the speckle energy parameter of the speckle in the unit area region. The reliability score is determined by the speckle density parameter and the speckle energy parameter of the speckle in the unit area region. The specific weight ratio or calculation method can be determined according to actual needs.

[0074] In an embodiment of the present application, the binocular camera is further configured to: in a case where the ratio of the first quantity to the second quantity is greater than a preset percentage threshold, determine that the speckle projection device needs to be adjusted.

[0075] The first quantity is the quantity of the unit area regions whose reliability scores are lower than a preset score threshold, and the second quantity is the total quantity of the unit area regions in the common field of view.

[0076] For each unit area region, a reliability score is calculated. The unit area region whose reliability score is lower than a preset score threshold is regarded as a low-score unit area region. If the ratio of the quantity of the low-score unit area regions to the total quantity of the unit area regions is greater than a preset percentage threshold, it indicates that the overall reliability of the common field of view is low, and it is determined that the speckle projection device needs to be adjusted. Specifically, the preset score threshold and the preset percentage threshold can be preset according to actual needs.

[0077] As can be seen from the above, the binocular speckle stereo camera system provided in the embodiments of the present application can adjust the speckle projection device according to the determined speckle density parameter and speckle energy parameter until the speckle projected by the speckle projection device can meet the needs, so that the quality of the depth image of the to-be-measured object collected is good. The adjustable speckle projection device based on specific adjustment values can make the quality of the depth image collected have high stability.

[0078] Referring to Figure 2 , Figure 2A second binocular speckle stereo camera system structure schematic diagram provided by the embodiment of the application, the system further comprises: a host computer, the host computer is connected with the binocular camera, the speckle projection device respectively;

[0079] The binocular camera is further used for: collecting first test image data of the to-be-measured object including the surface including speckles by using the first imaging unit, and collecting second test image data of the to-be-measured object including the surface including speckles by using the second imaging unit; and sending the first test image data and the second test image data to the host computer;

[0080] The host computer is used for: analyzing the speckles in the first test image data and the speckles in the second test image data, to obtain adjustment parameters of the speckle projection device, wherein the adjustment parameters include position adjustment parameters; and displaying the adjustment parameters, so that a user adjusts the position of the speckle projection device according to the position adjustment parameters in the adjustment parameters.

[0081] In the embodiment of the application, after the speckle projection device projects speckles into the common field of view of the to-be-measured object and the first imaging unit and the second imaging unit, the first imaging unit collects first test image data of the to-be-measured object including the surface including speckles, the second imaging unit collects second test image data of the to-be-measured object including the surface including speckles, and the first test image data and the second test image data are both sent to the host computer. The host computer analyzes the speckles in the first test image data and the speckles in the second test image data, to obtain position adjustment parameters included in the adjustment parameters of the speckle projection device, which are displayed to the user. At this time, the user can adjust the position of the speckle projection device according to the obtained position adjustment parameters.

[0082] As mentioned above, the speckle projection device can include one or more projectors, and the speckle projection device can project speckles by lighting only one projector or simultaneously lighting multiple speckle projectors. The host computer can simultaneously control the on-off state of the speckle projection device and the binocular camera, that is, control the lighting time of the speckle projection device and the exposure time of the binocular camera. Specifically, the host computer can control the speckle projection device to project speckles when the binocular camera is turned on, and the lighting time of the speckle projection device is not less than the exposure time of the binocular camera, to ensure that the image collected by the binocular camera can all have speckles meeting the requirements for analysis.

[0083] As can be seen from the above, the binocular speckle stereo camera system provided by the embodiment of the application, the host computer is connected with the speckle projection device and the binocular camera respectively and controls both, which can make the scheme of the binocular camera collecting depth images with the support of the speckle projection device more flexible, so as to obtain test image data meeting the requirements and further obtain depth images with stable quality on the premise of reducing the waste of speckle projection resources.

[0084] In one embodiment of the present application, the speckle projection device includes a plurality of projectors, and there are a plurality of speckles in the common field of view of the first imaging unit and the second imaging unit, and the adjustment parameters further include an energy adjustment parameter;

[0085] The host computer is specifically configured to: analyze the speckles in the first test image data and the speckles in the second test image data to obtain speckle density parameters and speckle energy parameters of the speckles in each unit area region in the common field of view; determine a first projector to be adjusted in position and a position adjustment parameter of the first projector according to the speckle density parameters, and determine a second projector to be adjusted in brightness and an energy adjustment parameter of the second projector according to the speckle energy parameters; and display the adjustment parameters, so that a user adjusts the position of the first projector according to the position adjustment parameter and adjusts the projection light brightness of the second projector according to the energy adjustment parameter.

[0086] As mentioned above, after the first imaging unit collects the first test image data of the to-be-measured object with a surface including speckles, and the second imaging unit collects the second test image data of the to-be-measured object with a surface including speckles, the first test image data and the second test image data not only include the to-be-measured object with a surface including speckles, but also include the image of the common field of view in which the to-be-measured object is located. At this time, the host computer analyzes the speckles in the first test image data and the second test image data to obtain speckle density parameters and speckle energy parameters of the speckles in the common field of view of the first imaging unit and the second imaging unit. The host computer can determine the first projector to be adjusted in position based on the determined speckle density parameters, and further determine the position adjustment parameter of the first projector, i.e., determine the information that the position of the speckle projection device needs to be adjusted, and then adjust the position of the speckle projection device based on this. Then the host computer adjusts the position of the speckle projection device according to the position adjustment parameter of the speckle projection device, so that the density of the projected speckles can meet the requirements.

[0087] Similarly, the host computer can determine the second projector to be adjusted in position based on the determined speckle energy parameters, and further determine the energy adjustment parameter of the second projector, i.e., determine the information that the energy of the speckle projection device needs to be adjusted, and then adjust the energy of the speckle projection device based on this. Then the host computer adjusts the energy of the speckle projection device according to the energy adjustment parameter of the speckle projection device, so that the projection light brightness of the projected speckles can meet the requirements.

[0088] In one example, when the speckle projection device has only one projector, the light intensity of the speckle in the image data can be adjusted by adjusting the energy of the speckle projection device, and the light intensity of the speckle in the image data can also be adjusted by adjusting the exposure parameters of the binocular camera. Therefore, when the speckle projection device has only one projector, after obtaining the energy adjustment parameters, the host computer can determine the exposure adjustment parameters corresponding to the adjustment effect of the binocular camera based on the energy adjustment parameters, and then adjust the exposure parameters of the binocular camera based on the exposure adjustment parameters, and the imaging units in the binocular camera adjusted based on the exposure parameters continue to collect images.

[0089] As can be seen from the above, the binocular speckle stereo camera system provided by the embodiments of the present application can analyze the speckle in the first test image data and the second test image data collected by the first imaging unit and the second imaging unit to obtain the position adjustment parameters and the energy adjustment parameters of the speckle projection device, and adjust the position and the energy of the speckle projection device based on the adjustment parameters, so that the density of the speckle projected on the surface of the object to be measured and the brightness of the projected light are more in line with the requirements, and a depth image with more stable quality is obtained.

[0090] In one embodiment of the present application, the host computer is further configured to determine that the adjustment of the speckle projection device is completed when the speckle density parameters indicate that the density of the speckle in each of the unit area regions is within the preset density interval, and the speckle energy parameters indicate that the average brightness of the speckle in each of the unit area regions is within the preset brightness interval.

[0091] As mentioned above, the host computer can adjust the position and the energy of the speckle projection device based on the determined density adjustment parameters and energy adjustment parameters, so that the density of the projected speckle and the brightness of the projected light meet the requirements. In this embodiment, the host computer adjusts the position and the energy of the speckle projection device based on the density adjustment parameters and the energy adjustment parameters until the density of the speckle projected by the speckle projection device meets the requirement that the density of the speckle in each of the unit area regions is within the preset density interval, and the energy of the speckle meets the requirement that the average brightness of each speckle is within the preset brightness interval, and then the adjustment of the speckle projection device by the host computer is completed.

[0092] The preset density interval is a preset interval value, which indicates that when the value of the speckle density is within this interval, the host computer can obtain accurate depth information of the object to be measured after analyzing the speckle projected on the surface of the object to be measured.

[0093] The preset brightness interval is also a preset interval value, which indicates that when the value of the average brightness of the speckle is within this interval, the host computer can obtain accurate depth information of the object to be measured after analyzing the speckle projected on the surface of the object to be measured.

[0094] In one embodiment of the present application, the host computer is further configured to: obtain speckle density parameters and speckle energy parameters of speckles in each unit area region in the common field of view; for each unit area region, calculate a reliability score of the unit area region according to the speckle density parameters and the speckle energy parameters of the speckles in the unit area region; and display the reliability scores of the unit area regions.

[0095] In one embodiment of the present application, the host computer is further configured to: determine that the speckle projection device needs to be adjusted when the ratio of the first number to the second number is greater than a preset percentage threshold.

[0096] The first number is the number of unit area regions whose reliability scores are lower than a preset score threshold, and the second number is the total number of unit area regions in the common field of view.

[0097] As can be seen from the above, the binocular speckle stereo camera system provided by the embodiments of the present application, the host computer adjusts the speckle projection device according to the determined speckle density parameters and speckle energy parameters until the speckles projected by the speckle projection device can meet the requirements, so that the quality of the depth image of the measured object collected is better, and the adjustable speckle projection device based on specific adjustment values can make the quality of the collected depth image have high stability.

[0098] In one embodiment of the present application, the host computer is further configured to: when an image data acquisition instruction is obtained, control the speckle projection device to start projecting speckles; after a first preset delay from the time when the image data acquisition instruction is obtained, control the binocular camera to collect image data; and after a second preset delay from the time when the image data acquisition instruction is obtained, turn off the speckle projection device.

[0099] The first preset delay is greater than the lighting duration of the speckle projection device, and the difference between the second preset delay and the first preset delay is greater than the shooting duration of the binocular camera.

[0100] As mentioned above, the host computer can control the on-off state of the speckle projection device and the binocular camera, so the host computer can control the speckle projection device to start projecting speckles when the image data acquisition instruction of the user is obtained, and then control the binocular camera to start collecting image data after a first preset delay after the speckle projection device starts projecting speckles; and control the speckle projection device to be turned off after a second preset delay from the time when the image data acquisition instruction of the user is obtained.

[0101] The first preset delay is a preset delay time, which is greater than the lighting duration of the speckle projection device, so as to ensure that the binocular camera starts to collect images only after the speckle projection device normally projects speckles. The second preset delay is a preset delay time, and the difference between the first preset delay and the second preset delay is greater than the shooting duration of the binocular camera, that is, the speckle projection device will be turned off only after the binocular camera finishes shooting image data. Therefore, the host computer can control the speckle projection device to project speckles meeting the requirements during the image data collection period of the binocular camera. Moreover, compared with the case that the speckle projection device is always on, the working duration of the speckle projection device can be effectively reduced, so as to reduce the energy consumption of the speckle projection device and increase the service life of the speckle projection device.

[0102] As can be seen from the above, the binocular speckle stereo camera system provided by the embodiments of the present application can control the speckle projection device to project speckles meeting the requirements during the image data collection period of the binocular camera. The collected image data can ensure that there are projected speckles, so that the collected image data can all have depth information meeting the requirements, and then a depth image with accurate depth information is obtained. Moreover, compared with the case that the speckle projection device is always on, the working duration of the speckle projection device can be effectively reduced, so as to reduce the energy consumption of the speckle projection device and increase the service life of the speckle projection device.

[0103] It should be noted that, in this document, the terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0104] The above description is only the preferred embodiments of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A binocular speckle stereoscopic camera system, characterized by, The system comprises: A binocular camera and a speckle projection device, wherein the relative position of the speckle projection device and the binocular camera is adjustable; the speckle projection device comprises a plurality of projectors, and a plurality of speckles exist in the common field of view of the first imaging unit and the second imaging unit; The speckle projection device is configured to project speckles onto the surface of the object to be measured. The binocular camera is configured to: acquire first test image data of the object to be measured including speckles on the surface by using the first imaging unit, acquire second test image data of the object to be measured including speckles on the surface by using the second imaging unit, analyze the speckles in the first test image data and the speckles in the second test image data to obtain speckle density parameters and speckle energy parameters of the speckles in each unit area region in the common field of view, determine a first projector to be adjusted in position and a position adjustment parameter of the first projector according to the speckle density parameters, determine a second projector to be adjusted in brightness and an energy adjustment parameter of the second projector according to the speckle energy parameters, display the adjustment parameters to enable a user to adjust the position of the first projector according to the position adjustment parameter and adjust the projection light brightness of the second projector according to the energy adjustment parameter, wherein the adjustment parameters comprise the position adjustment parameter and the energy adjustment parameter, acquire first image data of the object to be measured including speckles on the surface by using the first imaging unit, acquire second image data of the object to be measured including speckles on the surface by using the second imaging unit, and calculate the depth information of the object to be measured based on the speckles in the first image data and the speckles in the second image data.

2. The system of claim 1, wherein, The binocular camera is further configured to: determine that the adjustment of the speckle projection device is completed when the speckle density parameters of the speckles in each unit area region are within a preset density range and the speckle energy parameters of the speckles in each unit area region are within a preset brightness range.

3. The system of claim 1, wherein, The binocular camera is further configured to: acquire the speckle density parameters and the speckle energy parameters of the speckles in each unit area region in the common field of view, calculate a reliability score of each unit area region according to the speckle density parameters and the speckle energy parameters of the speckles in the unit area region, and display the reliability scores of the unit area regions.

4. The system of claim 3, wherein, The binocular camera is further configured to: determine that the speckle projection device needs to be adjusted when the ratio of a first number to a second number is greater than a preset percentage threshold, wherein the first number is the number of unit area regions with reliability scores lower than a preset score threshold, and the second number is the total number of unit area regions in the common field of view.

5. A binocular speckle stereoscopic camera system, characterized by, The system comprises: A host computer, a binocular camera, and a speckle projection device, the binocular camera comprising a first imaging unit and a second imaging unit, wherein a relative position of the speckle projection device and the binocular camera is adjustable; the host computer is connected with the binocular camera and the speckle projection device respectively; the speckle projection device comprises a plurality of projectors, and a plurality of speckles exist in a common field of view of the first imaging unit and the second imaging unit; The speckle projection device is configured to project speckles to a surface of an object to be measured. The binocular camera is configured to acquire first test image data of the object to be measured including the surface including speckles by using the first imaging unit, and acquire second test image data of the object to be measured including the surface including speckles by using the second imaging unit; and send the first test image data and the second test image data to the host computer. The host computer is configured to analyze the speckles in the first test image data and the speckles in the second test image data to obtain speckle density parameters and speckle energy parameters of speckles in each unit area region in the common field of view; determine a first projector to be adjusted in position and a position adjustment parameter of the first projector according to the speckle density parameters, and determine a second projector to be adjusted in brightness and an energy adjustment parameter of the second projector according to the speckle energy parameters; and display the adjustment parameters, so that a user adjusts the position of the first projector according to the position adjustment parameter and adjusts the projection light brightness of the second projector according to the energy adjustment parameter, wherein the adjustment parameters comprise the position adjustment parameter and the energy adjustment parameter. The binocular camera is further configured to acquire first image data of the object to be measured including the surface including speckles by using the first imaging unit, and acquire second image data of the object to be measured including the surface including speckles by using the second imaging unit; and calculate depth information of the object to be measured based on the speckles in the first image data and the speckles in the second image data.

6. The system of claim 5, wherein, The host computer is further configured to determine that the speckle projection device is adjusted completely, in a case where the speckle density parameters represent that the densities of the speckles in each unit area region are all within a preset density interval, and the speckle energy parameters represent that the average brightnesses of the speckles in each unit area region are all within a preset brightness interval.

7. The system of claim 5, wherein, The host computer is further configured to acquire the speckle density parameters and the speckle energy parameters of the speckles in each unit area region in the common field of view; for each unit area region, calculate a reliability score of the unit area region according to the speckle density parameter and the speckle energy parameter of the speckles in the unit area region; and display the reliability scores of the unit area regions.

8. The system of claim 5, wherein, The host computer is further configured to determine that the speckle projection device needs to be adjusted, in a case where a ratio of a first number and a second number is greater than a preset percentage threshold, wherein the first number is a number of unit area regions whose reliability scores are lower than a preset score threshold, and the second number is a total number of unit area regions in the common field of view.

9. The system of claim 5, wherein, The host computer is further configured to: control the speckle projection device to start projecting speckles when the image data acquisition instruction is acquired; control the binocular camera to collect image data after a first preset delay from the time when the image data acquisition instruction is acquired; and turn off the speckle projection device after a second preset delay from the time when the image data acquisition instruction is acquired, wherein the first preset delay is greater than the lighting duration of the speckle projection device, and the difference between the second preset delay and the first preset delay is greater than the shooting duration of the binocular camera.

Images