A TOF camera ambient light compensation auxiliary device, compensation method, and calibration method

Abstract

The present invention discloses an ambient light compensation auxiliary device, a compensation method and a calibration method for a TOF camera. Among them, the auxiliary device includes a device body, a host computer and a control module. The device body is of a square closed cavity structure. Inside the device body, there are a control box, a fixing box, a calibration box and a target box. A fixing device is installed in the fixing box, and a first through hole is provided on the side wall of the fixing box adjacent to the calibration box. The inner wall of the calibration box is coated with a light-absorbing material layer, and a supplementary light lamp is installed on the top of the calibration box. A target surface is rotatably installed in the target box. The host computer and the control module are installed in the control box. The input end of the control module is connected to the host computer, and its output end is connected to the supplementary light lamp. By using the auxiliary device of the present application, the camera is convenient to fix, the positions of the camera and the target surface are relatively fixed, reducing the installation error. By coating the inner wall of the calibration box with a light-absorbing material layer, the reflection of light on the inner wall is prevented from introducing errors.

Description

Technical Field

[0001] The present invention relates to the technical field of ambient light calibration of TOF cameras, and particularly relates to an ambient light compensation auxiliary device, a compensation method and a calibration method for a TOF camera. Background Art

[0002] 3D cameras based on the TOF (Time-Of-Flight) theory are one of the research hotspots in current technologies. TOF 3D cameras can simultaneously acquire grayscale images and distance images, and have gradually appeared in system fields such as people flow statistics, 3D measurement, gesture control, 3D modeling, automotive radar, and robot vision.

[0003] TOF camera technology is a relatively prominent one among many optical 3D measurement technologies. Its basic principle is that the light emitted by an active light source is reflected by an object to be photographed and then captured by the TOF camera. Further, the distance between the object to be photographed and the TOF camera is calculated based on the time or phase difference from the emission to the capture of the light. And while obtaining the depth information of the object to be photographed, the TOF camera also captures grayscale information. However, there are currently some systematic errors in TOF cameras, such as systematic errors caused by signal ripple, pixel process, signal delay, temperature drift, ambient light, etc.; among them, the intensity of outdoor ambient light (the light in the natural spectrum contains the light band of the camera light source) has a great impact on the measurement accuracy of TOF cameras. In some outdoor working scenarios in real use, the light intensity is stronger than that indoors and the light intensity is variable, which makes some cameras perform well indoors but have poor ranging effects outdoors.

[0004] How to perform ambient light compensation through ambient light calibration when the camera is working outdoors is particularly important for a TOF camera.

[0005] Currently, for ambient light compensation of TOF cameras, the TOF camera is usually placed in a darkroom, and external light sources are provided to simulate natural light. An object at a fixed distance is photographed. By turning on and off the external light sources, the distance change value is obtained, and combined with corresponding algorithms, the distance influence brought is compensated. The traditional method has the following several disadvantages:

[0006] 1. It is not convenient to fix. When the TOF camera changes the shooting distance, it is necessary to manually move the position of the photographed object or the camera. At the same time, when fixing the camera and the target object, installation errors are easily introduced.

[0007] 2. High cost. The construction of an optical darkroom environment requires special treatment of the darkroom walls, etc.

[0008] 3. Low efficiency. After testing the distance under a set of TOF camera intensities, it is necessary to manually adjust the light intensity setting. Summary of the Invention

[0009] Aiming at the defects in the prior art, the purpose of the present invention is to provide an ambient light compensation auxiliary device for a TOF camera.

[0010] Furthermore, the present application also provides a method for ambient light compensation of a TOF camera.

[0011] Furthermore, the present application also provides a calibration method for a TOF camera.

[0012] The technical solution adopted by the present invention is as follows:

[0013] An ambient light compensation auxiliary device for a TOF camera, comprising a device body, a host computer and a control module. The device body is a square closed cavity structure. Inside the device body, there are a control box, a fixing box, a calibration box and a target box. The calibration box is arranged between the fixing box and the target box. A fixing device for fixing the TOF camera is installed in the fixing box. A first through hole for the lens of the TOF camera to pass through and extend into the calibration box is provided on the side wall of the fixing box adjacent to the calibration box. The inner wall of the calibration box is coated with an absorbent material layer, and a supplementary light lamp is installed on the top of the calibration box. A target surface is rotatably installed in the target box. A second through hole for the target surface to be exposed is provided on the side wall of the target box adjacent to the calibration box. The host computer and the control module are installed in the control box. The input end of the control module is connected to the host computer, and its output end is connected to the supplementary light lamp.

[0014] Furthermore, the center of the target surface is aligned with the center of the first through hole.

[0015] Furthermore, there are multiple target surfaces, and the multiple target surfaces are installed in the target box at intervals.

[0016] Furthermore, a rotation driving device is also installed in the target box. A connecting arm is provided at the lower end of the target surface. The connecting arm is fixedly installed on the output shaft of the rotation driving device and is driven to rotate by the rotation driving device. The input end of the rotation driving device is connected to the control module.

[0017] Furthermore, the rotation driving device is a motor and a reduction box. The output shaft of the motor is connected to the input shaft of the reduction box, and the output shaft of the reduction box is connected to the connecting arm of the target surface.

[0018] Furthermore, the present application also provides a method for ambient light compensation of a TOF camera. Using the above-mentioned auxiliary device, the TOF camera is fixedly installed in the fixing box. The lens of the TOF camera extends into the calibration box through the first through hole and is directly opposite to the target surface exposed through the second through hole. The method includes the following steps:

[0019] S1. Turn off the supplementary light source, control the camera to work in the depth mode, and obtain the first distance d1 between the camera and the first target surface;

[0020] S2. Turn on the supplementary light source, control the camera to work in the depth mode, and obtain the second distance d2 between the camera and the first target surface;

[0021] S3. Adjust the exposure time to 1 - n, repeat steps S1 and S2, and obtain the first distance and the second distance d1 n and d2 n ;

[0022] S4. Determine whether there is a deviation between the first distance d1 n and the second distance d2 n for each group of different exposure times, and record the power of the supplementary light source and the exposure time threshold corresponding to the data with deviation;

[0023] S5. Turn on the supplementary light, set the power to P, control the camera to work in the grayscale mode, and obtain the intensity information A output at this time;

[0024] S6. Adjust the power of the supplementary light from P1 to Pn, repeat step S5, and obtain n groups of intensity information data An under the supplementary light with the set power Pn;

[0025] S7. Turn on the supplementary light, control the camera to work in the depth mode, set the power of the supplementary light to P1 - Pn, and obtain multiple groups of distance data Dn between the camera and the first target surface;

[0026] S8. Use S6 and S7 to obtain the compensation coefficient k by fitting the data Dn and An.

[0027] Further, after obtaining the compensation coefficient k, the following steps are also included:

[0028] S9. Turn on the supplementary light, control the camera to work in the grayscale mode, obtain the background light intensity at this time, then switch to the depth mode, use the compensation coefficient k to process the data, and calculate the new distance value, which is the compensated distance value.

[0029] Further, in step S9, the following formula is specifically used to calculate the compensated distance value:

[0030]

[0031]

[0032] Wherein, Q = phase3 - phase1, I = phase2 - phase0, c is the speed of light, phase0 is the phase value corresponding to 0°, phase1 is the phase value corresponding to 90°, phase2 is the phase value corresponding to 180°, phase3 is the phase value corresponding to 270°, Q' and I' are the compensated phase information, which is calculated using the following formula:

[0033] Q' = Q - BG * k

[0034] I' = I - BG * k

[0035] Wherein, BG is the data output in the grayscale mode, and k is the calibrated compensation coefficient.

[0036] Furthermore, the present application also provides a TOF camera calibration method. Switch the target surface to the second target surface, collect a set of distance data, calculate the compensated distance according to the above method, compare the compensated distance with the actual distance. If the two are consistent, the calibration is successful.

[0037] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be understood through the practice of the present application. Description of the Drawings

[0038] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0039] Figure 1 It is a cross-sectional view of the TOF camera ambient light compensation auxiliary device provided in Embodiment 1 of the present application;

[0040] Figure 2 It is a schematic structural diagram of the TOF camera ambient light compensation auxiliary device provided in Embodiment 1 of the present application.

[0041] Wherein, control box 1, fixed box 2, calibration box 3, target box 4, camera 5, fill light 6, target surface 7. Detailed Embodiments

[0042] The following will describe in detail the embodiments of the technical solutions of the present invention in combination with specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and therefore are only examples and cannot be used to limit the protection scope of the present invention.

[0043] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application shall have the ordinary meanings understood by those skilled in the art to which this invention pertains.

[0044] Example 1

[0045] See Figure 1 and Figure 2 In this embodiment, an ambient light compensation assistance device for a TOF camera 5 is provided, which includes a device body, a host computer, and a control module. The device body is a square closed cavity structure. Inside the device body, there is a control box 1, a fixing box 2, a calibration box 3, and a target box 4. The calibration box 3 is arranged between the fixing box 2 and the target box 4. A fixing device for fixing the TOF camera 5 is installed inside the fixing box 2. On the side wall adjacent to the calibration box 3 of the fixing box 2, there is a first through hole for the lens of the TOF camera 5 to pass through and extend into the calibration box 3. The inner wall of the calibration box 3 is coated with a light-absorbing material layer, and a fill light 6 is installed on the top of the calibration box 3. A target surface 7 is rotatably installed inside the target box 4. On the side wall adjacent to the calibration box 3 of the target box 4, there is a second through hole for the target surface 7 to be exposed. The host computer and the control module are installed inside the control box 1. The input end of the control module is connected to the host computer, and its output end is connected to the fill light 6.

[0046] During use, the TOF camera 5 is fixedly installed inside the fixing box 2. The lens of the TOF camera 5 extends into the calibration box 3 through the first through hole and is directly opposite to the target surface 7 exposed through the second through hole. By adjusting the powers of different fill lights 6 and the exposure time of the camera 5, the distance values in the states of different fill light 6 powers and camera 5 exposure times are measured. The compensation coefficient is calculated based on these distance values, and thus the compensation distance is obtained according to the compensation coefficient. With the assistance device of this application, the camera 5 is convenient to fix, the positions of the camera 5 and the target surface 7 are relatively fixed, reducing the installation error. By coating the inner wall of the calibration box 3 with a light-absorbing material layer, the error introduced by the light emission on the inner wall is prevented.

[0047] The device body can be a square closed cavity structure. The control box 1 is located at the bottom of the device body, and the fixing box 2, the calibration box 3, and the target box 4 are all located above the control box 1. The calibration box 3 is located between the fixing box 2 and the target box 4.

[0048] The TOF camera 5 can be installed inside the fixing box 2 through the fixing device arranged inside the fixing box 2, and its lens extends into the calibration box 3 through the first through hole.

[0049] Multiple fill lights 6 can be installed on the top of the calibration box 3. Each fill light 6 is connected to the control module. The control module adjusts the power of the fill light 6 according to the instructions of the host computer.

[0050] A plurality of target surfaces 7 are rotatably installed in the target box 4. The plurality of target surfaces 7 are installed at intervals in the target box 4. The center of each target surface 7 is directly opposite to the center of the first through hole, that is, the center of each target surface 7 is directly opposite to the center of the lens of the camera 5. By arranging a plurality of target surfaces 7 at intervals, when it is necessary to adjust the distance of the target surface 7, the front target surface 7 is rotated and moved away, so that the target surface 7 located at the rear side is exposed in the lens of the camera 5 through the second through hole, avoiding the installation error caused by manually moving the camera 5 or the target surface 7.

[0051] In order to facilitate the rotation of the target surface 7, a rotation driving device is further installed in the target box 4. A connecting arm is provided at the lower end of the target surface 7. The connecting arm is fixedly installed on the output shaft of the rotation driving device and is driven to rotate by the rotation driving device. The input end of the rotation driving device is connected to the control module.

[0052] Specifically, the rotation driving device is a motor and a reduction box. The output shaft of the motor is connected to the input shaft of the reduction box, and the output shaft of the reduction box is connected to the connecting arm of the target surface 7.

[0053] During rotation, after the motor is decelerated by the reduction box, the torque is transmitted to the connecting arm, thereby driving the target surface 7 fixed on the connecting arm to rotate, realizing the adjustment of the distance between the camera 5 and the target surface 7. Of course, the start and stop of the motor are controlled by the control module according to the control instructions of the host computer.

[0054] Embodiment 2

[0055] This embodiment provides a method for compensating ambient light of a TOF camera. Using the auxiliary device described in Embodiment 2, the TOF camera is fixedly installed in the fixed box. The lens of the TOF camera extends into the calibration box through the first through hole and is directly opposite to the target surface exposed through the second through hole. The method includes the following steps:

[0056] S1. Turn off the light source of the supplementary light, control the camera to work in the depth mode, and obtain the first distance d1 between the camera and the first target surface;

[0057] S2. Turn on the light source of the supplementary light, control the camera to work in the depth mode, and obtain the second distance d2 between the camera and the first target surface;

[0058] S3. Adjust the exposure time to 1 to n, repeat steps S1 and S2, and obtain the first distance and the second distance d1 n and d2 n under different exposure times; the first distance and the second distance d1 n and d2 n under different exposure times reflect the influence of different exposure times on the distance value. When the exposure time reaches a certain value, the influence of ambient light will become larger.

[0059] S4. Determine the first distance d1 under different exposure times for each group n and the second distance d2 n to check if there is any deviation between them, and record the power of the fill light source and the exposure time threshold corresponding to the data with deviation.

[0060] During actual operation, if the background light intensity value with the camera light source turned off is lower than the threshold intensity value, no compensation is required.

[0061] S5. Turn on the fill light, set the power to P, control the camera to work in grayscale mode, and obtain the intensity information A output at this time; this loudness information is calculated from the original data output by the TOF camera in grayscale mode and reflects to a certain extent the amplitude of the signal after photoelectric conversion. The calculation method of the intensity information A is a prior art and will not be elaborated here again.

[0062] S6. Adjust the fill light power P1 to Pn, repeat step S5, and obtain n groups of intensity information data An under the fill light with the set power Pn;

[0063] S7. Turn on the fill light, control the camera to work in depth mode, set the fill light power to P1 to Pn, set the exposure time to be greater than the exposure time threshold value, and obtain multiple groups of distance data Dn between the camera and the first target surface;

[0064] S8. Using S6 and S7, obtain the compensation coefficient k by fitting the data Dn and An; the fitting can adopt linear fitting, curve fitting, linear fitting of one variable, quadratic fitting of one variable, etc. Depending on the actual effect and different sensors, the fitting method used may also be different.

[0065] After obtaining the compensation coefficient k, the following steps are also included:

[0066] S9. Turn on the fill light, control the camera to work in grayscale mode, obtain the background light intensity at this time, then switch to depth mode, and use the compensation coefficient k to process the data and calculate the new distance value, which is the compensated distance value.

[0067] Further, in step S9, the following formula is specifically used to calculate the compensated distance value:

[0068]

[0069]

[0070] Wherein, Q = phase3 - phase1, I = phase2 - phase0, c is the speed of light, f is the modulation frequency. The data output in the depth mode of the TOF sensor is the phase relationship between the transfer tube of the pixel unit and the emitted modulated laser. Four phases are collected: the photo-generated charges of 0°, 90°, 180°, and 270° are phase0, phase1, phase2, and phase3 respectively. phase0 is the phase value corresponding to 0°, phase1 is the phase value corresponding to 90°, phase2 is the phase value corresponding to 180°, and phase3 is the phase value corresponding to 270°. Q' and I' are the compensated phase information, which is calculated using the following formulas:

[0071] Q' = Q - BG * k

[0072] I' = I - BG * k

[0073] Wherein, BG is the data output in the gray scale mode, that is, the data sampled by the ADC after the photoelectric signal conversion, which can also be called intensity data or gray scale data, and k is the calibrated compensation coefficient.

[0074] Embodiment 3

[0075] In this embodiment, the present application also provides a TOF camera calibration method. Switch the target surface to the second target surface, collect a set of distance data, calculate the compensated distance according to the method described in Embodiment 2, compare the compensated distance with the actual distance. If the two are consistent, the calibration is successful.

[0076] In the present application, unless otherwise clearly specified and defined, terms such as "connected", "connected to", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0077] In the description of the present invention, a large number of specific details are set forth. However, it can be understood that the embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, systems, and technologies are not shown in detail so as not to obscure the understanding of this specification.

[0078] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples", etc. means that the specific features, systems, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, systems, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention, and they should all be covered by the scope of the claims and the specification of the present invention.

Claims

1. An ambient light compensation auxiliary device for a TOF camera, characterized in that It includes a device body, a host computer and a control module. The device body is a square closed cavity structure. Inside the device body, there is a control box, a fixing box, a calibration box and a target box. The calibration box is arranged between the fixing box and the target box. A fixing device for fixing a TOF camera is installed in the fixing box. On the side wall of the fixing box adjacent to the calibration box, there is a first through hole for the lens of the TOF camera to pass through and extend into the calibration box. The inner wall of the calibration box is coated with a light-absorbing material layer, and a fill light is installed on the top of the calibration box. A rotatable target surface is installed in the target box. There are multiple target surfaces, and the multiple target surfaces are installed in the target box at intervals. On the side wall of the target box adjacent to the calibration box, there is a second through hole for the target surface to be exposed. The host computer and the control module are installed in the control box. The input end of the control module is connected to the host computer, and its output end is connected to the fill light.

2. The ambient light compensation auxiliary device for a TOF camera according to claim 1, characterized in that The center of the target surface is directly opposite to the center of the first through hole.

3. The ambient light compensation auxiliary device for a TOF camera according to claim 1, characterized in that A rotation driving device is also installed in the target box. A connecting arm is provided at the lower end of the target surface. The connecting arm is fixedly installed on the output shaft of the rotation driving device and is driven to rotate by the rotation driving device. The input end of the rotation driving device is connected to the control module.

4. The ambient light compensation auxiliary device for a TOF camera according to claim 3, characterized in that The rotation driving device is a motor and a reduction box. The output shaft of the motor is connected to the input shaft of the reduction box, and the output shaft of the reduction box is connected to the connecting arm of the target surface.

5. A method for ambient light compensation of a TOF camera, characterized in that Using the auxiliary device according to any one of claims 1-4, fixedly install the TOF camera in the fixing box. The lens of the TOF camera extends into the calibration box through the first through hole and is directly opposite to the target surface exposed through the second through hole. It includes the following steps: S1. Turn off the fill light source, control the camera to work in the depth mode, and obtain the first distance d1 between the camera and the first target surface. S2. Turn on the fill light source, control the camera to work in the depth mode, and obtain the second distance d2 between the camera and the first target surface. S3. Adjust the exposure time to 1 to n, repeat steps S1 and S2, and obtain multiple sets of the first distance and the second distance d1 at different exposure times n and d2 n ; S4. Determine the first distance d1 under each group of different exposure times n and the second distance d2 n to check if there is any deviation between them, and record the power of the fill light source and the exposure time threshold corresponding to the data with deviation S5. Turn on the fill light, set the power to P, control the camera to work in the grayscale mode, and obtain the intensity information A output at this time. S6. Adjust the fill light power P1~Pn, repeat step S5, and obtain n groups of intensity information data An under the fill light with the set power Pn. S7. Turn on the fill light, control the camera to work in the depth mode, set the fill light power to P1~Pn, and obtain multiple groups of distance data Dn between the camera and the first target surface. S8. Use S6 and S7 to obtain the compensation coefficient k by fitting the data Dn and An. S9. Turn on the fill light, control the camera to work in the grayscale mode, obtain the background light intensity at this time, then switch to the depth mode, use the compensation coefficient k to process the data, and calculate the new distance value, which is the compensated distance value.

6. The method for ambient light compensation of a TOF camera according to claim 5, characterized in that In step S9, the following formula is specifically used to calculate the compensated distance value: Wherein, Q = phase3 - phase1, I = phase2 - phase0, c is the speed of light, phase0 is the phase value corresponding to 0°, phase1 is the phase value corresponding to 90°, phase2 is the phase value corresponding to 180°, phase3 is the phase value corresponding to 270°, Q' and I' are the compensated phase information, which is calculated by the following formula: Q' = Q - BG * k I' = I - BG * k Wherein, BG is the data output in the grayscale mode, and k is the calibrated compensation coefficient.

7. A method for calibrating a TOF camera, characterized in that Switch the target surface to the second target surface, collect a set of distance data, calculate the compensated distance according to the method described in claim 6, compare the compensated distance with the actual distance, if the two are consistent, the calibration is successful.

Images