Rapid AE depth camera and robot

Abstract

A fast AE depth camera is characterized in that the fast AE depth camera comprises a speckle projector used for projecting speckles; the receiver is used for receiving reflection signals of the speckles and generating a speckle pattern; the processor is used for extracting a speckle center according to the speckle pattern, calculating an average amplitude value and determining an overexposure speckle proportion; and judging whether the overexposure pixel proportion is close to an ideal value, performing fine adjustment on the exposure parameters when the overexposure pixel proportion is close to the ideal value, and performing coarse adjustment on the exposure parameters when the overexposure pixel proportion is not close to the ideal value. According to the method, the effect of rapid convergence in a short time can be achieved through the speckle pattern.

Description

[0001] Technology Neighborhood

[0002] The present invention relates to the technical field of depth cameras, and in particular to a fast AE depth camera and a robot. Background Art

[0003] Before acquiring a depth image, the depth camera needs to obtain a speckle pattern that can be used for calculation through automatic exposure. The AE algorithm is the exposure time used by the depth camera to calculate the speckle pattern. The exposure time largely determines the quality of the captured image. In a motion scene, the scene captured by the module will change at any time, which means that the captured material and depth are changing, that is, the exposure time needs to change accordingly. In a motion scene, we can calculate the current exposure based on the current image and the previous information. The previous information here refers to the amplitude map before the current frame and its corresponding exposure time. Assuming that the scene has not changed very much in a short period of time, that is, the exposure time has not changed much, then the predicted exposure time can be used for the next frame. The assumption that the scene has not changed very much in a short period of time requires that the exposure time can be calculated in a very short time, and information from too long ago cannot be used, or even only the information of the current frame can be used.

[0004] Speckle patterns can be used for 3D reconstruction. The quality of the speckle pattern will affect the reconstruction result. The AE algorithm plays a big role in optimizing the quality of the speckle pattern. When the speckle pattern is overexposed, different chips will perform different processing, such as keeping it unchanged or reducing it. Therefore, the amplitude value of the overexposed point often has a large deviation, making it impossible to accurately evaluate the overexposure situation with the amplitude value on the image. This feature increases the difficulty of the AE algorithm, and the ordinary AE algorithm cannot take effect.

[0005] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0006] To this end, the present invention uses the speckle pattern and its corresponding exposure value to calculate the exposure time. When the exposure time is far from the ideal value, it first jumps to the vicinity of the ideal exposure time, and then gradually approaches the ideal exposure time when it is near the ideal value, thereby achieving the effect of rapid convergence in a short time.

[0007] In a first aspect, the present invention provides a fast AE depth camera, characterized by comprising:

[0008] A speckle projector, used for projecting speckles;

[0009] A receiver, used for receiving a reflection signal of the speckle to generate a speckle pattern;

[0010] The processor is used to extract the speckle center according to the speckle pattern, calculate the average amplitude value, and determine the overexposed speckle ratio; determine whether the overexposed pixel ratio is near an ideal value, and fine-tune the exposure parameter when it is near the ideal value, and coarse-tune the exposure parameter when it is not near the ideal value.

[0011] Optionally, the fast AE depth camera is characterized in that the processor includes:

[0012] Step S1: obtaining a speckle pattern and a corresponding exposure time;

[0013] Step S2: extracting the speckle center from the speckle pattern;

[0014] Step S3: calculating the average amplitude value and the overexposed pixel ratio according to the speckle center;

[0015] Step S4: determining whether the over-exposure speckle ratio is near an ideal value, and performing fine adjustment when it is near the ideal value; if it is not near the ideal value, obtaining an average amplitude value straight line according to the over-exposure speckle ratio of the first area of ​​the speckle pattern, and performing coarse adjustment according to the average amplitude value straight line.

[0016] Optionally, the fast AE depth camera is characterized in that, in step S2, the pixel points on the speckle pattern are traversed with a first radius, and the pixel values ​​of the center point and other points within the first radius are determined, and if the brightness of the center point is the maximum value within the first radius, the center point is marked as the speckle center.

[0017] Optionally, the fast AE depth camera is characterized in that the calculation of the average amplitude value in step S3 includes:

[0018] Step S31: Calculate the average amplitude value of each pixel area adjacent to the center of the speckle;

[0019] Step S32: obtaining the number of the average amplitude values ​​greater than the amplitude threshold, and dividing the number by the number of the speckle centers to obtain an over-exposed speckle ratio.

[0020] Optionally, the fast AE depth camera is characterized in that, when calculating the overexposed speckle ratio in step S3, the speckle center with an amplitude value greater than the amplitude threshold is marked as a preliminary overexposed speckle center, and then the preliminary overexposed speckle centers within the first distance are merged to obtain final overexposed speckle centers, and the number of the final overexposed speckle centers is divided by the number of the speckle centers to obtain the overexposed speckle ratio; wherein the first distance is greater than the first radius.

[0021] Optionally, the fast AE depth camera is characterized in that the coarse adjustment in step S4 includes:

[0022] Step S41: Obtaining an average amplitude value straight line;

[0023] Step S42: determining an ideal amplitude value according to the current distance and the over-exposed speckle ratio;

[0024] Step S43: Calculate the coarse exposure time according to the average amplitude value straight line and the ideal amplitude value.

[0025] Optionally, the fast AE depth camera is characterized in that step S41 comprises:

[0026] Step S411: acquiring speckles in a first area, and calculating an overexposure ratio a of the speckles in the first area;

[0027] Step S412: Calculate the average amplitude value straight line according to the formula y=a*b*x; wherein b is a coefficient related to the depth camera.

[0028] Optionally, the fast AE depth camera is characterized in that, when fine-tuning is performed in step S4, the fine-tuning exposure time is calculated according to the formula nt=ct+s2*(ir-cr) / pl, where s2 is a scaling factor greater than 0 and less than 1; wherein pl is the slope of the average amplitude value straight line, cr is the overexposure ratio, ct is the exposure time, ir is the ideal overexposure ratio, and nt is the fine-tuning exposure time.

[0029] In a second aspect, the present invention provides a robot, characterized in that it comprises a fast AE depth camera as described in any one of the above items.

[0030] Optionally, the robot is characterized in that, in the speckle pattern, the ground portion accounts for no less than 0.1.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention uses the speckle pattern and its corresponding exposure value to calculate the exposure time. When the exposure time is far from the ideal value, it first jumps to the vicinity of the ideal exposure time, and then gradually approaches the ideal exposure time when it is near the ideal value, so that the effect of rapid convergence in a short time can be achieved.

[0033] The present invention can quickly calculate the exposure time of the next frame using only the speckle pattern and its corresponding exposure value. The calculation relies on less data, has fast calculation and response, and is suitable for rapid adjustment of exposure of mobile scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. Other features, purposes and advantages of the present invention will become more obvious by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0035] Figure 1 Schematic diagram of the structure of a fast AE depth camera in an embodiment of the present invention;

[0036] Figure 2 A flowchart of the steps of a processor during processing in an embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of a speckle pattern in an embodiment of the present invention;

[0038] Figure 4 This is a flow chart of the steps of calculating the average amplitude value in an embodiment of the present invention;

[0039] Figure 5 A schematic diagram of an average amplitude value and an over-exposed speckle ratio in an embodiment of the present invention;

[0040] Figure 6 A schematic diagram of a proximity search range in an embodiment of the present invention;

[0041] Figure 7 A flowchart of a step of performing a rough adjustment in an embodiment of the present invention;

[0042] Figure 8 This is a flow chart of steps for obtaining an average amplitude value straight line in an embodiment of the present invention;

[0043] Fig. 9 Schematic diagram of a first area in an embodiment of the present invention.

[0044] 1- first area; DETAILED DESCRIPTION

[0045] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0046] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0047] A fast AE depth camera provided by an embodiment of the present invention is intended to solve the problems existing in the prior art.

[0048] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0049] The present invention uses the speckle pattern and its corresponding exposure value to calculate the exposure time. When the exposure time is far from the ideal value, it first jumps to the vicinity of the ideal exposure time, and then gradually approaches the ideal exposure time when it is near the ideal value, thereby achieving a rapid convergence effect in a short time.

[0050] Figure 1 FIG. 1 is a schematic diagram of the structure of a fast AE depth camera according to an embodiment of the present invention. Figure 1 As shown, a fast AE depth camera in an embodiment of the present invention includes:

[0051] A speckle projector, used for projecting speckles.

[0052] Specifically, the speckle projector can be any type of structural form in the prior art. For example, the speckle projector contains a laser light source and a metasurface optical element (or other optical element capable of generating speckles). The laser light source emits invisible infrared lasers of a specific wavelength, which are diffracted or modulated by the metasurface optical element to form light with a speckle structure. These speckle rays are projected onto the object being photographed to form a unique speckle pattern. The degree of distortion of the speckle pattern can reflect the position and depth information of the object, and is the basis for the depth camera to perform three-dimensional measurement.

[0053] The receiver is used to receive the reflected signal of the speckle and generate a speckle pattern.

[0054] Specifically, the receiver is usually an infrared camera that can capture the speckle light projected onto an object. When the speckle light encounters an object, it is reflected, and the reflected light is received by the receiver and converted into an electrical signal. After processing, these electrical signals form an image containing the speckle pattern, namely the speckle map. The speckle map is an important basis for the subsequent processor to extract the center of the speckle and calculate the depth.

[0055] The processor is used to extract the speckle center according to the speckle pattern, calculate the average amplitude value, and determine the overexposed speckle ratio; determine whether the overexposed pixel ratio is near an ideal value, and fine-tune the exposure parameter when it is near the ideal value, and coarse-tune the exposure parameter when it is not near the ideal value.

[0056] Specifically, the processor first pre-processes the speckle pattern, such as denoising, contrast enhancement, etc., and then uses a specific algorithm (such as centroid algorithm, correlation algorithm, etc.) to extract the center position of each speckle. The processor calculates the average amplitude value of each speckle, which reflects the overall or partial brightness level of the speckle pattern. The processor determines which speckles are overexposed (i.e., speckles whose amplitude values ​​exceed the amplitude threshold) by comparing the amplitude value of the center of each speckle with a preset threshold. Then, the ratio of the number of overexposed speckles to the total number of speckles, i.e., the overexposed speckle ratio, is calculated.

[0057] The processor determines whether the current exposure parameters are appropriate based on the over-exposed speckle ratio. If the over-exposed speckle ratio is close to the ideal value (the ideal value is usually set according to the actual application scenario and camera performance), the exposure parameters are fine-tuned to further optimize the image quality. If the over-exposed speckle ratio is far from the ideal value, the exposure parameters are roughly adjusted to quickly approach the ideal value. The adjustment of exposure parameters may include changing the power of the laser light source, adjusting the gain or exposure time of the receiver, etc.

[0058] Figure 2 FIG. 1 is a flowchart of the steps of a processor in processing according to an embodiment of the present invention. Figure 2 As shown, the steps of a processor in processing in an embodiment of the present invention include:

[0059] Step S1: Obtain a speckle pattern and a corresponding exposure time.

[0060] In this step, the current scene is captured by the image sensor to generate a speckle pattern. At the same time, the exposure time at this time is recorded as the exposure time. Figure 3 As shown in FIG. 1 , a speckle pattern is a pattern formed when a light spot projected by a speckle projector is reflected on a target object. Through the speckle pattern, depth information can be calculated using the parallax principle.

[0061] Step S2: extracting the speckle center from the speckle pattern.

[0062] In this step, the key feature point, i.e., the speckle center, is extracted from the speckle pattern for subsequent analysis and calculation. The speckle center can be located and extracted on the speckle pattern using image processing algorithms (such as image segmentation, centroid algorithm, edge detection algorithm, etc.). The speckle center is the most critical point of the speckle, and the speckle center can be used to characterize the characteristics of the speckle, such as the location of the speckle, and the number of speckles.

[0063] Step S3: calculating the average amplitude value and the overexposed pixel ratio according to the speckle center.

[0064] In this step, the average amplitude value and the ratio of overexposed pixels of the pixels around the center of the speckle are calculated based on the information of the center of the speckle to evaluate the exposure quality. The amplitude value is used to characterize the brightness change. The average amplitude value of the amplitude values ​​of multiple pixels around the center of the speckle is calculated as the average amplitude value to characterize the overall brightness of the current speckle. Since the amplitude value of the Infineon chip will decrease randomly after overexposure, it is impossible to directly use the overexposed pixels for judgment. In this embodiment, the amplitude values ​​of multiple points around the center of the speckle are used to judge whether it is overexposed. Overexposure means that the amplitude value is greater than a preset threshold. The ratio of pixels in the speckle image whose average amplitude value exceeds the preset threshold is counted as a quantitative indicator of the degree of overexposure. The calculation of the average amplitude value and the ratio of overexposed pixels needs to take into account the lighting conditions and camera parameters in different scenes to ensure the accuracy and reliability of the evaluation results.

[0065] Step S4: determining whether the over-exposure speckle ratio is near an ideal value, and performing fine adjustment when it is near the ideal value; if it is not near the ideal value, obtaining an average amplitude value straight line according to the over-exposure speckle ratio of the first area of ​​the speckle pattern, and performing coarse adjustment according to the average amplitude value straight line.

[0066] In this step, the exposure adjustment strategy is determined according to the over-exposure speckle ratio to achieve fast and accurate exposure control. An ideal over-exposure speckle ratio range is preset, and whether the current over-exposure speckle ratio is within the range is determined according to the calculation result. When the over-exposure speckle ratio is near the ideal value, a smaller exposure adjustment step is used for fine adjustment to gradually approach the optimal exposure state. When the over-exposure speckle ratio is far from the ideal value, a larger exposure adjustment step is used for rapid adjustment to quickly approach the optimal exposure range. When performing coarse adjustment, the average amplitude value straight line is first obtained according to the over-exposure speckle ratio of the first area of ​​the speckle pattern, and then the adjustment is performed according to the average amplitude value straight line. The first area is a preset area in the speckle pattern.

[0067] In some embodiments, in step S2, the pixels on the speckle pattern are traversed with a first radius, and the pixel values ​​of the center point and other points within the first radius are determined. If the brightness of the center point is the maximum value within the first radius, the center point is marked as the speckle center. An initial search radius, i.e., the first radius, is set. When the first radius is r, the calculation is performed within a range of r×r centered on the current position, where r can be 3, 5, 7, 9 or any other value. The size of the first radius needs to be determined according to the resolution of the speckle pattern and the average size of the speckle. An empty data structure (such as a list or an array) is created to store the extracted speckle center coordinates. Each pixel on the speckle pattern is traversed, and each pixel is temporarily regarded as a potential speckle center. For each potential speckle center point, take it as the center, take the set first radius as the search range, and check all the pixels within the range. Calculate the brightness value of each pixel within the range and compare it with the brightness value of the center point. If the brightness value of the center point is the maximum value within the search range (or exceeds a preset threshold and is the local maximum value within the range), the center point is marked as a valid speckle center. The coordinates of the center of the speckle are stored in the previously created data structure. After the traversal and search are completed, the data structure storing the coordinates of the center of the speckle is output for use in subsequent steps.

[0068] Figure 4 FIG. 1 is a flow chart of steps for calculating the average amplitude value in an embodiment of the present invention. Figure 4 As shown, in an embodiment of the present invention, a step of calculating an average amplitude value includes:

[0069] Step S31: Calculate the average amplitude value of each pixel region adjacent to the center of the speckle.

[0070] In this step, for each speckle center on the speckle pattern, the average amplitude value of each pixel in the adjacent pixel area is calculated. The amplitude value is the brightness value of the pixel or some processed measure that can reflect the importance of the pixel. The average amplitude value of each adjacent pixel area is stored for subsequent processing.

[0071] Step S32: obtaining the number of the average amplitude values ​​greater than the amplitude threshold, and dividing the number by the number of the speckle centers to obtain an over-exposed speckle ratio.

[0072] In this step, the average amplitude value calculated in step S31 is compared with the amplitude threshold to obtain the number N of average amplitude values ​​greater than the amplitude threshold. N is divided by the number M of speckle centers to obtain the overexposed speckle ratio N / M. The amplitude threshold is a preset value between 0 and 1. The amplitude threshold is set according to the application scenario and the characteristics of the depth camera, such as 1%, 2%, 3% or any other unlisted values.

[0073] This embodiment evaluates the amplitude through the neighborhood of the speckle center, fully considering the fact that the amplitude value is inaccurate after overexposure, but its neighborhood is generally not overexposed, and can accurately identify the overexposed area for subsequent calculations.

[0074] In some embodiments, when calculating the overexposed speckle ratio in step S3, the speckle center with an amplitude value greater than the amplitude threshold is marked as a preliminary overexposed speckle center, and then the preliminary overexposed speckle centers within the first distance are merged to obtain the final overexposed speckle center, and the number of the final overexposed speckle centers is divided by the total number of speckles to obtain the overexposed speckle ratio. First, an amplitude threshold is set according to the brightness distribution and exposure of the image. The amplitude threshold is used to distinguish between normal brightness and overexposed speckle centers. The ideal exposure time can keep the overexposed speckle ratio at a lower ratio, such as 1%, 1.5%, 2% or any other lower value. S1 represents the total number of speckles, t represents the ideal overexposed ratio, and the ideal overexposed number ir is t*s1. Traverse all speckle centers, and if the amplitude value of the adjacent pixel area corresponding to a speckle center is greater than the set amplitude threshold, it is marked as a preliminary overexposed speckle center. Set a first distance to determine which preliminary overexposed speckle centers are adjacent in space and can be merged into the final overexposed speckle center. For each preliminary overexposed speckle center, check other preliminary overexposed speckle centers within the first distance around it. If such centers exist, merge them into a final overexposed speckle center. The merge can be achieved by selecting one of these centers as a representative, or calculating their average position or properties. After the merge operation, a set of final overexposed speckle centers is obtained. Count the number of final overexposed speckle centers obtained after the merge. Divide the number of final overexposed speckle centers by the total number of speckles to obtain the overexposed speckle ratio.

[0075] Figure 5 Schematic diagram of an average amplitude value and an over-exposed speckle ratio in an embodiment of the present invention. Figure 5 The statistical data of different exposures at a distance of 40cm from the black wall is shown in Figure 1. The horizontal axis is the exposure time, and the vertical axis on the left is the amplitude value. The dotted line in the figure is the average amplitude value, and the solid line is the over-exposure speckle ratio. When testing different materials at different distances, the statistical data obtained are different, that is, Figure 5 The position and slope of the line in the image will be different. However, when testing different materials and distances, the following points remain unchanged: 1. The average amplitude value is close to a straight line and passes through the 0 point, but the slope is different for different materials and distances. 2. The overexposure ratio curve is close to a straight line near the ideal overexposure ratio value on the vertical axis, and is close to parallel to the average amplitude mean. 3. There is a point nr on the vertical axis, and its exposure value corresponding to the average amplitude value straight line is close to the ideal exposure value.

[0076] When judging whether the over-exposure speckle ratio is near the ideal value in step S4, the ideal value multiplied by a threshold value u greater than 1 is used as the upper limit, and multiplied by a value d less than 1 is used as the lower limit as the adjacent search range, and the exposure value corresponding to nr on the average amplitude value straight line is used as the adjacent point; when the over-exposure speckle ratio corresponding to the current image is within the adjacent search range, it is considered to be near the ideal value, otherwise it is considered to be not near the ideal value. Figure 6 As shown in FIG. 1 , k is within the monitoring search range, while j and i are not within the adjacent search range. When the overexposed speckle ratio corresponding to the current image is within the adjacent search range, the exposure value is fine-tuned, otherwise it is roughly adjusted.

[0077] Figure 7 FIG. 1 is a flow chart of a step of performing a rough adjustment in an embodiment of the present invention. Figure 7 As shown, a step of performing coarse adjustment in an embodiment of the present invention includes:

[0078] Step S41: Obtain the average amplitude value straight line.

[0079] In this step, when the over-exposure speckle ratio is not near the ideal value, the exposure value is roughly adjusted. At this time, the over-exposure speckle ratio is much larger or much smaller than the ideal value, which also means that the current exposure time is far from the ideal value. According to statistical information, such as Figure 5 and Figure 6 As shown in , the average amplitude mean straight line is a straight line passing through point 0. When the exposure parameters of the depth camera are fixed, such as Figure 3 As shown in FIG. 1 , in the lower part of the speckle pattern, the overexposure ratio is related to the slope of the average amplitude value straight line. The average amplitude value straight line can be calculated by the overexposure ratio of a specific part of the speckle pattern.

[0080] Step S42: determining an ideal amplitude value according to the current distance and the over-exposed speckle ratio.

[0081] In this step, the exposure time and the average amplitude mean at this time are known, that is, a point on the straight line passing through the 0 point is known, and the horizontal axis value corresponding to any vertical axis value can be calculated using the average amplitude value straight line, that is, the exposure time. According to the current distance and overexposure ratio, the ideal amplitude value can be obtained.

[0082] Step S43: Calculate the coarse exposure time according to the average amplitude value straight line and the ideal amplitude value.

[0083] In this step, the exposure time corresponding to 1000 is calculated by the average amplitude straight line as the exposure time of the next frame, so that it can jump to the ideal value in one step. Figure 6 The i and j points in the image need to be adjusted to the adjacent points first, so that fine-tuning can be performed in the next frame. Figure 6The ideal amplitude value in can be taken as 1000. By calculating the exposure time corresponding to 1000 on the average amplitude line as the exposure time of the next frame, it is possible to jump to the ideal value in one step.

[0084] This embodiment uses the average amplitude value to linearly adjust the exposure parameters with large deviations to the adjacent search range in one step, thereby greatly shortening the exposure adjustment time. Even in the case of inaccurate overexposure data, a more accurate ideal amplitude value and fast exposure adjustment capability can be obtained.

[0085] Figure 8 FIG. 1 is a flow chart of steps for obtaining an average amplitude value straight line in an embodiment of the present invention. Figure 8 As shown, in an embodiment of the present invention, a step of obtaining an average amplitude value straight line includes:

[0086] Step S411: acquiring speckles in a first area, and calculating an overexposure ratio a of the speckles in the first area.

[0087] In this step, the first area is a preset area in the speckle pattern. The size and position of the first area are determined according to the installation position of the depth camera. For example, when the depth camera is installed on a sweeping robot, the speckle at its bottom is irradiated on the ground, and a relatively strong spot signal is obtained. Fig. 9 As shown, the first area is located in the middle and lower part of the speckle pattern, and part of it includes the ground. The first area at least includes part of the known plane information, such as the ground, the wall, etc. The overexposure ratio change of the known plane information and the unknown part of the speckle can be used to evaluate the average amplitude value straight line.

[0088] Step S412: Calculate the average amplitude value straight line according to the formula y=a*b*x; wherein b is a coefficient related to the depth camera.

[0089] In this step, the average amplitude value straight line is a straight line passing through the origin O. b is determined by the intrinsic parameters of the depth camera, such as exposure parameters, projection intensity, baseline, etc. When the depth camera is fixed and the projection parameters are known, b is known. a is calculated by step S411, and thus the average amplitude value straight line y=a*b*x is calculated.

[0090] This embodiment calculates the average amplitude value straight line according to the overexposure ratio of the speckle in the first area, so that real-time calculation can be performed according to the current frame image to perform accurate adjustment, thereby ensuring the quality and stability of the adjustment.

[0091] In some embodiments, when fine-tuning is performed in step S4, the fine-tuning exposure time is calculated according to the formula nt=ct+s2*(ir-cr) / pl, where s2 is a scaling factor greater than 0 and less than 1; wherein pl is the slope of the average amplitude value straight line, cr is the overexposure ratio, ct is the exposure time, ir is the ideal overexposure ratio, and nt is the fine-tuning exposure time. According to statistical results, at this time, the overexposure speckle ratio is a straight line, and is nearly parallel to the amplitude mean straight line. The slope of the amplitude mean straight line can be used as the slope of the overexposure ratio straight line. Since the overexposure speckle ratio curve and the amplitude mean straight line are nearly parallel, but not completely parallel, and there is a certain difference, s2 can be used to prevent a large jump in one step, so that the exposure adjustment approaches the ideal value at a more reasonable speed. Figure 6 If the k points in the search are within the range of the nearest search, we can gradually jump to the ideal point.

[0092] The present invention also provides a robot, including a fast AE depth camera in the above embodiment. The following part of this specification will exemplarily illustrate the robot. Those skilled in the art will understand that the following robot is only used to help understand the effect of this embodiment, and does not constitute a limitation on the scope of protection of the present invention.

[0093] The robot system combines advanced fast AE depth camera technology with high-precision and high-speed environmental perception capabilities, and is suitable for tasks in a variety of complex scenarios.

[0094] The fast AE depth camera equipped on the robot has the following features:

[0095] Speckle projector: used to project speckle patterns into the environment. The degree of distortion of the speckle pattern can reflect the position and depth information of the object.

[0096] Receiver: Receives the reflected signal of the speckle pattern and generates a speckle image. The receiver has the characteristics of high sensitivity and low noise, which can ensure high-quality image acquisition.

[0097] Processor: Process the speckle image, extract the speckle center, calculate the average amplitude value, and determine the overexposed speckle ratio. According to the overexposed pixel ratio, the processor can automatically adjust the exposure parameters to ensure the stability and accuracy of image quality.

[0098] After the robot is started, the fast AE depth camera starts working, projecting speckle patterns and receiving reflected signals. The processor processes the received speckle images and extracts the position and depth information of the object. Based on this information, the robot builds a three-dimensional model of the environment, providing a basis for subsequent operations.

[0099] The robot plans the optimal operation path based on the task requirements and the three-dimensional model of the environment. During the planning process, the robot will consider various constraints, such as obstacle avoidance and the shortest path.

[0100] The robot performs specific operation tasks according to the planned path. During the operation, the fast AE depth camera continues to work and updates the 3D model of the environment in real time to ensure that the robot can complete the task accurately and stably.

[0101] On automated production lines, robots can be equipped with fast AE depth cameras for precise material handling and assembly. In warehousing environments, robots can use fast AE depth cameras for automatic sorting, warehousing, and outbound operations. In the smart home field, robots can be equipped with fast AE depth cameras for sensing and monitoring the home environment, improving the safety and convenience of family life. In the medical field, robots can use fast AE depth cameras for surgical assistance, patient care, and other operations to improve the efficiency and quality of medical services.

[0102] The fast AE depth camera can provide high-precision environmental perception information, ensuring that the robot can accurately complete tasks in complex scenarios. The robot can quickly respond to environmental changes, update the three-dimensional model of the environment in real time, and improve operational efficiency. The fast AE depth camera has an automatic exposure adjustment function, which can ensure that high-quality image information can be provided under different lighting conditions and improve the stability of the robot. The robot can adapt to a variety of different application scenarios and task requirements, and has a wide range of application prospects.

[0103] In some embodiments, in the speckle pattern, the ground portion accounts for no less than 0.1. The fast AE depth camera in the robot in this embodiment is relatively low in height, so that information about the ground can be obtained over a larger area. The robot in this embodiment can be a sweeping robot or other low-profile robots. The depth camera obtains information about the front and the ground. Since the position of the ground is fixed in the speckle pattern, the area of ​​the ground can be directly determined in the speckle pattern, so that it can be applied to the fast AE process to improve efficiency.

[0104] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. The various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0105] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A fast AE depth camera, characterized in that: include: A speckle projector, used for projecting speckles; A receiver, used for receiving a reflection signal of the speckle to generate a speckle pattern; A processor, configured to extract a speckle center according to the speckle pattern, calculate an average amplitude value, and determine an overexposed speckle ratio; It is determined whether the overexposed pixel ratio is near an ideal value, and when it is near the ideal value, the exposure parameter is finely adjusted, and when it is not near the ideal value, the exposure parameter is roughly adjusted.

2. A fast AE depth camera according to claim 1, characterized in that: The processor includes: Step S1: obtaining a speckle pattern and a corresponding exposure time; Step S2: extracting the speckle center from the speckle pattern; Step S3: calculating the average amplitude value and the overexposed pixel ratio according to the speckle center; Step S4: determining whether the over-exposure speckle ratio is near an ideal value, and performing fine adjustment when it is near the ideal value; if it is not near the ideal value, obtaining an average amplitude value straight line according to the over-exposure speckle ratio of the first area of ​​the speckle pattern, and performing coarse adjustment according to the average amplitude value straight line.

3. A fast AE depth camera according to claim 2, characterized in that: In step S2, the pixels on the speckle pattern are traversed with a first radius, and the pixel values ​​of the center point and other points within the first radius are determined. If the brightness of the center point is the maximum value within the first radius, the center point is marked as the speckle center.

4. The fast AE depth camera according to claim 2, characterized in that: The calculation of the average amplitude value in step S3 includes: Step S31: Calculate the average amplitude value of each pixel area adjacent to the center of the speckle; Step S32: obtaining the number of the average amplitude values ​​greater than the amplitude threshold, and dividing the number by the number of the speckle centers to obtain an over-exposed speckle ratio.

5. The fast AE depth camera according to claim 2, characterized in that: When calculating the overexposed speckle ratio in step S3, the speckle center with an amplitude value greater than the amplitude threshold is marked as a preliminary overexposed speckle center, and then the preliminary overexposed speckle centers within the first distance are merged to obtain final overexposed speckle centers, and the number of the final overexposed speckle centers is divided by the number of the speckle centers to obtain the overexposed speckle ratio; wherein the first distance is greater than the first radius.

6. The fast AE depth camera according to claim 2, characterized in that: The rough adjustment in step S4 includes: Step S41: Obtaining an average amplitude value straight line; Step S42: determining an ideal amplitude value according to the current distance and the over-exposed speckle ratio; Step S43: Calculate the coarse exposure time according to the average amplitude value straight line and the ideal amplitude value.

7. The fast AE depth camera according to claim 6, characterized in that: Step S41 includes: Step S411: acquiring speckles in a first area, and calculating an overexposure ratio a of the speckles in the first area; Step S412: Calculate the average amplitude value straight line according to the formula y=a*b*x; wherein b is a coefficient related to the depth camera.

8. The fast AE depth camera according to claim 2, characterized in that: When fine-tuning is performed in step S4, the fine-tuning exposure time is calculated according to the formula nt=ct+s2*(ir-cr) / pl, where s2 is a scaling factor greater than 0 and less than 1; wherein pl is the slope of the average amplitude value straight line, cr is the overexposure ratio, ct is the exposure time, ir is the ideal overexposure ratio, and nt is the fine-tuning exposure time.

9. A robot, characterized in that: A fast AE depth camera comprising any one of claims 1-8.

10. A robot according to claim 9, characterized in that: In the speckle pattern, the ground portion accounts for no less than 0.1.

Images