Rapid AE method, system and device based on amplitude value and storage medium

Abstract

The invention discloses a rapid AE method, system and device based on an amplitude value, and a storage medium. The method comprises the following steps: S1, obtaining a speckle pattern and corresponding exposure time; s2, extracting speckle centers and an adjacent pixel region of each speckle center on the speckle pattern; s3, calculating an average amplitude value of adjacent pixel areas of each speckle center, and determining an overexposure speckle proportion; and S4, judging whether the overexposure speckle proportion is close to an ideal value or not, performing fine adjustment when the overexposure speckle proportion is close to the ideal value, obtaining an average amplitude value straight line according to the overexposure speckle proportion of the first region of the speckle pattern when the overexposure speckle proportion is not close to the ideal value, and performing coarse adjustment according to the average amplitude value straight line. According to the method, the effect of rapid convergence in a short time can be achieved through the speckle pattern.

Description

[0001] Technology Neighborhood

[0002] The present invention relates to the field of image exposure adjustment technology, and in particular to a rapid AE method, system, device and storage medium based on amplitude value. Background Art

[0003] The AE algorithm is used to calculate the exposure time of the speckle pattern, and the exposure time largely determines the quality of the captured image. In a motion scene, the scene captured by the module will change at any time, which means that the captured material and depth are changing, that is, the exposure time needs to change accordingly. In a motion scene, we can calculate the current exposure based on the current image and the previous information. The previous information here refers to the amplitude map before the current frame and its corresponding exposure time. Assuming that the scene has not changed significantly in a short period of time, that is, the exposure time has not changed significantly, then the predicted exposure time can be used for the next frame. The assumption that the scene has not changed significantly in a short period of time requires that the exposure time can be calculated in a very short time, and information from too long ago cannot be used, or even only the information of the current frame can be used.

[0004] Speckle patterns can be used for 3D reconstruction. The quality of the speckle pattern will affect the reconstruction result. The AE algorithm plays a big role in optimizing the quality of the speckle pattern. When the speckle pattern is overexposed, different chips will perform different processing, such as keeping it unchanged or reducing it. Therefore, the amplitude value of the overexposed point often has a large deviation, making it impossible to accurately evaluate the overexposure situation with the amplitude value on the image. This feature increases the difficulty of the AE algorithm, and the ordinary AE algorithm cannot take effect.

[0005] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0006] To this end, the present invention uses the speckle pattern and its corresponding exposure value to calculate the exposure time. When the exposure time is far from the ideal value, it first jumps to the vicinity of the ideal exposure time, and then gradually approaches the ideal exposure time when it is near the ideal value, thereby achieving the effect of rapid convergence in a short time.

[0007] In a first aspect, the present invention provides a rapid AE method based on amplitude value, characterized by comprising:

[0008] Step S1: obtaining a speckle pattern and a corresponding exposure time;

[0009] Step S2: extracting the speckle center and the adjacent pixel area of ​​each speckle center on the speckle image;

[0010] Step S3: calculating the average amplitude value of the adjacent pixel area of ​​each speckle center, and determining the over-exposed speckle ratio;

[0011] Step S4: determining whether the over-exposure speckle ratio is near an ideal value, and performing fine adjustment when it is near the ideal value; if it is not near the ideal value, obtaining an average amplitude value straight line according to the over-exposure speckle ratio of the first area of ​​the speckle pattern, and performing coarse adjustment according to the average amplitude value straight line.

[0012] Optionally, the amplitude value-based rapid AE method is characterized in that, in step S2, if there are overlapping pixels in adjacent pixel areas of different speckle centers, the amplitude values ​​of adjacent pixel areas of the two speckle centers are calculated respectively, and the adjacent pixel area of ​​the speckle center with the smaller amplitude value is discarded.

[0013] Optionally, the amplitude value-based rapid AE method is characterized in that the calculation of the average amplitude value in step S3 includes:

[0014] Step S31: Calculate the average amplitude value of each pixel area adjacent to the center of the speckle;

[0015] Step S32: obtaining the number of the average amplitude values ​​greater than the amplitude threshold, and dividing the number by the number of the speckle centers to obtain an over-exposed speckle ratio.

[0016] Optionally, the amplitude value-based rapid AE method is characterized in that, when calculating the over-exposed speckle ratio in step S3, the speckle centers with amplitude values ​​greater than the amplitude threshold are marked as preliminary over-exposed speckle centers, and then the preliminary over-exposed speckle centers within the first distance are merged to obtain final over-exposed speckle centers, and the number of the final over-exposed speckle centers is divided by the number of the speckle centers to obtain the over-exposed speckle ratio.

[0017] Optionally, the amplitude value-based rapid AE method is characterized in that the rough adjustment in step S4 includes:

[0018] Step S41: Obtaining an average amplitude value straight line;

[0019] Step S42: determining an ideal amplitude value according to the current distance and the over-exposed speckle ratio;

[0020] Step S43: Calculate the coarse exposure time according to the average amplitude value straight line and the ideal amplitude value.

[0021] Optionally, the amplitude value-based rapid AE method is characterized in that step S41 comprises:

[0022] Step S411: acquiring speckles in a first area, and calculating an overexposure ratio a of the speckles in the first area;

[0023] Step S412: Calculate the average amplitude value straight line according to the formula y=a*b*x; wherein b is a coefficient related to the depth camera.

[0024] Optionally, the rapid AE method based on amplitude value is characterized in that, when fine-tuning is performed in step S4, the fine-tuning exposure time is calculated according to the formula nt=ct+s2*(ir-cr) / pl, where s2 is a scaling factor greater than 0 and less than 1; wherein pl is the slope of the average amplitude value straight line, cr is the overexposure ratio, ct is the exposure time, ir is the ideal overexposure ratio, and nt is the fine-tuning exposure time.

[0025] In a second aspect, the present invention provides a rapid AE system based on amplitude value, which is used to implement any of the rapid AE methods based on amplitude value described above, and is characterized by comprising:

[0026] An acquisition module, used for acquiring a speckle pattern and a corresponding exposure time;

[0027] An extraction module, used for extracting a speckle center and an adjacent pixel area of ​​each speckle center on the speckle image;

[0028] A calculation module, used to calculate the average amplitude value of the adjacent pixel area of ​​each speckle center and determine the over-exposed speckle ratio;

[0029] an adjusting module, for judging whether the over-exposed speckle ratio is near an ideal value, and performing fine adjustment when it is near the ideal value; and for obtaining an average amplitude value straight line according to the over-exposed speckle ratio of the first area of ​​the speckle pattern when it is not near the ideal value, and performing coarse adjustment according to the average amplitude value straight line.

[0030] In a third aspect, the present invention provides a rapid AE device based on amplitude value, characterized in that it includes:

[0031] processor;

[0032] a memory storing executable instructions of the processor;

[0033] The processor is configured to perform any of the steps of the aforementioned amplitude value-based fast AE method by executing the executable instructions.

[0034] In a fourth aspect, the present invention provides a computer-readable storage medium for storing a program, characterized in that when the program is executed, the steps of any of the aforementioned rapid AE methods based on amplitude values ​​are implemented.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention uses the speckle pattern and its corresponding exposure value to calculate the exposure time. When the exposure time is far from the ideal value, it first jumps to the vicinity of the ideal exposure time, and then gradually approaches the ideal exposure time when it is near the ideal value, so that the effect of rapid convergence in a short time can be achieved.

[0037] The present invention can quickly calculate the exposure time of the next frame using only the speckle pattern and its corresponding exposure value. The calculation relies on less data, has fast calculation and response, and is suitable for rapid adjustment of exposure of mobile scenes.

[0038] The present invention obtains an average amplitude value straight line according to the over-exposed speckle ratio of the first area of ​​the speckle pattern, and can automatically adjust according to the current speckle state, thereby avoiding the frame skipping phenomenon. The exposure can be adjusted to near the ideal value in the next frame according to the current state, thereby greatly shortening the exposure adjustment time. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. Other features, purposes and advantages of the present invention will become more obvious by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0040] Figure 1 This is a flowchart of a rapid AE method based on amplitude value in an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram of a speckle pattern in an embodiment of the present invention;

[0042] Figure 3 This is a flow chart of the steps of calculating the average amplitude value in an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of an average amplitude value and an over-exposed speckle ratio in an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of a proximity search range in an embodiment of the present invention;

[0045] Figure 6 A flowchart of a step of performing a rough adjustment in an embodiment of the present invention;

[0046] Figure 7 This is a flow chart of steps for obtaining an average amplitude value straight line in an embodiment of the present invention;

[0047] Figure 8 is a schematic diagram of a first area in an embodiment of the present invention;

[0048] Fig. 9 This is a schematic diagram of the structure of a rapid AE system based on amplitude value in an embodiment of the present invention;

[0049] Fig.10 is a schematic structural diagram of a rapid AE device based on amplitude value in an embodiment of the present invention; and

[0050] Fig.11 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention.

[0051] 1- first area; DETAILED DESCRIPTION

[0052] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0053] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] An embodiment of the present invention provides a rapid AE method based on amplitude value, aiming to solve the problems existing in the prior art.

[0055] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0056] The present invention uses the speckle pattern and its corresponding exposure value to calculate the exposure time. When the exposure time is far from the ideal value, it first jumps to the vicinity of the ideal exposure time, and then gradually approaches the ideal exposure time when it is near the ideal value, thereby achieving a rapid convergence effect in a short time.

[0057] Figure 1 FIG. 1 is a flowchart of a rapid AE method based on amplitude value in an embodiment of the present invention. Figure 1 As shown, the steps of a rapid AE method based on amplitude value in an embodiment of the present invention include:

[0058] Step S1: Obtain a speckle pattern and a corresponding exposure time.

[0059] In this step, the current scene is captured by the image sensor to generate a speckle pattern. At the same time, the exposure time at this time is recorded as the exposure time. Figure 2 As shown in FIG. 1 , a speckle pattern is a pattern formed when a light spot projected by a speckle projector is reflected on a target object. Through the speckle pattern, depth information can be calculated using the parallax principle.

[0060] Step S2: extracting the speckle center and the adjacent pixel area of ​​each speckle center from the speckle image.

[0061] In this step, the key feature point, i.e., the speckle center, is extracted from the speckle image. The speckle center can be located and extracted on the speckle image using image processing algorithms (such as image segmentation, centroid algorithm, edge detection algorithm, etc.). The number of speckles and the adjacent pixel area of ​​the speckle center can be obtained through the speckle center. The adjacent pixel area is the area adjacent to the speckle center. The selection of the adjacent pixel area can be made according to the size of the speckle and the characteristics of the depth camera, such as four neighborhoods, eight neighborhoods, etc.

[0062] Step S3: Calculate the average amplitude value of the pixel area adjacent to each speckle center, and determine the over-exposed speckle ratio.

[0063] In this step, the average amplitude value and the overexposed speckle ratio of the adjacent pixel area are calculated to evaluate the exposure quality. The amplitude value is used to characterize the brightness change. The average amplitude value of the amplitude values ​​of multiple pixels in the adjacent pixel area is calculated as the average amplitude value to characterize the overall brightness of the current speckle. Since the amplitude value of the image will be inaccurate or even reduced after being overexposed, directly using the overexposed pixel points for judgment will result in large deviations. In this embodiment, the amplitude values ​​of multiple points around the center of the speckle are used to determine whether it is overexposed. Overexposure means that the amplitude value is greater than a preset threshold. The ratio of pixels in the speckle image whose average amplitude value exceeds the preset threshold is counted as a quantitative indicator of the degree of overexposure. The calculation of the average amplitude value and the overexposed speckle ratio needs to take into account the lighting conditions and camera parameters in different scenes to ensure the accuracy and reliability of the evaluation results.

[0064] Step S4: determining whether the over-exposure speckle ratio is near an ideal value, and performing fine adjustment when it is near the ideal value; if it is not near the ideal value, obtaining an average amplitude value straight line according to the over-exposure speckle ratio of the first area of ​​the speckle pattern, and performing coarse adjustment according to the average amplitude value straight line.

[0065] In this step, the exposure adjustment strategy is determined according to the over-exposure speckle ratio to achieve fast and accurate exposure control. An ideal over-exposure speckle ratio range is preset, and whether the current over-exposure speckle ratio is within the range is determined according to the calculation result. When the over-exposure speckle ratio is near the ideal value, a smaller exposure adjustment step is used for fine adjustment to gradually approach the optimal exposure state. When the over-exposure speckle ratio is far from the ideal value, a larger exposure adjustment step is used for rapid adjustment to quickly approach the optimal exposure range. When performing coarse adjustment, the average amplitude value straight line is first obtained according to the over-exposure speckle ratio of the first area of ​​the speckle pattern, and then the adjustment is performed according to the average amplitude value straight line. The first area is a preset area in the speckle pattern.

[0066] In some embodiments, in step S2, if there are overlapping pixels in adjacent pixel regions of different speckle centers, the amplitude values ​​of adjacent pixel regions are calculated for the two speckle centers respectively, and the adjacent pixel region of the speckle center with the smaller amplitude value is discarded. When there are overlapping pixels in adjacent pixel regions, it means that the two speckles overlap to a large extent. At this time, the amplitude values ​​of adjacent pixel regions are calculated respectively, and the region with the smaller amplitude value is discarded. When calculating the amplitude values ​​of adjacent pixel regions, the overlapping pixels are calculated twice. The amplitude values ​​of each pixel in the adjacent pixel regions are summed, and the sum or mean (average value or weighted mean) is taken as the amplitude value of the adjacent pixel region.

[0067] Figure 3 FIG. 1 is a flow chart of steps for calculating the average amplitude value in an embodiment of the present invention. Figure 3As shown, a step of calculating the average amplitude value in an embodiment of the present invention includes:

[0068] Step S31: Calculate the average amplitude value of each pixel region adjacent to the center of the speckle.

[0069] In this step, for each speckle center on the speckle pattern, the average amplitude value of each pixel in the adjacent pixel area is calculated. The amplitude value is the brightness value of the pixel or some processed measure that can reflect the importance of the pixel. The average amplitude value of each adjacent pixel area is stored for subsequent processing.

[0070] Step S32: obtaining the number of the average amplitude values ​​greater than the amplitude threshold, and dividing the number by the number of the speckle centers to obtain an over-exposed speckle ratio.

[0071] In this step, the average amplitude value calculated in step S31 is compared with the amplitude threshold to obtain the number N of average amplitude values ​​greater than the amplitude threshold. N is divided by the number M of speckle centers to obtain the overexposed speckle ratio N / M. The amplitude threshold is a preset value between 0 and 1. The amplitude threshold is set according to the application scenario and the characteristics of the depth camera, such as 1%, 2%, 3% or any other unlisted values.

[0072] This embodiment evaluates the amplitude through the neighborhood of the speckle center, fully considering the fact that the amplitude value is inaccurate after overexposure, but its neighborhood is generally not overexposed, and can accurately identify the overexposed area for subsequent calculations.

[0073] In some embodiments, when calculating the overexposed speckle ratio in step S3, the speckle center with an amplitude value greater than the amplitude threshold is marked as a preliminary overexposed speckle center, and then the preliminary overexposed speckle centers within the first distance are merged to obtain the final overexposed speckle center, and the number of the final overexposed speckle centers is divided by the total number of speckles to obtain the overexposed speckle ratio. First, an amplitude threshold is set according to the brightness distribution and exposure of the image. This threshold is used to distinguish between normal brightness and overexposed speckle centers. The ideal exposure time can keep the overexposed speckle ratio at a lower ratio, such as 1%, 1.5%, 2% or any other lower value. S1 represents the total number of speckles, t represents the ideal overexposed ratio, and the ideal overexposed number ir is t*s1. Traverse all speckle centers, and if the amplitude value of the adjacent pixel area corresponding to a speckle center is greater than the set amplitude threshold, it is marked as a preliminary overexposed speckle center. Set a first distance to determine which preliminary overexposed speckle centers are adjacent in space and can be merged into the final overexposed speckle center. For each preliminary overexposed speckle center, check other preliminary overexposed speckle centers within the first distance around it. If such centers exist, merge them into a final overexposed speckle center. The merge can be achieved by selecting one of these centers as a representative, or calculating their average position or properties. After the merge operation, a set of final overexposed speckle centers is obtained. Count the number of final overexposed speckle centers obtained after the merge. Divide the number of final overexposed speckle centers by the total number of speckles to obtain the overexposed speckle ratio.

[0074] Figure 4 Schematic diagram of an average amplitude value and an over-exposed speckle ratio in an embodiment of the present invention. Figure 4 The statistical data of different exposures at a distance of 40cm from the black wall is shown in Figure 1. The horizontal axis is the exposure time, and the vertical axis on the left is the amplitude value. The dotted line in the figure is the average amplitude value, and the solid line is the over-exposure speckle ratio. When testing different materials at different distances, the statistical data obtained are different, that is, Figure 4 The position and slope of the line in the image will be different. However, when testing different materials and distances, the following points remain unchanged: 1. The average amplitude value is close to a straight line and passes through the 0 point, but the slope is different for different materials and distances. 2. The overexposure ratio curve is close to a straight line near the ideal overexposure ratio value on the vertical axis, and is close to parallel to the average amplitude mean. 3. There is a point nr on the vertical axis, and its exposure value corresponding to the average amplitude value straight line is close to the ideal exposure value.

[0075] When judging whether the over-exposure speckle ratio is near the ideal value in step S4, the ideal value multiplied by a threshold value u greater than 1 is used as the upper limit, and multiplied by a value d less than 1 is used as the lower limit as the adjacent search range, and the exposure value corresponding to nr on the average amplitude value straight line is used as the adjacent point; when the over-exposure speckle ratio corresponding to the current image is within the adjacent search range, it is considered to be near the ideal value, otherwise it is considered to be not near the ideal value. Figure 5 As shown in FIG. 1 , k is within the monitoring search range, while j and i are not within the adjacent search range. When the overexposed speckle ratio corresponding to the current image is within the adjacent search range, the exposure value is fine-tuned, otherwise it is roughly adjusted.

[0076] Figure 6 FIG. 1 is a flow chart of a step of performing a rough adjustment in an embodiment of the present invention. Figure 6 As shown, a step of performing coarse adjustment in an embodiment of the present invention includes:

[0077] Step S41: Obtain the average amplitude value straight line.

[0078] In this step, when the over-exposure speckle ratio is not near the ideal value, the exposure value is roughly adjusted. At this time, the over-exposure speckle ratio is much larger or much smaller than the ideal value, which also means that the current exposure time is far from the ideal value. According to statistical information, such as Figure 4 and Figure 5 As shown in , the average amplitude mean straight line is a straight line passing through point 0. When the exposure parameters of the depth camera are fixed, such as Figure 2 As shown in FIG. 1 , in the lower part of the speckle pattern, the overexposure ratio is related to the slope of the average amplitude value straight line. The average amplitude value straight line can be calculated by the overexposure ratio of a specific part of the speckle pattern.

[0079] Step S42: determining an ideal amplitude value according to the current distance and the over-exposed speckle ratio.

[0080] In this step, the exposure time and the average amplitude mean at this time are known, that is, a point on the straight line passing through the 0 point is known, and the horizontal axis value corresponding to any vertical axis value can be calculated using the average amplitude value straight line, that is, the exposure time. According to the current distance and overexposure ratio, the ideal amplitude value can be obtained.

[0081] Step S43: Calculate the coarse exposure time according to the average amplitude value straight line and the ideal amplitude value.

[0082] In this step, the exposure time corresponding to 1000 is calculated by the average amplitude straight line as the exposure time of the next frame, so that it can jump to the ideal value in one step. Figure 5 The i and j points in the image need to be adjusted to the adjacent points first, so that fine-tuning can be performed in the next frame. Figure 5The ideal amplitude value in can be taken as 1000. By calculating the exposure time corresponding to 1000 on the average amplitude line as the exposure time of the next frame, it is possible to jump to the ideal value in one step.

[0083] This embodiment uses the average amplitude value to linearly adjust the exposure parameters with large deviations to the adjacent search range in one step, thereby greatly shortening the exposure adjustment time. Even in the case of inaccurate overexposure data, a more accurate ideal amplitude value and fast exposure adjustment capability can be obtained.

[0084] Figure 7 FIG. 1 is a flow chart of steps for obtaining an average amplitude value straight line in an embodiment of the present invention. Figure 7 As shown, in an embodiment of the present invention, a step of obtaining an average amplitude value straight line includes:

[0085] Step S411: acquiring speckles in a first area, and calculating an overexposure ratio a of the speckles in the first area.

[0086] In this step, the first area is a preset area in the speckle pattern. The size and position of the first area are determined according to the installation position of the depth camera. For example, when the depth camera is installed on a sweeping robot, the speckle at its bottom is irradiated on the ground, and a relatively strong spot signal is obtained. Figure 8 As shown, the first area is located in the middle and lower part of the speckle pattern, and part of it includes the ground. The first area at least includes part of the known plane information, such as the ground, the wall, etc. The overexposure ratio change of the known plane information and the unknown part of the speckle can be used to evaluate the average amplitude value straight line.

[0087] Step S412: Calculate the average amplitude value straight line according to the formula y=a*b*x; wherein b is a coefficient related to the depth camera.

[0088] In this step, the average amplitude value straight line is a straight line passing through the origin O. b is determined by the intrinsic parameters of the depth camera, such as exposure parameters, projection intensity, baseline, etc. When the depth camera is fixed and the projection parameters are known, b is known. a is calculated by step S411, and thus the average amplitude value straight line y=a*b*x is calculated.

[0089] This embodiment calculates the average amplitude value straight line according to the overexposure ratio of the speckle in the first area, so that real-time calculation can be performed according to the current frame image to perform accurate adjustment, thereby ensuring the quality and stability of the adjustment.

[0090] In some embodiments, when fine-tuning is performed in step S4, the fine-tuning exposure time is calculated according to the formula nt=ct+s2*(ir-cr) / pl, where s2 is a scaling factor greater than 0 and less than 1; wherein pl is the slope of the average amplitude value straight line, cr is the overexposure ratio, ct is the exposure time, ir is the ideal overexposure ratio, and nt is the fine-tuning exposure time. According to statistical results, at this time, the overexposure speckle ratio is a straight line, and is nearly parallel to the amplitude mean straight line. The slope of the amplitude mean straight line can be used as the slope of the overexposure ratio straight line. Since the overexposure speckle ratio curve and the amplitude mean straight line are nearly parallel, but not completely parallel, and there is a certain difference, s2 can be used to prevent a large jump in one step, so that the exposure adjustment approaches the ideal value at a more reasonable speed. Figure 5 If the k points in the search are within the range of the nearest search, we can gradually jump to the ideal point.

[0091] Fig. 9 FIG. 1 is a schematic diagram of a structure of a rapid AE system based on amplitude value in an embodiment of the present invention. Fig. 9 As shown, a rapid AE system based on amplitude value in an embodiment of the present invention includes:

[0092] An acquisition module, used for acquiring a speckle pattern and a corresponding exposure time;

[0093] An extraction module, used for extracting a speckle center and an adjacent pixel area of ​​each speckle center on the speckle image;

[0094] A calculation module, used to calculate the average amplitude value of the adjacent pixel area of ​​each speckle center and determine the over-exposed speckle ratio;

[0095] an adjusting module, for judging whether the over-exposed speckle ratio is near an ideal value, and performing fine adjustment when it is near the ideal value; and for obtaining an average amplitude value straight line according to the over-exposed speckle ratio of the first area of ​​the speckle pattern when it is not near the ideal value, and performing coarse adjustment according to the average amplitude value straight line.

[0096] This embodiment uses the speckle pattern and its corresponding exposure value to calculate the exposure time. When the exposure time is far from the ideal value, it first jumps to the vicinity of the ideal exposure time, and then gradually approaches the ideal exposure time when it is near the ideal value, thereby achieving a rapid convergence effect in a short time.

[0097] The embodiment of the present invention also provides a rapid AE device based on amplitude value, comprising a processor and a memory, wherein executable instructions of the processor are stored, wherein the processor is configured to execute the steps of a rapid AE method based on amplitude value by executing the executable instructions.

[0098] As described above, this embodiment uses the speckle pattern and its corresponding exposure value to calculate the exposure time. When the exposure time is far from the ideal value, it first jumps to the vicinity of the ideal exposure time, and then gradually approaches the ideal exposure time when it is near the ideal value, thereby achieving a rapid convergence effect in a short time.

[0099] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0100] Fig.10 Schematic diagram of the structure of a rapid AE device based on amplitude value in an embodiment of the present invention. Fig.10 The electronic device 600 according to this embodiment of the present invention is described. Fig.10 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0101] like Fig.10 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0102] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps of various exemplary embodiments of the present invention described in the above-mentioned fast AE method based on amplitude value of this specification. For example, the processing unit 610 can execute the following steps: Figure 1 Follow the steps shown in .

[0103] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0104] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a grid environment.

[0105] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0106] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 600, and / or any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more grids (e.g., a local area network (LAN), a wide area network (WAN), and / or a public grid, such as the Internet) through a grid adapter 660. The grid adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Fig.10 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0107] In an embodiment of the present invention, a computer-readable storage medium is also provided for storing a program, and when the program is executed, the steps of a rapid AE method based on amplitude value are implemented. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary implementations of the present invention described in the above-mentioned rapid AE method based on amplitude value section of this specification.

[0108] As shown above, this embodiment uses the speckle pattern and its corresponding exposure value to calculate the exposure time. When the exposure time is far from the ideal value, it first jumps to the vicinity of the ideal exposure time, and then gradually approaches the ideal exposure time when it is near the ideal value, thereby achieving a rapid convergence effect in a short time.

[0109] Fig.11 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Fig.11As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0110] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0111] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0112] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of grid, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0113] This embodiment uses the speckle pattern and its corresponding exposure value to calculate the exposure time. When the exposure time is far from the ideal value, it first jumps to the vicinity of the ideal exposure time, and then gradually approaches the ideal exposure time when it is near the ideal value, thereby achieving a rapid convergence effect in a short time.

[0114] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. The various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0115] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A rapid AE method based on amplitude value, characterized in that: include: Step S1: obtaining a speckle pattern and a corresponding exposure time; Step S2: extracting the speckle center and the adjacent pixel area of ​​each speckle center on the speckle image; Step S3: calculating the average amplitude value of the adjacent pixel area of ​​each speckle center, and determining the over-exposed speckle ratio; Step S4: determining whether the over-exposure speckle ratio is near an ideal value, and performing fine adjustment when it is near the ideal value; if it is not near the ideal value, obtaining an average amplitude value straight line according to the over-exposure speckle ratio of the first area of ​​the speckle pattern, and performing coarse adjustment according to the average amplitude value straight line.

2. The rapid AE method based on amplitude value according to claim 1, characterized in that: In step S2, if there are overlapping pixels in adjacent pixel regions of different speckle centers, the amplitude values ​​of adjacent pixel regions of the two speckle centers are calculated respectively, and the adjacent pixel region of the speckle center with the smaller amplitude value is discarded.

3. The rapid AE method based on amplitude value according to claim 1, characterized in that: The calculation of the average amplitude value in step S3 includes: Step S31: Calculate the average amplitude value of each pixel area adjacent to the center of the speckle; Step S32: obtaining the number of the average amplitude values ​​greater than the amplitude threshold, and dividing the number by the number of the speckle centers to obtain an over-exposed speckle ratio.

4. The rapid AE method based on amplitude value according to claim 1, characterized in that: When calculating the over-exposed speckle ratio in step S3, the speckle centers with amplitude values ​​greater than the amplitude threshold are marked as preliminary over-exposed speckle centers, and then the preliminary over-exposed speckle centers within the first distance are merged to obtain final over-exposed speckle centers, and the number of the final over-exposed speckle centers is divided by the number of the speckle centers to obtain the over-exposed speckle ratio.

5. The rapid AE method based on amplitude value according to claim 1, characterized in that: The rough adjustment in step S4 includes: Step S41: Obtaining an average amplitude value straight line; Step S42: determining an ideal amplitude value according to the current distance and the over-exposed speckle ratio; Step S43: Calculate the coarse exposure time according to the average amplitude value straight line and the ideal amplitude value.

6. The rapid AE method based on amplitude value according to claim 5, characterized in that: Step S41 includes: Step S411: acquiring speckles in a first area, and calculating an overexposure ratio a of the speckles in the first area; Step S412: Calculate the average amplitude value straight line according to the formula y=a*b*x; wherein b is a coefficient related to the depth camera.

7. The rapid AE method based on amplitude value according to claim 1, characterized in that: When fine-tuning is performed in step S4, the fine-tuning exposure time is calculated according to the formula nt=ct+s2*(ir-cr) / pl, where s2 is a scaling factor greater than 0 and less than 1; wherein pl is the slope of the average amplitude value straight line, cr is the overexposure ratio, ct is the exposure time, ir is the ideal overexposure ratio, and nt is the fine-tuning exposure time.

8. A rapid AE system based on amplitude value, used to implement the rapid AE method based on amplitude value according to any one of claims 1 to 7, characterized in that: include: An acquisition module, used for acquiring a speckle pattern and a corresponding exposure time; An extraction module, used for extracting a speckle center and an adjacent pixel area of ​​each speckle center on the speckle image; A calculation module, used to calculate the average amplitude value of the adjacent pixel area of ​​each speckle center and determine the over-exposed speckle ratio; an adjusting module, for judging whether the over-exposed speckle ratio is near an ideal value, and performing fine adjustment when it is near the ideal value; and for obtaining an average amplitude value straight line according to the over-exposed speckle ratio of the first area of ​​the speckle pattern when it is not near the ideal value, and performing coarse adjustment according to the average amplitude value straight line.

9. A rapid AE device based on amplitude value, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to perform the steps of the amplitude value-based fast AE method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed, the steps of the amplitude value-based fast AE method described in any one of claims 1 to 7 are implemented.

Images