Binocular camera for quickly calculating hand depth and electronic equipment

Abstract

The invention discloses a binocular camera for quickly calculating hand depth and electronic equipment, and the camera comprises a first receiver which is used for receiving a first signal of a palm and generating a first image; the second receiver is used for receiving a second signal of the palm and generating a second image; and the processor is used for detecting a first hand region on the first image, reducing the first hand region to a preset size, reducing the second image at the same magnification as the first hand region, estimating a second hand region on the second image, and then carrying out depth calculation. According to the method, the calculation amount of search and reconstruction is greatly reduced, a special Asic chip or special hardware is not needed, the depth of the hand is calculated in real time, and help is provided for subsequent gesture recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of binocular cameras, and in particular to a binocular camera for quickly calculating hand depth and an electronic device. Background Art

[0002] In gesture recognition, depth information is very beneficial for improving the performance of gesture recognition. It can provide more spatial context, reduce the ambiguity of two-dimensional images, and improve the robustness of the algorithm under various environmental conditions.

[0003] The main functions of depth information are:

[0004] 1. Added spatial information: Depth data adds an extra dimension to gestures, helping the algorithm understand the shape and movement of gestures in space.

[0005] 2. Improve robustness: In environments with complex backgrounds or changing lighting, depth information can help the algorithm better distinguish between hands and background.

[0006] 3. Reduce ambiguity: When only two-dimensional information is available, some gestures may look similar. Depth data can help resolve this visual ambiguity and improve the accuracy of gesture recognition.

[0007] 4. Handling occlusion: Self-occlusion, which is common in gestures (e.g. one finger occluding another finger), is easier to distinguish and handle in 3D space.

[0008] Binocular cameras can obtain dense depth through binocular stereo matching. Unlike tof or monocular structured light depth cameras, binocular cameras do not require special hardware except for the two cameras, so their hardware cost is relatively low, and they have good autonomy and flexibility, so they are widely applicable. However, their disadvantages are also obvious. The binocular stereo matching algorithm is computationally intensive and requires a lot of computing resources. Therefore, in actual use, dedicated Asic chips or dedicated hardware are generally required for depth calculation, which increases the cost.

[0009] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0010] To this end, the present invention identifies the hand area, reduces the hand area, and obtains the depth value of the hand area by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. There is no need to use a dedicated Asic chip or dedicated hardware, and the depth of the hand is calculated in real time to provide assistance for subsequent gesture recognition.

[0011] In a first aspect, the present invention provides a binocular camera for fast hand depth calculation, characterized in that it includes:

[0012] A first receiver, used for receiving a first signal of a palm and generating a first image;

[0013] A second receiver, used for receiving a second signal of the palm and generating a second image;

[0014] A processor is used to detect a first hand area on the first image, reduce the first hand area to a preset size, reduce the second hand area by the same magnification as the first hand area on the second image, and perform depth calculation after estimating a second hand area on the second image.

[0015] Optionally, the binocular camera for fast hand depth calculation is characterized by further comprising:

[0016] A light source is used to illuminate the palm according to the brightness of the first image.

[0017] Optionally, the binocular camera for fast hand depth calculation is characterized in that the processor comprises:

[0018] A detection module, configured to detect a first hand region on the first image;

[0019] A first reduction module, used to reduce the first hand area to a preset size at a first magnification;

[0020] a second reduction module, configured to reduce the second image by the first magnification, and calculate a second hand region corresponding to the first hand region on the reduced second image;

[0021] A depth calculation module is used to perform depth calculation on the first hand area and the second hand area.

[0022] Optionally, the binocular camera for fast hand depth calculation is characterized in that the second reduction module comprises:

[0023] A reduction unit, configured to reduce the second image by the first magnification;

[0024] A symmetry unit is used to obtain a second hand region corresponding to the first hand region on the reduced second image according to the position of the first hand region in the first image.

[0025] Optionally, the binocular camera for fast hand depth calculation is characterized in that the depth calculation module estimates the depth of the hand area according to the first magnification, determines the search range, and then performs depth calculation.

[0026] Optionally, the aforementioned binocular camera for fast hand depth calculation is characterized in that the depth calculation module comprises:

[0027] A first estimating unit, configured to estimate a palm depth according to the first magnification;

[0028] A first disparity unit, configured to estimate a search range according to the estimated palm depth, and calculate a disparity between a first palm region and a second palm region; wherein the first palm region is a palm region in the first hand region; and the second palm region is a palm region in the second hand region;

[0029] The first calculation unit is used to calculate the depth value of each pixel in the first palm area according to the disparity and the parameters of the binocular camera.

[0030] Optionally, the aforementioned binocular camera for fast hand depth calculation is characterized in that the depth calculation module comprises:

[0031] a second estimating unit, configured to estimate a finger depth according to a shape of the first hand region and the first magnification;

[0032] A second disparity unit, configured to estimate a search range according to the estimated finger depth, and calculate a disparity between a first finger region and a second finger region; wherein the first finger region is a finger region in the first hand region; and the second finger region is a finger region in the second hand region;

[0033] A second calculation unit is configured to calculate a depth value of each pixel in the first finger area according to the disparity and parameters of the binocular camera.

[0034] Optionally, the binocular camera for fast hand depth calculation is characterized in that the second parallax unit calculates multiple fingers in sequence and adjusts the search range of subsequent fingers according to the parallax of the current finger.

[0035] Optionally, the binocular camera for fast hand depth calculation is characterized in that the detection module comprises:

[0036] a detection unit, configured to detect a third hand region and a fourth hand region on the first image, and to detect a fifth hand region and a sixth hand region on the second image;

[0037] A pairing unit is used to pair the third hand region, the fourth hand region, the fifth hand region and the sixth hand region according to the epipolar constraint.

[0038] In a second aspect, the present invention provides an electronic device, characterized in that it comprises a binocular camera for fast hand depth calculation as described in any one of the above items.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention first identifies the hand area, and in subsequent steps only processes the hand area, thereby reducing the amount of data processing and improving processing efficiency.

[0041] The present invention reduces the image before calculation, thereby making the calculation amount constant and avoiding the problem of exceeding the calculation capacity. At the same time, data conversion is performed when calculating the depth, so that the amount of data to be calculated is greatly reduced.

[0042] The present invention optimizes the hand area and shrinks it to greatly reduce the amount of calculation. The depth value of the hand area is obtained by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. There is no need to use a dedicated Asic chip or dedicated hardware. The depth of the hand is calculated in real time, which helps subsequent gesture recognition and is cost-effective.

[0043] The present invention only needs to detect the first hand area without detecting the second hand area, thereby reducing the amount of palm detection. While reducing the amount of calculation, it is also suitable for non-homologous binocular cameras, such as a non-homologous binocular camera composed of an RGB camera and an infrared camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following descriptions are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0045] Figure 1 This is a schematic diagram of the structure of a binocular camera for fast hand depth calculation in an embodiment of the present invention;

[0046] Figure 2This is a schematic diagram of the structure of a binocular camera for fast hand depth calculation in an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of the structure of a processor in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of a palm in an embodiment of the present invention;

[0049] Figure 5 is a schematic diagram of a structure for calculating a second hand area in an embodiment of the present invention;

[0050] Figure 6 A schematic diagram of a structure for performing depth calculation in an embodiment of the present invention;

[0051] Figure 7 A schematic diagram of palm area division in an embodiment of the present invention;

[0052] Figure 8 This is another structural schematic diagram of performing depth calculation in an embodiment of the present invention;

[0053] Fig. 9 This is a schematic diagram of a structure for detecting a hand area in an embodiment of the present invention;

[0054] Fig.10 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention.

[0055] 1--Hand area;

[0056] 2--Palm area;

[0057] 3--Finger area; DETAILED DESCRIPTION

[0058] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0059] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0060] For a stereo camera without a palm rest, the size of the hand in the image changes as the distance between the hand and the stereo camera changes. To ensure that the hand can be effectively recognized in a larger range, the resolution of the stereo camera is usually higher. When the hand is close to the stereo camera, the hand occupies a larger proportion in the image, and even if only the depth calculation is performed on the hand area, it still requires a large amount of calculation.

[0061] A binocular camera for fast hand depth calculation provided in an embodiment of the present invention is intended to be suitable for binocular depth calculation of hands at different distances, reduce the amount of calculation, and lower the requirements for chip computing power when the binocular camera is used for hand detection, thereby solving the problems existing in the prior art.

[0062] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0063] The present invention identifies and reduces the hand area, and obtains the depth value of the hand area by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. It does not need to use a dedicated Asic chip or dedicated hardware, and calculates the depth of the hand in real time, providing assistance for subsequent gesture recognition.

[0064] Figure 1 FIG. 1 is a schematic diagram of the structure of a binocular camera for fast hand depth calculation in an embodiment of the present invention. Figure 1 As shown, a binocular camera for fast hand depth calculation in an embodiment of the present invention includes:

[0065] The first receiver is used to receive a first signal from a palm and generate a first image.

[0066] Specifically, the first receiver is responsible for capturing the first signal of the palm, which generally refers to the image information of the palm obtained from a specific angle or position. The first receiver includes a camera sensor, such as a CMOS or CCD sensor, for converting the optical signal into an electrical signal to generate a digital image. The first image can be an RGB image, an infrared image, or any other type of image.

[0067] The second receiver is used to receive a second signal from the palm and generate a second image.

[0068] Specifically, similar to the first receiver, the second receiver captures a second signal of the palm from another angle or position to generate another image. The second receiver also includes a camera sensor for capturing images of the palm from different viewing angles. The second image and the first image can be of the same type, such as both are RGB images, both are infrared images, etc. The second image and the first image can also be of different types, such as the first image is an RGB image and the second image is an infrared image. When the first image and the second image are of different types, the first receiver and the second receiver need to be strictly calibrated so that the first image and the second image can be aligned.

[0069] A processor is used to detect a first hand area on the first image, reduce the first hand area to a preset size, reduce the second hand area by the same magnification as the first hand area on the second image, and perform depth calculation after estimating a second hand area on the second image.

[0070] Specifically, the processor is the core of the binocular camera system, responsible for processing images from two receivers and performing tasks such as hand area detection, image scaling, stereo matching, and depth calculation. First, the hand area is detected on the first image using image processing techniques (such as edge detection, color segmentation, machine learning algorithms, etc.). In order to simplify subsequent processing, the detected first hand area is reduced to a preset size. At the same time, the second image is scaled at the same magnification to ensure that the hand areas in the two images are consistent in size. Using the known correspondence between the first image and the second image and image processing technology, the approximate position of the hand area is estimated in the second image. By comparing the differences in the hand areas in the two images (such as pixel displacement), the depth information of the hand area is calculated using the principle of triangulation.

[0071] Figure 2 FIG. 1 is a schematic diagram of the structure of a binocular camera for fast hand depth calculation in an embodiment of the present invention. Figure 2 As shown, compared with the above-mentioned embodiment, a binocular camera for fast hand depth calculation in the embodiment of the present invention further includes:

[0072] A light source is used to illuminate the palm according to the brightness of the first image.

[0073] Specifically, the main function of the light source is to provide additional illumination to enhance the image quality of the palm or hand area in the first and second receivers. By adjusting the intensity or direction of the light source according to the brightness of the first image, it is possible to ensure that a clear, moderately bright hand image is obtained regardless of changes in ambient light.

[0074] The light source has an adjustable brightness function so that it can be automatically adjusted according to the real-time brightness of the first image. This helps to avoid overexposure or underexposure of the image, thereby improving the image quality. At the same time, the light source can provide uniform and soft lighting to reduce shadows and reflections in the hand area. This helps to capture the contours and features of the hand more accurately and improve the accuracy of hand area detection. The light source can be an LED light source or an infrared light source. LED light sources are widely used due to their high efficiency, low consumption, long life and easy control. They can produce light of various colors and brightness, suitable for different lighting needs. In some applications, such as at night or in scenes where visible light interference needs to be avoided, infrared light sources may be a better choice. Infrared light is invisible to the human eye, but can be captured by the camera, thereby achieving interference-free lighting. The light source can be a ring light source. Designing the light source in a ring shape and placing it around the camera lens can provide uniform and soft lighting effects and reduce shadows and reflections. The light source can also be controlled in groups to achieve different lighting modes, angles and brightness to meet the lighting needs of palms at different angles and positions.

[0075] Figure 3 FIG. 1 is a schematic diagram of the structure of a processor in an embodiment of the present invention. Figure 3 As shown, a processor in an embodiment of the present invention includes:

[0076] A detection module is used to detect a first hand area on the first image.

[0077] Specifically, a palm detection is required. A trained hand detection model is needed to identify the hand area in the first image. During processing, the first image is first preprocessed, such as denoising, contrast enhancement, etc., to facilitate subsequent hand detection. Image processing techniques (such as skin color detection, edge detection, contour recognition, or deep learning-based methods such as convolutional neural networks) are then used to locate and extract the hand area. The hand area is usually the area in the image that matches the shape and size of the hand, which is determined by identifying specific features (such as skin color, finger shape, etc.). Once the hand area is detected, these areas are marked with bounding boxes or other shapes for subsequent processing. Figure 4 As shown, the hand region is often a part of the image. The subsequent processing of this embodiment is all performed on the hand region, and there is no need to process other parts, thereby reducing the amount of calculation.

[0078] The first reduction module is used to reduce the first hand area to a preset size at a first magnification.

[0079] Specifically, in order to facilitate subsequent depth calculations and reduce the amount of calculation, the hand area is reduced to a uniform size. The preset size (such as fixed width and height) is a parameter smaller than the size of the first image, such as 128*128, 180*180 or other arbitrary sizes. The size of the preset size depends on the requirements of subsequent algorithms and the limitations of computing resources. The larger the preset size, the more depth values ​​that can be obtained subsequently; the smaller the preset size, the smaller the amount of calculation and the faster the calculation speed. The ratio of reducing the first hand area to the preset size is the first ratio. The first ratio is a number less than 1.

[0080] The second reduction module is used to reduce the second image at the first magnification and calculate a second hand area corresponding to the first hand area on the reduced second image.

[0081] Specifically, unlike the first image where only the first hand region is reduced, this step reduces the entire second image. The size of the reduced second image is larger than the preset size. However, at this time, the second hand region on the second image corresponds to the first hand region. According to the positional correspondence between the first hand region and the second hand region, the range of the second hand region can be obtained.

[0082] A depth calculation module is used to perform depth calculation on the first hand area and the second hand area.

[0083] Specifically, the depth information of the hand is calculated using the principle of binocular stereo vision. The depth calculation in this step is only performed for the first hand area and the second hand area, so that the reconstruction area can be very small. Since the binocular camera system has a fixed baseline distance and the two cameras shoot the same scene, the depth can be estimated by comparing the position difference (i.e., parallax) of the corresponding points in the two hand areas. For each point (or each pixel) in the first hand area, find the corresponding point in the second hand area and calculate the parallax between them. Then, using the known baseline distance, focal length and other parameters and parallax, the depth value of the point is calculated by the triangulation principle. Repeat this process to generate a depth map of the entire hand area. The obtained depth map can be used in a variety of applications, such as gesture recognition, virtual reality interaction, robot hand-eye coordination, etc. In addition, the depth map can be further processed or analyzed as needed.

[0084] The calculation area of ​​this embodiment is small, and the accuracy is higher than that of TOF technology. Taking the resolution of the RGB binocular camera as 1080P and the preset size as 128*128 as an example, when the image is directly reconstructed in depth, the search range is determined according to the limit length of the binocular camera and the reconstruction range. Here, it is assumed that the reconstruction range is 30mm-1m and the baseline length is 100mm. The calculation amount of the depth reconstruction is 1920*1080*384. When the solution of this embodiment is adopted, the palm is deeply reconstructed, and the calculation amount is 128*128*128*n, which is n / 380 when the image is directly reconstructed in depth, where n is the number of palms, which greatly reduces the calculation amount. If this embodiment only calculates the local area m of the palm, the calculation amount is n*128*128*128*(m / 128 / 128), where n is the number of palms and m is the total number of pixels in the specified calculation area. The calculation amount is m*n / (380*128*128) when the image is directly reconstructed. <n / 380。

[0085] In some embodiments, the depth calculation module estimates the depth of the hand area according to the first magnification, determines the search range, and then performs depth calculation. Using the image information obtained by the first magnification, combined with the detection of the hand region of interest (ROI), the approximate depth information of the hand can be predicted by a deep learning model or triangulation. Based on the approximate depth information, the search range of binocular reconstruction can be estimated. After the search and matching is completed, the disparity can be calculated to obtain the depth value.

[0086] Figure 5 FIG. 1 is a schematic diagram of a structure for calculating a second hand region in an embodiment of the present invention. Figure 5 As shown, in an embodiment of the present invention, a step of calculating a second hand area includes:

[0087] A reduction unit is used to reduce the second image at the first magnification.

[0088] Specifically, in order to maintain the consistency of the relative size and position relationship of the hand regions in the two images, the second image needs to be scaled at the same magnification. Since the first hand region has been reduced to a preset size in step S3, the same reduction operation needs to be performed on the second image here.

[0089] A symmetry unit is used to obtain a second hand region corresponding to the first hand region on the reduced second image according to the position of the first hand region in the first image.

[0090] Specifically, it is first necessary to determine the specific position of the first hand area in the first image. Since the detection module has detected the first hand area, the position of the first hand area relative to the first image can be directly obtained. The first hand area in this embodiment is a symmetrical shape, such as a rectangle, a circle, etc. The second hand area has the same shape as the first hand area. Both the first hand area and the second hand area are a larger area range including the hand, which is a rough range used for corresponding processing rather than accurately identifying the palm. For example, the first hand area is a rectangular area, the size of the first image is 1080*1080, the first hand area is in the range of (200, 300) to (700, 600), the size of the second image is 1080*1080, and calculation shows that the position of the first hand area in the first image is (200 / 1080, 700 / 1080) in the horizontal direction and (300 / 1080, 600 / 1080) in the vertical direction. The size of the reduced second image is x*y, then the second hand area is in the range of (x*200 / 1080, y*300 / 1080) to (x*700 / 1080, y*600 / 1080).

[0091] This embodiment calculates the position of the second hand area in the reduced second image based on the position of the first hand area in the first image, without the need for hand detection, thereby greatly reducing the amount of calculation and shortening the palm detection time.

[0092] Figure 6 FIG. 1 is a schematic diagram of a structure for performing depth calculation in an embodiment of the present invention. Figure 6 As shown, a step of performing depth calculation in an embodiment of the present invention includes:

[0093] The first estimating unit is configured to estimate the palm depth according to the first magnification.

[0094] Specifically, the palm area is further extracted from the detected hand area. Figure 7 As shown, the palm region and the finger region constitute the hand region. The palm region can usually be identified by analyzing features such as the hand contour, skin color distribution, and the geometric relationship between the fingers and the palm. For example, image segmentation techniques such as edge detection and region growing can be used to separate the palm and finger parts. Once the palm region is extracted, these regions are marked using bounding boxes, masks, or other shapes for subsequent processing.

[0095] The depth of the palm in three-dimensional space is preliminarily estimated using the first magnification (i.e., the scaling ratio of the hand area in the image) and prior knowledge (e.g., average hand size, average distance from the camera to the hand, etc.). This estimate will serve as a starting point or reference for subsequent calculations. First, determine the approximate size of the palm in the first image (which can be inferred from the detected hand area). Then, considering the effect of image scaling on the size of the object, use the first magnification to infer the original size (i.e., the size before scaling). Combined with camera parameters (e.g., focal length, baseline distance) and prior knowledge, a simple geometric relationship or lookup table is used to estimate the preliminary depth of the palm.

[0096] The first disparity unit is used to estimate the search range according to the estimated palm depth and calculate the disparity of the first palm area and the second palm area.

[0097] Specifically, the first palm area is a palm area in the first hand area; and the second palm area is a palm area in the second hand area.

[0098] Based on the preliminary depth estimation of the palm, determine the reasonable range of disparity values ​​for searching the corresponding palm area in the disparity map. Then, accurately match the palm area in the first image and the second image to calculate the disparity. According to the preliminary depth estimation of the palm and the camera parameters, calculate the possible disparity range (i.e., search range) in the disparity map. Use a binocular vision algorithm (such as block matching, SGBM, etc.) to accurately match between the first palm area and the second palm area to find the best corresponding point for each pixel in the two images. Calculate the horizontal position difference between the matching point pairs, i.e., the disparity.

[0099] The first calculation unit is used to calculate the depth value of each pixel in the first palm area according to the disparity and the parameters of the binocular camera.

[0100] Specifically, the disparity value is converted into depth information using the binocular vision principle. Based on the camera's internal parameters (focal length, photosensitive element size, etc.) and external parameters (baseline distance, camera relative posture, etc.), as well as the disparity and triangulation principles, the actual depth value of each pixel relative to the camera can be calculated. In this way, we can obtain the three-dimensional position information of all pixels in the first palm area, thereby realizing the depth perception of the palm.

[0101] Figure 8 FIG. 1 is another schematic diagram of a structure for performing depth calculation in an embodiment of the present invention. Figure 8 As shown, another step of performing depth calculation in an embodiment of the present invention includes:

[0102] The second estimating unit is configured to estimate the finger depth according to the shape of the first hand region and the first magnification.

[0103] Specifically, it is similar to extracting the palm region, but the goal is to identify and separate the finger parts in the hand image. This is usually more complicated because the fingers may have different lengths, degrees of curvature, and occlude each other. This can be achieved using methods based on shape analysis, contour tracking, or deep learning. Once the finger regions are identified, they are marked using appropriate methods (such as bounding boxes, masks, etc.) for subsequent processing.

[0104] The shape information of the first hand region (such as the length, width, curvature of the finger, etc.) and the first magnification of the image zoom are used to preliminarily estimate the depth of the finger in three-dimensional space. This estimated value will serve as a reference or starting point for the subsequent calculation of the finger depth. First, the position and shape of the finger region in the first hand region are identified through image processing techniques (such as edge detection, contour extraction, etc.). Then, the general anatomical structure of the hand and prior knowledge (such as the average length and width ratio of the finger, etc.) and the first magnification of the image zoom are combined to estimate the preliminary depth of the finger. This estimate is an average or a weighted average based on the shape and position of the finger.

[0105] The second disparity unit is used to estimate the search range according to the estimated finger depth and calculate the disparity between the first finger area and the second finger area.

[0106] Specifically, the first finger area is a finger area in the first hand area; the second finger area is a finger area in the second hand area.

[0107] Purpose: Based on the preliminary depth estimation of the finger, determine the reasonable range of disparity values ​​for searching the corresponding finger area in the disparity map. Then, accurately match the finger area in the first image and the second image to calculate the disparity. According to the preliminary depth estimation of the finger and the geometric parameters of the stereo camera (such as baseline distance, focal length, etc.), calculate the possible disparity range (i.e., search range) in the disparity map. Use a stereo vision algorithm (such as block matching, SGBM, etc.) to accurately match between the first finger area and the second finger area. Since the finger area may contain finer structures and higher texture changes, more complex matching strategies or algorithm parameters may be required to ensure the accuracy of the matching. Calculate the horizontal position difference between the matching point pairs, i.e., the disparity. For each pixel in the finger area, it is necessary to find its best corresponding point in the other image and calculate the disparity between them.

[0108] A second calculation unit is configured to calculate a depth value of each pixel in the first finger area according to the disparity and parameters of the binocular camera.

[0109] Specifically, the disparity value is converted into depth information using the binocular vision principle. Based on the camera's internal parameters (focal length, photosensitive element size, etc.) and external parameters (baseline distance, camera relative posture, etc.), as well as the disparity and triangulation principles, the actual depth value of each pixel relative to the camera can be calculated. In this way, we can obtain the three-dimensional position information of all pixels in the first finger area, thereby realizing the depth perception of the finger.

[0110] This embodiment can calculate the depth of each finger individually, which provides more detailed three-dimensional information for gesture recognition and hand motion capture. For example, this embodiment can estimate the finger posture individually, and through detection between multiple frames, it can detect the finger movement state, meeting the needs of various application scenarios.

[0111] In some embodiments, the second disparity unit calculates multiple fingers in sequence and adjusts the search range of subsequent fingers according to the disparity of the current finger. When processing a hand region containing multiple fingers, each finger needs to be calculated in sequence, and the search range of subsequent fingers in the disparity map can be optimized or adjusted according to the disparity result of the current finger. This strategy helps to improve the efficiency and accuracy of the depth calculation of the entire finger region. During execution, the number of fingers in the hand region and their positions in the first hand region are first determined. According to the estimation of the second estimation unit or other prior knowledge, an initial search range (in the disparity map) is set for each finger. Each finger in the hand region is traversed. For the current finger (assuming it is the i-th finger), the disparity is calculated between the first finger region and the second finger region corresponding to it using a binocular vision algorithm. Within the search range of the current finger, block matching, SGBM or other binocular vision algorithms are used to find the best matching point pair between the first finger region and the second finger region. The horizontal position difference between the matching point pairs, i.e., the disparity, is calculated. According to the disparity result of the current finger, the rationality of its depth value is evaluated (for example, whether it meets the constraints of the hand anatomical structure). If the disparity result of the current finger is reasonable, the possible disparity range of the adjacent fingers (especially the subsequent fingers) in the disparity map can be inferred based on the disparity and the geometric parameters of the binocular camera. Use this inferred disparity range to optimize or adjust the search range of the subsequent fingers in the disparity map. This helps to reduce the search space, improve matching efficiency, and may improve the accuracy of the disparity calculation of subsequent fingers. Repeat steps 2 to 4 for the next finger until all fingers in the hand area have been processed. Output the disparity map or disparity value of each finger, which will be used to calculate the depth value of each pixel in each finger area in the subsequent steps. By dynamically adjusting the search range, step S55 can more efficiently process the hand area containing multiple fingers and may improve the accuracy of the depth calculation. However, it should be noted that the adjustment of the search range should be based on reasonable assumptions and constraints to avoid introducing additional errors or uncertainties. The current finger is the finger whose disparity is currently being calculated, and the subsequent finger is the finger whose disparity has not yet been calculated.

[0112] Fig. 9 FIG. 1 is a schematic diagram of a structure for detecting a hand area in an embodiment of the present invention. Fig. 9 As shown, in an embodiment of the present invention, a step of detecting a hand area includes:

[0113] A detection unit is used to detect a third hand region and a fourth hand region on the first image, and to detect a fifth hand region and a sixth hand region on the second image.

[0114] Specifically, multiple hand regions are detected by using computer vision techniques, such as edge detection, contour extraction, morphological operations, or deep learning models. This step detects at least two hand regions on the first image and at least two hand regions on the second image. Each detected hand region should be correctly marked and identified so that pairing and disparity calculation can be performed correctly in subsequent steps.

[0115] A pairing unit is used to pair the third hand region, the fourth hand region, the fifth hand region and the sixth hand region according to the epipolar constraint.

[0116] Specifically, the epipolar constraint means that any point in the left image must be matched on the epipolar line of the right image, and vice versa. This constraint reduces the matching search space, thereby improving matching efficiency and accuracy. Using the epipolar geometry relationship, the area paired with the third hand area is determined in the fifth and sixth hand areas, and the area paired with the fourth hand area is determined in the fifth and sixth hand areas. This involves calculating the projection of each area on the corresponding epipolar line and finding the most likely matching point. At the same time, further verification and optimization are needed to ensure the accuracy of the pairing, such as by analyzing the size, shape and relative position of the hand area to confirm the validity of the pairing.

[0117] This embodiment processes multiple detected hand regions and pairs these regions based on epipolar constraints, which is critical for achieving accurate gesture recognition and depth perception, especially in complex scenarios that require processing multiple hands or fingers.

[0118] Fig.10 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. Fig.10 The electronic device 600 according to this embodiment of the present invention is described. Fig.10 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0119] like Fig.10 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0120] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps of various exemplary embodiments of the present invention described in the above-mentioned hand depth fast calculation binocular camera part of this specification. For example, the processing unit 610 can execute the following steps: Figure 1 Follow the steps shown in .

[0121] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0122] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a grid environment.

[0123] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0124] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 600, and / or any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more grids (e.g., a local area network (LAN), a wide area network (WAN), and / or a public grid, such as the Internet) through a grid adapter 660. The grid adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Fig.10 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0125] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0126] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A binocular camera for fast hand depth calculation, characterized in that: include: A first receiver, used for receiving a first signal of a palm and generating a first image; A second receiver, used for receiving a second signal of the palm and generating a second image; A processor is used to detect a first hand area on the first image, reduce the first hand area to a preset size, reduce the second hand area by the same magnification as the first hand area on the second image, and perform depth calculation after estimating the second hand area on the second image.

2. The binocular camera for fast hand depth calculation according to claim 1, characterized in that: Also includes: A light source is used to illuminate the palm according to the brightness of the first image.

3. The binocular camera for fast hand depth calculation according to claim 1, characterized in that: The processor comprises: A detection module, configured to detect a first hand region on the first image; A first reduction module, used to reduce the first hand area to a preset size at a first magnification; a second reduction module, configured to reduce the second image by the first magnification, and calculate a second hand region corresponding to the first hand region on the reduced second image; A depth calculation module is used to perform depth calculation on the first hand area and the second hand area.

4. The binocular camera for fast hand depth calculation according to claim 3, characterized in that: The second reduction module comprises: A reduction unit, configured to reduce the second image by the first magnification; A symmetry unit is used to obtain a second hand region corresponding to the first hand region on the reduced second image according to the position of the first hand region in the first image.

5. The binocular camera for fast hand depth calculation according to claim 3, characterized in that: The depth calculation module estimates the depth of the hand area according to the first magnification, determines the search range, and then performs depth calculation.

6. The binocular camera for fast hand depth calculation according to claim 3, characterized in that: The depth calculation module comprises: A first estimating unit, configured to estimate a palm depth according to the first magnification; A first disparity unit, configured to estimate a search range according to the estimated palm depth, and calculate a disparity between a first palm region and a second palm region; wherein the first palm region is a palm region in the first hand region; and the second palm region is a palm region in the second hand region; The first calculation unit is used to calculate the depth value of each pixel in the first palm area according to the disparity and the parameters of the binocular camera.

7. The binocular camera for fast hand depth calculation according to claim 3, characterized in that: The depth calculation module includes: a second estimating unit, configured to estimate a finger depth according to a shape of the first hand region and the first magnification; A second disparity unit, configured to estimate a search range according to the estimated finger depth, and calculate a disparity between a first finger region and a second finger region; wherein the first finger region is a finger region in the first hand region; and the second finger region is a finger region in the second hand region; A second calculation unit is configured to calculate a depth value of each pixel in the first finger area according to the disparity and parameters of the binocular camera.

8. The binocular camera for fast hand depth calculation according to claim 7, characterized in that: The second disparity unit calculates the multiple fingers in sequence, and adjusts the search range of subsequent fingers according to the disparity of the current finger.

9. The binocular camera for fast hand depth calculation according to claim 3, characterized in that: The detection module comprises: a detection unit, configured to detect a third hand region and a fourth hand region on the first image, and to detect a fifth hand region and a sixth hand region on the second image; A pairing unit is used to pair the third hand region, the fourth hand region, the fifth hand region and the sixth hand region according to the epipolar constraint.

10. An electronic device, characterized in that: A binocular camera for fast hand depth calculation comprising any one of claims 1 to 9.

Images