Hand depth calculation binocular camera and electronic equipment

Abstract

A binocular camera for hand depth calculation and an electronic device, comprising: a first receiver for receiving a first signal of a palm and generating a first image; the second receiver is used for receiving a second signal of the palm and generating a second image; the processor is used for detecting a first hand region on the first image, reducing the first hand region to a preset size at a first magnification, reducing the second image at the first magnification, and performing palm detection on the reduced second image to obtain a second hand region; and depth calculation is carried out on the first hand region and the second hand region. According to the method, the calculation amount of search and reconstruction is greatly reduced, a special Asic chip or special hardware is not needed, the depth of the hand is calculated in real time, and help is provided for subsequent gesture recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of binocular cameras, and in particular to a hand depth calculation binocular camera and electronic equipment. Background Art

[0002] In gesture recognition, depth information is very beneficial for improving the performance of gesture recognition. It can provide more spatial context, reduce the ambiguity of two-dimensional images, and improve the robustness of the algorithm under various environmental conditions.

[0003] The main functions of depth information are:

[0004] 1. Added spatial information: Depth data adds an extra dimension to gestures, helping the algorithm understand the shape and movement of gestures in space.

[0005] 2. Improve robustness: In environments with complex backgrounds or changing lighting, depth information can help the algorithm better distinguish between hands and background.

[0006] 3. Reduce ambiguity: When only two-dimensional information is available, some gestures may look similar. Depth data can help resolve this visual ambiguity and improve the accuracy of gesture recognition.

[0007] 4. Handling occlusion: Self-occlusion, which is common in gestures (e.g. one finger occluding another finger), is easier to distinguish and handle in 3D space.

[0008] Binocular cameras can obtain dense depth through binocular stereo matching. Unlike tof or monocular structured light depth cameras, binocular cameras do not require special hardware except for the two cameras, so their hardware cost is relatively low, and they have good autonomy and flexibility, so they are widely applicable. However, their disadvantages are also obvious. The binocular stereo matching algorithm is computationally intensive and requires a lot of computing resources. Therefore, in actual use, dedicated Asic chips or dedicated hardware are generally required for depth calculation, which increases the cost.

[0009] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0010] To this end, the present invention identifies the hand area, reduces the hand area, and obtains the depth value of the hand area by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. There is no need to use a dedicated Asic chip or dedicated hardware, and the depth of the hand is calculated in real time to provide assistance for subsequent gesture recognition.

[0011] In a first aspect, the present invention provides a hand depth calculation binocular camera, characterized in that it includes:

[0012] A first receiver, used for receiving a first signal of a palm and generating a first image;

[0013] A second receiver, used for receiving a second signal of the palm and generating a second image;

[0014] A processor is used to detect a first hand area on the first image, reduce the first hand area to a preset size at a first magnification, reduce the second image at the first magnification, perform palm detection on the reduced second image to obtain a second hand area, and then perform depth calculation on the first hand area and the second hand area.

[0015] Optionally, the palm-scanning binocular camera is characterized by further comprising:

[0016] A light source is used to illuminate the palm according to the brightness of the first image.

[0017] Optionally, the palm-scanning binocular camera is characterized in that the processor comprises:

[0018] A detection module, configured to detect a first hand region on the first image;

[0019] A first reduction module, used to reduce the first hand area to a preset size at a first magnification;

[0020] a second reduction module, configured to reduce the second image by the first magnification, and then perform palm detection on the reduced second image to obtain a second hand region;

[0021] A depth calculation module is used to perform depth calculation on the first hand area and the second hand area.

[0022] Optionally, the hand depth calculation binocular camera is characterized in that the second reduction module comprises:

[0023] A first reduction unit, configured to reduce the second image by the first magnification to obtain a third image;

[0024] an estimation region unit, configured to obtain a corresponding estimation region on the third image according to the first magnification, the position of the first hand region in the first image, and an epipolar constraint;

[0025] The hand region unit is used to perform palm detection in the estimated region to obtain a second hand region.

[0026] Optionally, the hand depth calculation binocular camera is characterized in that the depth calculation module comprises:

[0027] A palm unit, used for obtaining a first palm area in the first hand area, and obtaining a second palm area in the second hand area;

[0028] A first disparity unit, configured to estimate a search range according to a size of the first palm area or the second palm area, and calculate a disparity between the first palm area and the second palm area;

[0029] The first depth unit is used to calculate the depth value of each pixel in the first palm area according to the disparity and the parameters of the binocular camera.

[0030] Optionally, the hand depth calculation binocular camera is characterized in that the depth calculation module comprises:

[0031] A finger unit, used for obtaining a first finger region in the first hand region and obtaining a second finger region in the second hand region;

[0032] A second disparity unit, used to determine the number of each finger in the first finger area and the second finger area, and calculate the disparity for fingers with the same number;

[0033] The second depth unit is used to calculate the depth value of each pixel in the first finger area according to the disparity and the parameters of the binocular camera.

[0034] Optionally, the hand depth calculation binocular camera is characterized in that the second disparity unit calculates the search range of each finger according to the width of each finger, and then matches and calculates the disparity.

[0035] Optionally, the hand depth calculation binocular camera is characterized in that the second disparity calculates the search range according to the maximum width of the finger, and then matches and calculates the disparity.

[0036] Optionally, the hand depth calculation binocular camera is characterized in that the depth calculation module determines the search range according to the shape of the first hand area and the first magnification, and then matches the first hand area with the second hand area to calculate the disparity and depth value.

[0037] In a second aspect, the present invention provides an electronic device, characterized in that it comprises a hand depth calculation binocular camera as described in any one of the above items.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention first identifies the hand area, and in subsequent processing only processes the hand area, thereby reducing the amount of data processing and improving processing efficiency.

[0040] The present invention first identifies the first hand region on the first image, and then identifies the second hand region on the reduced second image, so that the amount of calculation when identifying the second hand region is greatly reduced, thereby improving efficiency and reducing the demand for chip processing capabilities.

[0041] The present invention reduces the image before calculation, thereby making the calculation amount constant and avoiding the problem of exceeding the calculation capacity. At the same time, data conversion is performed when calculating the depth, so that the amount of data to be calculated is greatly reduced.

[0042] The present invention optimizes the hand area and shrinks it to greatly reduce the amount of calculation. The depth value of the hand area is obtained by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. There is no need to use a dedicated Asic chip or dedicated hardware. The depth of the hand is calculated in real time, which helps subsequent gesture recognition and is cost-effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following descriptions are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0044] Figure 1 Schematic diagram of the structure of a binocular camera for hand depth calculation in an embodiment of the present invention;

[0045] Figure 2 Schematic diagram of the structure of another binocular camera for hand depth calculation in an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the structure of a processor in an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of a palm in an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of the structure of a second reduction module in an embodiment of the present invention;

[0049] Figure 6 This is a schematic diagram of the structure of a depth calculation module in an embodiment of the present invention;

[0050] Figure 7 A schematic diagram of palm area division in an embodiment of the present invention;

[0051] Figure 8 is a schematic diagram of the structure of another depth calculation module in an embodiment of the present invention;

[0052] Fig. 9 is a schematic diagram of shapes of different hand regions in an embodiment of the present invention;

[0053] Fig.10 The figure is a schematic diagram of the structure of an electronic device in an embodiment of the present invention.

[0054] 1--Hand area;

[0055] 2--Palm area;

[0056] 3--Finger area; DETAILED DESCRIPTION

[0057] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0058] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] For a stereo camera without a palm rest, the size of the hand in the image changes as the distance between the hand and the stereo camera changes. To ensure that the hand can be effectively recognized in a larger range, the resolution of the stereo camera is usually higher. When the hand is close to the stereo camera, the hand occupies a larger proportion in the image, and even if only the depth calculation is performed on the hand area, it still requires a large amount of calculation.

[0060] A binocular camera for hand depth calculation provided in an embodiment of the present invention is intended to be suitable for binocular depth calculation of hands at different distances, reduce the amount of calculation, and lower the requirements for chip computing power when the binocular camera is used for hand detection, thereby solving the problems existing in the prior art.

[0061] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0062] The present invention identifies and reduces the hand area, and obtains the depth value of the hand area by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. It does not need to use a dedicated Asic chip or dedicated hardware, and calculates the depth of the hand in real time, providing assistance for subsequent gesture recognition.

[0063] Figure 1 FIG. 1 is a schematic diagram of the structure of a binocular camera for hand depth calculation in an embodiment of the present invention. Figure 1 As shown, a hand depth calculation binocular camera in an embodiment of the present invention includes:

[0064] The first receiver is used to receive a first signal from a palm and generate a first image.

[0065] Specifically, the first receiver is usually a camera or an image sensor, which is responsible for capturing the first signal of the palm. The first receiver receives light from the palm through a lens and converts it into electrical signals, which are then processed into digital images. In order to accurately identify the details of the hand, the first receiver usually has a high-resolution sensor to capture clear images. The first image can be an RGB image, an infrared image, or any other type of image.

[0066] The second receiver is used to receive a second signal from the palm and generate a second image.

[0067] Specifically, similar to the first receiver, the second receiver captures the image of the palm from another perspective. The perspectives of the first receiver and the second receiver usually have a certain overlapping area, which is the basis for stereoscopic vision and depth calculation. In order to ensure that the first receiver and the second receiver capture the palm image at the same time point, the second receiver needs to maintain a high degree of synchronization with the first receiver. In order to simplify the subsequent image processing process, the resolution, focal length, exposure time and other parameters of the first receiver and the second receiver are usually set to be the same or similar. The second image is of the same type as the first image, such as both are RGB images, both are infrared images, etc.

[0068] A processor is used to detect a first hand area on the first image, reduce the first hand area to a preset size at a first magnification, reduce the second image at the first magnification, perform palm detection on the reduced second image to obtain a second hand area, and then perform depth calculation on the first hand area and the second hand area.

[0069] Specifically, the processor is the core of the entire system and is responsible for performing image processing and depth calculation tasks. First, image processing algorithms (such as edge detection, color segmentation, etc.) are used on the first image to identify the hand area. In order to match the perspective difference between the two images and simplify the calculation, the hand area in the first image needs to be adjusted to a preset size according to a certain scaling ratio. In order to ensure that the hand area size of the two images is consistent, the second image also needs to be adjusted according to the same scaling ratio. A similar palm area detection process is also performed in the resized second image to determine the palm position in the second image. Using the principle of binocular vision, the depth information of the hand is calculated by comparing the position difference of corresponding points in the two images. This usually involves stereo matching and triangulation techniques to calculate the three-dimensional coordinates of each point in the scene.

[0070] This embodiment restores the three-dimensional shape of the hand through two images from different perspectives, thereby achieving accurate measurement and analysis of gestures, hand movements or hand structures. This technology can be applied to a variety of fields such as virtual reality, augmented reality, game control, barrier-free communication, and medical rehabilitation.

[0071] Figure 2 FIG. 1 is a schematic diagram of the structure of another binocular camera for hand depth calculation in an embodiment of the present invention. Figure 2 As shown, compared with the above-mentioned embodiment, another hand depth calculation binocular camera in the embodiment of the present invention further includes:

[0072] A light source is used to illuminate the palm according to the brightness of the first image.

[0073] Specifically, the light source plays a vital role in the binocular camera system, especially in providing sufficient illumination and ensuring consistent brightness of the images captured by the two receivers. The main function of the light source is to supplement or adjust the light shining on the palm to ensure that the camera can capture images that are clear enough and of appropriate brightness regardless of the ambient lighting conditions. This helps to reduce shadows, overexposure or underexposure in the image, thereby improving the accuracy of hand detection and the reliability of depth calculation. The brightness and color temperature (such as warm white, cool white, etc.) of the light source are adjustable so that they can be optimized according to actual needs and environmental conditions. The light source can provide uniform light illumination to avoid obvious light spots or shadows on the palm, which helps maintain image consistency. The turning on and off of the light source may need to be precisely synchronized with the exposure time of the first receiver and the second receiver to ensure that the palm is optimally illuminated when the camera captures the image.

[0074] The light source is located on the central axis of the first receiver and the second receiver to ensure that the light can be evenly irradiated on the palm without causing unnecessary reflection or glare. The brightness and color temperature of the light source can be automatically adjusted according to the brightness of the first image to achieve dynamic compensation and balance. For example, if the first image shows that the palm is darker, the light source can automatically increase the brightness; if the palm is overexposed, the light source can reduce the brightness or adjust the color temperature.

[0075] By combining the use of light sources, the binocular camera system for hand depth calculation is able to work in a wider range of environmental conditions, improving overall performance and reliability.

[0076] Figure 3 FIG. 1 is a schematic diagram of the structure of a processor in an embodiment of the present invention. Figure 3 As shown, a processor in an embodiment of the present invention includes:

[0077] A detection module is used to detect a first hand area on the first image.

[0078] Specifically, a palm detection is required. The trained hand detection model needs to be used to identify the first hand area in the first image respectively. During processing, the first image is first preprocessed, such as denoising, contrast enhancement, etc., to facilitate subsequent hand detection. Then image processing techniques (such as skin color detection, edge detection, contour recognition, or deep learning-based methods such as convolutional neural networks) are used to locate and extract the hand area. The hand area is usually the area in the image that matches the shape and size of the hand, which is determined by identifying specific features (such as skin color, finger shape, etc.). Once the hand area is detected, these areas are marked with bounding boxes or other shapes for subsequent processing. Figure 4As shown, the hand region is often a part of the image. The subsequent processing of this embodiment is all performed on the hand region, and there is no need to process other parts, thereby reducing the amount of calculation.

[0079] The first reduction module is used to reduce the first hand area to a preset size at a first magnification.

[0080] Specifically, the first hand area already completely contains the information of the hand, so reducing the first hand area will not result in the loss of hand information. The preset size is usually a smaller size to achieve the purpose of reducing the amount of calculation. The setting of the preset size needs to take into account both the amount of calculation and the clarity. A preset size that is too large has limited effect on reducing the amount of calculation, while a preset size that is too small will cause the image to lose too much accuracy, and the final depth value is too small. The first reduction module reduces the first hand area to a preset size, which improves the processing speed and reduces the amount of calculation while maintaining the key visual features of the hand area.

[0081] The second reduction module is used to reduce the second image by the first magnification, and then perform palm detection on the reduced second image to obtain a second hand area.

[0082] Specifically, the second image as a whole is reduced at the same first magnification as the first hand region. Then, the same or similar palm detection algorithm as the first image is applied to the reduced second image to locate and extract the hand region as the second hand region. This step ensures that the hand regions in the two images are the same size, which facilitates subsequent depth calculations. It should be noted that the first hand region and the second hand region can be regions consisting of the edges of the hand, or regions including the hand and part of the background, such as a rectangular region, a circular region, etc.

[0083] A depth calculation module is used to perform depth calculation on the first hand area and the second hand area.

[0084] Specifically, the depth information of the hand is calculated using the principle of binocular stereo vision. The depth calculation module performs depth calculation only for the first hand area and the second hand area, so that the reconstruction area can be very small. Using the binocular matching algorithm, the disparity between the first hand area and the second hand area is found. The disparity value is inversely proportional to the depth of the hand from the camera. Through geometric transformation and triangulation principles, the disparity can be converted into the actual physical distance to obtain the depth information of the hand area. The obtained depth map can be used for a variety of applications, such as gesture recognition, human-computer interaction, etc. In addition, the depth map can be further processed or analyzed as needed.

[0085] The calculation area of ​​this embodiment is small, and the accuracy is higher than that of the TOF technology. For the second image, taking the resolution of the RGB binocular camera as 1080P and the preset size as 128*128 as an example, the calculation amount of this embodiment is 128*128*2, and the calculation amount of the second image is 1920*1080. Then the calculation amount of this embodiment is 128*128*2 / (1920*1080)=1 / 63.28 of the calculation amount of the original image. The calculation amount in this embodiment is proportional to the area of ​​the preset size. When the image is directly reconstructed in depth, taking the resolution of the RGB binocular camera as 1080P as an example, the search range is determined according to the limit length of the binocular camera and the reconstruction range. Here, it is assumed that the reconstruction range is 30mm-1m and the baseline length is 100mm. The calculation amount of the depth reconstruction is 1920*1080*384. When the solution of this embodiment is adopted, the palm is deeply reconstructed, and the amount of calculation is n*128*128*16, which is n / 3037 when the image is directly reconstructed in depth, where n is the number of palms, which greatly reduces the amount of calculation. If this embodiment only calculates the local area m of the palm, the amount of calculation is n*128*128*16*(m / 128 / 128), where n is the number of palms and m is the total number of pixels in the specified calculation area. The amount of calculation is m*n / (3037*128*128) when the image is directly reconstructed in depth. <n / 3037。

[0086] Figure 5 FIG. 1 is a schematic diagram of the structure of a second reduction module in an embodiment of the present invention. Figure 5 As shown, a second reduction module in an embodiment of the present invention includes:

[0087] The first reduction unit is used to reduce the second image by the first magnification to obtain a third image.

[0088] Specifically, the purpose of reducing the image is to reduce the amount of calculation and increase the processing speed, and the reduced image still retains sufficient details for hand area detection. The first reduction unit is crucial to the entire hand depth calculation process because it directly affects the size and clarity of the hand area on the third image. In order to keep consistent with the processing scale of the first hand area, the second image needs to be scaled at the same magnification. Use an image scaling algorithm (such as bilinear interpolation, bicubic interpolation, etc.) to reduce the second image as a whole by the first magnification to obtain the third image. The size of the third image is larger than the preset size.

[0089] An estimation region unit is used to obtain a corresponding estimation region on the third image according to the first magnification, the position of the first hand region in the first image and the epipolar constraint.

[0090] Specifically, the known camera geometry and epipolar geometry constraints, as well as the position information of the hand in the first image, are used to estimate the approximate area where the hand may appear in the third image. First, based on the position and size of the first hand area in the first image and the known first magnification, the size and position of the hand if it is in the same position in the second image (original size) are calculated. Then, considering the epipolar geometry constraints of the binocular camera, that is, the correspondence between the two camera views, this estimated position can be further adjusted to take into account the position offset caused by parallax. Finally, this adjusted position and size are applied to the third image to obtain the estimated area of ​​the hand in the third image.

[0091] The hand region unit is used to perform palm detection in the estimated region to obtain a second hand region.

[0092] Specifically, accurate palm detection is performed within the estimated area to reduce the search space and improve detection efficiency and accuracy. A palm detection algorithm (such as an algorithm based on color, shape, texture or depth information, or an algorithm based on machine learning) is used to search within the estimated area. The area that best matches the palm features is identified and extracted as the second hand area. If necessary, the detected second hand area can be further optimized, such as removing noise, smoothing edges, etc., to improve the accuracy of subsequent depth calculations.

[0093] This embodiment ensures the accuracy and efficiency of the binocular camera for hand depth calculation. They estimate the hand area by reducing the image, using the epipolar constraint, and performing accurate palm detection on the reduced third image, providing a reliable basis for subsequent depth calculation.

[0094] Figure 6 FIG. 1 is a schematic diagram of the structure of a depth calculation module in an embodiment of the present invention. Figure 6 As shown, a depth calculation module in an embodiment of the present invention includes:

[0095] A palm unit is used to obtain a first palm area in the first hand area and to obtain a second palm area in the second hand area.

[0096] Specifically, the palm area is further extracted from the detected hand area. Figure 7 As shown, the palm region and the finger region constitute the hand region. The palm region can usually be identified by analyzing features such as the hand contour, skin color distribution, and the geometric relationship between the fingers and the palm. For example, image segmentation techniques (such as threshold-based segmentation, region-based segmentation, or edge-based segmentation) can be used to separate the palm and finger parts. Once the palm region is extracted, these regions are marked using bounding boxes, masks, or other shapes for subsequent processing.

[0097] The first disparity unit is used to estimate the search range according to the size of the first palm area or the second palm area, and calculate the disparity of the first palm area and the second palm area.

[0098] Specifically, first, the search range in another view is estimated based on the size of the palm area, that is, the possible position range of the corresponding points of the palm area of ​​the first image is found in the second image. Then, a stereo matching algorithm (such as block matching, feature matching, etc.) is used to find the corresponding points between the first palm area and the second palm area, and their position difference in the horizontal direction, that is, disparity, is calculated. The search range is adjusted according to the size of the first palm area or the second palm area, but it is usually relatively small, such as 16, 24 pixels, etc.

[0099] The first depth unit is used to calculate the depth value of each pixel in the first palm area according to the disparity and the parameters of the binocular camera.

[0100] Specifically, the disparity value is converted into depth information using the binocular vision principle. Based on the camera's internal parameters (focal length, photosensitive element size, etc.) and external parameters (baseline distance, camera relative posture, etc.), as well as the disparity and triangulation principles, the actual depth value of each pixel relative to the camera can be calculated. In this way, we can obtain the three-dimensional position information of all pixels in the first palm area, thereby realizing the depth perception of the palm.

[0101] Figure 8 FIG. 1 is a schematic diagram of the structure of another depth calculation module in an embodiment of the present invention. Figure 8 As shown, another depth calculation module in an embodiment of the present invention includes:

[0102] The finger unit is used to obtain a first finger area in the first hand area and to obtain a second finger area in the second hand area.

[0103] Specifically, it is similar to extracting the palm region, but the goal is to identify and separate the finger parts in the hand image. This is usually more complicated because the fingers may have different lengths, degrees of curvature, and occlude each other. This can be achieved using methods based on shape analysis, contour tracking, or deep learning. Once the finger regions are identified, they are marked using appropriate methods (such as bounding boxes, masks, etc.) for subsequent processing.

[0104] The second disparity unit is used to determine the number of each finger in the first finger area and the second finger area, and calculate the disparity for the fingers with the same number.

[0105] Specifically, the detected fingers need to be numbered first so that the corresponding fingers can be found in the two views. The numbering can be done based on the position, size or other features of the fingers. Then, a stereo matching algorithm (such as block matching, feature matching, etc.) is used to find the corresponding points between the first finger area and the second finger area, and calculate their position difference in the horizontal direction, that is, disparity.

[0106] The second depth unit is used to calculate the depth value of each pixel in the first finger area according to the disparity and the parameters of the binocular camera.

[0107] Specifically, the disparity value is converted into depth information using the binocular vision principle. Based on the camera's internal parameters (focal length, photosensitive element size, etc.) and external parameters (baseline distance, camera relative posture, etc.), as well as the disparity and triangulation principles, the actual depth value of each pixel relative to the camera can be calculated. In this way, we can obtain the three-dimensional position information of all pixels in the first finger area, thereby realizing the depth perception of the finger.

[0108] This embodiment can calculate the depth of each finger individually, which provides more detailed three-dimensional information for gesture recognition and hand motion capture. For example, this embodiment can estimate the finger posture individually, and through detection between multiple frames, it can detect the finger movement state, meeting the needs of various application scenarios.

[0109] In some embodiments, the second disparity unit calculates the search range of each finger according to the width of each finger, and then performs matching to calculate the disparity. First, each finger needs to be identified and numbered in the first hand area and the second hand area. This is usually achieved through image processing technology, such as contour detection and fingertip recognition algorithm. Next, the search range in the second view is estimated based on the average width and direction of each finger. The search range of each finger is calculated separately, which can improve the accuracy of the search, reduce the search space in the stereo matching process, and improve the matching efficiency and accuracy. Use a stereo matching algorithm (such as block matching or feature matching) to find corresponding points between the first finger area and the second finger area. Since the width of the finger provides an additional constraint, the matching algorithm can be adjusted to take this feature into account, such as limiting the search range to the area defined by the finger width. After the match is successful, the horizontal position difference of each pair of matching points is recorded, that is, the disparity. The size of the disparity is related to the depth of the point. The larger the disparity, the closer the point is to the camera. Using the camera parameters and disparity information, the depth information of each finger can be calculated by triangulation. This step provides key three-dimensional data for subsequent gesture recognition or hand motion capture.

[0110] In some embodiments, the second disparity calculates the search range according to the maximum width of the finger, and then matches and calculates the disparity. In the first hand region, for each finger that has been identified and numbered, an image processing technique (such as edge detection, contour analysis, etc.) is used to detect the maximum width of the finger. This is usually done by finding the distance between the two farthest points in the finger contour, which correspond to the widest part of the finger. Based on the maximum width of each finger detected, a search range is calculated for the finger in the second hand region. This search range should be a rectangular or elliptical area centered on the estimated finger position and slightly larger than the maximum width of the finger. The size of the search range should be large enough to accommodate finger position offsets caused by changes in perspective, changes in hand posture, or depth differences, but should not be too large to avoid introducing unnecessary computational complexity and mismatching. Feature points (such as corner points, edge points, etc.) are extracted on each finger in the first hand region, and the best match of these feature points is found in the corresponding search range of the second hand region. This can be achieved by using feature matching algorithms (such as SIFT, SURF, ORB, etc.), which can find similar feature points between images and match them. Once a matching pair of feature points is found, the pixel position difference between them, i.e., disparity, is calculated. The size and direction of the disparity reflects the relative position of the finger in the two camera views, which can be used for subsequent 3D reconstruction or depth estimation.

[0111] In some embodiments, the depth calculation module determines a search range according to the shape of the first hand region and the first magnification, and then matches the first hand region with the second hand region to calculate the disparity and depth value. Fig. 9As shown, the shape of the first hand region indicates that the hand has different postures. Under different postures, the same depth will show a large area difference on the image. Since the size of the hands is not much different, the depth value of the hand can be estimated according to the first magnification of the first hand region, which is used to determine the search range, thereby reducing unnecessary matching calculations and improving matching efficiency and accuracy. When the shape of the first hand region is the first shape, the first search range is obtained according to the first magnification and the first formula. When the shape of the first hand region is the second shape, the second search range is obtained according to the first magnification and the second formula. The parameters of the first formula and the second formula are different. When the aspect ratio of the first hand region is larger, the search range corresponding to the same first magnification is smaller; when the aspect ratio of the first hand region is smaller, the search range corresponding to the same first magnification is larger. Within the estimated search range, a computer vision algorithm such as feature point matching or descriptor matching technology is used to find matching points between the first hand region and the second hand region. These matching points represent the projections of the same physical point in the binocular image under two different perspectives. The disparity value is calculated by comparing the position difference between the matching points of the first hand region and the second hand region. Parallax is the difference in horizontal coordinates of the same feature point in two images. A larger parallax means that the point is closer to the camera, while a smaller parallax means that it is farther away. Using the principles of geometric transformation and triangulation, the parallax can be converted into the actual physical distance to obtain the depth information of the hand area. This process usually involves the internal and external parameters of the camera, such as focal length, baseline distance, etc.

[0112] Fig.10 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. Fig.10 The electronic device 600 according to this embodiment of the present invention is described. Fig.10 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0113] like Fig.10 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0114] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps of various exemplary embodiments of the present invention described in the above-mentioned hand depth calculation binocular camera section of this specification. For example, the processing unit 610 can execute the following steps: Figure 3 Follow the steps shown in .

[0115] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0116] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a grid environment.

[0117] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0118] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 600, and / or any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more grids (e.g., a local area network (LAN), a wide area network (WAN), and / or a public grid, such as the Internet) through a grid adapter 660. The grid adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Fig.10 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0119] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0120] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A hand depth calculation binocular camera, characterized in that: include: A first receiver, used for receiving a first signal of a palm and generating a first image; A second receiver, used for receiving a second signal of the palm and generating a second image; A processor is used to detect a first hand area on the first image, reduce the first hand area to a preset size at a first magnification, reduce the second image at the first magnification, perform palm detection on the reduced second image to obtain a second hand area, and then perform depth calculation on the first hand area and the second hand area.

2. A palm-swiping binocular camera according to claim 1, characterized in that: Also includes: A light source is used to illuminate the palm according to the brightness of the first image.

3. The palm-swiping binocular camera according to claim 1, characterized in that: The processor comprises: A detection module, configured to detect a first hand region on the first image; A first reduction module, used to reduce the first hand area to a preset size at a first magnification; a second reduction module, configured to reduce the second image by the first magnification, and then perform palm detection on the reduced second image to obtain a second hand region; A depth calculation module is used to perform depth calculation on the first hand area and the second hand area.

4. The hand depth calculation binocular camera according to claim 3, characterized in that: The second reduction module comprises: A first reduction unit, configured to reduce the second image by the first magnification to obtain a third image; an estimation region unit, configured to obtain a corresponding estimation region on the third image according to the first magnification, the position of the first hand region in the first image, and an epipolar constraint; The hand region unit is used to perform palm detection in the estimated region to obtain a second hand region.

5. The hand depth calculation binocular camera according to claim 3, characterized in that: The depth calculation module includes: A palm unit, used for obtaining a first palm area in the first hand area, and obtaining a second palm area in the second hand area; A first disparity unit, configured to estimate a search range according to a size of the first palm area or the second palm area, and calculate a disparity between the first palm area and the second palm area; The first depth unit is used to calculate the depth value of each pixel in the first palm area according to the disparity and the parameters of the binocular camera.

6. The hand depth calculation binocular camera according to claim 3, characterized in that: The depth calculation module includes: A finger unit, used for obtaining a first finger region in the first hand region and obtaining a second finger region in the second hand region; A second disparity unit, used to determine the number of each finger in the first finger area and the second finger area, and calculate the disparity for fingers with the same number; The second depth unit is used to calculate the depth value of each pixel in the first finger area according to the disparity and the parameters of the binocular camera.

7. The hand depth calculation binocular camera according to claim 6, characterized in that: The second disparity unit calculates the search range of each finger according to the width of each finger, and then performs matching to calculate the disparity.

8. The hand depth calculation binocular camera according to claim 6, characterized in that: The second disparity calculates a search range according to the maximum width of the finger, and then performs matching to calculate the disparity.

9. The hand depth calculation binocular camera according to claim 3, characterized in that: The depth calculation module determines a search range according to the shape of the first hand region and the first magnification, and then matches the first hand region with the second hand region to calculate a disparity and a depth value.

10. An electronic device, characterized in that: A hand depth calculation binocular camera comprising any one of claims 1 to 9.

Images