Palm depth rapid calculation method, system and device and storage medium

Abstract

The invention discloses a palm depth rapid calculation method, system and device and a storage medium. The method comprises the steps that S1, a first image and a second image are obtained; wherein the first image and the second image are obtained by a binocular camera; s2, detecting a first hand region on the first image, and detecting a second hand region on the second image; s3, aligning the first hand part area with the second hand part area; s4, zooming the first hand region and the second hand region to a preset size at the same magnification; and S5, performing depth calculation on the first hand region and the second hand region. According to the method, the calculation amount of search and reconstruction is greatly reduced, a special Asic chip or special hardware is not needed, the depth of the hand is calculated in real time, and help is provided for subsequent gesture recognition.

Description

Background Art

[0002] In gesture recognition, depth information is very beneficial for improving the performance of gesture recognition. It can provide more spatial context, reduce the ambiguity of two-dimensional images, and improve the robustness of the algorithm under various environmental conditions.

[0003] The main functions of depth information are:

[0004] 1. Added spatial information: Depth data adds an extra dimension to gestures, helping the algorithm understand the shape and movement of gestures in space.

[0005] 2. Improve robustness: In environments with complex backgrounds or changing lighting, depth information can help the algorithm better distinguish between hands and background.

[0006] 3. Reduce ambiguity: When only two-dimensional information is available, some gestures may look similar. Depth data can help resolve this visual ambiguity and improve the accuracy of gesture recognition.

[0007] 4. Handling occlusion: Self-occlusion, which is common in gestures (e.g. one finger occluding another finger), is easier to distinguish and handle in 3D space.

[0008] Binocular cameras can obtain dense depth through binocular stereo matching. Unlike tof or monocular structured light depth cameras, binocular cameras do not require special hardware except for the two cameras, so their hardware cost is relatively low, and they have good autonomy and flexibility, so they are widely applicable. However, their disadvantages are also obvious. The binocular stereo matching algorithm is computationally intensive and requires a lot of computing resources. Therefore, in actual use, dedicated Asic chips or dedicated hardware are generally required for depth calculation, which increases the cost.

[0009] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0010] To this end, the present invention identifies the hand area, scales the hand area, and obtains the depth value of the hand area by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. There is no need to use a dedicated Asic chip or dedicated hardware, and the depth of the hand is calculated in real time to provide assistance for subsequent gesture recognition.

[0011] In a first aspect, the present invention provides a method for quickly calculating palm depth, characterized by comprising:

[0012] Step S1: obtaining a first image and a second image; wherein the first image and the second image are obtained by a binocular camera;

[0013] Step S2: detecting a first hand region on the first image, and detecting a second hand region on the second image;

[0014] Step S3: aligning the first hand area with the second hand area;

[0015] Step S4: scaling the first hand area and the second hand area to a preset size at the same magnification;

[0016] Step S5: performing depth calculation on the first hand region and the second hand region.

[0017] Optionally, the palm depth quick calculation method is characterized in that step S3 comprises:

[0018] Step S31: performing epipolar constraint processing on the first hand region and the second hand region in the y-axis direction;

[0019] Step S32: The first hand region and the second hand region are set to have the same pixel size range with the center point in the x-axis direction.

[0020] Optionally, the palm depth quick calculation method is characterized in that step S5 comprises:

[0021] Step S51: obtaining a first palm region in the first hand region, and obtaining a second palm region in the second hand region;

[0022] Step S52: estimating a search range according to the size of the first palm area or the second palm area, and calculating a disparity between the first palm area and the second palm area;

[0023] Step S53: Calculate the depth value of each pixel in the first palm area according to the parallax and the parameters of the binocular camera.

[0024] Optionally, the palm depth quick calculation method is characterized in that step S5 comprises:

[0025] Step S54: obtaining a first finger region in the first hand region, and obtaining a second finger region in the second hand region;

[0026] Step S55: determining the number of each finger in the first finger area and the second finger area, and calculating the disparity for fingers with the same number;

[0027] Step S56: Calculate the depth value of each pixel in the first finger area according to the parallax and the parameters of the binocular camera.

[0028] Optionally, the method for quickly calculating the palm depth is characterized in that, in step S55, the search range of each finger is calculated according to the width of each finger, and then matching is performed to calculate the disparity.

[0029] Optionally, the method for quickly calculating the palm depth is characterized in that, in step S55, the search range is calculated according to the maximum width of the finger, and then matching is performed to calculate the disparity.

[0030] Optionally, the palm depth quick calculation method is characterized in that step S2 comprises:

[0031] Step S21: detecting a third hand region and a fourth hand region on the first image, and detecting a fifth hand region and a sixth hand region on the second image;

[0032] Step S22: Pairing the third hand region, the fourth hand region, the fifth hand region and the sixth hand region according to the epipolar constraint.

[0033] In a second aspect, the present invention provides a palm depth fast calculation system, which is used to implement any of the aforementioned palm depth fast calculation methods, and is characterized by comprising:

[0034] An acquisition module, used to acquire a first image and a second image; wherein the first image and the second image are acquired by a binocular camera;

[0035] A detection module, configured to detect a first hand region on the first image and a second hand region on the second image;

[0036] an alignment module, configured to align the first hand region with the second hand region;

[0037] A scaling module, used for scaling the first hand area and the second hand area to a preset size at the same magnification;

[0038] A depth calculation module is used to perform depth calculation on the first hand area and the second hand area.

[0039] In a third aspect, the present invention provides a palm depth fast calculation device, characterized in that it includes:

[0040] processor;

[0041] a memory storing executable instructions of the processor;

[0042] The processor is configured to execute any one of the steps of the aforementioned method for quickly calculating palm depth by executing the executable instructions.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium for storing a program, characterized in that when the program is executed, the steps of any one of the aforementioned methods for quickly calculating palm depth are implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention first identifies the hand area, and in subsequent steps only processes the hand area, thereby reducing the amount of data processing and improving processing efficiency.

[0046] The present invention scales the image before calculation, thereby making the amount of calculation constant and avoiding the problem of exceeding the calculation capacity. At the same time, data conversion is performed when calculating the depth, so that the amount of data to be calculated is greatly reduced.

[0047] The present invention optimizes the hand area and scales it to greatly reduce the amount of calculation. The depth value of the hand area is obtained by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. There is no need to use a dedicated Asic chip or dedicated hardware. The depth of the hand is calculated in real time, which helps subsequent gesture recognition and is cost-effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following descriptions are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0049] Figure 1 A flowchart of a method for quickly calculating palm depth according to an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of a palm in an embodiment of the present invention;

[0051] Figure 3 is a flowchart of steps for aligning a first hand area with a second hand area in an embodiment of the present invention;

[0052] Figure 4 A flowchart of the steps of performing depth calculation in an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of palm area division in an embodiment of the present invention;

[0054] Figure 6 This is another flowchart of the steps of performing depth calculation in an embodiment of the present invention;

[0055] Figure 7 This is a flowchart of the steps of detecting a hand area in an embodiment of the present invention;

[0056] Figure 8 This is a schematic diagram of the structure of a palm depth fast calculation system according to an embodiment of the present invention;

[0057] Fig. 9 is a schematic diagram of the structure of a device for quickly calculating palm depth in an embodiment of the present invention; and

[0058] Fig.10 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention.

[0059] 1--Hand area;

[0060] 2--Palm area;

[0061] 3--Finger area; DETAILED DESCRIPTION

[0062] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0063] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0064] For a stereo camera without a palm rest, the size of the hand in the image changes as the distance between the hand and the stereo camera changes. To ensure that the hand can be effectively recognized in a larger range, the resolution of the stereo camera is usually higher. When the hand is close to the stereo camera, the hand occupies a larger proportion in the image, and even if only the depth calculation is performed on the hand area, it still requires a large amount of calculation.

[0065] A method for quickly calculating palm depth provided by an embodiment of the present invention is intended to be applicable to binocular depth calculation of the hand at different distances, reduce the amount of calculation, and lower the requirements for chip computing power when the binocular camera is used for hand detection, thereby solving the problems existing in the prior art.

[0066] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0067] The present invention identifies the hand area, scales the hand area, and obtains the depth value of the hand area by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. It does not need to use a dedicated Asic chip or dedicated hardware, and calculates the depth of the hand in real time, providing assistance for subsequent gesture recognition.

[0068] Figure 1 FIG. 1 is a flowchart of a method for quickly calculating palm depth according to an embodiment of the present invention. Figure 1 As shown, the steps of a method for quickly calculating palm depth in an embodiment of the present invention include:

[0069] Step S1: Obtain a first image and a second image.

[0070] In this step, the first image and the second image are obtained by a binocular camera. A binocular camera is used to obtain images of two different perspectives. The binocular camera is composed of two cameras arranged side by side or with a specific spacing, and the distance between them is called the baseline distance. The two cameras simultaneously shoot the same scene to obtain two images, namely the first image and the second image. The first image and the second image are slightly different images. These differences are caused by the horizontal displacement (baseline distance) between the cameras, which is crucial for subsequent three-dimensional reconstruction. The image in this embodiment can be an RGB image, an infrared image, or other types of images. This embodiment does not limit the image type.

[0071] Step S2: Detect a first hand region on the first image, and detect a second hand region on the second image.

[0072] In this step, two palm detections are required. The trained hand detection model needs to be used to identify the hand area in the first image and the second image respectively. During processing, the first and second images are first preprocessed, such as denoising, contrast enhancement, etc., to facilitate subsequent hand detection. Then image processing techniques (such as skin color detection, edge detection, contour recognition, or deep learning-based methods such as convolutional neural networks) are used to locate and extract the hand area. The hand area is usually the area in the image that matches the shape and size of the hand, which is determined by identifying specific features (such as skin color, finger shape, etc.). Once the hand area is detected, these areas are marked with bounding boxes or other shapes for subsequent processing. Figure 2 As shown, the hand region is often a part of the image. The subsequent processing of this embodiment is all performed on the hand region, and there is no need to process other parts, thereby reducing the amount of calculation.

[0073] Step S3: Align the first hand area with the second hand area.

[0074] In this step, due to the different shooting angles of the two cameras, the position and shape of the hand in the two images will be different. Therefore, we need to align the two hand regions through image registration technology for subsequent depth calculation.

[0075] During alignment, first, feature points (such as corner points, edge points, etc.) are extracted from the two hand regions respectively, and then the correspondence between the two sets of feature points is found using a feature matching algorithm (such as SIFT, SURF, ORB, etc.). Secondly, based on the correspondence between the feature points, a mathematical method (such as the RANSAC algorithm) is used to estimate the geometric transformation (such as translation, rotation, scaling, etc.) between the two hand regions, thereby calculating the transformation matrix from the first hand region to the second hand region. Finally, the calculated transformation matrix is ​​used to transform the first hand region to a position aligned with the second hand region.

[0076] Step S4: scaling the first hand area and the second hand area to a preset size at the same magnification.

[0077] In this step, the first hand area and the second hand area already completely contain the hand information, so scaling the first hand area and the second hand area will not result in the loss of hand information. Scaling the first hand area and the second hand area at the same magnification can ensure that the relative sizes of the first hand area and the second hand area remain unchanged, thereby ensuring the accuracy of the depth calculation. The preset size is usually a smaller size to achieve the purpose of reducing the amount of calculation. The setting of the preset size needs to take into account both the amount of calculation and the clarity. A preset size that is too large has limited effect on reducing the amount of calculation, while a preset size that is too small will cause the image to lose too much precision, and the final depth value is too small.

[0078] Step S5: performing depth calculation on the first hand region and the second hand region.

[0079] In this step, the depth information of the hand is calculated using the principle of binocular stereo vision. The depth calculation in this step is only performed for the first hand area and the second hand area, so that the reconstruction area can be very small. Since the binocular camera system has a fixed baseline distance and the two cameras shoot the same scene, the depth can be estimated by comparing the position difference (i.e., parallax) of the corresponding points in the two hand areas. For each point (or each pixel) in the first hand area, find the corresponding point in the second hand area and calculate the disparity between them. Then, using the known baseline distance, focal length and other parameters and the parallax, the depth value of the point is calculated by the triangulation principle. Repeat this process to generate a depth map of the entire hand area. The obtained depth map can be used for a variety of applications, such as gesture recognition, virtual reality interaction, robot hand-eye coordination, etc. In addition, the depth map can be further processed or analyzed as needed.

[0080] The calculation area of ​​this embodiment is small, and the accuracy is higher than that of TOF technology. Taking the resolution of the RGB binocular camera as 1080P and the preset size as 128*128 as an example, when the image is directly reconstructed in depth, the search range is determined according to the limit length of the binocular camera and the reconstruction range. Here, it is assumed that the reconstruction range is 30mm-1m and the baseline length is 100mm. The calculation amount of the depth reconstruction is 1920*1080*384. When the solution of this embodiment is adopted, the palm is deeply reconstructed, and the calculation amount is n*128*128*16, which is n / 3037 when the image is directly reconstructed in depth, where n is the number of palms, which greatly reduces the calculation amount. If this embodiment only calculates the local area m of the palm, the calculation amount is n*128*128*16*(m / 128 / 128), where n is the number of palms and m is the total number of pixels in the specified calculation area. The calculation amount is m*n / (3037*128*128) when the image is directly reconstructed. <n / 3037。

[0081] Figure 3 FIG. 1 is a flowchart of steps for aligning a first hand region with a second hand region in an embodiment of the present invention. Figure 3 As shown, in an embodiment of the present invention, a step of aligning a first hand area with a second hand area includes:

[0082] Step S31: performing epipolar constraint processing on the first hand region and the second hand region in the y-axis direction.

[0083] In this step, if Figure 2As shown, the y-axis is the epipolar line of the epipolar constraint. Epipolar constraint is a key concept in stereo vision. It describes that when two cameras observe the same object, the corresponding point of a point on one image on the other image must lie on a specific straight line, which is called the epipolar line. For a horizontally arranged stereo camera system, if the two cameras are perfectly aligned horizontally and their intrinsic parameters are the same, the epipolar line is parallel to the horizontal axis (x-axis) of the image plane.

[0084] In practical applications, the epipolar line may not be completely horizontal because the camera may have a slight pitch and yaw angle. Therefore, when aligning, it is necessary to ensure that the epipolar constraints of the first hand region and the second hand region in the vertical direction (y-axis) are satisfied, that is, to ensure that the corresponding feature points in the two images have the same position in the vertical direction or follow the epipolar geometry. The epipolar constraint is usually a transformation supported by the ISP, and can also be calculated by image mapping.

[0085] Step S32: The first hand region and the second hand region are set to have the same pixel size range with the center point in the x-axis direction.

[0086] In this step, if Figure 2 As shown, the x-axis is the axis of the image perpendicular to the y-axis. After the epipolar constraint processing, the next step is to align the first hand area and the second hand area in the x-axis direction. When searching for corresponding points in the two images, the search range is limited to the same horizontal scan line, thereby greatly reducing the amount of calculation. This step ensures that the first hand area and the second hand area are aligned in the x-axis direction with their center points as the reference. This helps to reduce the error caused by the position offset of the hand in the image. When executing, first, it is necessary to determine the center points of the first hand area and the second hand area respectively. This can be done by calculating the center of the hand area bounding box, the centroid of the hand contour, or other appropriate methods. Then, based on these center points, select the same pixel size range in the two hand areas as the object of subsequent processing.

[0087] Figure 4 FIG. 1 is a flowchart of a step of performing depth calculation in an embodiment of the present invention. Figure 4 As shown, a step of performing depth calculation in an embodiment of the present invention includes:

[0088] Step S51: obtaining a first palm region in the first hand region, and obtaining a second palm region in the second hand region.

[0089] In this step, the palm area is further extracted from the detected hand area. Figure 5As shown, the palm region and the finger region constitute the hand region. The palm region can usually be identified by analyzing features such as the hand contour, skin color distribution, and the geometric relationship between the fingers and the palm. For example, image segmentation techniques (such as threshold-based segmentation, region-based segmentation, or edge-based segmentation) can be used to separate the palm and finger parts. Once the palm region is extracted, these regions are marked using bounding boxes, masks, or other shapes for subsequent processing.

[0090] Step S52: estimating a search range according to the size of the first palm area or the second palm area, and calculating the disparity of the first palm area and the second palm area.

[0091] In this step, the search range in another view is first estimated based on the size of the palm area, that is, the possible position range of the corresponding points of the palm area of ​​the first image is found in the second image. Then, a stereo matching algorithm (such as block matching, feature matching, etc.) is used to find the corresponding points between the first palm area and the second palm area, and their position difference in the horizontal direction, i.e., disparity, is calculated. The search range is adjusted according to the size of the first palm area or the second palm area, but is usually relatively small, such as 16, 24 pixels, etc.

[0092] Step S53: Calculate the depth value of each pixel in the first palm area according to the parallax and the parameters of the binocular camera.

[0093] In this step, the disparity value is converted into depth information using the binocular vision principle. Based on the camera's internal parameters (focal length, photosensitive element size, etc.) and external parameters (baseline distance, camera relative posture, etc.), as well as the disparity and triangulation principles, the actual depth value of each pixel relative to the camera can be calculated. In this way, we can obtain the three-dimensional position information of all pixels in the first palm area, thereby realizing the depth perception of the palm.

[0094] Figure 6 FIG. 1 is another flow chart of the steps of performing depth calculation in an embodiment of the present invention. Figure 6 As shown, another step of performing depth calculation in an embodiment of the present invention includes:

[0095] Step S54: obtaining a first finger region in the first hand region, and obtaining a second finger region in the second hand region.

[0096] In this step, it is similar to extracting the palm region, but the goal is to identify and separate the finger parts in the hand image. This is usually more complicated because the fingers may have different lengths, degrees of curvature, and occlude each other. This can be achieved using methods based on shape analysis, contour tracking, or deep learning. Once the finger regions are identified, they are marked using appropriate methods (such as bounding boxes, masks, etc.) for subsequent processing.

[0097] Step S55: determining the number of each finger in the first finger area and the second finger area, and calculating the disparity for fingers with the same number.

[0098] In this step, the detected fingers need to be numbered first so that the corresponding fingers can be found in the two views. The numbering can be done based on the position, size or other features of the fingers. Then, a stereo matching algorithm (such as block matching, feature matching, etc.) is used to find the corresponding points between the first finger area and the second finger area, and calculate their horizontal position difference, i.e., disparity.

[0099] Step S56: Calculate the depth value of each pixel in the first finger area according to the parallax and the parameters of the binocular camera.

[0100] In this step, the disparity value is converted into depth information using the binocular vision principle. Based on the camera's internal parameters (focal length, photosensitive element size, etc.) and external parameters (baseline distance, camera relative posture, etc.), as well as the disparity and triangulation principles, the actual depth value of each pixel relative to the camera can be calculated. In this way, we can obtain the three-dimensional position information of all pixels in the first finger area, thereby realizing the depth perception of the finger.

[0101] This embodiment can calculate the depth of each finger individually, which provides more detailed three-dimensional information for gesture recognition and hand motion capture. For example, this embodiment can estimate the finger posture individually, and through detection between multiple frames, it can detect the finger movement state, meeting the needs of various application scenarios.

[0102] In some embodiments, in step S55, the search range of each finger is calculated according to the width of each finger, and then matching is performed to calculate the disparity. First, each finger needs to be identified and numbered in the first hand area and the second hand area. This is usually achieved through image processing technology, such as contour detection and fingertip recognition algorithm. Next, the search range in the second view is estimated based on the average width and direction of each finger. The search range of each finger is calculated separately, which can improve the accuracy of the search, reduce the search space in the stereo matching process, and improve the matching efficiency and accuracy. Use a stereo matching algorithm (such as block matching or feature matching) to find corresponding points between the first finger area and the second finger area. Since the width of the finger provides an additional constraint, the matching algorithm can be adjusted to take this feature into account, such as limiting the search range to the area defined by the finger width. After the match is successful, the horizontal position difference of each pair of matching points is recorded, that is, the disparity. The size of the disparity is related to the depth of the point. The larger the disparity, the closer the point is to the camera. Using the camera parameters and disparity information, the depth information of each finger can be calculated by triangulation. This step provides key three-dimensional data for subsequent gesture recognition or hand motion capture.

[0103] In some embodiments, in step S55, a search range is calculated based on the maximum width of the finger, and then matching is performed to calculate the disparity. In the first hand region, for each finger that has been identified and numbered, an image processing technique (such as edge detection, contour analysis, etc.) is used to detect the maximum width of the finger. This is usually done by finding the distance between the two farthest points in the finger contour, which correspond to the widest part of the finger. Based on the maximum width of each finger detected, a search range is calculated for the finger in the second hand region. This search range should be a rectangular or elliptical area centered on the estimated finger position and slightly larger than the maximum width of the finger. The size of the search range should be large enough to accommodate finger position offsets caused by changes in viewing angles, changes in hand postures, or depth differences, but should not be too large to avoid introducing unnecessary computational complexity and mismatching. Feature points (such as corner points, edge points, etc.) are extracted on each finger in the first hand region, and the best match of these feature points is found in the corresponding search range of the second hand region. This can be achieved by using feature matching algorithms (such as SIFT, SURF, ORB, etc.), which can find similar feature points between images and match them. Once a matching pair of feature points is found, the pixel position difference between them, i.e., disparity, is calculated. The size and direction of the disparity reflects the relative position of the finger in the two camera views, which can be used for subsequent 3D reconstruction or depth estimation.

[0104] Figure 7 FIG. 1 is a flow chart of steps for detecting a hand area in an embodiment of the present invention. Figure 7As shown, in an embodiment of the present invention, a step of detecting a hand area includes:

[0105] Step S21: Detect the third hand region and the fourth hand region on the first image, and detect the fifth hand region and the sixth hand region on the second image.

[0106] In this step, multiple hand regions are detected by using computer vision techniques, such as edge detection, contour extraction, morphological operations, or deep learning models. This step detects at least two hand regions on the first image and at least two hand regions on the second image. Each detected hand region should be correctly labeled and identified so that pairing and disparity calculation can be performed correctly in subsequent steps.

[0107] Step S22: Pairing the third hand region, the fourth hand region, the fifth hand region and the sixth hand region according to the epipolar constraint.

[0108] In this step, the epipolar constraint means that any point in the left image must be matched on the epipolar line of the right image, and vice versa. This constraint reduces the matching search space, thereby improving matching efficiency and accuracy. Using the epipolar geometry relationship, the area paired with the third hand area is determined in the fifth hand area and the sixth hand area, and the area paired with the fourth hand area is determined in the fifth hand area and the sixth hand area. This involves calculating the projection of each area on the corresponding epipolar line and finding the most likely matching point. At the same time, further verification and optimization are required to ensure the accuracy of the pairing, such as by analyzing the size, shape and relative position of the hand area to confirm the validity of the pairing.

[0109] This embodiment processes multiple detected hand regions and pairs these regions based on epipolar constraints, which is critical for achieving accurate gesture recognition and depth perception, especially in complex scenarios that require processing multiple hands or fingers.

[0110] Figure 8 FIG. 1 is a schematic diagram of the structure of a palm depth fast calculation system according to an embodiment of the present invention. Figure 8 As shown, a palm depth rapid calculation system in an embodiment of the present invention includes:

[0111] An acquisition module, used to acquire a first image and a second image; wherein the first image and the second image are acquired by a binocular camera;

[0112] A detection module, configured to detect a first hand region on the first image and a second hand region on the second image;

[0113] an alignment module, configured to align the first hand region with the second hand region;

[0114] A scaling module, used for scaling the first hand area and the second hand area to a preset size at the same magnification;

[0115] A depth calculation module is used to perform depth calculation on the first hand area and the second hand area.

[0116] This embodiment recognizes and scales the hand area, and obtains the depth value of the hand area by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. It does not need to use a dedicated Asic chip or dedicated hardware, and calculates the depth of the hand in real time, providing assistance for subsequent gesture recognition.

[0117] The present invention also provides a palm depth fast calculation device, comprising a processor and a memory, wherein executable instructions of the processor are stored, wherein the processor is configured to execute the steps of a palm depth fast calculation method by executing the executable instructions.

[0118] As described above, this embodiment recognizes the hand area, scales the hand area, and obtains the depth value of the hand area by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. There is no need to use a dedicated Asic chip or dedicated hardware, and the depth of the hand is calculated in real time to provide assistance for subsequent gesture recognition.

[0119] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0120] Fig. 9 Schematic diagram of the structure of a palm depth fast calculation device in an embodiment of the present invention. Fig. 9 The electronic device 600 according to this embodiment of the present invention is described. Fig. 9 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0121] like Fig. 9 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0122] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps of various exemplary embodiments of the present invention described in the above-mentioned method for quickly calculating the palm depth. For example, the processing unit 610 can execute the following steps: Figure 1 Follow the steps shown in .

[0123] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0124] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a grid environment.

[0125] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0126] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 600, and / or any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more grids (e.g., a local area network (LAN), a wide area network (WAN), and / or a public grid, such as the Internet) through a grid adapter 660. The grid adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Fig. 9 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0127] In an embodiment of the present invention, a computer-readable storage medium is also provided for storing a program, and when the program is executed, the steps of a method for quickly calculating the palm depth are implemented. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the above-mentioned method for quickly calculating the palm depth of this specification.

[0128] As shown above, this embodiment recognizes and scales the hand area, and obtains the depth value of the hand area by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. It does not require the use of dedicated Asic chips or dedicated hardware, and calculates the depth of the hand in real time, providing assistance for subsequent gesture recognition.

[0129] Fig.10 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Fig.10 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0130] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0131] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0132] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of grid, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0133] This embodiment recognizes and scales the hand area, and obtains the depth value of the hand area by calculating the parallax, which greatly reduces the amount of calculation for search and reconstruction. It does not need to use a dedicated Asic chip or dedicated hardware, and calculates the depth of the hand in real time, providing assistance for subsequent gesture recognition.

[0134] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0135] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for quickly calculating palm depth, characterized in that: include: Step S1: obtaining a first image and a second image; wherein the first image and the second image are obtained by a binocular camera; Step S2: detecting a first hand region on the first image, and detecting a second hand region on the second image; Step S3: aligning the first hand area with the second hand area; Step S4: scaling the first hand area and the second hand area to a preset size at the same magnification; Step S5: performing depth calculation on the first hand region and the second hand region.

2. A palm depth rapid calculation method according to claim 1, characterized in that: Step S3 includes: Step S31: performing epipolar constraint processing on the first hand region and the second hand region in the y-axis direction; Step S32: The first hand region and the second hand region are set to have the same pixel size range with the center point in the x-axis direction.

3. A palm depth rapid calculation method according to claim 1, characterized in that: Step S5 includes: Step S51: obtaining a first palm region in the first hand region, and obtaining a second palm region in the second hand region; Step S52: estimating a search range according to the size of the first palm area or the second palm area, and calculating a disparity between the first palm area and the second palm area; Step S53: Calculate the depth value of each pixel in the first palm area according to the parallax and the parameters of the binocular camera.

4. A palm depth rapid calculation method according to claim 1, characterized in that: Step S5 includes: Step S54: obtaining a first finger region in the first hand region, and obtaining a second finger region in the second hand region; Step S55: determining the number of each finger in the first finger area and the second finger area, and calculating the disparity for fingers with the same number; Step S56: Calculate the depth value of each pixel in the first finger area according to the parallax and the parameters of the binocular camera.

5. A palm depth rapid calculation method according to claim 4, characterized in that: In step S55, the search range of each finger is calculated according to the width of each finger, and then matching is performed to calculate the disparity.

6. A palm depth rapid calculation method according to claim 4, characterized in that: In step S55, the search range is calculated according to the maximum width of the finger, and then matching is performed to calculate the disparity.

7. A palm depth rapid calculation method according to claim 1, characterized in that: Step S2 includes: Step S21: detecting a third hand region and a fourth hand region on the first image, and detecting a fifth hand region and a sixth hand region on the second image; Step S22: Pairing the third hand region, the fourth hand region, the fifth hand region and the sixth hand region according to the epipolar constraint.

8. A palm depth rapid calculation system, used to implement the palm depth rapid calculation method according to any one of claims 1 to 7, characterized in that: include: An acquisition module, used to acquire a first image and a second image; wherein the first image and the second image are acquired by a binocular camera; A detection module, configured to detect a first hand region on the first image and a second hand region on the second image; an alignment module, configured to align the first hand region with the second hand region; A scaling module, used for scaling the first hand area and the second hand area to a preset size at the same magnification; A depth calculation module is used to perform depth calculation on the first hand area and the second hand area.

9. A palm depth rapid calculation device, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to execute the steps of the palm depth fast calculation method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed, the steps of the palm depth fast calculation method described in any one of claims 1 to 7 are implemented.

Images