A palm correction method based on non-homologous double purposes

Abstract

A palm correction method based on non-homologous binoculars, comprising the following steps: step S1: detecting a palm of a first image I ir and a second image I rgb respectively, and obtaining a palm region ROI ir of the first image I ir and a palm region ROI rgb of the second image I rgb respectively; wherein the first image and the second image are non-homologous images; step S2: estimating a palm orientation of the first image I ir or the second image I rgb by using a deep learning method; step S3: if the palm orientation does not meet a preset requirement, re-executing step S1; step S4: correcting the first image and the second image according to the palm orientation. The non-homologous binocular system composed of the first camera and the second camera is used to directly process the first image and the second image, so that the quality of image acquisition is improved, and the palm orientation is used for correction, so that a palm with a larger angle can be processed, and the accuracy of correction is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of palm recognition cameras, and more specifically, to a palm correction method based on non-homogeneous binoculars. Background Technology

[0002] Because the lines on a human palm are more stable, palm recognition is a more stable and secure biometric technology than facial recognition. Therefore, it can be used to identify individuals by scanning their palms, and can be applied to security checks, payments, and identity verification. Palm recognition is a technology with broad application prospects.

[0003] Existing non-contact palm recognition devices fall into two categories: one simultaneously recognizes palm prints and palm veins; the other identifies solely through palm vein information. The former greatly simplifies the system, and because palm veins are coarse-grained features, lower resolution is sufficient to preserve their information, but this also limits the accuracy of palm vein recognition. Furthermore, the image quality of palm veins is significantly affected by light source, hand distance, and posture, making it prone to capturing unstable images. While existing technologies improve data quality somewhat by binarizing the image and then comparing it with the original image, they still cannot meet the demands of commercial applications. Summary of the Invention

[0004] Therefore, this invention utilizes a non-homogeneous binocular system composed of a first camera and a second camera to directly process the first and second images, thereby improving the quality of image acquisition. Furthermore, it uses palm orientation for correction, enabling processing of palms at larger angles and significantly improving the accuracy of the correction.

[0005] In a first aspect, the present invention provides a hand correction method based on non-homogeneous binocular vision, characterized by comprising the following steps:

[0006] Step S1: For the first image I ir Second image I rgb The palms are detected separately, and the first image I is obtained separately. ir palm area ROI ir and the second image I rgb palm area ROI rgb Wherein, the first image and the second image are non-homogeneous images;

[0007] Step S2: Use deep learning methods to process the first image I ir Or the second image I rgb Estimate the orientation of the palm;

[0008] Step S3: If the palm orientation does not meet the preset requirements, then repeat step S1;

[0009] Step S4: Based on the palm orientation, compare the first image I ir and the second image I rgb Perform corrections.

[0010] Optionally, the aforementioned hand correction method based on non-homogeneous binoculars is characterized in that, before step S1, it further includes:

[0011] Step S0: Correct the distortion of the original first image and the original second image respectively, and then perform epipolar correction to obtain the corrected first image I. ir and the corrected second image I rgb .

[0012] Optionally, the hand correction method based on non-homogeneous binoculars is characterized in that, in step S1, the first image I is further processed. ir and the second image I rgb Based on the palm region ROI ir and the palm region ROI rgb Perform image segmentation and set the non-palm regions to zero.

[0013] Optionally, the hand correction method based on non-homogeneous binoculars is characterized in that, in step S2, only the first image I is corrected. ir and the second image I rgb One of the images was used to estimate the orientation of the hand.

[0014] Optionally, the hand correction method based on non-homogeneous binoculars is characterized in that, in step S2, the first image I... ir and the second image I rgb Deep learning is performed separately to obtain palm orientation estimates, and cross-validation is used to obtain the palm orientation.

[0015] Optionally, the hand correction method based on non-homogeneous binoculars is characterized in that, in step S2, the first image I... ir and the second image I rgb Simultaneously, deep learning is performed to determine the orientation of the hand.

[0016] Optionally, the hand correction method based on non-homogeneous binoculars is characterized in that step S2 includes:

[0017] Step S21: Acquire the first image I respectively ir and the second image I rgb The key points;

[0018] Step S22: Calculate the three-dimensional spatial information of the key points using parallax;

[0019] Step S23: Calculate the palm orientation based on the three-dimensional spatial information.

[0020] Optionally, the hand correction method based on non-homogeneous binoculars is characterized in that step S4 includes:

[0021] Step S41: Divide the image into a first stretching area, a first compression area, and a first repair area on the second image according to the palm orientation, grayscale value, and distance from the edge, and copy the area onto the first image; wherein, the first image is a texture image of the palm, and the second image is a vein image of the palm;

[0022] Step S42: On the first image, the first stretched area, the first compressed area and the first repaired area are finely adjusted according to the texture features to obtain the second stretched area, the second compressed area and the second repaired area respectively, and the second stretched area, the second compressed area and the second repaired area are copied onto the second image;

[0023] Step S43: On the first image and the second image, stretch the second stretching area by the same amount, compress the second compression area by the same amount, and repair the second repair area respectively.

[0024] Secondly, the present invention provides a hand correction device based on non-homogeneous binocular vision, characterized in that it includes:

[0025] processor;

[0026] A memory module that stores executable instructions of the processor;

[0027] The processor is configured to perform the steps of the non-homogeneous binocular hand correction method described above by executing the executable instructions.

[0028] Thirdly, the present invention provides a computer-readable storage medium for storing a program, characterized in that, when the program is executed, it implements the steps of the palm correction method based on non-homogeneous binocular vision described in any of the preceding claims.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] This invention uses the first and second images as raw data, eliminating the need for devices such as p-sensors. This reduces the input conditions for palm recognition, thereby simplifying the corresponding hardware devices, making them smaller, easier to integrate, and facilitating device miniaturization.

[0031] The images used in this invention can be shared with other palm recognition functions, allowing a single image to be used for multiple functions, thereby maximizing the functionality of an image and saving steps and equipment space. For example, when the first image is a color image, it can be used not only for correction and reconstruction but also for palm print recognition; when the second image is an infrared image, it can be used not only for correction and reconstruction but also for liveness detection. This invention corrects two different types of images, giving subsequent image processing more dimensional data and improving processing quality.

[0032] In existing technologies, palm print recognition and palm vein features are processed using the same image, but the results for both palm prints and veins are not good. This invention uses separate cameras to recognize palm prints and palm veins, resulting in high image quality and good performance. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0034] Figure 1 This is a flowchart illustrating the steps of a hand correction method based on non-homogeneous binocular vision in an embodiment of the present invention.

[0035] Figure 2 This is a hand detection image from an embodiment of the present invention;

[0036] Figure 3 This is a flowchart illustrating the steps of palm orientation estimation in an embodiment of the present invention;

[0037] Figure 4 The spatial positions of key points on the palm in this embodiment of the invention;

[0038] Figure 5 This is a flowchart illustrating the steps of image correction in an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of the structure of a non-homogeneous binocular palm correction device according to an embodiment of the present invention;

[0040] Figure 7 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. Detailed Implementation

[0041] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0042] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0043] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0044] The present invention provides a hand correction method based on non-homogeneous binocular vision, which aims to solve the problems existing in the prior art.

[0045] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0046] Figure 1 This is a flowchart illustrating the steps of a hand correction method based on non-homogeneous binocular vision according to an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a hand correction method based on non-homogeneous binocular vision, which includes the following steps:

[0047] Step S1: For the first image I ir Second image I rgb The palms are detected separately, and the first image I is obtained separately. ir palm area ROI ir and the second image I rgb palm area ROI rgb .

[0048] In this step, the first image and the second image are non-homogeneous images. A handprint detection model is used for the first image I. ir and the second image I rgb Each image is detected separately to determine if a hand is present. If a hand is present, the first image I is obtained. ir palm area ROI ir and the second image I rgb palm area ROI rgb .like Figure 2 As shown, the ROI of the palm region ir and palm area ROI rgb All rectangles are the smallest rectangles that include the palm, meaning that all four sides of the rectangle are tangent to the edge of the palm. The palm detection model used in this step can be any model that can perform palm detection; this embodiment does not impose any restrictions on it.

[0049] In some embodiments, the first image I is respectively... ir and the second image I rgb Based on the palm region ROI ir and the palm region ROI rgb Image segmentation is performed, and non-palm regions are zeroed out. By zeroing out non-palm regions, the contrast between the palm region and other regions is enhanced, resulting in better subsequent computation and processing with less processing power.

[0050] In some embodiments, this method is adapted for continuous shooting. The hand region in each frame is obtained by subtracting the previous frame from the subsequent frame. Since the hand is not fixed in place by a palm rest, it is difficult to keep the hand in the same position. Therefore, subtracting two frames taken at different times can quickly locate the edge of the hand, thereby obtaining the hand contour information and locating the hand region. At the same time, the image subtraction method is simple and fast.

[0051] In some embodiments, before performing the image processing in this step, the first image I is... ir Second image I rgb Compress the first image I. ir Second image I rgb The compression ratios differ. For example, before performing this step, the first image I... ir and the second image I rgbCompression is performed separately at ratios of 4 and 2, respectively, to make the image sizes more similar and ensure processing accuracy. Preferably, the closer the palm is to the camera, the higher the compression ratio; the farther the palm is from the camera, the lower the compression ratio. When the palm occupies 70% of the image area, the compression ratio of the first image is no less than 5; when the palm occupies 50% of the image area, the compression ratio of the first image is no less than 2. The compression ratio of the first image is more than three times that of the second image.

[0052] Step S2: Use deep learning methods to process the first image I ir Or the second image I rgb The orientation of the palm is estimated.

[0053] In this step, a deep learning model is built and trained using images labeled with hand orientation to obtain a hand orientation model. This model can identify the orientation of the hand in an image. Hand pose estimation can be divided into two categories based on the final generation method: detection-based methods and regression-based methods. Detection-based methods generate heatmaps to obtain the final predicted keypoints. Regression-based methods directly regress the position coordinates of the keypoints. The feature extraction network can use a ResNet-50 network, VGG-19, or an Hourglass network (hourglass network structure).

[0054] In some embodiments, only the first image I is considered. ir and the second image I rgb One image is used to estimate the hand orientation. For example, by using a deep learning model trained only on the first image, the hand orientation can be determined simply by recognizing the first image. This approach consumes fewer resources and can quickly obtain results when the hand is relatively standard, achieving a fast response time.

[0055] In some embodiments, the first image I ir and the second image I rgbDeep learning is performed separately to obtain hand orientation estimates, and cross-validation is used to obtain the final hand orientation. For example, deep learning models are trained separately on the training sets of the first and second images, resulting in a first model and a second model, respectively. The first model is used to identify the hand orientation in the first image, and the second model is used to identify the hand orientation in the second image. When calculating the hand orientation, the first orientation of the first image is obtained through the first model, and the second orientation of the second image is obtained through the second model. Theoretically, the first and second orientations are the same. When the first and second orientations are the same, the first orientation is output. When the deviation between the first and second orientations is within a first threshold, the average of the first and second orientations is calculated and output. When the deviation exceeds the first threshold, an error is reported, and the image is retaken and processed. This approach trains the model on two types of images separately, maximizing the training samples and allowing for cross-validation, resulting in more accurate results.

[0056] In some embodiments, the first image I ir and the second image I rgb Simultaneously, deep learning is performed to obtain the palm orientation. During the training of the deep learning model, the same model is used to train the sample sets that are from the same source as both the first and second images. This allows the deep learning model to combine the first and second images to estimate the palm orientation. This approach trains non-homogeneous images in pairs, resulting in a more accurate palm orientation estimate.

[0057] Step S3: If the palm orientation does not meet the preset requirements, then repeat step S1.

[0058] In this step, if the palm is facing within the preset range, then proceed to step S4; otherwise, repeat step S1.

[0059] Step S4: Correct the first image and the second image according to the palm orientation.

[0060] In this step, the hand image is corrected to a direction perpendicular to the camera lens, based on the palm's orientation. Since the pre-stored palm information is taken perpendicular to the lens, it is necessary to correct the palm to a direction perpendicular to the lens. In some embodiments, if the pre-stored palm information provides a palm orientation, the current hand image is corrected to the same palm orientation as the pre-stored palm information to improve consistency.

[0061] In some embodiments, the method further includes the following step before step S1:

[0062] Step S0: Correct the distortion of the original first image and the original second image respectively, and then perform epipolar correction to obtain the corrected first image I.ir and the corrected second image I rgb .

[0063] In this step, the original first image and the original second image are non-homogeneous images, meaning they were obtained using different techniques. Distortion correction is performed on the original first image, followed by epipolar correction, to obtain the corrected first image I. ir The original second image is distorted and then epipolarized to obtain the corrected second image I. rgb Since distortion is caused by the lens imaging principle, distortion correction for the original first and second images needs to be performed according to the parameters of each acquisition device. Epipolar correction is a correction for binocular systems. It involves rotating the two cameras and redefining a new image plane so that the epipolar pairs are collinear and parallel to a coordinate axis (usually the horizontal axis) of the image plane. This operation simultaneously establishes a new stereo image pair. After correction, the same matching point pair is located in the same row in both views, meaning they only differ in horizontal coordinates (or column coordinates), a difference called parallax. However, since the images used are the first and second images, the content they capture differs, making it impossible to directly solve for parallax using current technology. When the first camera captures an image of the palm, the veins absorb infrared light, resulting in darker areas around the veins. The second image, capturing the palm simultaneously, primarily images the surface texture of the palm, making direct matching of the palm prints difficult. Furthermore, the palms of people with different body types and builds vary significantly, further widening the difference between the infrared image and the second image, making effective matching even more challenging. This step corrects both images to make the data more accurate, thereby improving the accuracy of subsequent matching. It should be noted that the original first and second images used in this embodiment are typically acquired through a calibrated binocular system, where one camera is a first camera and the other a second camera. The first camera is used to acquire the first image, and the second camera is used to acquire the second image; both cameras acquire the target image simultaneously. For example, the first camera may be a near-infrared camera, resulting in a near-infrared image, while the second camera may be a color camera, resulting in a color image.

[0064] This step allows for better processing of images with significant distortion, enabling accurate results even when the palm is close to the imaging device (i.e., when the field of view is large), thus improving the effective recognition range for the palm.

[0065] Figure 3 This is a flowchart illustrating the steps involved in estimating palm orientation according to an embodiment of the present invention. Figure 3 As shown, unlike the aforementioned embodiments, a method for estimating palm orientation includes:

[0066] Step S21: Acquire the first image I respectively ir and the second image I rgb The key points.

[0067] In this step, only the palm area is identified, which reduces the range and amount of data to be processed, thereby improving response speed and reducing hardware requirements. Figure 4 The spatial locations of keypoints on the palm are shown. Keypoints include four at each finger joint and four at each end, plus one at the base of the palm, for a total of 21 keypoints. For example, a hand keypoint detection model trained on a ResNet50 network is used to extract hand keypoints from an input single image (IR / RGB). The hand keypoint detection model outputs the estimated XYZ coordinates of the keypoints.

[0068] In some embodiments, a palm keypoint detection model is used to estimate the palm orientation, obtaining key points of the palm in each image. The palm keypoint detection model is trained using a training set originating from the first image to obtain a first recognition model, which is then used to recognize the key points in the first image. Similarly, the palm keypoint detection model is trained using a training set originating from the second image to obtain a second recognition model, which is then used to recognize the key points in the second image.

[0069] In some embodiments, a hand keypoint detection model is used to estimate the hand orientation in the first image to obtain keypoints in the first image. Then, based on the matching relationship between the first and second images, keypoints in the second image are obtained. This method significantly reduces the time and resource consumption for training the model, improves efficiency, and can utilize existing models in the prior art, reducing the application cost of this embodiment.

[0070] Step S22: Calculate the three-dimensional spatial information of the key points using parallax.

[0071] In this step, only the 3D spatial information of key points is calculated, eliminating the need for 3D reconstruction of the entire image, thus minimizing the computational load of 3D space calculations. This step achieves 3D spatial information even for locations where depth data is unavailable by matching key points in non-originating images. The surface normal vector is calculated based on the 3D coordinates of the key points. Specifically, given a set of key points within a certain range, the normal vector of the optimal plane within that set is determined. Taking three points on the plane: P1(x1,y1,z1), P2(x2,y2,z2), and P3(x3,y3,z3), and using the normal vector (dx,dy,dz), the normal vector satisfies the following equation. The normal vector is then calculated using Cramer's rule.

[0072] Step S23: Calculate the palm orientation based on the three-dimensional spatial information.

[0073] In this step, a topological model of the hand key points is established based on the 21 key points provided by the aforementioned model. This topological model divides the key points into six planar fitting regions: palm, thumb, index finger, middle finger, ring finger, and little finger. Key points 2-4 are numbered for the thumb region, 6-8 for the index finger region, 10-12 for the middle finger region, 14-16 for the ring finger region, 18-19 for the little finger region, and the remaining key points 0, 1, 2, 5, 9, 13, and 17 for the palm region. Based on the obtained XYZ coordinates of the key points in these regions, the planar normal vectors are calculated. The results are used to determine whether the palm is in an upright, open state. The specific criterion is as follows: the normal vectors of the six regions are evaluated for error. If all the calculated planar normal vectors of the six regions are perpendicular and forward (allowing for a certain range of error), then the palm is considered to be in an upright, open state. If the normal vector in any region deviates significantly from the normal vectors of other regions, the palm is considered not to meet the standard for an upright, open state.

[0074] This embodiment obtains key points on the palm, matches key points on the first and second images, and then calculates the palm orientation using the 3D information of the key points. This embodiment uses key points on the palm to match points with different image features even in non-originating images, and utilizes the parallax principle to obtain depth data, thereby achieving accurate 3D reconstruction and obtaining accurate palm orientation. Compared to existing technologies, the data obtained in this embodiment is more accurate, and the obtained data can be used for processing more palms.

[0075] This invention eliminates the need to process the entire palm image; it only processes key points of the palm within the image. This significantly reduces the amount of data processed, thereby lowering the requirements for chips and other components. Furthermore, it eliminates the need for a separate on-chip system and can be integrated into the camera's built-in chip, simplifying the camera's structure, reducing costs, and enabling low-cost palm-swiping applications, which is beneficial for commercial promotion.

[0076] Figure 5 This is a flowchart illustrating the steps of image correction in an embodiment of the present invention. Figure 5 As shown, compared to the aforementioned embodiments, the image correction method provided in this embodiment of the invention includes the following steps:

[0077] Step S41: Divide the image into a first stretching area, a first compression area, and a first repair area on the second image according to the palm orientation, grayscale value, and distance from the edge, and copy the area onto the first image.

[0078] In this step, the first image is a texture image of the palm, and the second image is a vein image of the palm. The first stretching area, the first compression area, and the first repair area are defined based on the subsequent image processing method. After the palm is corrected for its current orientation, the area that becomes larger is the first stretching area, the area that becomes smaller is the first compression area, and the area that is occluded due to the angle and requires image content supplementation is the first repair area. The division of these three areas also applies to the first image.

[0079] Step S42: On the first image, the first stretched area, the first compressed area, and the first repaired area are finely adjusted according to the texture features to obtain the second stretched area, the second compressed area, and the second repaired area, respectively, and the second stretched area, the second compressed area, and the second repaired area are copied onto the second image.

[0080] In this step, the texture features on the palm are crucial for comparing the palm with pre-stored palm information; therefore, texture features are an important factor in palm correction. Important lines on the palm include the life line, success line, heart line, head line, and marriage line. These lines need to be kept clear and continuous during correction, so the division of each area needs to be fine-tuned based on these lines to make the division more reasonable.

[0081] Step S43: On the first image and the second image, stretch the second stretching area by the same amount, compress the second compression area by the same amount, and repair the second repair area respectively.

[0082] In this step, at the same location, the stretching or compression amplitude is the same for both the first and second images. The stretching amplitude can vary at different locations in the second stretching area, and the compression amplitude can also vary at different locations in the second compression area. For the second repair area, repair is performed separately for both the first and second images. Because the second repair area is located in different positions or the degree of finger bending may cause some content to be obscured, repair is needed on both the first and second images. During repair, palm print or vein information can be used.

[0083] This embodiment subdivides the palm into different areas, making the palm calibration operation more precise and effective. It also takes into account the occluded parts, can handle larger angles, expands the palm range supported by palm recognition, and makes palm recognition application more convenient and faster.

[0084] This invention also provides a non-homogeneous binocular hand correction device, including a processor and a memory storing executable instructions for the processor. The processor is configured to execute steps of a non-homogeneous binocular hand correction method via the executable instructions.

[0085] As described above, in this embodiment, a first image and a second image are acquired by a depth camera of a binocular system consisting of a first camera and a second camera. The two different types of images are corrected by the method described in the previous embodiment to overcome the differences between the different types of images and achieve the purpose of accurate correction.

[0086] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "platform."

[0087] Figure 6 This is a schematic diagram of a non-homogeneous binocular hand correction device according to an embodiment of the present invention. See below for reference. Figure 6 To describe an electronic device 600 according to this embodiment of the present invention. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0088] like Figure 6 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0089] The storage unit stores program code, which can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the above section of this specification regarding the non-homogeneous binocular palm correction method according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform, as follows: Figure 1 The steps are shown in the figure.

[0090] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0091] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0092] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0093] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although... Figure 6 As not shown in the diagram, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0094] This invention also provides a computer-readable storage medium for storing a program that, when executed, implements the steps of a non-homogeneous binocular hand correction method. In some possible implementations, various aspects of the invention can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the foregoing section of this specification regarding the non-homogeneous binocular hand correction method, according to various exemplary embodiments of the invention.

[0095] As shown above, when the program of the computer-readable storage medium of this embodiment is executed, it acquires a first image and a second image by using the depth camera of a binocular system composed of a first camera and a second camera, and corrects the two different types of images by using the method in the foregoing embodiment, thereby overcoming the differences between the different types of images and achieving the purpose of stable and fast correction.

[0096] Figure 7This is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. (Reference) Figure 7 As shown, a program product 800 for implementing the above-described method according to an embodiment of the present invention is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0097] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0098] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0099] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0100] In this embodiment of the invention, a first image and a second image are acquired by a depth camera of a binocular system consisting of a first camera and a second camera. The two different types of images are corrected by the method described in the foregoing embodiment, overcoming the differences between the different types of images and achieving the purpose of stable and fast correction.

[0101] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0102] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A hand correction method based on non-homogeneous binoculars, characterized in that, Includes the following steps: Step S1: For the first image Second image The first image is obtained by detecting the palm separately. palm area and the second image palm area Wherein, the first image and the second image are non-homogeneous images; Step S2: Use deep learning methods to process the first image. Or the second image Estimate the orientation of the palm; Step S3: If the palm orientation does not meet the preset requirements, then repeat step S1; Step S4: Based on the palm orientation, view the first image. and the second image Perform correction; Step S4 includes: Step S41: Divide the image into a first stretching area, a first compression area, and a first repair area on the second image according to the palm orientation, grayscale value, and distance from the edge, and copy the area onto the first image; wherein, the first image is a texture image of the palm, and the second image is a vein image of the palm; Step S42: On the first image, the first stretched area, the first compressed area and the first repaired area are finely adjusted according to the texture features to obtain the second stretched area, the second compressed area and the second repaired area respectively, and the second stretched area, the second compressed area and the second repaired area are copied onto the second image; Step S43: On the first image and the second image, stretch the second stretching area by the same amount, compress the second compression area by the same amount, and repair the second repair area respectively.

2. The hand correction method based on non-homogeneous binoculars according to claim 1, characterized in that, The procedure before step S1 also includes: Step S0: Correct the distortion of the original first image and the original second image respectively, and then perform epipolar correction to obtain the corrected first image. and the corrected second image .

3. The hand correction method based on non-homogeneous binoculars according to claim 1, characterized in that, In step S1, the first image is also processed respectively. and the second image According to the palm area and the palm area Perform image segmentation and set the non-palm regions to zero.

4. The hand correction method based on non-homogeneous binoculars according to claim 1, characterized in that, In step S2, only the first image is processed. and the second image One of the images was used to estimate the orientation of the hand.

5. The hand correction method based on non-homogeneous binoculars according to claim 1, characterized in that, In step S2, the first image and the second image Deep learning is performed separately to obtain palm orientation estimates, and cross-validation is used to obtain the palm orientation.

6. The hand correction method based on non-homogeneous binoculars according to claim 1, characterized in that, In step S2, the first image and the second image Simultaneously, deep learning is performed to determine the orientation of the hand.

7. The hand correction method based on non-homogeneous binoculars according to claim 1, characterized in that, Step S2 includes: Step S21: Acquire the first image respectively and the second image The key points; Step S22: Calculate the three-dimensional spatial information of the key points using parallax; Step S23: Calculate the palm orientation based on the three-dimensional spatial information.

8. A hand correction device based on non-homogeneous binocular vision, characterized in that, include: processor; A memory module that stores executable instructions of the processor; The processor is configured to perform the steps of the non-homogeneous binocular palm correction method according to any one of claims 1 to 7 by executing the executable instructions.

9. A computer-readable storage medium for storing a program, characterized in that, When the program is executed, it implements the steps of the non-homogeneous binocular palm correction method according to any one of claims 1 to 7.

Images