A two-image alignment method based on non-homologous double purposes

Abstract

A two-image alignment method for palm recognition based on a non-homologous binocular system, comprising the following steps: step S1: detecting a palm in a first image I ir and a second image I rgb respectively, and obtaining a palm region ROI ir of the first image I ir and a palm region ROI rgb of the second image I rgb respectively; wherein the first image and the second image are non-homologous images; step S2: estimating a palm posture of the palm region ROI ir and the palm region ROI rgb , and obtaining key points of the palm respectively; step S3: calculating a parallax of a palm center according to information of the key points; and step S4: aligning the first image I ir and the second image I rgb according to the parallax of the palm center. The application directly processes the first image and the second image by using a non-homologous binocular system, saves a p-sensor, aligns the first image and the second image by using the key points of the palm, greatly reduces a palm recognition calculation amount, reduces a cost, and is conducive to popularization of palm recognition.

Description

Technical Field

[0001] This invention relates to the field of palm recognition cameras, and more specifically, to a method for aligning two images in palm recognition based on non-homogeneous binoculars. Background Technology

[0002] Because the lines on a human palm are more stable, palm recognition is a more stable and secure biometric technology than facial recognition. Therefore, it can be used to identify individuals by scanning their palms, and can be applied to security checks, payments, and identity verification. Palm recognition is a technology with broad application prospects.

[0003] In existing technologies, some palm-scanning systems use a palm rest to fix the palm and determine its position, but this is inconvenient in practical applications. Some systems use p-sensors to estimate palm depth and then align the first and second images pixel-wise based on the p-sensor. In practice, due to considerations of measurement range and distance, it is generally necessary to capture the entire palm at close range and to have a clear image of the palm at a distance. This requires a sufficiently large field of view (FOV) for the camera. With a large FOV, the palm will not occupy a very large proportion of the image at a distance. If p-sensors are still used for image alignment, a large number of p-sensors would be required, leading to mutual interference, design incompatibility issues, and significantly increased costs.

[0004] Existing binocular systems all use the same source binoculars, which can obtain clear depth data of the target object. However, the computational load of this binocular system is large, and a separate on-chip system is required to process the relevant data, which leads to a series of problems such as transmission delay and recognition lag. Summary of the Invention

[0005] Therefore, this invention utilizes a non-homogeneous binocular system composed of a first camera and a second camera to directly process the first and second images, saving p-sensors. It also uses key points of the palm to align the first and second images, greatly reducing the computational load of palm recognition, requiring fewer devices, saving more device space, and having better compatibility and lower costs, which is conducive to the promotion of palm recognition applications.

[0006] In a first aspect, the present invention provides a method for aligning two images based on non-homogeneous binocular palm print recognition, characterized by comprising the following steps:

[0007] Step S1: For the first image I ir Second image I rgb The palms are detected separately, and the first image I is obtained separately. ir palm area ROI ir and the second image Irgb palm area ROI rgb Wherein, the first image and the second image are non-homogeneous images;

[0008] Step S2: Analyze the ROI of the palm region. ir and the palm region ROI rgb The hand posture was estimated, and the key points of the hand were obtained respectively;

[0009] Step S3: Calculate the parallax of the palm center based on the information of the key points;

[0010] Step S4: Based on the parallax of the center of the palm, adjust the first image I... ir and the second image I rgb Alignment.

[0011] Optionally, the method for aligning two images based on non-homogeneous binocular palm print recognition is characterized in that, before step S1, it further includes:

[0012] Step S0: Correct the distortion of the original first image and the original second image respectively, and then perform epipolar correction to obtain the corrected first image I. ir and the corrected second image I rgb .

[0013] Optionally, the method for aligning two images based on non-homogeneous binocular palm print recognition is characterized in that step S2 includes:

[0014] Step S21: Obtain the palm edge using the edge extraction algorithm, and then obtain the palm pose;

[0015] Step S22: Obtain key points based on the hand posture;

[0016] Step S23: Transfer the first image I ir and the second image I rgb Match the key points mentioned above.

[0017] Optionally, the method for aligning two images based on non-homogeneous binocular palm print recognition is characterized in that step S3 includes:

[0018] Step S31: Calculate the disparity d of the key points;

[0019] Step S32: Calculate the position of the center of the palm based on the key points at the base of the fingers and the base of the palm;

[0020] Step S33: Calculate the parallax b of the center of the palm based on the parallax of key points at the base of the fingers and the base of the palm.

[0021] Optionally, the method for aligning two images based on non-homogeneous binocular palm print recognition is characterized in that, in step S1, when aligning the first image I... ir and the second image I rgb Before detection, the first image I was also... ir and the second image I rgb Compress it.

[0022] Optionally, the method for aligning two images based on non-homogeneous binocular palm print recognition is characterized in that, for the first image I... ir Second image I rgb The compression ratios are different.

[0023] Optionally, the method for aligning two images based on non-homogeneous binocular palm print recognition is characterized in that, in step S22, a first deep learning model is used to align the first image I. ir The first key point is obtained by identifying the edge of the palm in the image; a second deep learning model is used to process the second image I. rgb The second key point is obtained by recognizing the edge of the palm in the image, wherein the first deep learning model is based on the first image I. ir The second deep learning model is obtained by training on homologous palm images, and is based on images I. rgb It was obtained by training with images of the same hand.

[0024] Optionally, the method for aligning two images based on non-homogeneous binocular palm print recognition is characterized in that, in step S22, the first image I... ir Subtract the second image I rgb Obtain the third image I0, and then obtain the key points based on the third image I0.

[0025] Secondly, the present invention provides a two-image alignment device based on non-homogeneous binocular palm recognition, characterized in that it includes:

[0026] processor;

[0027] A memory module that stores executable instructions of the processor;

[0028] The processor is configured to perform the steps of the above-described method for aligning two images based on non-homogeneous binocular palm recognition by executing the executable instructions.

[0029] Thirdly, the present invention provides a computer-readable storage medium for storing a program, characterized in that, when the program is executed, it implements the steps of the above-described method for aligning two images based on non-homogeneous binocular palm recognition.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] This invention uses the first and second images as raw data, eliminating the need for devices such as p-sensors. This reduces the input conditions for palm recognition, thereby simplifying the corresponding hardware devices, making them smaller, easier to integrate, and facilitating device miniaturization.

[0032] The images used in this invention can be shared with other palm recognition functions, allowing a single image to be used for multiple functions, thereby maximizing the functionality of an image and saving steps and equipment space. For example, when the first image is an infrared image, it can be used not only for alignment and reconstruction but also for liveness detection; when the second image is a color image, it can be used not only for alignment and reconstruction but also for palmprint recognition. This invention, by aligning two different types of images, makes steps such as palm detection, palm segmentation, edge extraction, edge matching, and depth reconstruction more convenient, saving subsequent image processing steps and improving processing efficiency.

[0033] This invention uses a first camera and a second camera to image a palm, achieving pixel-level alignment between the two images when a depth map is unavailable. The invention proposes using a first and second camera to form a binocular system, reconstructing the three-dimensional shape of the palm's edges based on binocular imaging theory, and then achieving pixel-level alignment between the first and second images based on the depth information of the palm's edges.

[0034] This invention constructs a binocular system using a first and second camera that are not from the same source. However, due to their non-homogeneous nature, existing technologies cannot effectively solve problems such as matching. This invention achieves matching and 3D reconstruction for non-homogeneous binocular systems. For example, when the first camera is a near-infrared camera and the second camera is a color camera, when capturing an image of a hand, the veins will appear darker because veins absorb infrared light. Furthermore, when the second camera simultaneously captures an image of the hand, it primarily images the surface texture of the hand, leading to greater differences between the first and second images, making effective processing impossible using existing technologies.

[0035] This invention eliminates the need to process the entire palm image; it only processes key points of the palm within the image. This significantly reduces the amount of data processed, thereby lowering the requirements for chips and other components. Furthermore, it eliminates the need for a separate on-chip system and can be integrated into the camera's built-in chip, simplifying the camera's structure, reducing costs, and enabling low-cost palm-swiping applications, which is beneficial for commercial promotion.

[0036] This invention calculates the disparity *b* at the center of the palm using key point information of the palm. This significantly reduces the amount of data processed, simplifies the computation steps, and improves efficiency, allowing for rapid calculation results and quick alignment, thus increasing image processing speed. Compared to other processing methods, this invention improves palm-swipe response speed, making it more suitable for applications requiring rapid response. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0038] Figure 1 This is a flowchart illustrating the steps of a method for aligning two images based on non-homogeneous binocular palm recognition in an embodiment of the present invention.

[0039] Figure 2 This is a hand detection image from an embodiment of the present invention;

[0040] Figure 3 The spatial positions of key points on the palm in this embodiment of the invention;

[0041] Figure 4 This is a set of palm-aligned images in an embodiment of the present invention;

[0042] Figure 5 This is a flowchart illustrating the steps for obtaining key points in an embodiment of the present invention;

[0043] Figure 6 This is a set of palm edge images in an embodiment of the present invention;

[0044] Figure 7 This is a flowchart illustrating the steps for calculating the parallax at the center of the palm in this embodiment;

[0045] Figure 8 This is a schematic diagram of the structure of a two-image alignment device based on non-homogeneous binocular palm recognition according to an embodiment of the present invention;

[0046] Figure 9 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. Detailed Implementation

[0047] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0048] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0049] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0050] This invention provides a method for aligning two images based on non-homogeneous binocular palm recognition, which aims to solve the problems existing in the prior art.

[0051] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0052] Figure 1 This is a flowchart illustrating the steps of a method for aligning two images based on non-homogeneous binocular palm print recognition, as described in an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a method for aligning two images based on non-homogeneous binocular palm print recognition, comprising the following steps:

[0053] Step S1: For the first image I ir Second image I rgb The palms are detected separately, and the first image I is obtained separately. ir palm area ROI ir and the second image I rgb palm area ROI rgb.

[0054] In this step, the first image and the second image are non-homogeneous images. A handprint detection model is used for the first image I. ir and the second image I rgb Each image is detected separately to determine if a hand is present. If a hand is present, the first image I is obtained. ir palm area ROI ir and the second image I rgb palm area ROI rgb .like Figure 2 As shown, the ROI of the palm region ir and palm area ROI rgb All rectangles are the smallest rectangles that include the palm, meaning that all four sides of the rectangle are tangent to the edge of the palm. The palm detection model used in this step can be any model that can perform palm detection; this embodiment does not impose any restrictions on it.

[0055] In some embodiments, this method is adapted for continuous shooting. The hand region in each frame is obtained by subtracting the previous frame from the subsequent frame. Since the hand is not fixed in place by a palm rest, it is difficult to keep the hand in the same position. Therefore, subtracting two frames taken at different times can quickly locate the edge of the hand, thereby obtaining the hand contour information and locating the hand region. At the same time, the image subtraction method is simple and fast.

[0056] In some embodiments, before performing the image processing in this step, the first image I is... ir Second image I rgb Compress the first image I. ir Second image I rgb The compression ratios differ. For example, before performing this step, the first image I... ir and the second image I rgb Compression is performed separately at ratios of 4 and 2, respectively, to make the image sizes more similar and ensure processing accuracy. Preferably, the closer the palm is to the camera, the higher the compression ratio; the farther the palm is from the camera, the lower the compression ratio. When the palm occupies 70% of the image area, the compression ratio of the first image is no less than 5; when the palm occupies 50% of the image area, the compression ratio of the first image is no less than 2. The compression ratio of the first image is more than three times that of the second image.

[0057] Step S2: Estimate the hand pose of the hand region in the first image and the hand region in the second image to obtain the key points of the hand.

[0058] In this step, only the palm area is identified, thus reducing the data range and volume, thereby improving response speed and lowering hardware requirements. Since the first and second images are not from the same source, they are difficult to process using the same model. Figure 3 The spatial locations of key points on the palm are shown. There are 21 key points in total, including four at each finger joint and four at each end, plus one at the base of the palm.

[0059] In some embodiments, a hand pose detection model is used to estimate the hand pose and obtain key points of the hand in each image. A first recognition model is trained using a training set originating from the first image, and then used to recognize the key points in the first image. Similarly, a second recognition model is trained using a training set originating from the second image, and then used to recognize the key points in the second image.

[0060] In some embodiments, a hand pose detection model is used to estimate the hand pose of the first image to obtain key points in the first image. Then, based on the matching relationship between the first image and the second image, the key points in the second image are obtained. This method greatly reduces the time and resource consumption for training the model, improves efficiency, and can utilize existing models in the prior art, reducing the application cost of this embodiment.

[0061] Step S3: Calculate the parallax of the palm center based on the information of the key points.

[0062] In this step, the keypoint information includes location, parallax, etc. Since the keypoints on the palm are located in various parts of the palm, and the center of the palm is always located in the middle of all the keypoints, only six keypoints on the palm are needed to calculate the parallax of the palm center. The palm center is the point with the smallest difference in distance to the six keypoints on the palm. For example, the parallax of the palm center is obtained by calculating the average of the parallaxes of the six keypoints.

[0063] Step S4: Based on the parallax of the center of the palm, adjust the first image I... ir and the second image I rgb Alignment.

[0064] In this step, the image is translated using the polar plane in the binocular system. Specifically, the image is translated using the polar plane located at the center of the palm, so that the first image I... ir and the second image I rgb The centers of the palms overlap to complete the alignment. For example... Figure 4 As shown, the aligned images have better consistency, thus enabling better recognition and processing of the palm. When translating the images, only the first image I can be translated.ir Alternatively, you can simply translate the second image I. rgb It can also be used for the first image I ir Second image I rgb Translate them all so that the centers of the palms coincide.

[0065] It should be noted that the first image I used in this embodiment ir and the second image I rgb It has been registered. If the first image I ir and the second image I rgb If the data is not registered, it needs to be registered before this plan can be executed.

[0066] In some embodiments, the method further includes the following step before step S1:

[0067] Step S0: Correct the distortion of the original first image and the original second image respectively, and then perform epipolar correction to obtain the corrected first image I. ir and the corrected second image I rgb .

[0068] In this step, the original first image and the original second image are non-homogeneous images, meaning they were obtained using different techniques. Distortion correction is performed on the original first image, followed by epipolar correction, to obtain the corrected first image I. ir The original second image is distorted and then epipolarized to obtain the corrected second image I. rgbSince distortion is caused by the lens imaging principle, distortion correction for the original first and second images needs to be performed according to the parameters of each acquisition device. Epipolar correction is a correction for binocular systems. It involves rotating the two cameras and redefining a new image plane so that the epipolar pairs are collinear and parallel to a coordinate axis (usually the horizontal axis) of the image plane. This operation simultaneously establishes a new stereo image pair. After correction, the same matching point pair is located in the same row in both views, meaning they only differ in horizontal coordinates (or column coordinates), a difference called parallax. However, since the images used are the first and second images, the content they capture differs, making it impossible to directly solve for parallax using current technology. When the first camera captures an image of the palm, the veins absorb infrared light, resulting in darker areas around the veins. The second image, capturing the palm simultaneously, primarily images the surface texture of the palm, making direct matching of the palm prints difficult. Furthermore, the palms of people with different body types and builds vary significantly, further widening the difference between the infrared image and the second image, making effective matching even more challenging. This step corrects both images to make the data more accurate, thereby improving the accuracy of subsequent matching. It should be noted that the original first and second images used in this embodiment are typically acquired through a calibrated binocular system, where one camera is a first camera and the other a second camera. The first camera is used to acquire the first image, and the second camera is used to acquire the second image; both cameras acquire the target image simultaneously. For example, the first camera may be a near-infrared camera, resulting in a near-infrared image, while the second camera may be a color camera, resulting in a color image.

[0069] This step allows for better processing of images with significant distortion, enabling accurate results even when the palm is close to the imaging device (i.e., when the field of view is large), thus improving the effective recognition range for the palm.

[0070] Figure 5 A flowchart outlining the steps involved in obtaining key points. (For example...) Figure 5 As shown, compared to the previous embodiment, the method for obtaining key points in this embodiment includes the following steps:

[0071] Step S21: Obtain the palm edge according to the edge extraction algorithm, and then obtain the palm pose.

[0072] In this step, the edges of the palm are extracted using an edge extraction algorithm. Various edge extraction algorithms can be employed, such as those based on designed edge extraction operators (convolutional templates). These edge extraction operators include, but are not limited to, Sobel, Prewitt, Robert, and LoG. Edge extraction algorithms can also be obtained using adaptive algorithms or machine learning-trained models; this embodiment does not impose any limitations on these methods. Figure 6 As shown, the identified palm edge has a certain width, and because the grasping object is the same palm, the edge also has very good consistency compared to the inside of the palm, which can achieve better processing results.

[0073] Step S22: Obtain key points based on the hand posture.

[0074] In this step, since the keypoints are located at specific positions on the palm, their locations can be determined through the shape and posture of the palm. For example, the keypoint locations can be determined according to different proportions based on finger length. The proportions of human finger joints are usually fixed. Based on the proportional relationship between the fingers and the palm, the location of the base of the fingers can be determined. Then, based on the shape of the base of the palm, the keypoints at the base of the palm can be determined, thus determining the locations of all keypoints.

[0075] For example, a hand pose detection model can be used to detect the hand pose in a first image and a second image, obtaining key points of the hand in each image. A first recognition model is then trained using a training set originating from the first image, which is used to recognize the key points in the first image. Similarly, a second recognition model is trained using a training set originating from the second image, which is used to recognize the key points in the second image.

[0076] For example, a hand pose detection model can be used to detect the hand pose in the first image, obtaining key points in the first image. Then, based on the matching relationship between the first and second images, key points in the second image can be obtained. This method significantly reduces the time and resource consumption for training the model, improving efficiency, and can utilize existing models in the prior art, reducing the application cost of this embodiment.

[0077] For example, the first image I ir Subtract the second image I rgb Obtain the third image I0, and then obtain the key points based on the third image I0. Since the first image I... ir Second image I rgb Since they are not homologous, subtracting the two images will result in a third image I0 after removing some background information, which makes the features of the palm region more obvious, so that other models can be used for processing to improve the processing effect.

[0078] Step S23: Transfer the first image I ir and the second image I rgb Match the key points mentioned above.

[0079] In this step, key points at different locations are uniquely labeled, and key points with the same label are matched.

[0080] This embodiment determines the position of key points on the palm based on the edge of the palm, and then applies this information to the first image I. ir Second image I rgb By matching key points in the data, key information can be quickly obtained, enabling a rapid response.

[0081] Figure 7 This is a flowchart illustrating the steps for calculating the parallax of the palm center in this embodiment. Figure 7 As shown, compared to the previous embodiment, the method for calculating the parallax of the center of the palm in this embodiment includes the following steps:

[0082] Step S31: Calculate the disparity d of the key points.

[0083] In this step, since the number and location of keypoints are the same in both the first and second images, the disparity between the two images can be calculated based on the location of the corresponding palm keypoints. This step does not require calculating the disparity of all keypoints; it only needs to calculate the disparity of the six keypoints located on the palm, namely the keypoints at the base of the fingers and the base of the palm.

[0084] Step S32: Calculate the position of the center of the palm based on the key points at the base of the fingers and the base of the palm.

[0085] In this step, the position of the palm center is calculated based on the key points at the base of the fingers and the base of the palm. The palm center is the point where the difference in distance to the six key points on the palm is the smallest. Since the positions of these six key points are relatively fixed, the position of the palm center is also relatively fixed. Therefore, when calculating the position of the palm center, it can be performed within a pre-defined area to ultimately determine the position of the palm center.

[0086] In some embodiments, calculating the position of the center of the palm includes the following steps:

[0087] Step S321: Determine the range of the center of the palm based on the key points at the base of the fingers and the base of the palm.

[0088] Step S322: Randomly select three points on the edge of the range, and the area of ​​the triangle formed by the three points is not less than half of the area of ​​the range.

[0089] Step S323: Calculate the average difference between the distances of the three points from the key points at the finger root and the palm root respectively, and record the minimum average difference f.

[0090] Step S324: Move the three points towards the center point of the triangle, recalculate the average difference between the distances of the three points from the key points at the finger root and the palm root, and record the current minimum average difference g.

[0091] Step S325: If g < f, assign g to f, and continue to move the point corresponding to g in the original direction, and move the other two points along the direction of the line connecting with the point corresponding to g.

[0092] Step S326: If g >= f, move the three points towards the direction of the point corresponding to f.

[0093] Repeat Step S325 and Step S326 until the points with the smallest difference in distances from the 6 key points on the palm are converged, which is the palm center.

[0094] Step S33: Calculate the parallax b of the palm center based on the parallax of the key points at the finger root and the palm root.

[0095] In this step, different weights are assigned according to the distances between the key points at the finger root and the palm root and the palm center, and thus the parallax b of the palm center is calculated by weighted calculation. The closer the distance between the key points at the finger root and the palm root and the palm center is, the greater the weight. The sum of the weight values of the 6 key points is 1.

[0096] An embodiment of the present invention also provides a non - homologous binocular palm - swiping recognition two - image alignment device, including a processor and a memory, in which the executable instructions of the processor are stored. The processor is configured to execute the steps of the non - homologous binocular palm - swiping recognition two - image alignment method by executing the executable instructions.

[0097] As above, in this embodiment, the first image and the second image are obtained by using the depth camera of the binocular system composed of the first camera and the second camera, and the two different types of images are aligned by the method in the foregoing embodiment, overcoming the differences between different types of images and achieving the purpose of stable and fast alignment.

[0098] Those skilled in the art can understand that various aspects of the present invention can be implemented as a system, a method or a program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module" or "platform" here.

[0099] Figure 8 This is a schematic diagram of a two-image alignment device based on non-homogeneous binocular palm recognition according to an embodiment of the present invention. See below for reference. Figure 8 To describe an electronic device 600 according to this embodiment of the present invention. Figure 8 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0100] like Figure 8 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0101] The storage unit stores program code, which can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the section on the method for aligning two images based on non-homogeneous binocular palm recognition according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform, as follows: Figure 1 The steps are shown in the figure.

[0102] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0103] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0104] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0105] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although... Figure 8 As not shown in the diagram, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0106] This invention also provides a computer-readable storage medium for storing a program that, when executed, implements the steps of a two-image alignment method based on non-homogeneous binocular palm recognition. In some possible implementations, various aspects of this invention can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the above-described section of this specification regarding the two-image alignment method based on non-homogeneous binocular palm recognition, according to various exemplary embodiments of the invention.

[0107] As shown above, when the program of the computer-readable storage medium of this embodiment is executed, it acquires a first image and a second image by using the depth camera of a binocular system composed of a first camera and a second camera, and aligns the two different types of images by using the method in the foregoing embodiment, thereby overcoming the differences between the different types of images and achieving the purpose of stable and fast alignment.

[0108] Figure 9 This is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. (Reference) Figure 9 As shown, a program product 800 for implementing the above-described method according to an embodiment of the present invention is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0109] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0110] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0111] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0112] In this embodiment of the invention, a first image and a second image are acquired using a depth camera of a binocular system consisting of a first camera and a second camera. The two different types of images are aligned using the method described in the foregoing embodiment, overcoming the differences between the different types of images and achieving stable and fast alignment.

[0113] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0114] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for aligning two images based on non-homogeneous binocular palm print recognition, characterized in that, Includes the following steps: Step S1: For the first image Second image The first image is obtained by detecting the palm separately. palm area and the second image palm area Wherein, the first image and the second image are non-homogeneous images; Step S2: For the palm area and the palm area The hand posture was estimated, and the key points of the hand were obtained respectively; Step S3: Calculate the parallax of the palm center based on the information of the key points; Step S4: Based on the parallax of the center of the palm, the first image is... and the second image Alignment; Step S3 includes: Step S31: Calculate the disparity d of the key points; Step S32: Calculate the position of the center of the palm based on the key points at the base of the fingers and the base of the palm; Step S33: Calculate the parallax b of the center of the palm based on the parallax of key points at the base of the fingers and the base of the palm.

2. The method for aligning two images based on non-homogeneous binocular palm print recognition according to claim 1, characterized in that, The procedure before step S1 also includes: Step S0: Correct the distortion of the original first image and the original second image respectively, and then perform epipolar correction to obtain the corrected first image. and the corrected second image .

3. The method for aligning two images based on non-homogeneous binocular palm print recognition according to claim 1, characterized in that, Step S2 includes: Step S21: Obtain the palm edge using the edge extraction algorithm, and then obtain the palm pose; Step S22: Obtain key points based on the hand posture; Step S23: Transfer the first image and the second image Match the key points mentioned above.

4. The method for aligning two images based on non-homogeneous binocular palm print recognition according to claim 1, characterized in that, In step S1, when processing the first image and the second image Before the detection, the first image was also... and the second image Compress it.

5. The method for aligning two images based on non-homogeneous binocular palm print recognition according to claim 4, characterized in that, For the first image Second image The compression ratios are different.

6. The method for aligning two images based on non-homogeneous binocular palm print recognition according to claim 3, characterized in that, In step S22, a first deep learning model is used to process the first image. The first key point is obtained by identifying the edge of the palm in the second image; a second deep learning model is then used to process the second image. The second key point is obtained by recognizing the edge of the palm in the image, wherein the first deep learning model is based on the first image. The second deep learning model is obtained by training on homologous palm images, and is based on images related to the second image. It was obtained by training with images of the same hand.

7. The method for aligning two images based on non-homogeneous binocular palm print recognition according to claim 3, characterized in that, In step S22, the first image Subtract the second image Obtain the third image Then, based on the third image Obtain the key points.

8. A device for aligning two images based on non-homogeneous binocular palm recognition, characterized in that, include: processor; A memory module that stores executable instructions of the processor; The processor is configured to perform the steps of the two-image alignment method based on non-homogeneous binocular palm recognition as described in any one of claims 1 to 7 by executing the executable instructions.

9. A computer-readable storage medium for storing a program, characterized in that, When the program is executed, it implements the steps of the two-image alignment method based on non-homogeneous binocular palm recognition as described in any one of claims 1 to 7.

Images