Non-homologous binocular camera with alignment function

Abstract

The application discloses a non-homologous binocular camera with alignment function, which is characterized by comprising a first camera for shooting an original first image of a palm, a second camera for shooting an original second image of the palm, wherein the first camera and the second camera are non-homologous cameras and are simultaneously shot, and a processor for processing the original first image and the second image, acquiring palm key points, and then obtaining palm center information and performing alignment. The application directly processes the first image and the second image by using a non-homologous binocular system, saves a p-sensor, aligns the first image and the second image by using the palm key points, greatly reduces the calculation amount of palm brushing recognition, reduces the cost, and is favorable for popularization of palm brushing application.

Description

Technical Field

[0001] This invention relates to the field of palm recognition cameras, and more specifically, to a non-homogeneous binocular camera with alignment functionality. Background Technology

[0002] Because the lines on a human palm are more stable, palm recognition is a more stable and secure biometric technology than facial recognition. Therefore, it can be used to identify individuals by scanning their palms, and can be applied to security checks, payments, and identity verification. Palm recognition is a technology with broad application prospects.

[0003] In existing technologies, some palm-scanning systems use a palm rest to fix the palm and determine its position, but this is inconvenient in practical applications. Some systems use p-sensors to estimate palm depth and then align the first and second images pixel-wise based on the p-sensor. In practice, due to considerations of measurement range and distance, it is generally necessary to capture the entire palm at close range and to have a clear image of the palm at a distance. This requires a sufficiently large field of view (FOV) for the camera. With a large FOV, the palm will not occupy a very large proportion of the image at a distance. If p-sensors are still used for image alignment, a large number of p-sensors would be required, leading to mutual interference, design incompatibility issues, and significantly increased costs.

[0004] Existing binocular systems all use the same source binoculars, which can obtain clear depth data of the target object. However, the computational load of this binocular system is large, and a separate on-chip system is required to process the relevant data, which leads to a series of problems such as transmission delay and recognition lag. Summary of the Invention

[0005] Therefore, this invention utilizes a non-homogeneous binocular system composed of a first camera and a second camera to directly process the first and second images, saving p-sensors. It also uses key points of the palm to align the first and second images, greatly reducing the computational load of palm recognition, requiring fewer devices, saving more device space, and having better compatibility and lower costs, which is conducive to the promotion of palm recognition applications.

[0006] In a first aspect, the present invention provides a non-homogeneous binocular camera with alignment function, characterized in that it comprises:

[0007] The first camera was used to capture the original first image of the palm;

[0008] A second camera is used to capture a raw second image of the palm; wherein the first camera and the second camera are non-co-located cameras and capture images simultaneously;

[0009] The processor is used to process the original first image and the second image to obtain key points of the palm, thereby obtaining the center information of the palm, and then aligning them.

[0010] Optionally, the non-homogeneous binocular camera with alignment function is characterized in that the processor includes:

[0011] The detection module is used to detect the first image I. ir Second image I rgb The palms are detected separately, and the first image I is obtained separately. ir palm area ROI ir and the second image I rgb palm area ROI rgb Wherein, the first image and the second image are non-homogeneous images;

[0012] Key point module, used for the ROI of the palm region. ir and the palm region ROI rgb The hand posture was estimated, and the key points of the hand were obtained respectively;

[0013] A center parallax module is used to calculate the parallax of the center of the palm based on the information of the key points;

[0014] Alignment module, used to align the first image I according to the parallax at the center of the palm. ir and the second image I rgb Alignment.

[0015] Optionally, the aforementioned non-homogeneous binocular camera with alignment function is characterized by further comprising:

[0016] The correction module is used to correct the distortion of the original first image and the original second image respectively, and then perform epipolar correction to obtain the corrected first image I. ir and the corrected second image I rgb .

[0017] Optionally, the non-homogeneous binocular camera with alignment function is characterized in that the key point module includes:

[0018] The extraction unit is used to obtain the palm edge according to the edge extraction algorithm, and then obtain the palm pose;

[0019] Key point unit, used to obtain key points based on the palm posture;

[0020] Matching unit, used to match the first image I ir and the second image I rgb Match the key points mentioned above.

[0021] Optionally, the non-homogeneous binocular camera with alignment function is characterized in that the central parallax module includes:

[0022] A key disparity unit is used to calculate the disparity d of the key point;

[0023] The center position unit is used to calculate the position of the center of the palm based on key points at the base of the fingers and the base of the palm;

[0024] The central parallax unit is used to calculate the parallax b of the center of the palm based on the parallax of key points at the base of the fingers and the base of the palm.

[0025] Optionally, the non-homogeneous binocular camera with alignment function is characterized in that, in the detection module, when processing the first image I... ir and the second image I rgb Before detection, the first image I was also... ir and the second image I rgb Compress it.

[0026] Optionally, the non-homogeneous binocular camera with alignment function is characterized in that, for the first image I ir Second image I rgb The compression ratios are different.

[0027] Optionally, the non-homogeneous binocular camera with alignment function is characterized in that, in the key point unit, a first deep learning model is used to process the first image I. ir The first key point is obtained by identifying the edge of the palm in the image; a second deep learning model is used to process the second image I. rgb The second key point is obtained by recognizing the edge of the palm in the image, wherein the first deep learning model is based on the first image I. ir The second deep learning model is obtained by training on homologous palm images, and is based on images I. rgb It was obtained by training with images of the same hand.

[0028] Optionally, the non-homogeneous binocular camera with alignment function is characterized in that the center position unit includes:

[0029] The range sub-unit is used to determine the range of the center of the palm based on the key point positions of the base of the fingers and the base of the palm.

[0030] Select a sub-unit to randomly select three points on the edge of the range, wherein the area of ​​the triangle formed by the three points is not less than half the area of ​​the range.

[0031] A difference sub-unit for separately calculating the average difference between the distances of the three points from the key points at the finger root and the palm root, and recording the minimum average difference f.

[0032] An update sub-unit for moving the three points towards the center point of the triangle, recalculating the average difference between the distances of the three points from the key points at the finger root and the palm root, and recording the current minimum average difference g.

[0033] A first movement sub-unit for, if g < f, assigning g to f and continuing to move the point corresponding to g in the original direction, and moving the other two points in the direction of the line connecting them to the point corresponding to g.

[0034] A second movement sub-unit for, if g >= f, moving the three points in the direction of the point corresponding to f.

[0035] Optionally, in the key point unit of the non-homologous binocular camera with an alignment function, subtracting the second image I from the first image I to obtain a third image I0, and then obtaining key points based on the third image I0. ir Subtracting the second image I from the first image I rgb to obtain a third image I0, and then obtaining key points based on the third image I0.

[0036] In a second aspect, the present invention provides a palm brushing device, which comprises a non-homologous binocular camera with an alignment function as described in any one of the above.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention uses the first image and the second image as raw data, and does not require devices such as p-sensors, reducing the input conditions for palm brushing recognition, thereby simplifying the corresponding hardware devices, making them smaller in size and easier to integrate, which is beneficial to the miniaturization of the device.

[0039] The images used in the present invention can be shared with other palm recognition functions, enabling one image to be used for multiple functions, thus maximizing the function of one image and saving device space. For example, when the first image is an infrared image, it can not only be used for alignment and reconstruction, but also for live detection; when the second image is a color image, it can not only be used for alignment and reconstruction, but also for palmprint recognition. By aligning two different types of images in the present invention, steps such as palm detection, palm segmentation, edge extraction, edge matching, and depth reconstruction are made more convenient, saving subsequent image processing steps and improving processing efficiency.

[0040] This invention uses a first camera and a second camera to image a palm, achieving pixel-level alignment between the two images when a depth map is unavailable. The invention proposes using a first and second camera to form a binocular system, reconstructing the three-dimensional shape of the palm's edges based on binocular imaging theory, and then achieving pixel-level alignment between the first and second images based on the depth information of the palm's edges.

[0041] This invention constructs a binocular system using a first and second camera that are not from the same source. However, due to their non-homogeneous nature, existing technologies cannot effectively solve problems such as matching. This invention achieves matching and 3D reconstruction for non-homogeneous binocular systems. For example, when the first camera is a near-infrared camera and the second camera is a color camera, when capturing an image of a hand, the veins will appear darker because veins absorb infrared light. Furthermore, when the second camera simultaneously captures an image of the hand, it primarily images the surface texture of the hand, leading to greater differences between the first and second images, making effective processing impossible using existing technologies.

[0042] This invention eliminates the need to process the entire palm image; it only processes key points of the palm within the image. This significantly reduces the amount of data processed, thereby lowering the requirements for chips and other components. Furthermore, it eliminates the need for a separate on-chip system and can be integrated into the camera's built-in chip, simplifying the camera's structure, reducing costs, and enabling low-cost palm-swiping applications, which is beneficial for commercial promotion.

[0043] This invention calculates the disparity *b* at the center of the palm using key point information of the palm. This significantly reduces the amount of data processed, simplifies and simplifies the calculation, and allows for rapid results, enabling quick alignment and improving image processing speed. Compared to other processing methods, this invention improves the palm-swipe response speed, making it more suitable for applications requiring rapid response. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0045] Figure 1 This is a schematic diagram of the structure of a non-homogeneous binocular camera with alignment function in an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of a processor structure in an embodiment of the present invention;

[0047] Figure 3 This is a hand detection image from an embodiment of the present invention;

[0048] Figure 4 The spatial positions of key points on the palm in this embodiment of the invention;

[0049] Figure 5 This is a set of palm-aligned images in an embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram of the structure of a key point module in an embodiment of the present invention;

[0051] Figure 7 This is a set of palm edge images in an embodiment of the present invention;

[0052] Figure 8 This is a schematic diagram of the structure of a central parallax module in this embodiment;

[0053] Figure 9 This is a schematic diagram of the structure of a palm-brushing device according to an embodiment of the present invention. Detailed Implementation

[0054] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0055] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0056] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0057] The present invention provides a non-homogeneous binocular camera with alignment function, which aims to solve the problems existing in the prior art.

[0058] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0059] Figure 1 This is a schematic diagram of the structure of a non-homogeneous binocular camera with alignment function according to an embodiment of the present invention. Figure 1 As shown, a non-homogeneous binocular camera with alignment function includes:

[0060] The first camera was used to capture the original first image of the palm;

[0061] A second camera is used to capture a raw second image of the palm; wherein the first camera and the second camera are non-co-located cameras and capture images simultaneously;

[0062] The processor is used to process the original first image and the second image to obtain key points of the palm, thereby obtaining the center information of the palm, and then aligning them.

[0063] Specifically, during filming, the first and second cameras simultaneously acquire information about the target object. Since the target object acquired by the first and second cameras is the same object at the same time, a binocular system can be formed using the first and second cameras. The processor processes the first and second images, identifies key points of the palm, and then obtains the parallax at the center of the palm, thereby achieving alignment between the first and second images.

[0064] Figure 2 This is a schematic diagram of a processor structure according to an embodiment of the present invention. Figure 2 As shown, unlike the previous embodiments, the processor in this embodiment of the invention includes:

[0065] Detection module 100 is used to detect the first image I ir Second image I rgb The palms are detected separately, and the first image I is obtained separately. ir palm area ROI ir and the second image I rgb palm area ROI rgb .

[0066] Specifically, the first image and the second image are non-originating images. A handprint detection model is used for the first image I. ir and the second image I rgbEach image is detected separately to determine if a hand is present. If a hand is present, the first image I is obtained. ir palm area ROI ir and the second image I rgb palm area ROI rgb .like Figure 3 As shown, the ROI of the palm region ir and palm area ROI rgb All rectangles are the smallest rectangles that include the palm, meaning that all four sides of the rectangle are tangent to the edge of the palm. The palm detection model used in this module can be any model that can achieve palm detection functionality; this embodiment does not impose any restrictions on it.

[0067] In some embodiments, this method is adapted for continuous shooting. The hand region in each frame is obtained by subtracting the previous frame from the subsequent frame. Since the hand is not fixed in place by a palm rest, it is difficult to keep the hand in the same position. Therefore, subtracting two frames taken at different times can quickly locate the edge of the hand, thereby obtaining the hand contour information and locating the hand region. At the same time, the image subtraction method is simple and fast.

[0068] In some embodiments, this module processes the first image I before processing the image. ir Second image I rgb Compress the first image I. ir Second image I rgb The compression ratios are different. For example, before processing, the first image I... ir and the second image I rgb Compression is performed separately at ratios of 4 and 2, respectively, to make the image sizes more similar and ensure processing accuracy. Preferably, the closer the palm is to the camera, the higher the compression ratio; the farther the palm is from the camera, the lower the compression ratio. When the palm occupies 70% of the image area, the compression ratio of the first image is no less than 5; when the palm occupies 50% of the image area, the compression ratio of the first image is no less than 2. The compression ratio of the first image is more than three times that of the second image.

[0069] The key point module 200 is used to estimate the hand pose of the hand region in the first image and the hand region in the second image, and obtain the key points of the hand respectively.

[0070] Specifically, only the palm area is identified, thus reducing the data range and volume, thereby improving response speed and lowering hardware requirements. Since the first and second images are not from the same source, they are difficult to process using the same model. Figure 4The spatial locations of key points on the palm are shown. There are 21 key points in total, including four at each finger joint and four at each end, plus one at the base of the palm.

[0071] In some embodiments, a hand pose detection model is used to estimate the hand pose and obtain key points of the hand in each image. A first recognition model is trained using a training set originating from the first image, and then used to recognize the key points in the first image. Similarly, a second recognition model is trained using a training set originating from the second image, and then used to recognize the key points in the second image.

[0072] In some embodiments, a hand pose detection model is used to estimate the hand pose of the first image to obtain key points in the first image. Then, based on the matching relationship between the first image and the second image, the key points in the second image are obtained. This approach significantly reduces the time and resource consumption for training the model, improves efficiency, and can utilize existing models in the prior art, thus reducing the application cost of this embodiment.

[0073] The center parallax module 300 is used to calculate the parallax of the center of the palm based on the information of the key points.

[0074] Specifically, the keypoint information includes location, parallax, etc. Since the keypoints on the palm are located in various parts of the palm, and the center of the palm is always located between all the keypoints, only six keypoints on the palm are needed to calculate the parallax of the palm center. The palm center is the point with the smallest difference in distance to the six keypoints. For example, the parallax of the palm center is obtained by calculating the average of the parallaxes of these six keypoints.

[0075] Alignment module 400 is used to align the first image I according to the parallax of the center of the palm. ir and the second image I rgb Alignment.

[0076] Specifically, the image is translated using the polar plane in the binocular system, specifically using the polar plane located at the center of the palm, so that the first image I... ir and the second image I rgb The centers of the palms overlap to complete the alignment. For example... Figure 5 As shown, the aligned images have better consistency, thus enabling better recognition and processing of the palm. When translating the images, only the first image I can be translated. ir Alternatively, you can simply translate the second image I. rgb It can also be used for the first image I ir Second image I rgbTranslate them all so that the centers of the palms coincide.

[0077] It should be noted that the first image I used in this embodiment ir and the second image I rgb It has been registered. If the first image I ir and the second image I rgb If the data is not registered, it needs to be registered before this plan can be executed.

[0078] In some embodiments, it also includes:

[0079] The correction module 500 is used to correct the distortion of the original first image and the original second image respectively, and then perform epipolar correction to obtain the corrected first image I. ir and the corrected second image I rgb .

[0080] Specifically, the original first image and the original second image are non-originating images, meaning they were obtained using different techniques. Distortion correction is performed on the original first image, followed by epipolar correction, to obtain the corrected first image I. ir The original second image is distorted and then epipolarized to obtain the corrected second image I. rgbSince distortion is caused by the lens imaging principle, distortion correction for the original first and second images needs to be performed according to the parameters of each acquisition device. Epipolar correction is a correction for binocular systems. It involves rotating the two cameras and redefining a new image plane so that the epipolar pairs are collinear and parallel to a coordinate axis (usually the horizontal axis) of the image plane. This operation simultaneously establishes a new stereo image pair. After correction, the same matching point pair is located in the same row in both views, meaning they only differ in horizontal coordinates (or column coordinates), a difference called parallax. However, since the images used are the first and second images, the content they capture differs, making it impossible to directly solve for parallax using current technology. When the first camera captures an image of the palm, the veins absorb infrared light, resulting in darker areas around the veins. The second image, capturing the palm simultaneously, primarily images the surface texture of the palm, making direct matching of the palm prints difficult. Furthermore, the palms of people with different body types and builds vary significantly, further widening the difference between the infrared image and the second image, making effective matching even more challenging. This module corrects the two types of images to make the data more accurate, thereby improving the accuracy of subsequent matching. It should be noted that the original first image and original second image used in this embodiment are typically acquired through a calibrated binocular system, where one camera is a first camera and the other is a second camera. The first camera is used to acquire the first image, and the second camera is used to acquire the second image; both cameras acquire the target image simultaneously. For example, the first camera may be a near-infrared camera, resulting in a near-infrared image, while the second camera may be a color camera, resulting in a color image.

[0081] The correction module 500 enables images with significant distortion to be processed well, thus allowing for accurate results even when the palm is close to the shooting device, i.e., when the field of view (FOV) is large, thereby improving the effective recognition distance range for the palm.

[0082] Figure 6 This is a schematic diagram of the structure of a key point module in an embodiment of the present invention. Figure 6 As shown, compared to the previous embodiment, the key point module 200 in this embodiment includes:

[0083] Extraction unit 210 is used to obtain the palm edge according to the edge extraction algorithm, and then obtain the palm pose.

[0084] Specifically, the edges of the palm are extracted using an edge extraction algorithm. This algorithm can be implemented using various methods, such as those based on designed edge extraction operators (convolutional templates). These operators include, but are not limited to, Sobel, Prewitt, Robert, and LoG. Edge extraction algorithms can also be obtained using adaptive algorithms or machine learning-trained models; this embodiment does not impose any limitations on these methods. Figure 7 As shown, the identified palm edge has a certain width, and because the grasping object is the same palm, the edge also has very good consistency compared to the inside of the palm, which can achieve better processing results.

[0085] Key point unit 220 is used to obtain key points based on the palm posture.

[0086] Specifically, since key points are located at specific positions on the palm, their locations can be determined through the shape and posture of the palm. For example, key point locations can be determined according to different proportions based on finger length. The proportions of human finger joints are usually fixed. Based on the proportional relationship between the fingers and the palm, the location of the base of the fingers can be determined, and then based on the shape of the base of the palm, the key points at the base of the palm can be determined, thus determining the locations of all key points.

[0087] For example, a hand pose detection model can be used to detect the hand pose in a first image and a second image, obtaining key points of the hand in each image. A first recognition model is then trained using a training set originating from the first image, which is used to recognize the key points in the first image. Similarly, a second recognition model is trained using a training set originating from the second image, which is used to recognize the key points in the second image.

[0088] For example, a hand pose detection model can be used to detect the hand pose in the first image, obtaining key points in the first image. Then, based on the matching relationship between the first and second images, key points in the second image can be obtained. This method significantly reduces the time and resource consumption for training the model, improving efficiency, and can utilize existing models in the prior art, reducing the application cost of this embodiment.

[0089] For example, the first image I ir Subtract the second image I rgb Obtain the third image I0, and then obtain the key points based on the third image I0. Since the first image I... ir Second image I rgb Since they are not homologous, subtracting the two images will result in a third image I0 after removing some background information, which makes the features of the palm region more obvious, so that other models can be used for processing to improve the processing effect.

[0090] Matching unit 230, used to match the first image I ir and the second image I rgb Match the key points mentioned above.

[0091] Specifically, key points at different locations are uniquely labeled, and key points with the same label are matched.

[0092] This embodiment determines the position of key points on the palm based on the edge of the palm, and then applies this information to the first image I. ir Second image I rgb By matching key points in the data, key information can be quickly obtained, enabling a rapid response.

[0093] Figure 8 This is a schematic diagram of the structure of a central parallax module in this embodiment. Figure 8 As shown, compared to the previous embodiment, the center parallax module 300 in this embodiment includes:

[0094] The key disparity unit 310 is used to calculate the disparity d of the key point.

[0095] Specifically, since the number and position of key points are the same in both the first and second images, the disparity between the two images can be calculated based on the position of the corresponding palm key points. The key disparity unit 310 does not need to calculate the disparity of all key points, but only needs to calculate the disparity of the six key points located on the palm, namely the key points at the base of the fingers and the base of the palm.

[0096] The center position unit 320 is used to calculate the position of the center of the palm based on key points at the base of the fingers and the base of the palm.

[0097] Specifically, the position of the palm center is calculated based on the locations of key points at the base of the fingers and the base of the palm. The palm center is the point where the difference in distance to the six key points on the palm is minimal. Since the positions of these six key points are relatively fixed, the position of the palm center is also relatively fixed. Therefore, the calculation of the palm center can be performed within a pre-defined area to ultimately determine its position.

[0098] In some embodiments, the center position unit 320 includes:

[0099] Range subunit 321 is used to determine the range of the center of the palm based on the key point positions of the base of the fingers and the base of the palm.

[0100] Select subunit 322 to randomly select three points on the edge of the range, and the area of ​​the triangle formed by the three points is not less than half of the area of ​​the range.

[0101] A difference sub - unit 323 is configured to calculate the average difference between the distances of the three points from the key points at the finger root and the palm root respectively, and record the minimum average difference f.

[0102] An update sub - unit 324 is configured to move the three points towards the center point of the triangle, recalculate the average difference between the distances of the three points from the key points at the finger root and the palm root, and record the current minimum average difference g.

[0103] A first movement sub - unit 325 is configured to, if g < f, assign g to f, continue to move the point corresponding to g in the original direction, and move the other two points along the direction of the line connecting to the point corresponding to g.

[0104] A second movement sub - unit 326 is configured to, if g >= f, move the three points towards the direction of the point corresponding to f.

[0105] Repeat running the first movement sub - unit 325 and the second movement sub - unit 326 until the point with the smallest difference in distances from the 6 key points on the palm is converged, which is the palm center.

[0106] A central parallax unit 330 is configured to calculate the parallax b of the palm center according to the parallax of the key points at the finger root and the palm root.

[0107] Specifically, different weights are assigned according to the distances between the key points at the finger root and the palm root and the palm center, so as to calculate the parallax b of the palm center by weighted calculation. The closer the distance between the key points at the finger root and the palm root and the palm center is, the greater the weight. The sum of the weight values of the 6 key points is 1.

[0108] Figure 9 It is a schematic structural diagram of a palm - brushing device in an embodiment of the present invention. The following refers to Figure 9 to describe the palm - brushing device 700 according to this embodiment of the present invention. Figure 9 The shown palm - brushing device 700 is only an example, and should not bring any limitation to the functions and usage scope of the embodiments of the present invention.

[0109] As Figure 9 shown, the palm - brushing device 700 includes a palm - brushing area 710, a non - homologous binocular camera 720, a display screen 730, and a key area 740.

[0110] The palm-brush area 710 is the area for placing the palm. The palm-brush area 710 is typically a light-transmitting area, allowing light from the non-isomorphic stereo camera 720 to pass through and thus obtain palm information. The palm-brush area can be on the same plane as other areas, or it can be at an angle to other areas. The interior of the palm-brush area 710 is usually a glass or plastic plate, which, while allowing light to pass through, also serves to protect the non-isomorphic stereo camera 720.

[0111] The non-homogeneous binocular camera 720 is any one of those described in the foregoing embodiments, used to acquire palm information and align the image. The non-homogeneous binocular camera 720 is located at the bottom of the palm brushing area 710, and the light from the non-homogeneous binocular camera 720 shines onto the palm through the palm brushing area 720, thereby receiving the reflected signal from the palm and obtaining palm information.

[0112] Display screen 730 is typically an LCD screen used to display the recognition results from button area 740 and the non-homogeneous binocular camera 720. Display screen 730 is the core display content of the palm-swiping device 700. Display screen 730 can be any of three types: static, simple matrix, or active matrix. Due to its small size, display screen 730 is usually a single, complete display.

[0113] The button area 740 contains multiple physical buttons for inputting information into the palm-swiping device 700, such as the amount to be paid. The button area 740 includes the numbers "0" through "9," as well as a ".", "clear," and "confirm" button. The buttons can be either contact-type switches, such as mechanical switches or conductive rubber switches, or contactless switches, such as electrical switches or magnetic induction switches. The former is cheaper, while the latter has a longer lifespan.

[0114] In this embodiment of the invention, a first image and a second image are acquired by a depth camera of a binocular system consisting of a first camera and a second camera. The two different types of images are aligned using the scheme described in the foregoing embodiment, overcoming the differences between the different types of images and achieving the purpose of stable and fast alignment.

[0115] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0116] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A non-homogeneous binocular camera with alignment function, characterized in that, Comprising: A first camera for capturing an original first image of a palm; A second camera for capturing an original second image of the palm; wherein, the first camera and the second camera are non-homologous cameras and capture simultaneously; A processor for processing the original first image and the second image to obtain palm key points, further obtaining palm center information, and then performing alignment; The processor comprises: The detection module is used to detect the first image. Second image The first image is obtained by detecting the palm of the hand separately. palm area and the second image palm area Wherein, the first image and the second image are non-homogeneous images; Key point module, used for the palm area and the palm area The hand posture was estimated, and the key points of the hand were obtained respectively; A central parallax module for calculating the parallax of the palm center according to the information of the key points; Alignment module, used to align the first image according to the parallax at the center of the palm. and the second image Alignment; The central parallax module comprises: A key parallax unit for calculating the parallax d of the key points; A central position unit for calculating the position of the palm center according to the key points of the finger roots and the palm root; A central parallax unit for calculating the parallax b of the palm center according to the parallax of the key points of the finger roots and the palm root.

2. A non-homogeneous binocular camera with alignment function according to claim 1, characterized in that, Further comprising: The correction module is used to correct the distortion of the original first image and the original second image respectively, and then perform epipolar correction to obtain the corrected first image. and the corrected second image .

3. A non-homogeneous binocular camera with alignment function according to claim 1, characterized in that, The key point module comprises: An extraction unit for obtaining the palm edge according to an edge extraction algorithm, and further obtaining the palm posture; A key point unit for obtaining key points according to the palm posture; Matching unit, used to match the first image and the second image Match the key points mentioned above.

4. A non-homogeneous binocular camera with alignment function according to claim 1, characterized in that, In the detection module, when processing the first image... and the second image Before the detection, the first image was also... and the second image Compress it.

5. A non-homogeneous binocular camera with alignment function according to claim 4, characterized in that, For the first image Second image The compression ratios are different.

6. A non-homogeneous binocular camera with alignment function according to claim 1, characterized in that, The central position unit comprises: A range sub-unit for determining the range of the palm center according to the positions of the key points of the finger roots and the palm root; A selection sub-unit for randomly selecting three points at the edge of the range, and the area of the triangle formed by the three points is not less than half of the area of the range; A difference sub-unit for respectively calculating the average difference between the distances of the three points from the key points of the finger roots and the palm root, and recording the minimum average difference f; An update sub-unit for moving the three points in the direction of the center point of the triangle, recalculating the average difference between the distances of the three points from the key points of the finger roots and the palm root, and recording the current minimum average difference g; A first movement sub-unit for if g < f, assigning g to f and continuing to move the point corresponding to g in the original direction, and moving the other two points in the direction of the line connecting with the point corresponding to g; A second movement sub-unit for if g >= f, moving the three points in the direction of the point corresponding to f.

7. A non-homogeneous binocular camera with alignment function according to claim 3, characterized in that, In the key point unit, the first image Subtract the second image Obtain the third image Then, based on the third image Obtain the key points.

8. A palm-swiping device, characterized in that, Comprising a non-homologous binocular camera with an alignment function according to any one of claims 1 to 7.

Images