Mobile terminal and identity recognition method
By acquiring and processing mixed images of palm prints and palm veins on a mobile terminal, and separating RGB and NIR information for feature recognition, the security and adaptability issues of single biometric recognition methods are solved, achieving highly secure and high-recognition-rate identity verification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-08
- Publication Date
- 2026-03-13
AI Technical Summary
Existing single biometric identification methods have limited security, poor adaptability, and low recognition rates on mobile terminals.
A camera module is used to simultaneously capture mixed images of palm prints and palm veins. The image processing module separates the RGB and NIR information, and the control module performs palm print and palm vein feature recognition respectively. Finally, the two are combined to determine the identity.
It achieves highly secure and accurate identity recognition, improving environmental adaptability and user experience.
Smart Images
Figure CN121661683A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of identity recognition technology, and more particularly to a mobile terminal and an identity recognition method. Background Technology
[0002] With the popularization of mobile internet and the continuous improvement of information security requirements, biometric identification technology has been widely used in the field of mobile terminal identity recognition due to its uniqueness, stability and convenience.
[0003] Common biometric identification methods include facial recognition, iris recognition, fingerprint recognition, palm print recognition, and palm vein recognition. However, these methods all employ a single identification mode, and a single biometric feature is easily copied or forged. For example, identity verification can be achieved by creating fingerprint films, using photographs, or masks, resulting in limited security. Furthermore, single identification modes have poor environmental adaptability; for example, wet or damaged fingers, facial injuries or makeup, and insufficient lighting conditions reduce the recognition rate.
[0004] Therefore, how to achieve highly secure and accurate identity verification on mobile terminals has become an urgent problem to be solved. Summary of the Invention
[0005] This application provides a mobile terminal and an identity recognition method to solve the problems of limited security, poor adaptability and low recognition rate of current single biometric recognition methods, thereby achieving high-security and high-recognition-rate identity recognition on mobile terminals.
[0006] In a first aspect, this application provides a mobile terminal, which includes: a camera module, an image processing module, and a control module; the image processing module is connected to both the camera module and the control module. The camera module is used to capture one or more frames of raw mixed images. The raw mixed images include the user's palm print information and palm vein information. The raw mixed images are composed of multiple mixed arrays, which are used to record red, green and blue (RGB) information and near-infrared (NIR) information. The image processing module is used to process each frame of the original mixed image to obtain RGB recognition images and NIR recognition images corresponding to one or more frames of the original mixed image, respectively. The control module is used to perform palmprint feature recognition based on RGB recognition images to obtain palmprint recognition results, and to perform palm vein feature recognition based on NIR recognition images to obtain palm vein recognition results. Based on the palmprint recognition results and palm vein recognition results, the user's identity is determined.
[0007] In one possible design, the camera module includes a white light, an infrared light, an optical lens, a filter, and an image sensor; the white light, infrared light, and optical lens are disposed on a first plane, the filter is disposed on a second plane, and the image sensor is disposed on a third plane. The first, second, and third planes are parallel to each other, with the first plane above the second plane and the third plane below the second plane. The centers of the optical lens, the filter, and the image sensor are located on the same straight line perpendicular to the first plane. An optical lens is used to receive a focused light signal from the user's palm when the user's palm is located in the recognition area of the optical lens; A white light is used to provide a first light source when the user's palm is located in the recognition area of the optical lens; an infrared light is used to provide a second light source when the user's palm is located in the recognition area of the optical lens. The filter is used to process the converged optical signal to obtain a mixed optical signal, which includes a first band signal and a second band signal; the first band signal is used to provide RGB information, and the second band signal is used to provide NIR information. An image sensor is used to convert mixed light signals to obtain one or more frames of original mixed image.
[0008] In one possible design, the filter is a bimodal transmission filter; the filter allows light signals from 380 nm to 780 nm and 850 nm to pass through, or the filter allows light signals from 380 nm to 780 nm and 940 nm to pass through.
[0009] In one possible design, the image processing module includes an image processing front-end and an image processing engine; the image processing front-end is connected to the camera module, the image processing front-end is connected to the image processing engine, and the image processing engine is connected to the control module. The image processing front end is used to split the blending array for each blending array in each frame of the original blending image to obtain the initial RGB array and the initial NIR array corresponding to the blending array; The image processing engine is used to compensate the initial RGB array to obtain a compensated RGB array, and to compensate the initial NIR array to obtain a compensated NIR array. The image processing engine is also used to perform YUV encoding on the compensated RGB array and compensated NIR array corresponding to each mixing array in each frame of the original mixed image, respectively, to obtain the RGB recognition image and the NIR recognition image corresponding to one or more frames of the original mixed image, respectively.
[0010] In one possible design, the control module includes a trusted execution environment and a rich execution environment. The trusted execution environment includes secure memory and a communication agent, and the rich execution environment includes a trusted application and a client application. The secure memory is connected to the image processing module, and the secure memory, the communication agent, the trusted application, and the client application are connected in sequence. The trusted application and the client application share memory. Secure memory is used for encrypting RGB and NIR recognition images; A communication agent is used to transmit encrypted RGB recognition images and encrypted NIR recognition images to the rich execution environment; The trusted application performs signature verification on encrypted RGB recognition images, extracts palmprint features from the verified RGB recognition images, compares these features with preset palmprint features to obtain palmprint recognition results, and performs signature verification on encrypted NIR recognition images. It then extracts palm vein features from the verified NIR recognition images, compares these features with preset palm vein features to obtain palm vein recognition results, and determines the user's identity based on both palmprint and palm vein recognition results. The preset palmprint and palm vein features are those entered during user registration. The client application is used to receive and output the results of user identity verification.
[0011] In one possible design, the image processing module is further configured to process each frame of the original mixed image to obtain an RGB preview image corresponding to one or more frames of the original mixed image and an NIR preview image corresponding to one or more frames of the original mixed image; the resolution of the RGB preview image and the NIR preview image is a first resolution, and the resolution of the RGB recognition image and the NIR recognition image is a second resolution, wherein the first resolution is smaller than the second resolution; The secure memory is also used for encrypting RGB and NIR preview images; A communication proxy is used to transmit encrypted RGB preview images and encrypted NIR preview images to the rich execution environment; The client application is also used to receive and output encrypted RGB preview images and encrypted NIR preview images.
[0012] In one possible design, the mobile terminal also includes: a touch screen; the touch screen is connected to the control module; The touchscreen is used to receive and display encrypted RGB preview images and encrypted NIR preview images to the user.
[0013] In one possible design, the control module is also connected to the white light and the infrared light respectively. The control module is also used to adjust the light intensity of the white light according to the RGB recognition image and to adjust the light intensity of the infrared light according to the NIR recognition image.
[0014] The mobile terminal provided in the first aspect includes a camera module, an image processing module, and a control module. The camera module acquires one or more frames of raw mixed images. These raw mixed images include the user's palmprint and palm vein information. The raw mixed images consist of multiple mixed arrays that record RGB (red, green, blue) information and NIR (near-infrared) information, thus enabling simultaneous acquisition of the user's palmprint and palm vein information for multimodal identity recognition. Furthermore, the user only needs to place their palm on the recognition area once for the camera module to acquire both palmprint and palm vein biometrics, simplifying the recognition process and improving the user experience while achieving multimodal identity recognition. The image processing module processes each frame of the raw mixed image to obtain RGB recognition images and NIR recognition images corresponding to each frame of the raw mixed image, resulting in independent RGB and NIR recognition images. This allows the control module to perform multimodal identity recognition based on the RGB and NIR recognition images respectively. The control module performs palmprint feature recognition based on the RGB recognition image to obtain the palmprint recognition result, and performs palm vein feature recognition based on the NIR recognition image to obtain the palm vein recognition result. Based on the palmprint recognition result and the palm vein recognition result, the user's identity is determined, thereby realizing the simultaneous use of two biometric features, palmprint and palm vein, for identity recognition, which improves the security, environmental adaptability and recognition rate of mobile terminal identity recognition.
[0015] Secondly, this application provides an identity verification method, the method comprising: Acquire one or more frames of raw mixed image; the raw mixed image includes the user's palm print information and palm vein information, and the raw mixed image is composed of multiple mixed arrays, which are used to record red, green and blue (RGB) information and near-infrared (NIR) information. Each frame of the original mixed image is processed to obtain an RGB recognition image corresponding to one or more frames of the original mixed image and an NIR recognition image corresponding to one or more frames of the original mixed image. Palmprint feature recognition is performed based on RGB recognition images to obtain palmprint recognition results, and palm vein feature recognition is performed based on NIR recognition images to obtain palm vein recognition results. The user's identity is determined based on the palm print recognition results and palm vein recognition results.
[0016] In one possible design, each frame of the original mixed image is processed to obtain an RGB recognition image corresponding to one or more frames of the original mixed image and an NIR recognition image corresponding to one or more frames of the original mixed image, including: For each blending array in each frame of the original blended image, the blending array is split to obtain the initial RGB array and the initial NIR array corresponding to the blending array; The initial RGB array is compensated to obtain a compensated RGB array, and the initial NIR array is compensated to obtain a compensated NIR array; YUV encoding is performed on the compensated RGB array and compensated NIR array corresponding to each mixing array in each frame of the original mixed image to obtain the RGB recognition image and the NIR recognition image corresponding to one or more frames of the original mixed image, respectively.
[0017] The beneficial effects of the methods provided in the second aspect and the various possible designs of the second aspect can be found in the first aspect and the various possible implementations of the first aspect, and will not be repeated here.
[0018] Thirdly, this application provides an electronic device including a first processor, which implements an identity recognition method as described in the second aspect and any possible design of the second aspect when executing a computer-executable program or instructions in a memory.
[0019] Fourthly, this application provides an electronic device including at least one memory and at least one second processor. The memory stores a computer-executable program or instructions, and the second processor, when executing the computer-executable program or instructions, implements an identity recognition method as described in the second aspect and any possible design of the second aspect.
[0020] Fifthly, this application provides a computer-readable storage medium storing a computer-executable program or instructions, which, when executed by a processor, implement an identity recognition method as described in the second aspect and any possible design of the second aspect.
[0021] In a sixth aspect, this application provides a computer program product comprising: execution instructions stored in a readable storage medium, at least one processor of an electronic device being able to read the execution instructions from the readable storage medium, and the at least one processor executing the execution instructions causing the electronic device to implement an identification method as described in the second aspect and any possible design of the second aspect.
[0022] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description
[0023] Figure 1 A schematic diagram of the structure of a mobile terminal provided in an embodiment of this application. Figure 1 .
[0024] Figure 2 This is a schematic diagram of the structure of a camera module provided in one embodiment of this application.
[0025] Figure 3 This is a schematic diagram of the structure of an image processing module provided in an embodiment of this application.
[0026] Figure 4 This is a schematic diagram of the structure of a control module provided in one embodiment of this application.
[0027] Figure 5 A schematic diagram of the structure of a mobile terminal provided in an embodiment of this application. Figure 2 .
[0028] Figure 6 This is a flowchart of an identity recognition method provided in an embodiment of this application.
[0029] Figure 7 This is a flowchart illustrating a method for obtaining RGB recognition images and NIR recognition images according to an embodiment of this application.
[0030] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 1 .
[0031] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 2 . Detailed Implementation
[0032] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c alone can mean: a alone, b alone, c alone, a combination of a and b, a combination of a and c, a combination of b and c, or a, b, and c, where a, b, and c can be single or multiple. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0033] The terms “center,” “longitudinal,” “lateral,” “up,” “down,” “left,” “right,” “front,” and “rear,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.
[0034] The terms "connected" and "connected" should be interpreted broadly. For example, in circuit structures, "connected" or "connected" can refer not only to physical connections but also to electrical or signal connections. This could be a direct connection (physical connection) or an indirect connection via at least one intermediate component, as long as the circuit is connected. It could also refer to the internal connection between two components. Similarly, a signal connection can refer to a connection via a circuit or a medium, such as radio waves. Those skilled in the art will understand the specific meaning of these terms in this application based on the specific circumstances.
[0035] For example, this application provides a mobile terminal and an identity recognition method. The camera module simultaneously acquires a raw mixed image including palmprint information and palm vein information. The image processing module separates the red-green-blue (RGB) recognition image for palmprint recognition and the near-infrared (NIR) recognition image for palm vein recognition from the raw mixed image. The control module uses the RGB recognition image to identify the user's identity through palmprint recognition and obtains the palmprint recognition result. It also uses the NIR recognition image to identify the user's identity through palm vein recognition and obtains the palm vein recognition result. Based on both the palmprint recognition result and the palm vein recognition result, identity recognition is achieved by simultaneously using both palmprint and palm vein biometrics, thereby improving the security, environmental adaptability, and recognition rate of mobile terminal identity recognition.
[0036] The following is combined Figures 1 to 5 The mobile terminal provided in the embodiments of this application will be described.
[0037] Please see Figure 1 , Figure 1 A schematic diagram of the structure of a mobile terminal provided in an embodiment of this application. Figure 1 .like Figure 1 As shown, the mobile terminal includes: a camera module 10, an image processing module 20, and a control module 30.
[0038] The image processing module 20 is connected to the camera module 10 and the control module 30, respectively.
[0039] The camera module 10 is used to capture one or more frames of raw mixed images and input one or more frames of raw mixed images to the image processing module 20.
[0040] When identity verification is required, the user places their palm on the recognition area of the camera module 10. The camera module 10 has a sensing function, and when it senses an object in the recognition area, it will begin to acquire the original mixed image.
[0041] The original mixed image includes the user's palm print information and palm vein information.
[0042] Palm print information refers to the skin lines, folds, and other palm print features on the surface of the palm. Palm print information originates from the visible light reflection signal formed after visible light shines on the palm. Palm vein information refers to the palm vein features such as the bifurcation and endpoints of the veins under the skin of the palm. Palm vein information originates from the NIR reflection or transmission signal formed after NIR irradiation of the palm.
[0043] When the camera module 10 begins to acquire the original mixed image, it can simultaneously emit NIR and visible light to illuminate the recognition area and simultaneously acquire visible light reflection signals and NIR reflection or transmission signals, thereby obtaining one or more frames of original mixed image.
[0044] A single frame of raw blended image consists of multiple blending arrays arranged sequentially. Each blending array consists of multiple pixels, which are also arranged sequentially. The arrangement of the multiple blending arrays and the arrangement of the multiple pixels are related to the arrangement of the color filters in the camera module 10, with a one-to-one correspondence between pixels and color filters.
[0045] The hybrid array is used to record RGB and NIR information. RGB information refers to the intensity values of the three visible light channels: red (R), green (G), and blue (B). NIR information refers to the intensity values of the NIR channel.
[0046] Each pixel in the blending array records the intensity value of one color channel or the NIR channel. Thus, multiple pixels in the blending array collectively record the corresponding RGB and NIR information. All RGB information in a single frame of the original blended image can reflect the user's palm print information, and all NIR information in a single frame can reflect the user's palm vein information. In a single frame of the original blended image, the spatial positions of the palm corresponding to the RGB and NIR information are perfectly aligned. The camera module 10 can simultaneously acquire palm print and palm vein information based on the same original blended image.
[0047] In some examples, the hybrid array can be a 4×4 Bayer pattern, comprising 16 pixels.
[0048] A traditional Bayer array is a 2×2 array, with red and blue pixels each accounting for 25%, i.e., one red pixel and one blue pixel. Since the human eye is more sensitive to green, green pixels account for 50%, i.e., two green pixels. For example, a traditional Bayer array can be shown in Table 1 below: Table 1
[0049] In Table 1, B 00 G represents the intensity value of the blue channel. 01 G 10 R represents the intensity value of the green channel. 11 This represents the intensity value of the red channel. Accordingly, based on Table 1, the traditional arrangement of color filters for a Bayer array, from left to right and from top to bottom, is: blue filter, green filter, green filter, and red filter.
[0050] Considering the need to acquire both RGB and NIR information simultaneously, the hybrid array can be a 4×4 Bayer array, including blue, red, green, and NIR pixels. For example, a hybrid array can be shown in Table 2 below: Table 2
[0051] In Table 2, G 00 G 02 G 11 G 13 G 20 G 22 G 31 G 33 Indicates the intensity value of the green channel, IR 01 IR 03 IR 21 IR 23 B represents the intensity value of the NIR channel. 10 B 32 R represents the intensity value of the blue channel. 12 R 30 This indicates the intensity value of the red channel. Accordingly, based on Table 2, the color filters corresponding to a hybrid array in the camera module 10 are arranged from left to right and from top to bottom as follows: green filter, NIR filter, green filter, NIR filter, blue filter, green filter, red filter, green filter, green filter, NIR filter, green filter, NIR filter, red filter, green filter, blue filter, green filter.
[0052] Among them, the green color filter only allows green visible light to pass through, the red color filter only allows red visible light to pass through, the blue color filter only allows blue visible light to pass through, and the NIR color filter only allows NIR light to pass through.
[0053] Based on this, by setting corresponding color filters in the camera module 10, when the user places their palm on the recognition area of the camera module 10, the camera module 10 can collect corresponding visible light and NIR through the sequentially arranged color filters, obtaining multiple mixed arrays and a raw mixed image. This allows for the simultaneous acquisition of the user's palmprint and palm vein information, facilitating multimodal identity recognition using both palmprint and palm vein features. Furthermore, the user only needs to place their palm on the recognition area once for the camera module 10 to complete the acquisition of both palmprint and palm vein biometrics, thus simplifying the recognition process and improving the user experience while achieving multimodal identity recognition.
[0054] The image processing module 20 receives one or more frames of raw mixed images input from the camera module 10. The image processing module 20 processes each frame of raw mixed image to obtain RGB recognition images and NIR recognition images corresponding to the one or more frames of raw mixed images, respectively.
[0055] Considering that RGB and NIR information are intertwined in the original mixed image, for a frame of the original mixed image, the image processing module 20 can first split each mixed array according to RGB pixels and NIR pixels, splitting it into separate RGB arrays and NIR arrays. After splitting, there are blank pixels in the RGB arrays and NIR arrays. The image processing module 20 compensates for the blank pixels using interpolation algorithms and other methods, compensating each RGB array and each NIR array into a complete array, thereby obtaining multiple compensated complete RGB arrays and multiple NIR arrays. The image processing module 20 encodes each RGB array to obtain an RGB recognition image, and encodes each NIR array to obtain an NIR recognition image.
[0056] In one identity recognition process, if the original mixed image is one frame, the image processing module 20 outputs one frame of RGB recognition image and one frame of NIR recognition image.
[0057] In a single identity recognition process, if the original mixed image consists of multiple frames, the image processing module 20 outputs corresponding multi-frame RGB recognition images and corresponding multi-frame NIR recognition images. The multi-frame RGB recognition images can form an RGB video stream, which the control module 30 uses for palmprint recognition. This ensures that even when the user's hand moves or the palm shifts, the control module 30 can obtain an RGB recognition image from the RGB video stream that records complete palmprint information, thereby improving the recognition rate and user experience. Similarly, the multi-frame NIR recognition images can form an NIR video stream, which the control module 30 uses for palm vein recognition. This ensures that even when the user's hand moves or the palm shifts, the control module 30 can obtain an NIR recognition image from the NIR video stream that records complete palm vein information, thereby improving the recognition rate and user experience.
[0058] Based on this, the image processing module 20 separates, compensates, and encodes the original mixed image where RGB and NIR information are intertwined, thereby obtaining independent RGB recognition images and NIR recognition images, so that the control module 30 can realize multimodal identity recognition based on the RGB recognition images and NIR recognition images respectively.
[0059] The control module is used to perform palmprint feature recognition based on RGB recognition images to obtain palmprint recognition results, and to perform palm vein feature recognition based on NIR recognition images to obtain palm vein recognition results. Based on the palmprint recognition results and palm vein recognition results, the user's identity is determined.
[0060] For a given RGB recognition image frame, the control module 30 processes the RGB recognition image using a preset palmprint recognition algorithm, extracting palmprint features such as main lines, folds, and textures. These palmprint features are then compared with preset palmprint features provided during user registration to obtain the palmprint recognition result. The palmprint recognition result can be a similarity score or confidence level, used to quantify the degree of matching between the palmprint features extracted from the RGB recognition image and the preset palmprint features.
[0061] For a single frame of NIR recognition image, the control module 30 processes the NIR recognition image using a preset palm vein recognition algorithm, extracting palm vein features such as vascular network structure, bifurcation points, and endpoints. These palm vein features are then compared with preset palm vein features provided during user registration to obtain the palm vein recognition result. The palm vein recognition result can be a similarity score or confidence level, used to quantify the degree of matching between the palm vein features extracted from the NIR recognition image and the preset palm vein features.
[0062] In one identity recognition process, if the RGB recognition image and the NIR recognition image are each a frame, the control module 30 performs palm print feature recognition based on the RGB recognition image of that frame and palm vein feature recognition based on the NIR recognition image of that frame.
[0063] In a single identity verification process, if both the RGB and NIR recognition images are multiple frames, the control module 30 can select the image that provides complete palmprint information from the multiple RGB recognition images to identify the user's identity via palmprint recognition. Similarly, it can select the image that provides complete palm vein information from the multiple NIR recognition images to identify the user's identity via palm vein recognition. This ensures that the control module 30 can complete palmprint and palm vein recognition even when the user's hand moves or the palm shifts, improving the recognition rate and enhancing the user experience.
[0064] After obtaining the palmprint recognition results and palm vein recognition results, the control module 30 can determine the user's identity in multiple ways.
[0065] As a feasible implementation, the control module 30 determines that the user identity recognition is successful when the palmprint recognition result indicates that the palmprint matching is successful and the palm vein recognition result indicates that the palm vein matching is successful.
[0066] For example, both palmprint recognition and palm vein recognition results are similarity scores. If the similarity score of the palmprint recognition result is greater than a first preset score, the palmprint recognition result indicates a successful match; otherwise, the match fails. If the similarity score of the palm vein recognition result is greater than a second preset score, the palm vein recognition result indicates a successful match; otherwise, the match fails. The control module 30 determines that user identification has passed when both the palmprint recognition result's similarity score and the palm vein recognition result's similarity score are greater than the first preset score and the second preset score, respectively. It determines that user identification has failed when either the palmprint recognition result's similarity score is less than or equal to the first preset score, or the palm vein recognition result's similarity score is less than or equal to the second preset score, respectively.
[0067] In this method, the control module 30 only determines that the user's identity has been successfully identified when both the palm print and palm vein are successfully matched, thereby improving the security of mobile terminal identity recognition.
[0068] As another feasible implementation, the control module 30 determines that the user identity recognition is successful when the palm print recognition result indicates that the palm print matching is successful or the palm vein recognition result indicates that the palm vein matching is successful.
[0069] For example, both palmprint recognition and palm vein recognition results are similarity scores. If the similarity score of the palmprint recognition result is greater than a first preset score, the palmprint recognition result indicates a successful match; otherwise, the match fails. Similarly, if the similarity score of the palm vein recognition result is greater than a second preset score, the palm vein recognition result indicates a successful match; otherwise, the match fails. The control module 30 determines that user identification has passed when it determines that the similarity score of the palmprint recognition result is greater than the first preset score, or the similarity score of the palm vein recognition result is greater than the second preset score. It determines that user identification has failed when it determines that the similarity score of the palmprint recognition result is less than or equal to the first preset score and the similarity score of the palm vein recognition result is less than or equal to the second preset score.
[0070] In this approach, when the quality of palmprint or palm vein information acquisition decreases due to environmental factors, the control module 30 performs identity recognition based on either palmprint or palm vein. This improves the recognition pass rate and environmental adaptability while ensuring the security of mobile terminal identity recognition, thereby enhancing the user experience.
[0071] As another feasible implementation method, the control module 30 performs weighted fusion based on the palmprint recognition result and the palm vein recognition result to obtain the fused recognition result. When the fused recognition result indicates a successful match, it determines that the user identity recognition has passed.
[0072] For example, both palmprint recognition and palm vein recognition results are similarity scores. The control module 30 performs a weighted fusion of the similarity scores of the palmprint recognition and palm vein recognition results to obtain a fused similarity score. When the fused similarity score is greater than a preset fused score, the fused recognition result indicates a successful match; otherwise, it indicates a failed match. Therefore, the control module 30 determines that user identification has passed when the fused similarity score is greater than the preset fused score, and that user identification has failed when the fused similarity score is less than or equal to the preset fused score.
[0073] The weights of palmprint recognition and palm vein recognition results in the weighted fusion can be set according to the environment and requirements.
[0074] In this approach, the control module 30 makes a fusion decision based on palmprint recognition and palm vein recognition, which improves the security of mobile terminal identity recognition. The palmprint information and palm vein information can provide redundancy to each other, thereby improving the recognition success rate.
[0075] Based on this, the control module 30 performs palmprint feature recognition based on the RGB recognition image to obtain palmprint recognition results, and performs palm vein feature recognition based on the NIR recognition image to obtain palm vein recognition results. Based on the palmprint recognition results and palm vein recognition results, the user's identity is determined, thereby realizing the simultaneous use of two biometric features, palmprint and palm vein, for identity recognition, which improves the security of mobile terminal identity recognition.
[0076] In this embodiment, the mobile terminal includes a camera module, an image processing module, and a control module. The camera module acquires one or more frames of raw mixed images. These raw mixed images include the user's palmprint and palm vein information. The raw mixed images consist of multiple mixed arrays that record RGB (red, green, blue) information and NIR (near-infrared) information, thus enabling simultaneous acquisition of the user's palmprint and palm vein information for multimodal identity recognition. Furthermore, the user only needs to place their palm on the recognition area once for the camera module to acquire both palmprint and palm vein biometrics, simplifying the recognition process and improving the user experience while achieving multimodal identity recognition. The image processing module processes each frame of the raw mixed image to obtain an RGB recognition image and an NIR recognition image corresponding to each frame of the raw mixed image, resulting in independent RGB and NIR recognition images. This allows the control module to perform multimodal identity recognition based on the RGB and NIR recognition images respectively. The control module performs palmprint feature recognition based on the RGB recognition image to obtain the palmprint recognition result, and performs palm vein feature recognition based on the NIR recognition image to obtain the palm vein recognition result. Based on the palmprint recognition result and the palm vein recognition result, the user's identity is determined, thereby realizing the simultaneous use of two biometric features, palmprint and palm vein, for identity recognition, which improves the security, environmental adaptability and recognition rate of mobile terminal identity recognition.
[0077] Based on the above exemplary description, the following, in conjunction with Figure 2 This section introduces a specific structure of the camera module 10.
[0078] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of a camera module provided in one embodiment of this application. Figure 2 As shown, the camera module 10 includes a white light 101, an infrared light 102, an optical lens 103, a filter 104, and an image sensor 105.
[0079] The white light 101, infrared light 102, and optical lens 103 are disposed on a first plane, the filter 104 is disposed on a second plane, and the image sensor 105 is disposed on a third plane. The first, second, and third planes are parallel to each other, the first plane is located above the second plane, and the third plane is located below the second plane. The center of the optical lens 103, the center of the filter 104, and the center of the image sensor 105 are located on the same straight line perpendicular to the first plane.
[0080] Based on this, the optical lens 103, filter 104 and image sensor 105 are arranged in a coaxial layout to ensure image quality.
[0081] An optical lens 103 is used to receive a focused light signal from the user's palm when the user's palm is located within the recognition area of the optical lens 103. A white light 101 is used to provide a first light source when the user's palm is located within the recognition area of the optical lens 103; an infrared light 102 is used to provide a second light source when the user's palm is located within the recognition area of the optical lens 103.
[0082] The first light source is a visible light source, whose spectral range covers the visible light band. Its main function is to illuminate the surface of the palm, allowing palm print features such as textures and creases to enter the optical lens 103 through reflected visible light. The second light source is a NIR light source. NIR light has penetrability to biological tissues, can be absorbed by hemoglobin in the blood and reflected by other surrounding tissues of blood vessels, allowing palm vein features to enter the optical lens 103 through reflected near-infrared light. Based on this, under the supplementary lighting of the white light lamp 101 and the infrared light lamp 102, the first light source, the second light source, and the ambient light source will be reflected from the user's palm into the optical lens 103, forming a converged light signal, which includes palm print information and palm vein information.
[0083] In some examples, the wavelength of the first light source is 380 nanometers to 780 nanometers, and the wavelength of the second light source is 850 nanometers or 940 nanometers.
[0084] The filter 104 is used to process the converged optical signal to obtain a mixed optical signal, which includes a first band signal and a second band signal; the first band signal is used to provide RGB information, and the second band signal is used to provide NIR information.
[0085] Filter 104 is an optical device that enables transmission of specific wavelengths through a special coating technology. Its core principle is based on the design of a multilayer interference film system. By controlling the film thickness and refractive index, transmission is enhanced at specific wavelengths, while interference effects are used to suppress other wavelengths, thereby enabling the projection of optical signals in the desired wavelength band.
[0086] In some examples, filter 104 is a bimodal transmission filter; filter 104 allows light signals from 380 nm to 780 nm and 850 nm to pass through, or filter 104 allows light signals from 380 nm to 780 nm and 940 nm to pass through. Thus, the first band is 380 nm to 780 nm, and the second band is either 850 nm or 940 nm.
[0087] Based on this, the filter 104 can precisely select the spectral range entering the image sensor 105, ensuring that the image sensor 105 acquires pure RGB and NIR information, avoiding interference from light signals in other bands to the imaging, thereby helping to improve the quality of the acquired original mixed image.
[0088] Image sensor 105 is used to convert mixed optical signals to obtain one or more frames of original mixed images.
[0089] The image sensor 105 includes multiple color filters corresponding to each pixel in the hybrid array. When the user places his palm on the recognition area of the camera module 10, the color filters arranged in sequence receive the hybrid light signal in one exposure. The blue, red, and green color filters respond to the first band signal, which is the visible light part, and the NIR color filter responds to the second band signal, which is the NIR part, thereby obtaining multiple hybrid arrays and obtaining a frame of original hybrid image.
[0090] Based on this, by setting an optical lens 103 and an image sensor 105 in the camera module 10, palm print information and palm vein information can be obtained simultaneously, eliminating the need to set up two cameras and two image sensors to collect palm print information and palm vein information respectively, thus saving the cost of the mobile terminal.
[0091] Based on the above exemplary description, the following, in conjunction with Figure 3 This section introduces a specific structure of the image processing module 20.
[0092] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an image processing module provided in one embodiment of this application. Figure 3 As shown, the image processing module 20 includes an image processing front-end 201 and an image processing engine 202.
[0093] The image processing front end 201 is connected to the camera module 10, the image processing front end 201 is connected to the image processing engine 202, and the image processing engine 202 is connected to the control module 30.
[0094] The image processing front end 201 is used to split the blending array for each blending array in each frame of the original blending image to obtain the initial RGB array and the initial NIR array corresponding to the blending array.
[0095] Based on Table 2, after splitting, the initial RGB array obtained from the mixed array in Table 2 can be shown in Table 3 below: Table 3
[0096] After splitting, the initial NIR array obtained from the hybrid array in Table 2 can be shown in Table 4 below: Table 4
[0097] Based on this, the image processing front end 201 splits each hybrid array into an initial RGB array and an initial NIR array, so that palm print recognition and palm vein recognition can be performed independently in the future.
[0098] In addition, the image processing front end 201 can also perform noise reduction, white balance, color calibration and other processing on the initial RGB array and the initial NIR array, thereby optimizing the image quality and helping to perform feature recognition in the subsequent process.
[0099] Image processing engine 202 is used to compensate the initial RGB array to obtain a compensated RGB array and to compensate the initial NIR array to obtain a compensated NIR array.
[0100] There are blank pixels in the initial RGB array and initial NIR array after splitting, so the image processing engine 20 needs to compensate for them.
[0101] For the initial RGB array, the image processing engine 202 can use methods such as simple interpolation and bilinear interpolation to compensate for the initial RGB array. For example, based on Table 3, the compensated RGB array can be shown in Table 5 below: Table 5
[0102] For the initial NIR array, the image processing engine 202 can use methods such as interpolation to compensate for the initial NIR array. For example, based on Table 4, the compensated NIR array can be shown in Table 6 below: Table 6
[0103] Based on this, the image processing engine 202 compensates for each initial RGB array and each initial NIR array to obtain a complete compensated RGB array and a complete compensated NIR array, so as to obtain a high-quality RGB recognition image and NIR recognition image.
[0104] The image processing engine 202 is also used to perform YUV encoding on the compensated RGB array and compensated NIR array corresponding to each mixing array in each frame of the original mixed image, respectively, to obtain the RGB recognition image and the NIR recognition image corresponding to one or more frames of the original mixed image, respectively.
[0105] Considering that images formed by multiple compensated RGB arrays and multiple compensated NIR arrays are not convenient for feature recognition, the image processing engine 202 can use YUV encoding to process the image into RGB recognition images and NIR recognition images that are more in line with the characteristics of human vision and are easier for recognition algorithms to process, so as to facilitate feature recognition.
[0106] Among them, YUV encoding can be YUV420 encoding, etc.
[0107] Specifically, for the compensated RGB array, the image processing engine 202 calculates the red, green and blue pixel components in the compensated RGB array into Y, U and V components using the following formulas one to three, thereby achieving YUV encoding.
[0108] Formula 1; Formula 2; Formula 3; Where Y represents the Y component, U represents the U component, V represents the V component, R represents the intensity value of the red channel, G represents the intensity value of the green channel, and B represents the intensity value of the blue channel.
[0109] For compensated NIR arrays, since they lack chromaticity, YUV encoding can be achieved by directly assigning values to the Y component.
[0110] When performing YUV encoding, the image processing engine 202 can use multiple resolutions for encoding, such as 640x480, 1280x720, etc.
[0111] Based on this, in the image processing module 20, the image processing front-end 201 splits each hybrid array into an initial RGB array and an initial NIR array, so as to facilitate independent palmprint recognition and palm vein recognition later. The image processing engine 202 compensates for each initial RGB array and each initial NIR array to obtain a complete compensated RGB array and a complete compensated NIR array. Then, it performs YUV encoding on the compensated RGB array and compensated NIR array to obtain high-quality RGB recognition images and NIR recognition images. The image processing engine 202 performs YUV encoding on the compensated RGB array and compensated NIR array corresponding to each hybrid array in each frame of the original hybrid image, respectively, to process the image into RGB recognition images and NIR recognition images that are more in line with the characteristics of human vision and are easier for recognition algorithms to process, so as to facilitate feature recognition.
[0112] Based on the above exemplary description, the following, in conjunction with Figure 4 This section introduces a specific structure of the control module 30.
[0113] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a control module provided in one embodiment of this application. Figure 4 As shown, the control module 30 includes a trusted execution environment 301 and a rich execution environment 302.
[0114] The trusted execution environment 301 includes secure memory 3011 and a communication agent 3012, while the rich execution environment 302 includes trusted applications 3021 and client applications 3022. The control module 30 consists of a central processing unit (CPU), a memory module, and a storage module. The trusted execution environment 301 is a hardware-protected secure area within the control module 30, isolated from the rich execution environment 302. The code and data running within it are invisible and inaccessible in the rich execution environment 302, protecting the confidentiality and integrity of data and code. The rich execution environment 302 is a normal operating environment, running regular applications.
[0115] Among them, the secure memory 3011 is connected to the image processing module 20, and the secure memory 3011, the communication agent 3012, the trusted application 3021 and the client application 3022 are connected in sequence, with the trusted application 3021 and the client application 3022 sharing memory.
[0116] The secure memory 3011 receives RGB and NIR recognition images from the image processing module 20. The secure memory 3011 is used to encrypt the RGB and NIR recognition images and then input the encrypted RGB and NIR recognition images to the communication proxy 3012.
[0117] When the secure memory 3011 receives an RGB recognition image / NIR recognition image, it encrypts the RGB recognition image / NIR recognition image. Performing encryption within the secure memory 3011 can prevent data from being tampered with, thereby improving the security of the mobile terminal.
[0118] Communication proxy 3012 is used to transmit encrypted RGB recognition images and encrypted NIR recognition images to rich execution environment 302. Communication proxy 3012 is a controlled data channel between trusted execution environment 301 and rich execution environment 302. When rich execution environment 302 needs to access data, communication proxy 3012 verifies the identity of rich execution environment 302 to ensure the confidentiality and integrity of encrypted RGB recognition images and encrypted NIR recognition images during transmission.
[0119] Trusted application 3021 receives encrypted RGB recognition images and encrypted NIR recognition images. Trusted application 3021 performs signature verification on the encrypted RGB recognition image, extracts the palmprint features to be recognized from the verified RGB recognition image, compares the palmprint features to be recognized with preset palmprint features to obtain the palmprint recognition result, and performs signature verification on the encrypted NIR recognition image, extracts the palm vein features to be recognized from the verified NIR recognition image, compares the palm vein features to be recognized with preset palm vein features to obtain the palm vein recognition result. Based on the palmprint recognition result and the palm vein recognition result, the user's identity is determined.
[0120] Among them, the preset palm print features are the palm print features entered during user registration, and the preset palm vein features are the palm vein features entered during user registration.
[0121] Trusted application 3021 is an application or component trusted by the Trusted Execution Environment 301. User identification is achieved through trusted application 3021, which improves the security of mobile terminals.
[0122] Client application 3022 is used to receive and output the result of user identity verification.
[0123] The 3022 client application is powerful and easily expandable, but its complex design makes it vulnerable to attacks. The 3022 client application only receives and outputs the results of user identity verification, and does not directly access palm vein information, palm print information, preset palm vein features and preset palm print features, thereby avoiding data leakage and improving the security of mobile terminals.
[0124] Based on this, by setting a trusted execution environment 301 and a rich execution environment 302 in the control module 30, the identity recognition process is executed in a secure area, preventing data leakage and malicious attacks, thereby improving the security of the mobile terminal.
[0125] Based on the above exemplary description, the image processing module 20 is further configured to process each frame of the original mixed image to obtain an RGB preview image corresponding to one or more frames of the original mixed image and an NIR preview image corresponding to one or more frames of the original mixed image.
[0126] Similar to generating RGB and NIR recognition images, the image processing module 20 can also generate RGB and NIR preview images to provide real-time feedback on the recognition status to the user. Furthermore, the corresponding RGB / NIR preview images and RGB / NIR recognition images originate from the same original mixed image frame, ensuring consistency between the feedback content and the currently being recognized content, thus improving the user experience.
[0127] The RGB and NIR preview images have a first resolution, while the RGB and NIR recognition images have a second resolution. The first resolution is smaller than the second resolution. For example, the first resolution is 640×480, and the second resolution is 1280×720.
[0128] Based on this, the image processing module 20 can obtain higher resolution RGB and NIR recognition images, thereby improving image quality and increasing the recognition rate during biometric identification. The image processing module 20 can also obtain lower resolution RGB and NIR preview images, saving computational resources while providing real-time feedback to the user on the recognition status.
[0129] Security Memory 3011 is also used for encrypting RGB and NIR preview images.
[0130] Considering that although the RGB and NIR preview images have low resolution, they still contain the user's biometric information, the Secure Memory 3011 encrypts the RGB and NIR preview images, thereby avoiding the risk of data leakage and improving security.
[0131] The communication agent 3012 also connects to the client application 3022 and is also used to transmit encrypted RGB preview images and encrypted NIR preview images to the rich execution environment 302.
[0132] Customer application 3022 is also used to receive and output encrypted RGB preview images and encrypted NIR preview images.
[0133] After receiving the encrypted RGB preview image and the encrypted NIR preview image, the client application 3022 uses the agreed key to decrypt the preview image to obtain the RGB preview image and the NIR preview image. The client application 3022 can then display the RGB preview image and the NIR preview image through the screen or other means to provide intuitive feedback to the user. This allows the user to adjust their palm position based on the displayed image, ensuring that the appropriate image is captured for recognition, thereby improving the efficiency of identity recognition and the user experience.
[0134] Based on the above exemplary description, the mobile terminal may also include a touch screen 40.
[0135] Please see Figure 5 , Figure 5 A schematic diagram of the structure of a mobile terminal provided in an embodiment of this application. Figure 2 .like Figure 5 As shown, the mobile terminal also includes a touch screen 40.
[0136] The touch screen 40 is connected to the control module 30.
[0137] The touch screen 40 is used to receive and display encrypted RGB preview images and encrypted NIR preview images to the user.
[0138] Based on this, users can intuitively view the palm recognition status through the touch screen 40. With real-time visual feedback, users can quickly complete the collection of correct biometric features without guessing or repeated attempts, thereby improving the efficiency of identity recognition and enhancing the user experience.
[0139] In addition, users can send commands to the mobile terminal via the touch screen 40, such as re-identification, starting the identity verification process, and viewing the recognition results.
[0140] Based on the above exemplary description, the control module 30 is also connected to the white light 101 and the infrared light 102 respectively.
[0141] The control module 30 is also used to adjust the light intensity of the white light lamp according to the RGB recognition image, and to adjust the light intensity of the infrared lamp according to the NIR recognition image.
[0142] After receiving the RGB recognition image / NIR recognition image, the control module 30 can first analyze the imaging quality of the RGB recognition image / NIR recognition image and detect whether the user's palm is present in the RGB recognition image / NIR recognition image.
[0143] If the user's palm is found in the RGB / NIR recognition image, the control module 30 determines whether the brightness of the RGB / NIR recognition image is within a preset brightness range. If the user's palm image is not found in the RGB / NIR recognition image or the palm image is incomplete, the control module 30 sends a prompt to the user, prompting the user to adjust the position of the palm in the recognition area.
[0144] When the brightness of the RGB recognition image / NIR recognition image is determined to be within the preset brightness range, the control module 30 begins palm print feature recognition / palm vein feature recognition. When the brightness of the RGB recognition image / NIR recognition image is determined to be outside the preset brightness range, the control module 30 adjusts the light source intensity of the white light 101 / infrared light 102 until the brightness of the RGB recognition image / NIR recognition image is determined to be within the preset brightness range.
[0145] Based on this, the control module 30 dynamically adjusts the brightness of the white light and infrared light to provide supplemental lighting to the palm, which can improve the imaging quality, thereby improving the accuracy of feature extraction and the success rate of final identity recognition.
[0146] Furthermore, it should be noted that the mobile terminal provided in this application embodiment can also recognize faces. The steps for face recognition are similar to those for palm print recognition, and will not be described again here. During face recognition, the control module 30 can adjust the light intensity of the touch screen 40 to provide supplementary lighting for the face.
[0147] For example, this application also provides an identity recognition method.
[0148] Please see Figure 6 , Figure 6 This is a flowchart illustrating an identity recognition method provided in one embodiment of this application. Figure 6 As shown, the method includes: S601, acquire one or more frames of raw mixed image.
[0149] The original mixed image includes the user's palm print information and palm vein information. The original mixed image consists of multiple mixed arrays, which are used to record red, green and blue (RGB) information and near-infrared (NIR) information. S602. Process each frame of the original mixed image to obtain the RGB recognition image corresponding to one or more frames of the original mixed image and the NIR recognition image corresponding to one or more frames of the original mixed image.
[0150] S603. Perform palmprint feature recognition based on the RGB recognition image to obtain palmprint recognition results, and perform palm vein feature recognition based on the NIR recognition image to obtain palm vein recognition results.
[0151] S604. Determine the user's identity based on the palm print recognition results and palm vein recognition results.
[0152] It should be noted that the identity recognition method in this application embodiment can be executed through the aforementioned mobile terminal, and its implementation principle and technical effect are similar, so it will not be described again here.
[0153] Based on the above exemplary description, a method for obtaining RGB recognition images and NIR recognition images is provided below.
[0154] Please see Figure 7 , Figure 7 This is a flowchart illustrating a method for obtaining RGB and NIR recognition images according to an embodiment of this application. Figure 7 As shown, the method includes: S701. For each blending array in each frame of the original blending image, split the blending array to obtain the initial RGB array and the initial NIR array corresponding to the blending array.
[0155] S702. The initial RGB array is compensated to obtain a compensated RGB array, and the initial NIR array is compensated to obtain a compensated NIR array.
[0156] S703. Perform YUV encoding on the compensated RGB array and compensated NIR array corresponding to each mixing array in each frame of the original mixed image to obtain the RGB recognition image and the NIR recognition image corresponding to one or more frames of the original mixed image, respectively.
[0157] By way of example, this application also provides an electronic device.
[0158] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 1 .like Figure 8 As shown, the electronic device may include a first processor 801, which, when executing a computer-executable program or instruction stored in a memory, implements the embodiments of this application. Figures 6 to 7 The identity recognition method shown.
[0159] The electronic device can be used to perform the various steps and / or processes corresponding to the electronic devices in the above method embodiments.
[0160] By way of example, this application also provides an electronic device.
[0161] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 2 .like Figure 9 As shown, the electronic device may include a second processor 901 and a memory 902. The memory 902 stores a computer program. When the second processor 901 executes the computer program, it implements the embodiments of this application. Figures 6 to 7 The identity recognition method shown.
[0162] The electronic device can be used to perform the various steps and / or processes corresponding to the electronic devices in the above method embodiments.
[0163] The electronic device of this application can be used to execute the technical solutions of the method embodiments described above. Its implementation principle and technical effects are similar. The operations implemented by each module can be further referred to the relevant descriptions of the method embodiments, which will not be repeated here. The modules here can also be replaced by components or circuits.
[0164] This application can divide electronic devices into functional modules based on the above method examples. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.
[0165] Another embodiment of this application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the embodiments of this application. Figures 6 to 7 The identity recognition method shown.
[0166] This application also provides a program product including executable instructions stored in a computer-readable storage medium. At least one processor of an electronic device can read the executable instructions from the computer-readable storage medium, and the at least one processor executes the executable instructions to cause the electronic device to implement embodiments of this application. Figures 6 to 7 The identity recognition method shown.
[0167] This application also provides a chip that is connected to a memory, or a chip that integrates a memory. When a software program stored in the memory is executed, it implements the embodiments of this application. Figures 6 to 7 The identity recognition method shown.
[0168] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.
[0169] Those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.
[0170] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A mobile terminal, characterized in that, The mobile terminal includes: a camera module, an image processing module, and a control module; the image processing module is connected to both the camera module and the control module. The camera module is used to capture one or more frames of raw mixed images; the raw mixed image includes the user's palm print information and palm vein information, and the raw mixed image is composed of multiple mixed arrays, which are used to record red, green and blue (RGB) information and near-infrared (NIR) information. The image processing module is used to process each frame of the original mixed image to obtain an RGB recognition image corresponding to one or more frames of the original mixed image and an NIR recognition image corresponding to one or more frames of the original mixed image. The control module is used to perform palmprint feature recognition based on the RGB recognition image to obtain a palmprint recognition result, and to perform palm vein feature recognition based on the NIR recognition image to obtain a palm vein recognition result, and to determine the user's identity based on the palmprint recognition result and the palm vein recognition result.
2. The mobile terminal according to claim 1, characterized in that, The camera module includes a white light, an infrared light, an optical lens, a filter, and an image sensor. The white light, the infrared light, and the optical lens are disposed on a first plane, the filter is disposed on a second plane, and the image sensor is disposed on a third plane. The first plane, the second plane, and the third plane are parallel to each other. The first plane is located above the second plane, and the third plane is located below the second plane. The centers of the optical lens, the filter, and the image sensor are located on the same straight line perpendicular to the first plane. The optical lens is used to receive a focused light signal from the user's palm when the user's palm is located in the recognition area of the optical lens. The white light is used to provide a first light source when the user's palm is located in the recognition area of the optical lens; The infrared lamp is used to provide a second light source when the user's palm is located in the recognition area of the optical lens; The filter is used to process the converged optical signal to obtain a mixed optical signal, the mixed optical signal including a first band signal and a second band signal; the first band signal is used to provide RGB information, and the second band signal is used to provide NIR information; The image sensor is used to convert the mixed optical signal to obtain one or more frames of the original mixed image.
3. The mobile terminal according to claim 2, characterized in that, The filter is a dual-peak transmission filter; the filter allows light signals from 380 nm to 780 nm and 850 nm to pass through, or the filter allows light signals from 380 nm to 780 nm and 940 nm to pass through.
4. The mobile terminal according to claim 1, characterized in that, The image processing module includes an image processing front-end and an image processing engine; the image processing front-end is connected to the camera module, the image processing front-end is connected to the image processing engine, and the image processing engine is connected to the control module. The image processing front end is used to split the hybrid array for each hybrid array in each frame of the original hybrid image to obtain the initial RGB array and the initial NIR array corresponding to the hybrid array; The image processing engine is used to compensate the initial RGB array to obtain a compensated RGB array, and to compensate the initial NIR array to obtain a compensated NIR array; The image processing engine is further configured to perform YUV encoding on the compensated RGB array and compensated NIR array corresponding to each mixing array in each frame of the original mixed image, respectively, to obtain RGB recognition images and NIR recognition images corresponding to the one or more frames of the original mixed image respectively.
5. The mobile terminal according to claim 1, characterized in that, The control module includes a trusted execution environment and a rich execution environment. The trusted execution environment includes secure memory and a communication agent. The rich execution environment includes a trusted application and a client application. The secure memory is connected to the image processing module. The secure memory, the communication agent, the trusted application, and the client application are connected in sequence. The trusted application and the client application share memory. The secure memory is used to encrypt the RGB recognition image and the NIR recognition image; The communication proxy is used to transmit the encrypted RGB recognition image and the encrypted NIR recognition image to the rich execution environment; The trusted application is used to perform signature verification on the encrypted RGB recognition image, extract palmprint features to be identified from the verified RGB recognition image, compare the palmprint features to be identified with preset palmprint features to obtain the palmprint recognition result, and perform signature verification on the encrypted NIR recognition image, extract palm vein features to be identified from the verified NIR recognition image, compare the palm vein features to be identified with preset palm vein features to obtain the palm vein recognition result, and determine the user's identity based on the palmprint recognition result and the palm vein recognition result; the preset palmprint features are the palmprint features entered by the user during registration, and the preset palm vein features are the palm vein features entered by the user during registration; The client application is used to receive and output the result of the user's identity determination.
6. The mobile terminal according to claim 5, characterized in that, The image processing module is further configured to process each frame of the original mixed image to obtain an RGB preview image corresponding to one or more frames of the original mixed image and an NIR preview image corresponding to one or more frames of the original mixed image. The resolution of the RGB preview image and the NIR preview image is a first resolution, and the resolution of the RGB recognition image and the NIR recognition image is a second resolution, wherein the first resolution is smaller than the second resolution; The secure memory is also used to encrypt the RGB preview image and the NIR preview image; The communication proxy is used to transmit the encrypted RGB preview image and the encrypted NIR preview image to the rich execution environment; The client application is also used to receive and output the encrypted RGB preview image and the encrypted NIR preview image.
7. The mobile terminal according to claim 6, characterized in that, The mobile terminal further includes: a touch screen; the touch screen is connected to the control module; The touchscreen is used to receive and display encrypted RGB preview images and encrypted NIR preview images to the user.
8. The mobile terminal according to claim 2, characterized in that, The control module is also connected to the white light and the infrared light respectively. The control module is also used to adjust the light source intensity of the white light according to the RGB recognition image and to adjust the light source intensity of the infrared light according to the NIR recognition image.
9. An identity recognition method, characterized in that, The method includes: Acquire one or more frames of raw mixed image; the raw mixed image includes the user's palm print information and palm vein information, the raw mixed image is composed of multiple mixed arrays, the mixed arrays are used to record red, green and blue (RGB) information and near-infrared (NIR) information; Each frame of the original mixed image is processed to obtain an RGB recognition image corresponding to one or more frames of the original mixed image and an NIR recognition image corresponding to one or more frames of the original mixed image; Palmprint feature recognition is performed based on the RGB recognition image to obtain palmprint recognition results, and palm vein feature recognition is performed based on the NIR recognition image to obtain palm vein recognition results. The user's identity is determined based on the palm print recognition results and the palm vein recognition results.
10. The method according to claim 9, characterized in that, The step of processing each frame of the original mixed image to obtain an RGB recognition image corresponding to one or more frames of the original mixed image and an NIR recognition image corresponding to one or more frames of the original mixed image includes: For each blending array in the original blending image of each frame, the blending array is split to obtain the initial RGB array and the initial NIR array corresponding to the blending array; The initial RGB array is compensated to obtain a compensated RGB array, and the initial NIR array is compensated to obtain a compensated NIR array; YUV encoding is performed on the compensated RGB array and compensated NIR array corresponding to each mixing array in each frame of the original mixed image to obtain the RGB recognition image and the NIR recognition image corresponding to the one or more frames of the original mixed image, respectively.