Palmprint image generation method and apparatus, device, storage medium, and program product

Abstract

Embodiments of the present application provide a palmprint image generation method and apparatus, a device, a storage medium, and a program product. The method comprises: determining a plurality of control points on the basis of a predetermined palmprint curve template, and generating a Bézier curve on the basis of the plurality of control points; on the basis of the Bézier curve, generating a palm crease energy map having crease information, the palm crease energy map comprising a palm crease having a crease distribution same as that of the Bézier curve and a crease type different from that of the Bézier curve, wherein the crease type is used for describing the crease information and corresponds to crease direction energy of each pixel on the palm crease; and on the basis of the palm crease energy map, generating a photorealistic palmprint image having detailed texture information.

Description

Palmprint image generation method, device, equipment, storage medium and program product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on November 17, 2023, with application number 202311546177.7 and application name “Palmprint image generation method, device, equipment and storage medium”. Technical Field

[0002] The present application relates to the field of artificial intelligence, and more specifically, to a palmprint image generation method, apparatus, device, storage medium, and program product.

[0003] Background of the Invention

[0004] In the field of palmprint recognition, large-scale palmprint datasets available for training and evaluation are extremely limited, severely hindering the development and performance improvement of palmprint recognition technology. Palmprints refer to the unique and stable lines on the inside of a person's palm, making them useful for individual identification and identity verification. However, the acquisition and annotation of palmprint data are relatively complex and time-consuming, resulting in a limited number of available palmprint datasets, especially large-scale ones.

[0005] In fields like deep learning, data volume is crucial for model training and performance. Large-scale datasets can provide more samples and variations, helping models learn and generalize better. However, the lack of large-scale palmprint datasets has limited the performance of traditional deep learning methods in palmprint recognition.

[0006] While the scarcity of large-scale palmprint datasets presents a challenge, care must be taken to protect user privacy when collecting such datasets. Palmprints are a sensitive and private part of a person's physical characteristics. Therefore, when collecting and using palmprint data, relevant privacy laws and regulations must be adhered to, and appropriate security measures must be implemented to protect user privacy.

[0007] Therefore, there is a need for an efficient palmprint image generation method that can generate diverse and realistic palmprint images while protecting user privacy.

[0008] Summary of the Invention

[0009] In order to solve the above problems, this application introduces a new palm crease energy (PCE) domain. First, the Bezier curve is converted into the palm crease energy domain to generate a palm crease line energy map with realistic creases. Then, a realistic palm print image with realistic texture is generated based on the palm crease line energy map, thereby generating diverse and realistic palm print images.

[0010] The embodiments of the present application provide a palmprint image generation method, apparatus, device, storage medium, and program product.

[0011] In one aspect, an embodiment of the present application provides a palmprint image generation method, which is performed by an electronic device and includes:

[0012] Determining a plurality of control points based on a predetermined palmprint curve template, and generating a Bezier curve based on the plurality of control points;

[0013] generating a palm line energy map having wrinkle information based on the Bezier curve, wherein the palm line energy map includes palm lines having the same line distribution as the Bezier curve but different line types, wherein the line types are used to describe the wrinkle information and correspond to the line direction energy of each pixel on the palm line; and

[0014] Based on the palmprint line energy map, a realistic palmprint image with detailed texture information is generated.

[0015] On the other hand, an embodiment of the present application provides a palmprint image generation device, comprising:

[0016] a curve generating module configured to determine a plurality of control points based on a predetermined palmprint curve template, and generate a Bezier curve based on the plurality of control points;

[0017] a wrinkle generation module configured to generate a palm line energy map having wrinkle information based on the Bezier curve, wherein the palm line energy map includes palm lines having the same line distribution as the Bezier curve but different line types, wherein the line types are used to describe the wrinkle information and correspond to the line direction energy of each pixel on the palm line; and

[0018] The texture generation module is configured to generate a realistic palm print image with detailed texture information based on the palm print line energy map.

[0019] On the other hand, an embodiment of the present application provides an electronic device, comprising: one or more processors; and one or more memories, wherein a computer executable program is stored in the one or more memories, and when the computer executable program is executed by the processor, the palmprint image generation method described above is executed.

[0020] On the other hand, an embodiment of the present application provides a computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by a processor, are used to implement the palmprint image generation method described above.

[0021] In another aspect, embodiments of the present application provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the palmprint image generation method according to embodiments of the present application.

[0022] BRIEF DESCRIPTION OF THE DRAWINGS

[0023] FIG1 is a schematic diagram illustrating the significant difference between a Bezier curve and a real palm print in terms of wrinkle distribution and texture;

[0024] FIG2 is a flow chart illustrating a palmprint image generating method according to an embodiment of the present application;

[0025] FIG3 is a schematic diagram illustrating a palmprint image generation system according to an embodiment of the present application;

[0026] FIG4A is a schematic diagram illustrating an example palmprint curve template according to an embodiment of the present application;

[0027] FIG4B is a schematic diagram showing a comparison between a Gaussian-MFRAT kernel according to an embodiment of the present application and a traditional MFRAT filter;

[0028] FIG5 is a schematic diagram illustrating domain conversion in joint training according to an embodiment of the present application;

[0029] FIG6 is a schematic diagram illustrating a first generation phase in joint training according to an embodiment of the present application;

[0030] FIG7 is a schematic diagram showing a palm line energy extractor according to an embodiment of the present application;

[0031] FIG8 is a schematic diagram illustrating a second generation phase in joint training according to an embodiment of the present application;

[0032] FIG9 is a comparison diagram showing palmprint generation results using different palmprint generation methods according to an embodiment of the present application;

[0033] FIG10 is a diagram showing a result of verification of the effectiveness of a line energy feature enhancement block according to an embodiment of the present application;

[0034] FIG11 is a schematic diagram showing a palmprint image generating device according to an embodiment of the present application;

[0035] FIG12 shows a schematic diagram of an electronic device according to an embodiment of the present application; and

[0036] FIG13 is a schematic diagram showing the architecture of an exemplary computing device according to an embodiment of the present application.

[0037] Implementation Method

[0038] In order to make the purpose, technical solutions and advantages of this application more apparent, the following will describe in detail an exemplary embodiment of this application with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this application, rather than all the embodiments of this application, and it should be understood that this application is not limited to the exemplary embodiments described herein.

[0039] In this specification and the accompanying drawings, substantially the same or similar steps and elements are denoted by the same or similar reference numerals, and repeated descriptions of these steps and elements will be omitted. Furthermore, in the description of this application, the terms "first," "second," etc. are used only to distinguish descriptions and are not to be understood as indicating or implying relative importance or ranking.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art. The terms used herein are for the purpose of describing the embodiments of the present invention only and are not intended to limit the present invention.

[0041] To facilitate the description of the present application, concepts related to the present application are introduced below.

[0042] The palmprint image generation method of the present application can be implemented based on artificial intelligence (AI). Artificial intelligence uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, enabling machines to have the capabilities of perception, reasoning, and decision-making. Specifically, in the embodiments of the present application, AI studies the design principles and implementation methods of various intelligent machines to enable the palmprint image generation method of the present application to achieve the following functions: based on control points determined from a palmprint curve template, a realistic palmprint image with realistic creases and textures is generated.

[0043] The palmprint image generation method of the present application can also be implemented based on computer vision (CV) technology. Computer vision technology can obtain information from images or multidimensional data. Specifically, the palmprint image generation method of the present application can use CV technology to generate a palmprint line energy map with a style close to that of a real palmprint from a Bezier curve. Then, based on the palmprint line energy map, a palmprint image with diverse texture information is generated to achieve diverse palmprint image output, which can be used, for example, in the pre-training process of a palmprint recognition model.

[0044] The palmprint image generation method of the present application can be implemented based on Bezier curves. Bezier curves are used to describe smooth curves. In the embodiments of the present application, Bezier curves can be applied to the drawing of palmprint lines. The characteristic of Bezier curves is that the shape of the curve can be controlled by control points. Therefore, the curvature and shape of the palmprint line can be adjusted by control points. Specifically, the palm can be divided into several segments, each segment is described by a Bezier curve. By adjusting the position and number of control points, palmprint lines of different shapes can be obtained.

[0045] In summary, the solutions provided in the embodiments of the present application involve technologies such as artificial intelligence and computer vision. The embodiments of the present application will be further described below in conjunction with the accompanying drawings.

[0046] FIG1 is a schematic diagram showing a significant difference between the Bezier curve and the real palm print in terms of wrinkle distribution and texture.

[0047] Palmprint recognition, as a stable and privacy-friendly biometric recognition technology, has recently shown great potential in recognition applications. In recent years, deep learning-based palmprint recognition methods have become the mainstream palmprint recognition technology. Deep learning-based palmprint recognition methods train neural networks to extract palmprint features with improved classification or pairing loss. However, a major difficulty in the research and application of deep learning-based palmprint recognition is the scarcity of large-scale palmprint datasets. Collecting large-scale palmprint datasets may pose the risk of violating user privacy. To overcome this problem, researchers can currently use a number of data synthesis techniques to generate simulated palmprint data, thereby expanding the dataset.

[0048] Currently, some palmprint generation methods have been applied to generate pseudo-palmprint samples in the field of palmprint recognition. For example, the Bezier palmprint generation method uses parameterized Bezier curves to synthesize pseudo-palmprint lines. However, as shown in Figure 1, the Bezier palmprint 101 generated by the Bezier palmprint generation method has significant differences from the real palmprint image 102 in terms of wrinkles and texture. It cannot reflect the wrinkle distribution of a real palmprint, nor can it present the various detailed textures in a real palmprint. Therefore, Bezier palmprint still requires a certain amount of real palmprint data for fine-tuning.

[0049] Furthermore, with limited sample data, models based on generative adversarial networks (GANs) often face challenges such as discriminator overfitting and imbalance between discrete data space and continuous latent distribution, which lead to reduced fidelity and unstable training process.

[0050] Moreover, some methods using few samples, such as data augmentation, regularization, and transfer learning, lack control over identity when generating palmprints.

[0051] Based on this, an embodiment of the present application provides a method for using an intermediate domain connecting the Bezier palmprint domain and the real palmprint image domain. A new Palm Crease Energy (PCE) domain is introduced as this intermediate domain. First, the Bezier curve is converted into the PCE domain to generate a palmprint line energy map with realistic wrinkles (abbreviated as a PCE image). Then, based on the PCE map, a realistic palmprint image with realistic texture is generated, resulting in diverse and realistic palmprint images.

[0052] As shown in FIG1 , the PCE image 103 is an intermediate state close to the real palm print. It has wrinkle consistency with the Bezier curve 101, that is, the line distribution of its wrinkles (i.e., palm lines) is consistent with the Bezier curve 101, and has appearance similarity with the real palm print image 102, that is, the appearance of the palm lines of the PCE image 103 is consistent with the appearance of the real palm lines.

[0053] Compared to traditional palmprint generation methods, the method provided in the embodiments of the present application decomposes the Bezier-Real difference into wrinkle difference and texture difference, thereby reducing the generation difficulty. Specifically, by introducing the palmprint line energy domain, the generation of palmprint wrinkles and texture is decoupled. Realistic wrinkles are generated by converting the Bezier curve in the Bezier palmprint domain to a palmprint line energy map in the palmprint line energy domain, and realistic textures are generated by converting the palmprint line energy map in the palmprint line energy domain to a palmprint image in the palmprint image domain. This reduces the difficulty of generating realistic palmprint images from Bezier curves and enables the generation of diverse realistic palmprint images.

[0054] The method provided in the embodiment of the present application uses control points determined from a predetermined palmprint curve template to generate a Bezier curve, and converts the Bezier curve into a palmprint line energy map with wrinkle information, wherein the wrinkle information includes the same line distribution as the Bezier curve but a different line type, and the line type is determined by the line energy characteristics of each pixel. Then, based on the palmprint line energy map with wrinkle information, a palmprint image with texture information is generated. The palmprint lines of the palmprint image are consistent with the wrinkle information of the palmprint line energy map, thereby generating a realistic palmprint image with realistic wrinkles and texture.

[0055] The method provided in the embodiments of the present application introduces a palmprint line energy domain as an intermediate domain connecting the Bezier palmprint domain and the palmprint image domain, thereby avoiding directly generating a palmprint image with wrinkle information and texture information from a Bezier curve, thereby reducing the difficulty of generating a palmprint image. In addition, in the process of generating a palmprint image from a palmprint line energy map, detailed texture information is generated while ensuring that the palmprint image has consistent palmprint lines. This enables the generation of realistic palmprint images with diverse textures while retaining the same identity information. Therefore, the dependence on real data is reduced, and the method is suitable for palmprint recognition training in the absence of large-scale palmprint datasets.

[0056] Fig. 2 is a flow chart showing a palmprint image generation method 200 according to an embodiment of the present application. Fig. 3 is a schematic diagram showing a palmprint image generation system according to an embodiment of the present application.

[0057] As shown in FIG3 , in this embodiment of the present application, the palmprint image generation system may include a first generation stage 310 associated with a first generator 304, and a second generation stage 320 associated with a second generator 307. The first generation stage 310 and the second generation stage 320 are connected by a PCE image 305 in the PCE domain.

[0058] In the first generation stage 310 , the Bezier palmprint generator 302 generates a Bezier curve 303 based on the control points 301 , and the first generator 304 generates a PCE image 305 based on the Bezier curve 303 .

[0059] In the second generation stage 320, the second generator 307 generates a simulated palm print image 308 based on the PCE image 305 and the control vector 306. The generation process will be described in detail below with reference to FIG2 to FIG4A.

[0060] First, as shown in FIG2 , in step S201 , a plurality of control points may be determined based on a predetermined palmprint curve template, and a Bezier curve may be generated based on the plurality of control points.

[0061] Optionally, since the Bezier curve is a curve generated based on multiple control points, in an embodiment of the present application, the palmprint image generation method can use prior knowledge obtained from human skin texture to improve the control point generation mechanism, for example, to improve the palmprint curve template used to generate control points.

[0062] According to an embodiment of the present application, the palmprint curve template can be pre-determined based on statistical information obtained from real human palmprint lines. Because it takes into account the diversity and individual differences of real human palmprints, the palmprint curve template determined is more representative. Therefore, the control point generation range determined based on this palmprint curve template is more accurate, thereby generating a Bezier curve that more closely resembles the distribution of real palmprint wrinkles.

[0063] Figure 4A is a schematic diagram illustrating example palmprint curve templates according to an embodiment of the present application. As shown in Figure 4A , five example palmprint curve templates are provided, divided based on statistical information derived from real human palmprint lines, corresponding to the five most representative human palmprint line distributions. Each of these five example palmprint curve templates provides a generation range for control points, as indicated by the dotted box. Different example palmprint curve templates can have different numbers of palmprint lines and, therefore, can be determined using different numbers of generation ranges and control points.

[0064] It should be understood that the example palmprint curve template in FIG4A is not intended to limit the palmprint curve template used to generate control points in the present application, and the present application may also adopt various other palmprint curve templates.

[0065] According to an embodiment of the present application, determining multiple control points based on a predetermined palmprint curve template may include:

[0066] Based on the palmprint curve template, determining the area range for generating the plurality of control points;

[0067] Sampling is performed within the region to obtain the plurality of control points.

[0068] As an example, sampling methods may include random sampling, etc., to increase randomness and individual differences while maintaining the shape and style of the palm print, making the generated palm print image more realistic and diverse. Other sampling methods may also be used to achieve different data generation effects, and this application does not limit this.

[0069] Therefore, by generating palmprint curve templates based on statistical information obtained from real human palmprint lines, these palmprint curve templates can provide a more accurate generation range of control points, so that the Bezier curves generated based on these control points are closer to the distribution of real wrinkles, thereby reducing the difference between the generated palmprints and the real palmprints.

[0070] Next, in step S202, a palmprint line energy map with wrinkle information can be generated based on the Bezier curve. The palmprint line energy map includes palmprint lines with the same line distribution as the Bezier curve but different line types, wherein the line type is used to describe the wrinkle information and corresponds to the line direction energy of each pixel on the palmprint line.

[0071] Alternatively, as shown in FIG3 , in the first generation phase corresponding to step S202, the Bezier curve 303 can be converted to the PCE domain to generate a PCE image (i.e., a palm line energy map) 305. Both the Bezier curve 303 and the PCE image 305 are binary line-based images. The goal of the first generation phase is to convert the curved lines in the Bezier palm print image into a PCE image having wrinkles (i.e., palm lines) that are closer to real human palm prints.

[0072] Among them, wrinkles (i.e., palm lines) can refer to the deep and shallow concave and convex lines in the palm lines, which are usually formed due to the folding and bending of the skin. They are used to describe the overall shape of the palm lines, such as the main lines of the palm, curved edges, etc., and they play a role in segmenting and defining different areas in the palm lines. The shape and distribution of palm lines can be used in fields such as individual identity recognition.

[0073] Therefore, compared with the Bezier curve, the PCE image can include palm lines with the same line distribution (i.e., the position and direction of the main lines of the palm are consistent), but the palm lines in the PCE image have different line types (e.g., lines of different thickness and depth) to simulate the wrinkles of real human palm lines.

[0074] Optionally, the different line types used on the palm line may depend on the line energy characteristics of the pixels at the corresponding positions on the palm line, wherein the line energy characteristics of each pixel may be obtained by enhancing the line energy characteristics of the Bezier curve, which may describe the directional distribution of the line energy characteristics at the pixel.

[0075] Specifically, according to an embodiment of the present application, generating a palm line energy map based on the Bezier curve may include:

[0076] Performing feature extraction on the Bezier curve to obtain a multi-channel feature map of the Bezier curve;

[0077] Based on the multi-channel feature map, determining the line energy feature of each pixel in each channel feature map;

[0078] enhancing the line energy features of each pixel to generate an enhanced multi-channel feature map; and

[0079] The palmprint line energy map is generated based on the enhanced multi-channel feature map.

[0080] As an example, the Bezier curve has N channels. By processing the Bezier curve into a multidimensional matrix, the feature map of each channel can be obtained. For example, the feature map of the i-th channel can be represented as a three-dimensional matrix X i ∈R h×w×c,i=1,…,N. Then, the line energy feature enhancement can be performed on the feature map on each channel so that the wrinkle generation process focuses on the line energy feature.

[0081] Optionally, the line energy feature enhancement may include subtracting the average value of all features in the feature map on each channel to obtain high-frequency component features in these feature maps, and then, the line energy features may be extracted from these high-frequency component features.

[0082] According to an embodiment of the present application, determining the line energy feature of each pixel in each channel feature map based on the multi-channel feature map may include:

[0083] For each channel feature map, the following processing is performed:

[0084] Extracting line orientation energy of each pixel from the channel feature map using a linear convolution layer, wherein the linear convolution layer includes a Gaussian modified finite Radon transform kernel along a plurality of predetermined directions, and the line orientation energy includes line orientation energies of the pixel along the plurality of predetermined directions;

[0085] The line energy feature of each pixel is obtained according to the line direction energy, wherein the line energy feature includes the maximum line direction energy of the pixel in the plurality of predetermined directions and a predetermined direction corresponding to the maximum line direction energy.

[0086] For example, the linear convolution layer may include several Gaussian Modified Finite RAdon Transform (MFRAT) kernels along different directions (ie, the aforementioned multiple predetermined directions).

[0087] Then, the line energy feature of the feature map can be obtained according to the maximum response operation. Specifically, for each pixel in the feature map, the maximum line direction energy of the pixel in the plurality of predetermined directions and a predetermined direction corresponding to the maximum line direction energy are selected as the line energy feature of the pixel.

[0088] In palmprint images, palmprint lines usually have certain directional characteristics, and line direction energy can be used to describe the line direction information at different pixels in the palmprint image. For example, the line direction information at different pixels (or specific areas) can be quantified and converted into a set of numerical features to represent the line direction energy of the pixel.

[0089] In addition, considering that the size of the Gaussian-MFRAT kernel is usually large, in the embodiment of the present application, a dilated convolution based on the Gaussian-MFRAT kernel can be used to reduce the computational and time costs.

[0090] Finally, the line energy feature of the feature map can be multiplied by the preset learning parameter S, and the product is added to the original feature map to obtain a feature map with enhanced line energy features.

[0091] Alternatively, the Gaussian-MFRAT kernel can be improved based on the traditional MFRAT method. For example, specifically, FIG4B is a schematic diagram showing a comparison between a Gaussian-MFRAT kernel 420 according to an embodiment of the present application and a traditional MFRAT filter 410.

[0092] The traditional MFRAT method uses a linear filter with a constant value, which is sensitive to noise or small changes, while the Gaussian-MFRAT kernel in this application is calculated as follows:

[0093] Where (x,y)∈L(θ) represents the coordinates on the kernel, (x0,y0) represents the center point of the kernel, L(θ) represents the line with an angle θ defined on the two-dimensional image plane (as shown by the white line on the black background in Figure 4B), and σ is a hyperparameter. In addition, when When f(x, y) = 0. For example, in the embodiment of the present application, 12 Gaussian-MFRAT kernels may be designed, with a size of 31×31, θ ranging from 0° to 165°, and an interval of 15°.

[0094] As shown in Figure 4B, by adopting the Gaussian-MFRAT kernel 420 along different predetermined directions (for example, the 6 different predetermined directions shown by the lower left white line in Figure 4B), the line direction energy can be extracted from the feature map along these different predetermined directions, and the line energy feature of each pixel in the feature map includes the maximum line direction energy selected from these line direction energies along different predetermined directions.

[0095] 4B also shows filtering results 411 and 421 for line energy features generated based on the MFRAT filter and the Gaussian-MFRAT kernel. Compared with the filtering result 411 of the MFRAT filter, the line energy feature extraction result based on the Gaussian-MFRAT kernel (i.e., filtering result 421) is clearer and has less noise.

[0096] Therefore, the Gaussian-MFRAT kernel 420 can be used to replace the traditional MFRAT filter 410, avoiding the use of the inefficient response suppression denoising strategy in the traditional MFRAT filter 410, simplifying the enhancement operation of the line energy feature, and achieving the differentiability of the enhancement operation.

[0097] So, as an example, for the feature map X on the i-th channel in the multi-channel feature map X i , and its line energy characteristics are enhanced, which can be calculated as follows:

[0098] Among them, μ i Represents X i The mean value, f MAX Indicates the maximum response operation, represents the kth Gaussian-MFRAT kernel, and the total number of Gaussian-MFRAT kernels is N k , for example, set to 12, and s i represents the preset learning parameter of the i-th channel, which is used to adjust the feature enhancement degree of the i-th channel.

[0099] Therefore, in the first generation stage, a PCE image with realistic palm lines (ie, wrinkles) can be generated based on the Bezier curve, wherein the palm lines appear as different line types depending on the line energy characteristics of each pixel.

[0100] According to an embodiment of the present application, the line direction energy of each pixel in the palm line can include the line direction energy of the pixel in all directions. Optionally, the line direction energy of each pixel in all directions can be determined based on a feature-enhanced multi-channel feature map of a Bezier curve, and these line direction vectors can be represented using different line types to simulate real wrinkles.

[0101] According to an embodiment of the present application, generating a palm line energy map with wrinkle information based on the Bezier curve may include: generating a palm line energy map based on the Bezier curve using a pre-trained first generator.

[0102] Alternatively, as shown in FIG3 , a first generator 304G may be used B→P , convert the Bezier curve domain (B) to the PCE domain (P), G B→P The main structure of may include an encoder-decoder network based on residual blocks (RBs), and its specific structure and operation will be introduced below with reference to FIG6 .

[0103] After the PCE image with wrinkle information is generated in the PCE domain, in step S203 , a simulated palmprint image with detailed texture information may be generated based on the palmprint line energy map.

[0104] In this step, the simulated palmprint image has palmprint lines that are consistent with the palmprint line energy map and has detailed texture information.

[0105] Optionally, in the second generation stage, preserving wrinkle content can be decoupled from generating detail textures.

[0106] According to an embodiment of the present application, generating a simulated palmprint image based on the palmprint line energy map may include: using multiple control vectors to generate the simulated palmprint image with multiple detail texture information based on the palmprint line energy map.

[0107] The palm lines in the palm line energy map serve as identity information and are used to indicate the identity of the palm print image. That is, a realistic palm print image with detailed texture information can be generated while retaining the palm lines in the palm line energy map. Retaining the palm lines in the palm line energy map can be interpreted as maintaining consistent identity information, as palm lines can be used to indicate identity (ID).

[0108] According to an embodiment of the present application, detail texture information may be used to describe subtle texture features in a palm print. For example, the detail texture information may include one or more of light information, shadow information, and skin texture information.

[0109] Among them, skin texture information may include spots and fine textures on the skin, which are usually formed based on factors such as tiny details of the skin, arrangement and tissue structure of skin cells, and can be used to generate more realistic palm print images.

[0110] In addition to skin texture information, detail texture information can also include information related to the style of the palmprint image, such as light, shadow, etc. of the generated palmprint image, so as to achieve diversified palmprint image generation.

[0111] Optionally, the generation of detail texture information may be achieved using a control vector.

[0112] According to an embodiment of the present application, the control vector may be a random noise vector. As shown in FIG3 , a control vector 306 may be involved in the generation of a realistic palm print image 308 from a PCE image 305 . Optionally, the control vector may be a random noise vector (e.g., Gaussian white noise conforming to a standard normal distribution) to control the generation of diverse palm print images in the second generation phase.

[0113] It should be understood that, in addition to the above-mentioned random noise, other forms of control vectors can also be generated to generate diversified palmprints, and this application does not impose any limitation on this.

[0114] According to an embodiment of the present application, generating a realistic palmprint image with detailed texture information based on the palmprint line energy map may include: generating a realistic palmprint image based on the palmprint line energy map using a pre-trained second generator.

[0115] Optionally, as shown in FIG3 , a second generator 307G may be used P→R, the PCE image 305 of the PCE domain (P) is converted to the palmprint image domain (R), so as to use the PCE image 305 as the ID condition to generate realistic palmprints. In order to generate diverse palmprint images, a random noise vector can be used as a control vector 306 and input into G in the form of a latent vector. P→R to reproduce various detailed textures such as light, shadow, and skin texture.

[0116] According to an embodiment of the present application, the first generator and the second generator can be jointly trained using real palmprint images. Therefore, the joint training process of the model used in the palmprint image generation method of the present application will be introduced with reference to Figures 5 to 8.

[0117] FIG5 is a schematic diagram illustrating domain conversion in joint training according to an embodiment of the present application.

[0118] As shown in FIG5 , similar to the palmprint image generation process described above with reference to FIG2 to FIG4B , the joint training process of the present application also involves a conversion from the Bessel palmprint domain to the PCE domain in the first generation stage, and a conversion from the PCE domain to the palmprint image domain in the second generation stage.

[0119] In the first generation phase, PCE image samples 5031 are converted from Bezier curve samples 501. Since there is no supervision information, this conversion is an unpaired domain conversion 502.

[0120] In the second generation stage, a cycle conditional generation model is introduced. Using the real palmprint image 506 as supervision information, the PCEE 505 is used to extract the real PCE image 5032 from the real palmprint image 506. Then, the PCE image samples 5031 and 5032 are paired to generate a realistic palmprint image (not shown) in the palmprint image domain, thereby achieving the purpose of using real palmprint images for supervised learning.

[0121] According to an embodiment of the present application, in the joint training, the first generator can take Bezier curve samples as input and palmprint line energy map samples as output, wherein the palmprint line energy map samples include wrinkle information.

[0122] FIG6 is a schematic diagram illustrating the first generation stage in joint training according to an embodiment of the present application.

[0123] As shown in FIG6 (a), in the first generation stage of the joint training, similar to the description of the first generation stage above, the first generator G to be trained can be used based on the Bezier curve sample 601 in the Bezier palmprint domain. B→PPCE image samples 602 are generated in the PCE domain.

[0124] Figure 6(b) shows the first generator G B→P An exemplary structure, wherein the first generator G B→P The main structure can be an encoder-decoder network based on the residual block RB 610, and according to an embodiment of the present application, the first generator may include a line energy feature enhancement block (Line Feature Enhancement Block, LFEB) 620, which is used to enhance the line energy features of the multi-channel feature map of the Bezier curve sample 601.

[0125] Specifically, as shown in FIG6(b), the first generator G B→P Each layer of the encoder structure may include an LFEB 620, a convolution layer (e.g., Conv 7x7, Conv 3x3, etc.) and an activation layer (e.g., BN+ReLU), and each layer of the decoder structure may include a deconvolution layer (e.g., DeConv 3x3) and an activation layer (e.g., BN+ReLU, Tanh). The residual block RB 610 includes a convolution layer (e.g., Conv 3x3), an activation layer (e.g., BN+ReLU, BN), and a dropout layer (e.g., Dropout).

[0126] Specifically, the line energy feature enhancement block LFEB 620 may be used to implement the line energy feature enhancement process described above with reference to step S202, as shown in the above formula (2), which may include:

[0127] 621, for the feature map of each channel, subtract the average value of the feature map to obtain the high-frequency component features in the feature map;

[0128] 622, extracting line energy features from high-frequency component features through dilated convolution based on Gaussian-MFRAT kernel;

[0129] 623, obtaining the line energy features of the feature map from these line energy features through a maximum response operation;

[0130] Specifically, the maximum line energy feature among the extracted multiple line energy features is used as the line energy feature of the feature map;

[0131] 624 , multiplying the line energy feature of the feature map by a preset learning parameter S, and adding the product to the original feature map (i.e., the feature map before enhancement), thereby obtaining a feature map after the line energy feature is enhanced.

[0132] In the embodiment of the present application, the line energy feature enhancement block LFEB 620 is a lightweight plug-and-play block that can facilitate domain transmission and improve recognition performance.

[0133] In addition, as shown in FIG6( a ), in the first generation stage of the joint training, a real PCE image 604 is obtained based on a real palmprint image 603 to supervise the training of the PCE image sample 602 .

[0134] According to an embodiment of the present application, in the joint training, a palmprint line energy extractor (PCE Extractor, PCEE) is used to extract the corresponding real PCE image from the real palmprint image.

[0135] In an embodiment of the present application, a PCEE can be designed to extract a PCE image from a real palmprint image to generate supervisory information for joint training, making the generated palmprint image more realistic. Figure 7 is a schematic diagram of a palmprint line energy extractor PCEE according to an embodiment of the present application. As shown in Figure 7, similar to the description of the line energy feature enhancement block above, the palmprint line energy extractor PCEE of the present application may include a mean filter 710, a linear convolution layer 720, a maximum response operation layer 730, and an adaptive binarization layer 740. It takes a real palmprint image 700 as input and ultimately generates a binarized PCE image 750.

[0136] According to an embodiment of the present application, extracting a real palmprint line energy map from the real palmprint image using a palmprint line energy extractor may include:

[0137] Utilizing the palmprint line energy extractor, extracting the real palmprint line in the real palmprint image, and determining the line energy feature of each pixel in the real palmprint line;

[0138] Based on the real palm line and the line energy characteristics of each pixel in the real palm line, the real palm line energy map is obtained through binarization processing.

[0139] Optionally, as shown in FIG7 , a mean filter 710 is applied to subtract the filtering result from the original real palmprint image 700 to obtain the high-frequency component; and a linear convolution layer 720 is used to obtain the line energy feature in the real palmprint image 700 , that is, the line distribution of the real palmprint line. The linear convolution layer can be composed of several Gaussian-MFRAT kernels along different directions.

[0140] Optionally, through the maximum response operation 730, for each pixel in the real palmprint line, based on the line direction energy of the pixel in each direction corresponding to each Gaussian-MFRAT kernel, the maximum line direction energy is determined as the line energy feature of the pixel.

[0141] Optionally, the final PCE image can be obtained using an adaptive binarization process 740. For example, to highlight the main lines, a binarization threshold T can be set based on the top 10% of the line energy feature values ​​across the entire image.

[0142] According to an embodiment of the present application, in joint training, the second generator can take the real palmprint line energy map of the real palmprint image as input and the simulated palmprint image corresponding to the real palmprint image as output, and the simulated palmprint image includes detailed texture information.

[0143] FIG8 is a schematic diagram illustrating the second generation phase in joint training according to an embodiment of the present application.

[0144] As shown in FIG8 , the second generation phase of the joint training uses a real palmprint image 801 to participate in the training. Similar to the above, PCEE is also used to extract a real PCE image 802 from the real palmprint image 801. For example, a real PCE image f is generated from a real palmprint image A through PCEE. PCEE (A), then the real PCE image f PCEE (A) Input to the second generator G to be trained P→ R , and obtain the image f with the same PCE as the real one PCEE (A) Simulated palmprint image A with consistent palmprint lines * 803.

[0145] Optionally, in the second generation phase of the joint training, the encoder E can be used to map the input real palmprint image A to a matrix with a mean μ Q and variance The latent space Q(z|A) can obey the normal distribution Thus, a latent vector (ie, control vector) for controlling the generation of texture information is generated based on the latent space.

[0146] Optionally, the divergence of the latent vector can be constrained during the training phase, for example, the distribution of the latent space can be made Close to the standard normal distribution N(0,1), that is, the latent space is approximated to the standard normal space, so that the random noise z~N(0,1) can be easily sampled as a latent vector to generate diverse palmprint details.

[0147] In addition, considering that directly training with a small number of samples (few-shot training) may cause the discriminator to have an overfitting problem, in an embodiment of the present application, the generated simulated palmprint image 803 and the real palmprint image 801 can be expanded using a data augmentation module AUG before being fed to the discriminator D to generate more diverse training samples, thereby improving the generalization ability and robustness of the discriminator D. Even if only a small number of samples are used for training, the overfitting risk of the discriminator D can be reduced, making it better adapted to different palmprint image samples.

[0148] According to an embodiment of the present application, in the joint training, the generation process of the first generator and the second generator can be supervised based on the real PCE image, wherein the palm line energy extractor can be jointly trained with the first generator and the second generator.

[0149] Optionally, as shown in Figure 6(a), the real PCE image 604 can be used to supervise the generation of the PCE image sample 602. For example, the real PCE image 604 extracted from the real palmprint image 603 can be used as an adversarial sample to adopt an adversarial loss to drive the result generated by the first generator to be closer to the real PCE image 604, thereby optimizing the generation quality of the PCE image.

[0150] Optionally, as shown in FIG8 , a loop structure with PCEE can be used in the second generation phase of the joint training to generate the simulated palm print image A * Remap back to the PCE domain f PCEE (A * ), that is, the simulated PCE image 804 is obtained, and by minimizing the gap between the real PCE image 803 and the simulated PCE image 804, it is ensured that the simulated palm print image and the original real palm print image have consistent identity information. For example, by minimizing f PCEE (A) and the generated f PCEE (A * ) to constrain the generated realistic palmprint image A * distortion to strictly preserve the input f PCEE (A), and the discriminator D can be applied to enhance the generated realistic palmprint image A * authenticity.

[0151] According to an embodiment of the present application, the loss function of the joint training may include a first loss function associated with the first generator and a second loss function associated with the second generator;

[0152] The first loss function includes: a contrast loss for maintaining structural consistency between the palmprint line energy map sample and the Bezier curve sample, and an adversarial loss for making the palmprint line energy map sample similar to the real palmprint line energy map;

[0153] The second loss function includes: a distribution control loss for making the distribution of the control vector close to a standard normal distribution, an identity consistency loss between the identity of the simulated palmprint image and the identity of the real palmprint image, a distortion loss of the simulated palmprint image, and a loss for enhancing the authenticity of the simulated palmprint image.

[0154] As described above, the joint training of the present application may include joint training of the first generator, the second generator, and the PCEE, wherein the PCEE serves as a connection between the first generator and the second generator. Therefore, the loss function of the entire training process may include a loss function corresponding to the generation process of the first generator and the generation process of the second generator.

[0155] Optionally, as shown in FIG6( a ), the loss function corresponding to the generation process of the first generator may include a contrast loss L for maintaining structural consistency. CL , and the adversarial loss L for making the palm print image more realistic adv .

[0156] For example, as shown in Figure 6(a), in order to constrain the Bezier curve sample B and PCE image sample G B→P (B), we can use contrast loss. Specifically, we can get the structural consistency between B and G. B→P (B) Cut a series of tiles from both. B→ P The query image patch q in (B), the corresponding image patch at the same position in B can be used as the positive sample k of the query image patch q + , and other tiles in B can be used as negative samples of the query tile Therefore, the contrastive loss function can be constructed by making positive samples closer together and negative samples farther apart, for example, the contrastive loss L applying the normalized mutual information neural estimation (InfoNCE) loss function CL It can be expressed as follows:

[0157] where τ is the temperature hyperparameter.

[0158] Therefore, the loss function corresponding to the generation process of the first generator can be expressed as follows:

[0159] Among them, fPCEE represents the PCEE processing performed on image A, where A represents a random real palmprint image, and are two weights.

[0160] Optionally, as shown in FIG8 , the loss function corresponding to the generation process of the second generator may include a distribution control loss L for making the distribution of the control vector N(z) close to the standard normal distribution Q(z|A) KL , an identity consistency loss L for maintaining the identity consistency between the identity of the simulated palmprint image 803 generated by the second generator based on the real PCE image 802 and the identity of the real palmprint image 801 cyc , and a generation loss L for constraining the distortion of the simulated palmprint image 803 generated by the second generator and enhancing the authenticity G .

[0161] Optionally, the distribution control loss L KL It can be expressed as follows:

[0162] Optionally, the identity consistency loss L cyc It can be expressed as follows:

[0163] Optionally, generate the loss L G The distortion loss for the realistic palm print image 803 may be included and the loss of authenticity of the simulated palmprint image 803 for the discriminator Among them, T() represents the processing of the data enhancement module T. Therefore, the loss L is generated G It can be expressed as follows:

[0164] Therefore, the loss function corresponding to the generation process of the second generator can be expressed as follows:

[0165] in, Represents the weights of different loss terms.

[0166] Of course, the calculation method of the loss function given above is only used as an example and not as a limitation in this application. This application can also adopt other forms of loss functions.

[0167] Therefore, by optimizing the loss function of the entire training process, all parameters in the first generator, the second generator and the PCEE can be determined, and thus applied to the palmprint image generation process of this application.

[0168] Next, the performance verification of the palmprint image generation method of the present application will be presented with reference to FIG9 and FIG10 .

[0169] Optionally, in this performance verification, the same experimental dataset and open set evaluation scheme as used in the Bessel palmprint generation method and the RPG-Palm (Realistic Pseudo-data Generation-Palm) palmprint generation method can be followed. For example, the performance of recognition models pre-trained on palmprint images generated by various palmprint generation methods can be evaluated based on TAR and FAR, where TAR and FAR represent "True Acceptance Rate" and "False Acceptance Rate," respectively. That is, TAR represents the proportion of correctly identified samples that are correctly accepted, and can also be understood as the accuracy of the recognition model, while FAR represents the proportion of incorrectly identified samples that are falsely accepted, and can also be understood as the false recognition rate of the recognition model. Furthermore, the FID (Fréchet Inception Distance) metric can be used to evaluate the quality of the generated palmprint images.

[0170] Alternatively, the Bessel palmprint generation method and the RPG palmprint generation method can be followed. In this performance verification, 13 public datasets are used, which can come from various devices, with a total of 3,268 IDs and 59,162 images. Among them, the region of interest (ROI) can be extracted by following the detect-then-crop scheme.

[0171] Optionally, in this performance verification, 4000 identities can be generated according to the Bessel palmprint generation method and the RPG palmprint generation method, and each identity has 100 samples by default. For the first generation stage, and τ can be set to 1.0, 1.0, and 1.0, and the learning rate is 0.0002 in the first 30 training cycles and linearly decays to 1e-6 in the last 30 training cycles. For the second generation phase, and are set to 1.0, 10.0, 0.01, and 1.0, respectively, and the learning rate is 0.0002 in the first 50 training epochs and linearly decays to 1e-8 in the last 50 training epochs. During the joint training phase, the Adaptive Moment Estimation (Adam) optimizer parameters are set to (0.5, 0.99). The resolution of all images in the above training can be set to 256×256.

[0172] Furthermore, to achieve fair performance comparisons, the same recognition model backbone as the palmprint recognition model corresponding to the Bessel palmprint generation method, namely ResNet (Residual Network) 50 and MobileFaceNet (Mobile Face Recognition Network), can be used, with an input image resolution of 224×224. The recognition model is first pre-trained on synthetic data for 25 training cycles and then fine-tuned on the real dataset for 50 training cycles. The baseline model for comparison is trained on the real dataset for 50 training cycles. ArcFace (Arc Face Recognition Method) with a margin m = 0.5 and a scaling factor s = 48 is used for pre-training, fine-tuning, and baseline training supervision. The maximum and minimum learning rates for pre-training and fine-tuning can be set to 1e-2 and 1e-6, respectively. All recognition models can be trained using mini-batch stochastic gradient descent (SGD) with a batch size of 128.

[0173] Therefore, based on the above experimental settings, in this application, we first verify the performance of the recognition model under the open set protocol in which the training identity and the test identity are completely isolated. Alternatively, two different ratios of training ID and test ID can be used, such as 1:1 and 1:3 (for example, training:testing is 1634:1632 and 818:2448), and the quantitative results are shown in Table 1 below, where "MB" represents MobileFaceNet and "R50" represents ResNet50.

[0174] Table 1 Quantitative results under the open set protocol

[0175] As shown in Table 1, the method described in the embodiments of the present application can improve the RPG palmprint generation method by a clear margin and achieves the highest performance level in both the settings of 1:1 and 1:3 training ID to test ID ratios. In addition, the improvement of the method described in the embodiments of the present application in the setting of 1:3 training ID to test ID ratio is greater than that in the setting of 1:1 ratio, that is, the method described in the embodiments of the present application has significant effectiveness in the case of less real data.

[0176] In addition, to verify the performance of the method described in the embodiments of this application under limited training identities, models with different numbers of training identities (IDs) can be tested under an open set protocol with a 1:1 ratio of training ID to test ID. Specifically, a total of 4000 pseudo IDs were synthesized during this verification process, and each ID contained 100 pseudo palmprints. The same MobileFaceNet was used as the recognition model skeleton for different methods, and the quantitative results are shown in Table 2 below.

[0177] Table 2 Performance under different numbers of real training identities

[0178] As shown in Table 2, when trained with very few real IDs, ArcFace, Bessel palmprint generation, and RPG palmprint generation methods all become unusable, while the method described in the embodiments of the present application is still able to maintain performance. When trained with only 2.5% of the real IDs (i.e., 40), the method described in the embodiments of the present application is still better than the results of ArcFace trained with 100% of the real IDs (i.e., 1600). When trained with only 1% of the real IDs, the TAR of the method described in the embodiments of the present application is still comparable to the TAR of ArcFace trained with 50% of the real IDs (i.e., 800).

[0179] Alternatively, the palmprint generation performance can be compared using four generation methods: pix2pixHD, CycleGAN, BicycleGAN, and RPG-Palm, all of which are retrained using 40 real IDs and unpaired data from RPG-Palm. The quantitative results are shown in Table 3.

[0180] Table 3 Quantitative recognition results using different palmprint generation methods under the open set protocol

[0181] As shown in Table 3 above, the palmprint recognition model pre-trained based on the method described in the embodiment of the present application outperforms other methods. In addition, the method described in the embodiment of the present application can achieve an FID score of 40.3, showing a significant improvement.

[0182] Figure 9 is a comparison chart showing palmprint generation results using different palmprint generation methods according to an embodiment of the present application. (a) corresponds to the Bézier palm method, (b) corresponds to the pix2pixHD method, (c) corresponds to the CycleGAN method, (d) corresponds to the BicycleGAN method, (e) corresponds to the RPG-Palm method, (f) corresponds to the PCE image, and (g)-(j) correspond to the diverse palmprint images generated by the methods described in the embodiments of the present application.

[0183] As shown in Figure 9, with a small amount of training data, the palm print images generated by RPG-Palm exhibit severe blurring and inconsistent lines, while the method described in the embodiments of the present application still maintains overall clarity and ID consistency. In addition, the method described in the embodiments of the present application can recover better detailed information about the palm print lines, including variations in thickness and the interweaving of multiple wrinkles, rather than just migrating Bezier curves. In addition, the results of the pix2pixHD, CycleGAN, and BicycleGAN methods show more severe blurring problems than RPG-Palm.

[0184] In addition, in order to further enhance the line energy characteristics of the input palmprint, the palmprint image generation method of the present application can also apply the proposed LFEB to the palmprint recognition model, for example, incorporating a plug-and-play LFEB before the first convolutional layer of the skeleton of the palmprint recognition model to enhance the line energy characteristics of the input palmprint image.

[0185] This performance verification may also include a study of ablation. Among them, the main components of the method described in the embodiment of the present application may include PCEE, a data amplification (DA) module for few-sample training, an improved Bezier curve synthesis, a generation model with LFEB, and a recognition model with LFEB, which can be represented by "P", "A", "I", "G+L" and "R+L" respectively. For the baseline generation model, a two-stage training method can be used and the above components can be removed. Therefore, optionally, an ablation experiment can be trained using 40 IDs, and the test set is fixed under an open set protocol with a ratio of 1:1 between training ID and test ID. The results of the ablation experiment are shown in Table 4 below.

[0186] Table 4 Ablation of different components

[0187] As shown in Table 4, the model with PCEE achieves the largest performance improvement at 13.81% @ FAR = 1e-6 (i.e., TAR is 13.81% when FAR = 1e-6), reflecting the superiority of the model with PCEE in generating realistic palmprint samples with limited data. Compared with the baseline generation and recognition models, the LFEB module brings a significant and consistent performance gain of 6% @ FAR = 1e-6 by enhancing the palmprint line energy features. The DA module achieves an improvement of 5.52% @ FAR = 1e-6 by effectively expanding the intra-class diversity with a small number of training samples. In addition, the improvement to the Bezier curve also achieves better performance by introducing a more reasonable palmprint line distribution.

[0188] Finally, the performance verification may also include the effectiveness verification of the LFEB. FIG10 is a diagram showing the results of the effectiveness verification of the line energy feature enhancement block according to an embodiment of the present application.

[0189] Figure 10 visualizes the features of the first block in the middle layer of MobileFaceNet with and without LFEB, where Figure 10(a) is the input palmprint image, Figure 10(b) is the palmprint image after LFEB, Figure 10(c) corresponds to the feature visualization without LFEB, and Figure 10(d) corresponds to the feature visualization with LFEB. Therefore, it can be seen that the model focuses on both palmprint lines and non-line areas without LFEB, while by adding the LFEB component, the model can be biased to focus on palmprint lines.

[0190] As described above, through the palmprint image generation method of the present application, a Bezier curve can be generated using control points determined from a predetermined palmprint curve template, and the Bezier curve can be converted into a palmprint line energy map with wrinkle information, wherein the wrinkle information includes the same line distribution as the Bezier curve but a different line type, and its line type is determined by the line energy characteristics of each pixel. Then, a palmprint image with texture information is further generated based on the palmprint line energy map with the wrinkle information, and the palmprint lines of the palmprint image are consistent with the wrinkle information of the palmprint line energy map, thereby generating a realistic palmprint image with realistic wrinkles and texture. Among them, by introducing the palmprint line energy domain as the intermediate domain connecting the Bezier palmprint domain and the palmprint image domain, the direct generation of palmprint images with wrinkle information and texture information from the Bezier curve is avoided, which reduces the difficulty of generating palmprint images. In the process of generating palmprint images from the palmprint line energy map, while ensuring that the palmprint image has consistent palmprint lines, detailed texture information is generated, so that realistic palmprint images with diverse textures can be generated while retaining the same identity information. Therefore, the dependence on real data is reduced, which is suitable for palmprint recognition training in the absence of large-scale palmprint datasets.

[0191] FIG11 is a schematic diagram illustrating a palmprint image generating device 1100 according to an embodiment of the present application.

[0192] According to an embodiment of the present application, the palmprint image generating device 1100 may include a curve generating module 1101 , a wrinkle generating module 1102 , and a texture generating module 1103 .

[0193] The curve generation module 1101 may be configured to determine a plurality of control points based on a predetermined palmprint curve template, and generate a Bezier curve based on the plurality of control points. Optionally, the curve generation module 1101 may perform the operations described above with reference to step S201.

[0194] Wrinkle generation module 1102 can be configured to generate a palm line energy map containing wrinkle information based on the Bezier curve. The palm line energy map includes palm lines having the same line distribution as the Bezier curve but with different line types. The line types are used to describe the wrinkle information and correspond to the line direction energy of each pixel on the palm line. Optionally, wrinkle generation module 1102 can perform the operations described above with reference to step S202.

[0195] The texture generation module 1103 may be configured to generate a realistic palmprint image with detailed texture information based on the palmprint line energy map. Optionally, the texture generation module 1103 may perform the operations described above with reference to step S203.

[0196] Optionally, the wrinkle generation module 1102 is configured to perform feature extraction on the Bezier curve to obtain a multi-channel feature map of the Bezier curve; determine the line energy features of each pixel in each channel feature map based on the multi-channel feature map; enhance the line energy features of each pixel to generate an enhanced multi-channel feature map; and generate the palm print line energy map based on the enhanced multi-channel feature map.

[0197] Optionally, the wrinkle generation module 1102 is configured to perform the following processing for each channel feature map:

[0198] Extracting line direction energy of each pixel from the channel feature map using a linear convolution layer, wherein the linear convolution layer includes a Gaussian modified finite Radon transform kernel along multiple predetermined directions, and the line direction energy includes line direction energy of the pixel along the multiple predetermined directions;

[0199] The line energy feature of each pixel is obtained according to the line direction energy, wherein the line energy feature includes the maximum line direction energy of the pixel in the plurality of predetermined directions and a predetermined direction corresponding to the maximum line direction energy.

[0200] The texture generation module 1103 is configured to use multiple control vectors to generate the simulated palmprint image with multiple detail texture information based on the palmprint line energy map.

[0201] Optionally, the palm lines in the palm line energy map are used as identity information to indicate the identity of the simulated palm print image.

[0202] Optionally, the control vector is a random noise vector, and the detail texture information includes one or more of light information, shadow information and skin texture information.

[0203] Optionally, the wrinkle generation module 1102 is configured to generate the palm line energy map based on the Bezier curve using a pre-trained first generator;

[0204] The texture generation module 1103 is configured to generate the realistic palmprint image based on the palmprint line energy map using a pre-trained second generator;

[0205] The first generator and the second generator are jointly trained using real palmprint images.

[0206] Optionally, in the joint training,

[0207] The first generator takes a Bezier curve sample as input and a palm line energy map sample as output, wherein the palm line energy map sample includes wrinkle information;

[0208] The second generator takes a real palmprint line energy map of a real palmprint image as input and outputs a simulated palmprint image corresponding to the real palmprint image, wherein the simulated palmprint image includes detail texture information;

[0209] extracting the real palmprint line energy map from the real palmprint image using a palmprint line energy extractor;

[0210] Supervising the first generator and the second generator based on the real palmprint line energy map;

[0211] The palm line energy extractor is jointly trained with the first generator and the second generator.

[0212] Optionally, the extracting the real palmprint line energy map from the real palmprint image using a palmprint line energy extractor includes:

[0213] Utilizing the palmprint line energy extractor, extracting the real palmprint line in the real palmprint image, and determining the line energy feature of each pixel in the real palmprint line;

[0214] Based on the real palm line and the line energy characteristics of each pixel in the real palm line, the real palm line energy map is obtained through binarization processing.

[0215] Optionally, the first generator includes a line energy feature enhancement block for enhancing the line energy features of the multi-channel feature map of the Bezier curve sample.

[0216] Optionally, the loss function of the joint training includes a first loss function associated with the first generator and a second loss function associated with the second generator;

[0217] The first loss function includes: a contrast loss for maintaining structural consistency between the palmprint line energy map sample and the Bezier curve sample, and an adversarial loss for making the palmprint line energy map sample similar to the real palmprint line energy map;

[0218] The second loss function includes: a distribution control loss for making the distribution of the control vector close to a standard normal distribution, an identity consistency loss between the identity of the simulated palmprint image and the identity of the real palmprint image, a distortion loss of the simulated palmprint image, and a loss for enhancing the authenticity of the simulated palmprint image.

[0219] Optionally, the palmprint curve template is predetermined based on statistical information obtained from real human palmprint lines;

[0220] The curve generating module 1101 is configured to determine a region for generating the plurality of control points based on the palmprint curve template; and perform sampling within the region to obtain the plurality of control points.

[0221] According to another aspect of the present application, an electronic device is also provided. FIG12 shows a schematic diagram of an electronic device 2000 according to an embodiment of the present application.

[0222] As shown in FIG12 , the electronic device 2000 may include one or more processors 2010 and one or more memories 2020. The memory 2020 may store computer-readable codes, which, when executed by the one or more processors 2010, may execute the palmprint image generation method described above.

[0223] The processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can be an X86 architecture or an ARM architecture.

[0224] In general, the various example embodiments of the present application can be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects can be implemented in hardware, while other aspects can be implemented in firmware or software that can be executed by a controller, microprocessor or other computing device. When various aspects of the embodiments of the present application are illustrated or described as block diagrams, flow charts or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controller or other computing device, or some combination thereof.

[0225] For example, the method or apparatus according to the embodiments of the present application can also be implemented with the aid of the architecture of the computing device 3000 shown in FIG13 . As shown in FIG13 , the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, and the like. The storage device in the computing device 3000, such as the ROM 3030 or the hard disk 3070, can store various data or files used for processing and / or communication of the palmprint image generation method provided in the present application, as well as program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, the architecture shown in FIG13 is merely exemplary. When implementing different devices, one or more components in the computing device shown in FIG13 may be omitted according to actual needs.

[0226] According to another aspect of the present application, a computer-readable storage medium is also provided. The computer storage medium has computer-readable instructions stored thereon. When the computer-readable instructions are executed by a processor, the palm print image generation method according to the embodiment of the present application described with reference to the above figures can be executed. The computer-readable storage medium in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0227] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the palmprint image generation method according to the present application.

[0228] Embodiments of the present application provide a palmprint image generation method, apparatus, device, and computer-readable storage medium.

[0229] Compared with traditional palmprint generation methods, the method provided in the embodiments of the present application can decouple the generation of palmprint wrinkles and textures by introducing the palmprint line energy domain, so as to generate realistic wrinkles by converting the Bezier curve in the Bezier palmprint domain into a palmprint line energy map in the palmprint line energy domain, and generate realistic textures by converting the palmprint line energy map in the palmprint line energy domain into a palmprint image in the palmprint image domain, thereby reducing the difficulty of generating realistic palmprint images from Bezier curves and realizing diversified realistic palmprint image generation.

[0230] The method provided in the embodiment of the present application uses control points determined from a predetermined palmprint curve template to generate a Bezier curve, and converts the Bezier curve into a palmprint line energy map with wrinkle information, wherein the wrinkle information includes the same line distribution as the Bezier curve but a different line type, and its line type is determined by the line energy characteristics of each pixel. Then, based on the palmprint line energy map with the wrinkle information, a palmprint image with texture information is generated. The palmprint lines of the palmprint image are consistent with the wrinkle information of the palmprint line energy map, thereby generating a realistic palmprint image with realistic wrinkles and texture. The method provided in the embodiment of the present application introduces the palmprint line energy domain as an intermediate domain connecting the Bezier palmprint domain and the palmprint image domain, thereby avoiding directly generating a palmprint image with wrinkle information and texture information from the Bezier curve, thereby reducing the difficulty of generating the palmprint image. In addition, in the process of generating the palmprint image from the palmprint line energy map, detailed texture information is generated while ensuring that the palmprint image has consistent palmprint lines. This makes it possible to generate realistic palmprint images with diverse textures while retaining the same identity information. Therefore, the dependence on real data is reduced, and the method is suitable for palmprint recognition training in the absence of large-scale palmprint datasets.

[0231] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of the code, and the module, program segment, or a part of the code contains at least one executable instruction for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0232] In general, the various example embodiments of the present application can be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects can be implemented in hardware, while other aspects can be implemented in firmware or software that can be executed by a controller, microprocessor or other computing device. When various aspects of the embodiments of the present application are illustrated or described as block diagrams, flow charts or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controller or other computing device, or some combination thereof.

[0233] The exemplary embodiments of the present application described in detail above are merely illustrative and not restrictive. Those skilled in the art should understand that various modifications and combinations of these embodiments or their features may be made without departing from the principles and spirit of the present application, and such modifications should fall within the scope of the present application.

Claims

1. A palmprint image generation method, executed by an electronic device, comprising: Based on a predetermined palmprint curve template, a plurality of control points are determined, and a Bezier curve is generated based on the plurality of control points; Based on the Bezier curve, a palm line energy map with wrinkle information is generated, wherein the palm line energy map includes palm lines having the same line distribution as the Bezier curve but different line types, wherein the line type is used to describe the wrinkle information and corresponds to the line direction energy of each pixel on the palm line; and, Based on the palmprint line energy map, a simulated palmprint image with detailed texture information is generated.

2. The method of claim 1, wherein: The step of generating a palm line energy map with wrinkle information based on the Bezier curve comprises: Performing feature extraction on the Bezier curve to obtain a multi-channel feature map of the Bezier curve; Based on the multi-channel feature map, determining the line energy feature of each pixel in each channel feature map; enhancing the line energy feature of each pixel to generate an enhanced multi-channel feature map; and The palm line energy map is generated based on the enhanced multi-channel feature map.

3. The method of claim 2, wherein: The step of determining the line energy feature of each pixel in each channel feature map based on the multi-channel feature map includes: For each channel feature map, the following processing is performed: Extracting the line direction energy of each pixel from the channel feature map using a linear convolution layer, wherein the linear convolution layer includes a Gaussian modified finite Radon transform kernel along a plurality of predetermined directions, and the line direction energy includes the line direction energy of the pixel along the plurality of predetermined directions; The line energy feature of each pixel is obtained according to the line direction energy, and the line energy feature includes the maximum line direction energy of the pixel in the plurality of predetermined directions and a predetermined direction corresponding to the maximum line direction energy.

4. The method according to any one of claims 1 to 3, wherein: The step of generating a simulated palmprint image with detailed texture information based on the palmprint line energy map comprises: The simulated palm print image with multiple detail texture information is generated based on the palm print line energy map using multiple control vectors.

5. The method of claim 4, wherein: The palm lines in the palm line energy map are used as identity information to indicate the identity of the simulated palm line image.

6. The method according to claim 4 or 5, wherein: The control vector is a random noise vector, and the detail texture information includes one or more of light information, shadow information and skin texture information.

7. The method according to any one of claims 4 to 6, wherein: The step of generating a palm line energy map with wrinkle information based on the Bezier curve comprises: Based on the Bezier curve, using a pre-trained first generator, generating the palm line energy map; The step of generating a simulated palmprint image with detailed texture information based on the palmprint line energy map comprises: Based on the palmprint line energy map, using a pre-trained second generator, generating the simulated palmprint image; Wherein, the first generator and the second generator are jointly trained using real palmprint images.

8. The method of claim 7, wherein: In the joint training, The first generator takes a Bezier curve sample as input and a palm line energy map sample as output, wherein the palm line energy map sample includes wrinkle information; The second generator takes the real palmprint line energy map of the real palmprint image as input, and takes the simulated palmprint image corresponding to the real palmprint image as output, wherein the simulated palmprint image includes detail texture information; Using a palmprint line energy extractor, extracting the real palmprint line energy map from the real palmprint image; Based on the real palmprint line energy map, supervising the first generator and the second generator; and, The palm line energy extractor is jointly trained with the first generator and the second generator.

9. The method of claim 8, wherein: The method of extracting the real palmprint line energy map from the real palmprint image by using a palmprint line energy extractor comprises: Utilizing the palmprint line energy extractor, extracting the real palmprint line in the real palmprint image, and determining the line energy feature of each pixel in the real palmprint line; Based on the real palm line and the line energy characteristics of each pixel in the real palm line, the real palm line energy map is obtained through binarization processing.

10. The method according to any one of claims 7 to 9, wherein: The first generator comprises a line energy feature enhancement block for enhancing the line energy feature of the multi-channel feature map of the Bezier curve sample.

11. The method according to any one of claims 8 to 10, wherein: The loss function of the joint training includes a first loss function associated with the first generator and a second loss function associated with the second generator; The first loss function includes: a contrast loss for maintaining the structural consistency between the palm line energy map sample and the Bezier curve sample, and an adversarial loss for making the palm line energy map sample similar to the real palm line energy map; The second loss function includes: a distribution control loss for making the distribution of the control vector close to a standard normal distribution, an identity consistency loss between the identity of the simulated palmprint image and the identity of the real palmprint image, a distortion loss of the simulated palmprint image, and a loss for strengthening the authenticity of the simulated palmprint image.

12. The method according to any one of claims 1 to 11, wherein: The palm print curve template is predetermined based on statistical information obtained from real human palm print lines; Wherein, the determining of multiple control points based on a predetermined palmprint curve template includes: Based on the palmprint curve template, determining the area range for generating the plurality of control points; Sampling is performed within the region to obtain the plurality of control points.

13. A palmprint image generating device, comprising: A curve generating module is configured to determine a plurality of control points based on a predetermined palmprint curve template, and generate a Bezier curve based on the plurality of control points; A wrinkle generation module is configured to generate a palm line energy map with wrinkle information based on the Bezier curve, wherein the palm line energy map includes palm lines having the same line distribution as the Bezier curve but different line types, wherein the line type is used to describe the wrinkle information and corresponds to the line direction energy of each pixel on the palm line; and The texture generation module is configured to generate a simulated palm print image with detailed texture information based on the palm print line energy map.

14. The device of claim 13, wherein: The wrinkle generation module is configured to perform feature extraction on the Bezier curve to obtain a multi-channel feature map of the Bezier curve; based on the multi-channel feature map, determine the line energy feature of each pixel in each channel feature map; Enhance the line energy features of each pixel to generate an enhanced multi-channel feature map; And, based on the enhanced multi-channel feature map, the palm line energy map is generated.

15. The device according to claim 13 or 14, wherein: The texture generation module is configured to generate the simulated palm print image with multiple detail texture information based on the palm print line energy map using multiple control vectors.

16. The device of claim 13, wherein: The wrinkle generation module is configured to generate the palm line energy map based on the Bezier curve using a pre-trained first generator; The texture generation module is configured to generate the simulated palm print image based on the palm print line energy map using a pre-trained second generator; Wherein, the first generator and the second generator are jointly trained using real palmprint images.

17. The device according to any one of claims 13 to 16, wherein: The palm print curve template is predetermined based on statistical information obtained from real human palm print lines; The curve generating module is configured to determine the region range for generating the plurality of control points based on the palmprint curve template; and perform sampling within the region range to obtain the plurality of control points.

18. An electronic device comprising: one or more processors; as well as One or more memories, wherein a computer executable program is stored, and when the computer executable program is executed by the processor, the method of any one of claims 1-12 is performed.

19. A computer program product, stored on a computer-readable storage medium, and comprising computer instructions, which, when executed by a processor, cause a computer device to perform the method of any one of claims 1 to 12.

20. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the instructions are used to implement the method according to any one of claims 1 to 12 when executed by a processor.

Images