Controllable palmprint sample generation method based on stochastic differential equation

Abstract

The embodiment of the disclosure provides a controllable palmprint sample generation method based on a stochastic differential equation; it relates to the field of computer vision. The method comprises constructing an unconditional diffusion model network architecture and defining a forward process and a reverse process; training an unconditional diffusion model according to the forward process and a palmprint image to obtain a noise image; according to the reverse process, the trained unconditional diffusion model is used for sampling the noise image to generate an unconditional palmprint sample; a conditional encoder is introduced into the trained unconditional diffusion model; the weights of the unconditional diffusion model are frozen, and the conditional encoder is trained according to a conditional image; when the conditional encoder loss function converges, the frozen weights are unlocked, and the unconditional diffusion model introduced with the conditional encoder is trained according to the forward process, the palmprint image and the conditional image; according to the reverse process, the trained unconditional diffusion model with the conditional encoder is used for sampling to generate a conditional palmprint sample. Thus, the palmprint sample authenticity, diversity and controllability are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer vision, and in particular to a controllable palmprint sample generation method based on stochastic differential equation. BACKGROUND

[0002] With the rapid development of science and technology, biometric recognition technology has gradually become a key means of identity verification and security. Among many biometrics, palmprints have become an important basis for identity recognition due to their unique texture and details. However, due to the scarcity of public palmprint datasets, the development of palmprint recognition technology has been greatly limited, and therefore palmprint sample generation methods have emerged.

[0003] Existing palmprint sample generation methods include a palmprint sample generation method based on Bezier curves, a palmprint sample generation method based on generative adversarial networks, and a palmprint sample generation method based on diffusion models. The palmprint samples generated by the palmprint sample generation method based on Bezier curves have a large gap with real palmprint samples. The palmprint sample generation method based on generative adversarial networks has problems such as mode collapse and unstable training, and it is still difficult to generate palmprint samples with diversity. Although the palmprint samples generated by the palmprint sample generation method based on diffusion models are more realistic, the generated palmprint sample images are prone to have abnormal color backgrounds, excessive noise points, missing details, and other defects. Moreover, the palmprint sample generation method cannot accurately control the specific attributes or features of the palmprint samples, which greatly limits the diversity and practicality of the palmprint samples. Therefore, it is of great significance to research a palmprint sample generation method that can avoid the above shortcomings. SUMMARY

[0004] The present disclosure provides a controllable palmprint sample generation method based on stochastic differential equation, a device, an apparatus, and a storage medium.

[0005] According to a first aspect of the present disclosure, a controllable palmprint sample generation method based on stochastic differential equation is provided. The method comprises:

[0006] constructing an unconditional diffusion model network architecture based on a Unet structure;

[0007] defining a forward process and a reverse sampling process based on stochastic differential equation;

[0008] taking palmprint images in a palmprint database as first training samples, and training an unconditional diffusion model based on the network architecture according to the defined forward process to obtain a noise image;

[0009] generating an unconditional palmprint sample by reverse sampling the noise image using the trained unconditional diffusion model according to the defined reverse sampling process;

[0010] introducing a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to a decoder in the network architecture through a zero convolution layer;

[0011] freezing the weights of the unconditional diffusion model, training the conditional control encoder according to the conditional control images in the conditional control dataset, so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through the zero convolution layer;

[0012] when the loss function corresponding to the conditional control encoder converges, unlocking the weights of the unconditional diffusion model, taking the palmprint image and the conditional control image as the second training sample, training the unconditional diffusion model introduced with the conditional control encoder according to the defined forward process;

[0013] generating the conditional palmprint sample by using the trained unconditional diffusion model introduced with the conditional control encoder according to the defined backward sampling process.

[0014] In some implementable manners of the first aspect, the definition of the forward process is as follows:

[0015] ;

[0016] wherein, , ; denotes the change amount of the palmprint image state over time , is a function about the palmprint image state and time , denotes the time change amount, denotes a noise function related to time, denotes a random noise term.

[0017] In some implementable manners of the first aspect, the training of the unconditional diffusion model according to the defined forward process to obtain the noise image comprises:

[0018] adding noise to the palmprint image in the palmprint database by using an image noise adding formula to generate a corresponding noise image; wherein the image noise adding formula is:

[0019] ;

[0020] denotes the palmprint image at time t, the noise image after adding random noise, denotes a random noise variable subject to a Gaussian distribution, denotes a Gaussian distribution.

[0021] In some implementable manners of the first aspect, the definition of the reverse sampling process is as follows:

[0022] ;

[0023] wherein, is a score function, denotes a noise image relative to a standard deviation of a palmprint image .

[0024] In some implementable manners of the first aspect, the noise image is reverse sampled according to the defined reverse sampling process by using the trained unconditional diffusion model to generate an unconditional palmprint sample, comprising:

[0025] obtaining an initial solution of the reverse sampling process by using a numerical method according to the defined reverse sampling process;

[0026] substituting the noise image at the i-th sampling point into the denoising function in the initial solution to calculate the noise value of the noise image at the i-th sampling point; generating the corresponding unconditional palmprint sample of the noise image at the i-th sampling point according to the noise value of the noise image at the i-th sampling point;

[0027] repeating the generation of the corresponding unconditional palmprint sample of the noise image at the next sampling point until the sampling time step is reached; calculating the weight of the unconditional diffusion model according to the standard deviation of the noise image relative to the corresponding palmprint image;

[0028] calculating the loss function according to the denoising function and the weight of the unconditional diffusion model;

[0029] evaluating the gap between the generated unconditional palmprint sample and the corresponding real palmprint image by using the loss function, and training the loss function by using the Adam optimizer to perform the gradient descent strategy until the loss function converges.

[0030]

[0031]

[0032] In some implementable manners of the first aspect, the initial solution is represented as:

[0033]

[0034] wherein, is a denoising function.

[0035] In some implementable manners of the first aspect, the denoising function is represented as:

[0036] ;​​​​

[0037] wherein, is a residual scaling factor, is an output transform scaling factor, is an input transform scaling factor, is a noise level factor, is a function of the input transform scaling factor and the noise level factor.

[0038] According to a second aspect of the present disclosure, a controllable palmprint sample generation device based on a stochastic differential equation is provided. The device comprises:

[0039] a network architecture construction module configured to construct an unconditional diffusion model network architecture based on a Unet structure;

[0040] a forward / backward process definition module configured to define a forward process and a backward sampling process based on a stochastic differential equation;

[0041] an unconditional diffusion model training module configured to take palmprint images in a palmprint database as first training samples, and train an unconditional diffusion model based on the network architecture according to the defined forward process to obtain noise images;

[0042] an unconditional palmprint sample generation module configured to generate unconditional palmprint samples by using the trained unconditional diffusion model to perform backward sampling on the noise images according to the defined backward sampling process;

[0043] a conditional control encoder introduction module configured to introduce a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to a decoder in the network architecture through a zero convolution layer;

[0044] a conditional control encoder training module configured to freeze the weights of the unconditional diffusion model, and train the conditional control encoder according to conditional control images in a conditional control dataset, so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through the zero convolution layer;

[0045] a model joint training module configured to, when a loss function corresponding to the conditional control encoder converges, unlock the weights of the unconditional diffusion model, take the palmprint images and the conditional control images as second training samples, and train the unconditional diffusion model introduced with the conditional control encoder according to the defined forward process;

[0046] a conditional palmprint sample generation module configured to generate conditional palmprint samples by using the trained unconditional diffusion model introduced with the conditional control encoder to perform backward sampling according to the defined backward sampling process.

[0047] According to a third aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processor, and a memory connected with the at least one processor in communication; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0048] According to a fourth aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, the computer instructions being used to cause a computer to perform the method as described above.

[0049] In the present disclosure, an unconditional diffusion model network architecture is constructed based on a Unet structure; a forward process and a reverse sampling process are defined based on a stochastic differential equation; palmprint images in a palmprint database are taken as first training samples, and an unconditional diffusion model is trained based on the network architecture to obtain a noise image according to the defined forward process; the trained unconditional diffusion model is used to perform reverse sampling on the noise image according to the defined reverse sampling process to generate an unconditional palmprint sample; a conditional control encoder is introduced into the trained unconditional diffusion model; the conditional control encoder is connected to a decoder in the network architecture through a zero convolution layer; the weights of the unconditional diffusion model are frozen, and the conditional control encoder is trained according to conditional control images in a conditional control dataset, so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through the zero convolution layer; when the loss function corresponding to the conditional control encoder converges, the weights of the unconditional diffusion model are unlocked, the palmprint images and the conditional control images are taken as second training samples, and the unconditional diffusion model with the introduced conditional control encoder is trained according to the defined forward process; the trained unconditional diffusion model with the introduced conditional control encoder is used to perform reverse sampling according to the defined reverse sampling process to generate a conditional palmprint sample. In this way, the authenticity and the variety of the generated palmprint samples can be improved, and the generation of the palmprint samples is more controllable by introducing the conditional control encoder.

[0050] It should be understood that the content described in the summary section is not intended to limit or define key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0051] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail the following embodiments with reference to the attached drawings. The attached drawings are intended to better understand the present disclosure and do not limit the present disclosure. In the drawings, the same or similar elements are denoted by the same or similar reference numerals, and:

[0052] Figure 1A flow chart of a controllable palmprint sample generation method based on a stochastic differential equation is shown.

[0053] Figure 2 An unconditional control reverse sampling result map provided by the embodiments of the present disclosure is shown.

[0054] Figure 3 A conditional control reverse sampling result map provided by the embodiments of the present disclosure is shown.

[0055] Figure 4 A structure diagram of a controllable palmprint sample generation device based on a stochastic differential equation provided by the embodiments of the present disclosure is shown.

[0056] Figure 5 A structure diagram of an exemplary electronic device capable of implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0057] To make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present disclosure.

[0058] In addition, the term “and / or” in this document is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character “ / ” in this document generally represents that the front and rear associated objects are in an “or” relationship.

[0059] To solve the problems in the background art, the embodiment of the present disclosure provides a controllable palmprint sample generation method and device based on stochastic differential equation. Specifically, an unconditional diffusion model network architecture is constructed based on Unet structure; a forward process and a reverse sampling process are defined based on stochastic differential equation; palmprint images in a palmprint database are taken as first training samples, and an unconditional diffusion model is trained based on the network architecture to obtain a noise image according to the defined forward process; the trained unconditional diffusion model is used to perform reverse sampling on the noise image according to the defined reverse sampling process to generate an unconditional palmprint sample; a conditional control encoder is introduced into the trained unconditional diffusion model; the conditional control encoder is connected to a decoder in the network architecture through a zero convolution layer; the weights of the unconditional diffusion model are frozen, and the conditional control encoder is trained according to conditional control images in a conditional control dataset, so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through the zero convolution layer; when the loss function corresponding to the conditional control encoder converges, the weights of the unconditional diffusion model are unlocked, the palmprint images and the conditional control images are taken as second training samples, and the unconditional diffusion model with the introduced conditional control encoder is trained according to the defined forward process; the trained unconditional diffusion model with the introduced conditional control encoder is used to perform reverse sampling according to the defined reverse sampling process to generate a conditional palmprint sample. In this way, the authenticity and the variety of the generated palmprint samples can be improved, and the generation of the palmprint samples is more controllable by introducing the conditional control encoder.

[0060] The controllable palmprint sample generation method and device based on stochastic differential equation provided by the embodiment of the present disclosure will be described in detail below with reference to the specific embodiments and the accompanying drawings.

[0061] Figure 1 A flowchart of a controllable palmprint sample generation method based on stochastic differential equation provided by the embodiment of the present disclosure is shown, and the method 100 includes the following steps:

[0062] S110, an unconditional diffusion model network architecture is constructed based on Unet structure.

[0063] In some embodiments, the unconditional diffusion model network architecture includes an encoder and a decoder.

[0064] In some embodiments, the encoder extracts high-level semantic features of the palmprint image by gradually reducing the resolution of the palmprint image in the palmprint database, and then converts the palmprint image into a noise image;

[0065] Furthermore, the encoder includes 5 blocks, each of which includes 4 encoder blocks. When the image passes through each block, the number of channels of the feature map remains unchanged, and the length and width are doubled to ensure the consistency and stability of the information. Specifically, the number of channels of each block is set to [128, 128, 256, 256, 256], which can effectively extract and retain the key features of the palmprint image.

[0066] In some embodiments, the decoder outputs a palmprint sample image with the same resolution as the palmprint image in the palmprint database by gradually upsampling the low-resolution image features and fusing them with the features of the encoder;

[0067] Furthermore, the decoder also includes 5 blocks, and the input of each block is spliced ​​to achieve information fusion and complementarity. This not only makes full use of the feature information extracted in the encoding stage, but also enables more accurate and detailed palmprint sample reconstruction during the decoding process.

[0068] S120, defining a forward process and a reverse sampling process based on a stochastic differential equation.

[0069] In some embodiments, the forward process is defined as follows:

[0070] ;

[0071] in, , ; Represents the state of the palmprint image over time The amount of change, Regarding the palm print image status and time function, represents the time variation, represents the time-dependent noise function, represents the random noise term.

[0072] In some embodiments, according to the defined forward process, the marginal distribution of the forward process is obtained as:

[0073] ;

[0074] express Palm print image at any moment Noise image after adding random noise, is the marginal distribution of the forward process, is the identity matrix, Represents a noisy image Relative to palm print image a standard deviation of denotes a Gaussian distribution with mean and variance

[0075] According to the edge distribution of the forward process, the image plus noise formula can be further obtained as:

[0076] ;

[0077] denotes a random noise variable subject to a Gaussian distribution, denotes a Gaussian distribution.

[0078] In some embodiments, the definition of the backward sampling process is as follows:

[0079] ;

[0080] wherein, is a score function, denotes a noise image with respect to a standard deviation of a palmprint image ;

[0081] Further, a denoising function can be used for estimation, and the estimation formula is as follows:

[0082] ;

[0083] wherein, the denoising function can be calculated by the following formula:

[0084] ;

[0085] ;

[0086] is a residual scaling factor, is an output transform scaling factor, is an input transform scaling factor, is a noise level factor, is a function about the input transform scaling factor and the noise level factor.

[0087] In some embodiments, the backward sampling process adopts skip sampling, and the sampling step is only performed for tens of times, greatly accelerating the sampling process.

[0088] S130, taking the palmprint images in the palmprint database as first training samples, training the unconditional diffusion model to obtain a noise image according to the defined forward process based on the network architecture.

[0089] ​In some embodiments, the unconditionally diffused model is trained to generate noise according to a defined forward process

[0090] images, comprising:

[0091] The palmprint images in the palmprint database are added with noise by using the image noise adding formula to generate corresponding noise images;

[0092] Further, the noise is randomly selected from a Gaussian distribution , the corresponding standard deviation is obtained according to the calculation formula of the standard deviation , and the standard deviation is substituted into the image noise adding formula to obtain the noise image ; wherein the calculation formula of the standard deviation is as follows:

[0093] ;

[0094] S140, according to the defined reverse sampling process, the trained unconditionally diffused model is used to perform reverse sampling on the noise image to generate an unconditional palmprint sample.

[0095] In some embodiments, according to the defined reverse sampling process, the trained unconditionally diffused

[0096] model is used to perform reverse sampling on the noise image to generate an unconditional palmprint sample, comprising:

[0097] According to the defined reverse sampling process, a numerical solution method is used to obtain an initial solution of the reverse sampling process;

[0098] The noise image at the i-th sampling point is substituted into the denoising function in the initial solution to calculate the noise value of the noise image at the i-th sampling point;

[0099] According to the noise value of the noise image at the i-th sampling point, the corresponding unconditional palmprint sample of the noise image is generated;

[0100] The corresponding unconditional palmprint sample of the noise image at the next sampling point is repeatedly obtained until the sampling time step is reached;

[0101] According to the standard deviation of the noise image relative to the corresponding palmprint image, the weight of the unconditionally diffused model is calculated;

[0102] According to the denoising function and the weight of the unconditionally diffused model, the loss function is calculated;

[0103] The loss function is used to evaluate the gap between the generated unconditional palmprint sample and the corresponding real palmprint image, and the Adam optimizer is used to execute the gradient descent strategy to train the loss function until the loss function converges.

[0104] In some embodiments, Substitute the defined reverse sampling process and use numerical solutions (such as the first-order Euler method and the second-order Henu Law ） The initial solution of the reverse sampling process is obtained; the initial solution is expressed as:

[0105] ;

[0106] In some embodiments, the noise image at the i-th sampling point is Substitute the denoising function in the initial solution and calculate the noise image at the i-th sampling point The noise value is based on the noise image at the i-th sampling point The noise value generates a noisy image The corresponding unconditional palmprint samples include:

[0107] Set the sampling sequence i, let i=[0,1,2,......17], and assume the maximum standard deviation is 80, and the sampling time step is calculated according to the following formula :

[0108] ;

[0109] When i is greater than or equal to 17, the noise image is updated using the first-order Euler method according to the following formula Corresponding palmprint samples , get the updated palmprint sample :

[0110] ;

[0111] ;

[0112] in, is the palmprint image of the i-th sampling point, represents the i-th sampling point moment, Represented as a palmprint sample The rate of change of the palmprint image at the i-th sampling point compared to the i-th sampling point;

[0113] When i is less than 17, the second-order Henu The method updates the noise image according to the following formula Corresponding palmprint samples , get the updated palmprint sample :

[0114] ;

[0115] ;

[0116] wherein, is the palmprint sample is the palmprint image of the i+1th sampling point is the change rate at the i+1th sampling point;

[0117] According to the above method, the next sampling point corresponding to the unconditional palmprint sample is repeatedly iterated until the sampling time step is reached.

[0118] The loss function is calculated and the Adam optimizer is used to perform gradient descent strategy to train the loss function until the loss function converges.

[0119] Further, the loss function is calculated by the following formula:

[0120] ;

[0121] is the weight corresponding to the loss function (i.e. the weight of the unconditional diffusion model), and the calculation formula is:

[0122] ;

[0123] In some embodiments, Figure 2 shows the unconditional palmprint samples generated by using the trained unconditional diffusion model.

[0124] S150, introducing a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to the decoder in the network architecture through a zero convolution layer.

[0125] In some embodiments, the result of the conditional control encoder is the same as the encoder structure.

[0126] In some embodiments, the zero convolution layer is a 1x1 convolution layer with both weight and bias initialized to zero. This structure can ensure that the unconditional diffusion model with the introduction of the conditional control encoder maintains stable output in the initial stage of training, and will not significantly change the initial output value of the unconditional diffusion model because of the zero convolution structure. As the iteration of the training process proceeds, the zero convolution structure will gradually fit the data set, gradually improving the performance of the unconditional diffusion model with the introduction of the conditional control encoder.

[0127] S160, freezing the weight of the unconditional diffusion model, training the condition control encoder according to the condition control image in the condition control data set, so that the condition control encoder maps the condition control image into the network architecture of the unconditional diffusion model through the zero convolution layer.

[0128] In some embodiments, the loss function used in the training process of the condition control encoder is the same as the loss function used in the training process of the unconditional diffusion model, to ensure consistency and stability of the training process.

[0129] In some embodiments, the condition control data set is a data set describing the features of the palm print image, such as a hand-drawn palm print image data set, a multi-spectral palm print image data set, etc.

[0130] S170, when the loss function corresponding to the condition control encoder converges, the weight of the unconditional diffusion model is unlocked, the palm print image and the condition control image are taken as the second training sample, and the unconditional diffusion model with the condition control encoder is trained according to the defined forward process.

[0131] In some embodiments, the method of training the unconditional diffusion model with the condition control encoder is consistent with the training method of the unconditional diffusion model;

[0132] By training the unconditional diffusion model with the condition control encoder, the unconditional diffusion model with the condition control encoder can generate palm print sample images that meet the features of the condition control image according to the palm print images in the palm print database and the condition control images in the condition control data set, so that the generated palm print samples have controllability, further improving the generation effect and realizing better output.

[0133] S180, according to the defined reverse sampling process, the trained unconditional diffusion model with the condition control encoder is used for reverse sampling to generate condition control palm print samples.

[0134] In some embodiments, the method of reverse sampling using the trained unconditional diffusion model with the condition control encoder is the same as the method of reverse sampling using the trained unconditional diffusion model.

[0135] In some embodiments, Figure 3 The condition control palm print samples generated by the trained unconditional diffusion model with the condition control encoder are shown;

[0136] In some embodiments, the condition control data set is a hand-drawn palm print image data set, so Figure 3The image feature of one hand-drawn palmprint image in the hand-drawn palmprint image dataset is represented by the image located at the leftmost side, and the trained unconditional diffusion model of the introduced conditional control encoder can generate a palmprint sample image conforming to the image feature according to the image feature, so that the generation of the palmprint sample image is controllable, and more kinds of palmprint sample images can be generated.

[0137] According to an embodiment of the present disclosure, an unconditional diffusion model network architecture is constructed based on an Unet structure; a forward process and a reverse sampling process are defined based on a stochastic differential equation; a palmprint image in a palmprint database is taken as a first training sample, and a noise image is obtained by training the unconditional diffusion model based on the defined forward process based on the network architecture; a conditional palmprint sample is generated by reverse sampling the trained unconditional diffusion model on the noise image according to the defined reverse sampling process; a conditional control encoder is introduced into the trained unconditional diffusion model; the conditional control encoder is connected to a decoder in the network architecture through a zero convolution layer; the weights of the unconditional diffusion model are frozen, and the conditional control encoder is trained according to a conditional control image in a conditional control dataset, so that the conditional control encoder maps the conditional control image to the unconditional diffusion model network architecture through the zero convolution layer; when the loss function corresponding to the conditional control encoder converges, the weights of the unconditional diffusion model are unlocked, the palmprint image and the conditional control image are taken as a second training sample, and the unconditional diffusion model introduced with the conditional control encoder is trained according to the defined forward process; a conditional palmprint sample is generated by reverse sampling the trained unconditional diffusion model introduced with the conditional control encoder according to the defined reverse sampling process. In this way, the authenticity and the variety of the generated palmprint sample can be improved, and the generation of the palmprint sample is more controllable by introducing the conditional control encoder.

[0138] It should be noted that, for each of the foregoing method embodiments, in order to simply describe, each is described as a combination of a series of actions, but those skilled in the art should know that the present disclosure is not limited by the order of the described actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0139] The above is the introduction of the method embodiment, and the scheme of the present disclosure will be further described through the device embodiment.

[0140] Figure 4 A structure diagram of a controllable palmprint sample generation device based on a stochastic differential equation is shown. The device 400 includes:

[0141] The network architecture constructing module 410 is configured to construct the unconditional diffusion model network architecture based on a Unet structure.

[0142] The forward / reverse process defining module 420 is configured to define a forward process and a reverse sampling process based on a stochastic differential equation.

[0143] In some embodiments, the module 420 is specifically configured to:

[0144] The definition of the forward process is as follows:

[0145] ;

[0146] wherein, , ; denotes the change amount of the palmprint image state over time , is a function about the palmprint image state and time , denotes the time change amount, denotes a noise function related to time, denotes a random noise item.

[0147] In some embodiments, the module 420 is specifically further configured to:

[0148] The definition of the reverse sampling process is as follows:

[0149] ;

[0150] wherein, is a score function, denotes the standard deviation of the noise image relative to the palmprint image .

[0151] The unconditional diffusion model training module 430 is configured to take the palmprint images in the palmprint database as first training samples, and train the unconditional diffusion model to obtain noise images based on the network architecture according to the defined forward process.

[0152] In some embodiments, the module 430 is specifically configured to:

[0153] Training the unconditional diffusion model to obtain noise images according to the defined forward process includes:

[0154] Adding noise to the palmprint images in the palmprint database by using an image noise adding formula to generate corresponding noise images; wherein the image noise adding formula is:

[0155] ;

[0156] denotes palmprint image noise image after adding random noise, denotes a random noise variable subject to a Gaussian distribution, denotes a Gaussian distribution.

[0157] The unconditional palmprint sample generation module 440 is configured to generate an unconditional palmprint sample by performing reverse sampling on the noise image according to a defined reverse sampling process and using the trained unconditional diffusion model.

[0158] In some embodiments, the module 440 is specifically configured to:

[0159] generate an unconditional palmprint sample by performing reverse sampling on the noise image according to a defined reverse sampling process and using the trained unconditional diffusion model, including:

[0160] obtaining an initial solution of the reverse sampling process according to a numerical solution method of the defined reverse sampling process;

[0161] substituting the noise image at the i-th sampling point into the denoising function in the initial solution to calculate the noise value of the noise image at the i-th sampling point;

[0162] generate a noise image corresponding to the unconditional palmprint sample from the noise value of the noise image at the i-th sampling point;

[0163] repeating the generation of the unconditional palmprint sample corresponding to the noise image at the next sampling point until the sampling time step is reached;

[0164] calculating the weight of the unconditional diffusion model according to the standard deviation of the noise image relative to the corresponding palmprint image;

[0165] calculating a loss function according to the denoising function and the weight of the unconditional diffusion model;

[0166] evaluating the gap between the generated unconditional palmprint sample and the corresponding real palmprint image by the loss function, and training the loss function by using an Adam optimizer to perform a gradient descent strategy until the loss function converges.

[0167] In some embodiments, the module 440 is specifically configured to:

[0168] the initial solution is expressed as:

[0169] ;

[0170] wherein, ​​​​is the denoising function.

[0171] In some embodiments, module 440 is further configured to:

[0172] The denoising function is expressed as:

[0173] ;

[0174] in, is the residual scaling factor, is the output transform scaling factor, is the input transform scaling factor, is the noise level factor, is a function of the input transform scaling factor and the noise level factor.

[0175] The conditional control encoder introduction module 450 is used to introduce a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to the decoder in the network architecture through a zero convolution layer.

[0176] The conditional control encoder training module 460 is used to freeze the weights of the unconditional diffusion model and train the conditional control encoder based on the conditional control images in the conditional control dataset so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through zero convolution layers.

[0177] The model joint training module 470 is used to unlock the weights of the unconditional diffusion model when the loss function corresponding to the conditional control encoder converges, use the palm print image and the conditional control image as the second training samples, and train the unconditional diffusion model introduced with the conditional control encoder according to the defined forward process.

[0178] The conditional palmprint sample generating module 480 is used to perform reverse sampling using the trained unconditional diffusion model with the conditional control encoder according to the defined reverse sampling process to generate conditional palmprint samples.

[0179] It is understandable that Figure 4 Each module / unit in the illustrated device 400 has the function of implementing each step in the method 100 provided in the embodiment of the present disclosure and can achieve its corresponding technical effect. For the sake of brevity, they will not be described in detail here.

[0180] Figure 5A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. The electronic device 500 is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device 500 can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0181] As shown in Figure 5 The electronic device 500 includes a computing unit 501 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 502 or a computer program loaded into a random access memory (RAM) 503 from a storage unit 508. Various programs and data required for the operation of the electronic device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An I / O interface 505 is also connected to the bus 504.

[0182] Various components in the electronic device 500 are connected to the I / O interface 505, including an input unit 506, such as a keyboard, a mouse, and the like; an output unit 507, such as various types of displays, speakers, and the like; a storage unit 508, such as a magnetic disk, an optical disk, and the like; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0183] The computing unit 501 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs various methods and processes described above, such as the method 100. For example, in some embodiments, the method 100 can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded onto the RAM 503 and executed by the computing unit 501, one or more steps of the method 100 described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the method 100 by any other suitable means, such as by means of firmware.

[0184] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0185] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0186] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0187] It should be noted that the present disclosure also provides a non-transitory computer readable storage medium having computer instructions stored therein, wherein the computer instructions are used to make a computer execute the method 100 and achieve the corresponding technical effects achieved by the embodiments of the present disclosure executing the method. For brevity, the description will not be repeated here.

[0188] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0189] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0190] The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0191] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0192] It should be understood that the various forms of flow shown above can be re-ordered, added to, or have steps deleted, using the steps. For example, the steps described in the present disclosure can be executed in parallel, in series, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, which are not limited herein.

[0193] The above detailed description does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A controllable palmprint sample generation method based on stochastic differential equations, characterized in that: include: Construct an unconditional diffusion model network architecture based on the Unet structure; Define the forward process and the reverse sampling process based on stochastic differential equations; Using a palmprint image in a palmprint database as a first training sample, and training an unconditional diffusion model according to a defined forward process based on the network architecture to obtain a noise image; Perform reverse sampling on the noise image using the trained unconditional diffusion model according to the defined reverse sampling process to generate an unconditional palmprint sample; Introducing a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to the decoder in the network architecture through a zero convolutional layer; Freeze the weights of the unconditional diffusion model and train the conditional control encoder based on the conditional control images in the conditional control dataset so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through zero convolutional layers; When the loss function corresponding to the conditional control encoder converges, the weights of the unconditional diffusion model are unlocked, and the palm print image and the conditional control image are used as the second training samples. The unconditional diffusion model with the conditional control encoder is trained according to the defined forward process. According to the defined reverse sampling process, the trained unconditional diffusion model with the conditional control encoder is used for reverse sampling to generate conditional palmprint samples.

2. The method according to claim 1, characterized in that The forward process is defined as follows: ； in, Represents the state of the palmprint image over time The amount of change, Regarding the palm print image status and time function, represents the time variation, represents the time-dependent noise function, represents the random noise term.

3. The method according to claim 1, characterized in that The forward pass according to the definition The unconditional diffusion model is trained to obtain a noisy image, including: The palmprint image in the palmprint database is denoised using an image denoising formula to generate a corresponding noise image; wherein the image denoising formula is: ； express Palm print image at any moment Noise image after adding random noise, represents a random noise variable that follows a Gaussian distribution, Represents a Gaussian distribution.

4. The method according to claim 3, characterized in that The reverse sampling process The meaning is as follows: ； in, is the score function, Represents a noisy image Relative to palm print image The standard deviation of Represents the state of the palmprint image over time The amount of change, Regarding the palm print image status and time function, represents the time variation, represents the time-dependent noise function.

5. The method according to claim 4, characterized in that The reverse sampling according to the definition The sampling process uses the trained unconditional diffusion model to reversely sample the noise image to generate an unconditional palmprint sample, including: According to the defined reverse sampling process, an initial solution of the reverse sampling process is obtained by using a numerical solution method; The noise image at the i-th sampling point Substitute the denoising function in the initial solution and calculate the noise image at the i-th sampling point The noise value of According to the noise image at the i-th sampling point The noise value generates a noisy image Corresponding unconditional palmprint samples; Repeatedly obtain the unconditional palmprint sample corresponding to the noise image at the next sampling point until the sampling time step is reached; The weights of the unconditional diffusion model are calculated based on the standard deviation of the noise image relative to the corresponding palmprint image; Calculate the loss function based on the weights of the denoising function and the unconditional diffusion model; The loss function is used to evaluate the gap between the generated unconditional palmprint sample and the corresponding real palmprint image, and the Adam optimizer is used to execute the gradient descent strategy to train the loss function until the loss function converges.

6. The method according to claim 5, characterized in that The initial solution is expressed as: ； in, is the denoising function.

7. The method according to claim 6, characterized in that The denoising function is expressed as: ； in, is the residual scaling factor, is the output transform scaling factor, is the input transform scaling factor, is the noise level factor, is a function of the input transform scaling factor and the noise level factor.

8. A controllable palmprint sample generation device based on stochastic differential equations, characterized in that: include: Network architecture building module, used to build the unconditional diffusion model network architecture based on the Unet structure; Forward / reverse process definition module, used to define the forward process and reverse sampling process based on stochastic differential equations; An unconditional diffusion model training module is used to use a palmprint image in a palmprint database as a first training sample, and train an unconditional diffusion model according to a defined forward process based on the network architecture to obtain a noise image; An unconditional palmprint sample generation module is used to perform reverse sampling on the noise image using a trained unconditional diffusion model according to a defined reverse sampling process to generate an unconditional palmprint sample; A conditional control encoder introduction module is used to introduce a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to the decoder in the network architecture through a zero convolution layer; A conditional control encoder training module is used to freeze the weights of the unconditional diffusion model and train the conditional control encoder based on the conditional control images in the conditional control dataset so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through zero convolutional layers; The model joint training module is used to unlock the weights of the unconditional diffusion model when the loss function corresponding to the conditional control encoder converges. The palm print image and the conditional control image are used as the second training sample, and the unconditional diffusion model introduced with the conditional control encoder is trained according to the defined forward process. The conditional palmprint sample generation module is used to perform reverse sampling using the trained unconditional diffusion model that introduces a conditional control encoder according to a defined reverse sampling process to generate conditional palmprint samples.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images