Controllable palm print sample generation method based on stochastic differential equation

Abstract

The embodiment of the invention provides a controllable palm print sample generation method based on a stochastic differential equation. Relates to the field of computer vision. The method comprises the following steps: constructing an unconditional diffusion model network architecture and defining a forward process and a reverse process; training an unconditional diffusion model according to the forward process and the palm print image to obtain a noise image; sampling a noise image by using the trained unconditional diffusion model according to a reverse process to generate an unconditional palm print sample; introducing a conditional encoder into the trained unconditional diffusion model; freezing the weight of the unconditional diffusion model, and training a condition encoder according to the condition image; unlocking the frozen weight when a condition encoder loss function converges, and training an unconditional diffusion model introducing the condition encoder according to the forward process, the palmprint image and the condition image; and according to a reverse process, sampling by using an unconditional diffusion model of the trained conditional encoder, and generating a conditional palmprint sample. Therefore, the authenticity, diversity and controllability of palm print samples are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision, and in particular to a controllable palmprint sample generation method based on stochastic differential equations. Background Art

[0002] With the rapid development of science and technology, biometric recognition technology has gradually become a key means of identity verification and security. Among many biometric features, palmprints, with their unique texture and details, have become an important basis for identity recognition. However, due to the scarcity of public palmprint datasets, the development of palmprint recognition technology has been greatly restricted, so palmprint sample generation methods have emerged.

[0003] The existing palmprint sample generation methods include palmprint sample generation methods based on Bezier curves, palmprint sample generation methods based on generative adversarial networks, and palmprint sample generation methods based on diffusion models. There is a large gap between the palmprint samples generated by the palmprint sample generation method based on Bezier curves and the real palmprint samples. The palmprint sample generation method based on generative adversarial networks has problems such as mode collapse and training instability, and it is still difficult to generate palmprint samples with diversity. Although the palmprint samples generated by the palmprint sample generation method based on diffusion models are more realistic, the generated palmprint sample images are prone to defects such as abnormal color background, too many noise points, and missing details and textures. In addition, when generating palmprint samples, it is impossible to accurately control the specific attributes or characteristics of the palmprint samples, which greatly limits the diversity and practicality of the palmprint samples. Therefore, it is of great significance to study a palmprint sample generation method that can avoid the above shortcomings. Summary of the invention

[0004] The present disclosure provides a controllable palmprint sample generation method, device, equipment and storage medium based on stochastic differential equations.

[0005] According to a first aspect of the present disclosure, a controllable palmprint sample generation method based on stochastic differential equations is provided. The method comprises: Construct an unconditional diffusion model network architecture based on the Unet structure; Define the forward process and the reverse sampling process based on stochastic differential equations; Using the palmprint image in the palmprint database as the first training sample, training the unconditional diffusion model according to the defined forward process based on the network architecture to obtain a noise image; According to the defined reverse sampling process, the trained unconditional diffusion model is used to reversely sample the noise image to generate an unconditional palmprint sample; Introducing a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to a decoder in the network architecture through a zero convolution layer; Freeze the weights of the unconditional diffusion model, and train the conditional control encoder based on the conditional control images in the conditional control dataset so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through zero convolutional layers; When the loss function corresponding to the conditional control encoder converges, the weight of the unconditional diffusion model is unlocked, the palm print image and the conditional control image are used as the second training sample, and the unconditional diffusion model with the conditional control encoder introduced is trained according to the defined forward process; According to the defined reverse sampling process, the trained unconditional diffusion model with conditional control encoder is used for reverse sampling to generate conditional palmprint samples.

[0006] In some implementations of the first aspect, the forward process is defined as follows: ; in, , ; Represents the state of the palmprint image over time The amount of change, For palmprint image status With time The function of represents the time variation, represents the time-dependent noise function, represents the random noise term.

[0007] In some implementations of the first aspect, training an unconditional diffusion model according to a defined forward process to obtain a noisy image includes: The palmprint image in the palmprint database is denoised using an image denoising formula to generate a corresponding noise image; wherein the image denoising formula is: ; express Palm print image at all times Noise image after adding random noise, represents a random noise variable that follows a Gaussian distribution, Represents a Gaussian distribution.

[0008] In some implementations of the first aspect, the reverse sampling process is defined as follows: ; in, is the score function, Represents a noisy image Relative to palm print image The standard deviation of .

[0009] In some implementations of the first aspect, reverse sampling is performed on the noise image using a trained unconditional diffusion model according to a defined reverse sampling process to generate an unconditional palmprint sample, including: According to the defined reverse sampling process, an initial solution of the reverse sampling process is obtained by using a numerical solution method; The noise image at the i-th sampling point Substitute the denoising function in the initial solution and calculate the noise image at the i-th sampling point The noise value of According to the noise image at the i-th sampling point The noise value generates a noisy image The corresponding unconditional palmprint samples; Repeatedly obtain the unconditional palmprint sample corresponding to the noise image at the next sampling point until the sampling time step is reached; The weights of the unconditional diffusion model are calculated based on the standard deviation of the noise image relative to the corresponding palmprint image; Calculate the loss function based on the weights of the denoising function and the unconditional diffusion model; The loss function is used to evaluate the gap between the generated unconditional palmprint sample and the corresponding real palmprint image, and the Adam optimizer is used to execute the gradient descent strategy to train the loss function until the loss function converges.

[0010] In some implementations of the first aspect, the initial solution is expressed as:

[0011] in, is the denoising function.

[0012] In some implementations of the first aspect, the denoising function is expressed as: ; in, is the residual scaling factor, is the output transform scaling factor, is the input transform scaling factor, is the noise level factor, is a function of the input transform scaling factor and the noise level factor.

[0013] According to a second aspect of the present disclosure, a controllable palmprint sample generation device based on stochastic differential equations is provided. The device comprises: Network architecture building module, used to build the unconditional diffusion model network architecture based on the Unet structure; The forward / reverse process definition module is used to define the forward process and reverse sampling process based on stochastic differential equations; An unconditional diffusion model training module is used to use the palmprint image in the palmprint database as a first training sample, and train the unconditional diffusion model according to the defined forward process based on the network architecture to obtain a noise image; An unconditional palmprint sample generation module, used to perform reverse sampling on the noise image using a trained unconditional diffusion model according to a defined reverse sampling process to generate an unconditional palmprint sample; A conditional control encoder introduction module, used to introduce a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to the decoder in the network architecture through a zero convolution layer; A conditional control encoder training module, used to freeze the weights of the unconditional diffusion model, and train the conditional control encoder according to the conditional control images in the conditional control dataset, so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through a zero convolution layer; A model joint training module is used to unlock the weights of the unconditional diffusion model when the loss function corresponding to the conditional control encoder converges, use the palm print image and the conditional control image as the second training sample, and train the unconditional diffusion model introduced with the conditional control encoder according to the defined forward process; The conditional palmprint sample generation module is used to perform reverse sampling using the trained unconditional diffusion model with the conditional control encoder introduced according to the defined reverse sampling process to generate conditional palmprint samples.

[0014] According to a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: at least one processor; and a memory connected to the at least one processor in communication; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method described above.

[0015] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method as described above.

[0016] In the present disclosure, an unconditional diffusion model network architecture is constructed based on a Unet structure; a forward process and a reverse sampling process are defined based on a stochastic differential equation; a palmprint image in a palmprint database is used as a first training sample, and an unconditional diffusion model is trained based on the network architecture according to a defined forward process to obtain a noise image; the noise image is reversely sampled using the trained unconditional diffusion model according to a defined reverse sampling process to generate an unconditional palmprint sample; a conditional control encoder is introduced into the trained unconditional diffusion model; wherein the conditional control encoder is connected to a decoder in the network architecture through a zero convolution layer; the weight of the unconditional diffusion model is frozen, and the conditional control encoder is trained according to a conditional control image in a conditional control data set so that the conditional control encoder maps the conditional control image to the unconditional diffusion model network architecture through a zero convolution layer; when the loss function corresponding to the conditional control encoder converges, the weight of the unconditional diffusion model is unlocked, the palmprint image and the conditional control image are used as second training samples, and the unconditional diffusion model with the conditional control encoder introduced is trained according to a defined forward process; reverse sampling is performed using the trained unconditional diffusion model with the conditional control encoder introduced according to a defined reverse sampling process to generate a conditional palmprint sample. In this way, the authenticity and variety of the generated palmprint samples can be improved, and the generation of palmprint samples can be made more controllable by introducing a conditional control encoder.

[0017] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which: Figure 1 A flowchart of a controllable palmprint sample generation method based on stochastic differential equations provided by an embodiment of the present disclosure is shown; Figure 2 A diagram showing the unconditional control reverse sampling result provided by an embodiment of the present disclosure is shown; Figure 3 A conditionally controlled reverse sampling result diagram provided by an embodiment of the present disclosure is shown; Figure 4 A structural diagram of a controllable palmprint sample generation device based on stochastic differential equations provided in an embodiment of the present disclosure is shown; Figure 5 A block diagram of an exemplary electronic device capable of implementing an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0020] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0021] In view of the problems in the background technology, the embodiments of the present disclosure provide a controllable palmprint sample generation method and device based on stochastic differential equations. Specifically, an unconditional diffusion model network architecture is constructed based on a Unet structure; a forward process and a reverse sampling process are defined based on a stochastic differential equation; a palmprint image in a palmprint database is used as a first training sample, and an unconditional diffusion model is trained based on the network architecture according to a defined forward process to obtain a noise image; the noise image is reversely sampled using the trained unconditional diffusion model according to a defined reverse sampling process to generate an unconditional palmprint sample; a conditional control encoder is introduced into the trained unconditional diffusion model; wherein the conditional control encoder is connected to a decoder in the network architecture through a zero convolution layer; the weight of the unconditional diffusion model is frozen, and the conditional control encoder is trained according to the conditional control image in the conditional control data set so that the conditional control encoder maps the conditional control image to the unconditional diffusion model network architecture through a zero convolution layer; when the loss function corresponding to the conditional control encoder converges, the weight of the unconditional diffusion model is unlocked, the palmprint image and the conditional control image are used as second training samples, and the unconditional diffusion model with the conditional control encoder introduced is trained according to a defined forward process; reverse sampling is performed using the trained unconditional diffusion model with the conditional control encoder introduced according to a defined reverse sampling process to generate a conditional palmprint sample. In this way, the authenticity and variety of the generated palmprint samples can be improved, and the generation of palmprint samples can be made more controllable by introducing a conditional control encoder.

[0022] The controllable palmprint sample generation method and device based on stochastic differential equations provided by the embodiments of the present disclosure are described in detail below with reference to the accompanying drawings through specific embodiments.

[0023] Figure 1A flowchart of a controllable palmprint sample generation method based on stochastic differential equations provided in an embodiment of the present disclosure is shown. The method 100 includes the following steps: S110, build an unconditional diffusion model network architecture based on the Unet structure.

[0024] In some embodiments, the unconditional diffusion model network architecture includes an encoder and a decoder.

[0025] In some embodiments, the encoder extracts high-level semantic features of the palmprint image by gradually reducing the resolution of the palmprint image in the palmprint database, thereby converting the palmprint image into a noise image; Furthermore, the encoder includes 5 blocks, each block includes 4 encoder blocks; when the image passes through each block, the number of channels of the feature map remains unchanged, and the length and width are doubled to ensure the consistency and stability of the information; specifically, the number of channels of each block is set to [128, 128, 256, 256, 256], which can effectively extract and retain the key features of the palmprint image.

[0026] In some embodiments, the decoder outputs a palmprint sample image with the same resolution as the palmprint image in the palmprint database by gradually upsampling the low-resolution image features and fusing them with the features of the encoder; Furthermore, the decoder also includes 5 blocks, and the input of each block realizes information fusion and complementarity through splicing. This not only makes full use of the feature information extracted in the encoding stage, but also realizes more accurate and detailed palmprint sample reconstruction in the decoding process.

[0027] S120, defining a forward process and a reverse sampling process based on a stochastic differential equation.

[0028] In some embodiments, the forward process is defined as follows: ; in, , ; Represents the state of the palmprint image over time The amount of change, For palmprint image status With time The function of represents the time variation, represents the time-dependent noise function, represents the random noise term.

[0029] In some embodiments, according to the defined forward process, the marginal distribution of the forward process is obtained as: ; express Palm print image at all times Noise image after adding random noise, is the marginal distribution of the forward process, is the identity matrix, Represents a noisy image Relative to palm print image The standard deviation of It means that the mean is The variance is Gaussian distribution of According to the edge distribution of the forward process, the image noise addition formula can be further obtained as follows: ; represents a random noise variable that follows a Gaussian distribution, Represents a Gaussian distribution.

[0030] In some embodiments, the reverse sampling process is defined as follows: ; in, is the score function, Represents a noisy image Relative to palm print image The standard deviation of Further, You can use the denoising function Make an estimate, the estimation formula is as follows: ; Among them, the denoising function can be calculated using the following formula: ; ; is the residual scaling factor, is the output transform scaling factor, is the input transform scaling factor, is the noise level factor, is a function of the input transform scaling factor and the noise level factor.

[0031] In some embodiments, the reverse sampling process uses skip-step sampling, and the sampling steps are performed only dozens of times, which greatly speeds up the sampling process.

[0032] S130, taking the palmprint image in the palmprint database as the first training sample, and training the unconditional diffusion model according to the defined forward process based on the network architecture to obtain a noise image.

[0033] In some embodiments, the unconditional diffusion model is trained according to a defined forward process to obtain noise Images, including: Using the image noise addition formula to add noise to the palmprint image in the palmprint database, a corresponding noise image is generated; Furthermore, from the Gaussian distribution Randomly select noise , according to the standard deviation The calculation formula is used to obtain the corresponding standard deviation, and the standard deviation is substituted into the image noise addition formula to obtain the noisy image ; Among them, the standard deviation The calculation formula is as follows: ; S140, reverse sampling the noise image using the trained unconditional diffusion model according to the defined reverse sampling process to generate an unconditional palmprint sample.

[0034] In some embodiments, the trained unconditional diffusion The model performs reverse sampling on the noise image to generate unconditional palmprint samples, including: According to the defined reverse sampling process, an initial solution of the reverse sampling process is obtained by using a numerical solution method; The noise image at the i-th sampling point Substitute the denoising function in the initial solution and calculate the noise image at the i-th sampling point The noise value of According to the noise image at the i-th sampling point The noise value generates a noisy image The corresponding unconditional palmprint samples; Repeatedly obtain the unconditional palmprint sample corresponding to the noise image at the next sampling point until the sampling time step is reached; The weights of the unconditional diffusion model are calculated based on the standard deviation of the noise image relative to the corresponding palmprint image; Calculate the loss function based on the weights of the denoising function and the unconditional diffusion model; The loss function is used to evaluate the gap between the generated unconditional palmprint sample and the corresponding real palmprint image, and the Adam optimizer is used to execute the gradient descent strategy to train the loss function until the loss function converges.

[0035] In some embodiments, Substitute the defined reverse sampling process and use numerical solutions (such as first-order Euler method and second-order Henu Law ）The initial solution of the reverse sampling process is obtained; the initial solution is expressed as: ; In some embodiments, the noise image at the i-th sampling point is Substitute the denoising function in the initial solution and calculate the noise image at the i-th sampling point The noise value is based on the noise image at the i-th sampling point The noise value generates a noisy image The corresponding unconditional palmprint samples include: Set the sampling sequence i, let i=[0,1,2,......17], and assume the maximum value of the standard deviation is 80, and the sampling time step is calculated according to the following formula : ; When i is greater than or equal to 17, the noise image is updated using the first-order Euler method according to the following formula Corresponding palm print samples , get the updated palmprint sample : ; ; in, is the palmprint image of the i-th sampling point, represents the i-th sampling point moment, Represented as a palm print sample The change rate of the palmprint image at the i-th sampling point compared to the i-th sampling point; When i is less than 17, the second order Henu The method updates the noise image according to the following formula Corresponding palm print samples , get the updated palmprint sample : ; ; in, Represented as a palm print sample Compared with the palmprint image of the i+1th sampling point The rate of change at the i+1th sampling point; Repeat the above method to iterate and continuously obtain the unconditional palmprint sample corresponding to the next sampling point until the sampling time step is reached; Calculate the loss function and use the Adam optimizer to perform gradient descent strategy to train the loss function until the loss function converges; Furthermore, the loss function Calculated by the following formula: ; represents the weight corresponding to the loss function (i.e., the weight of the unconditional diffusion model), and its calculation formula is: ; In some embodiments, Figure 2 Unconditional palmprint samples generated using the trained unconditional diffusion model are shown.

[0036] S150, introducing a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to the decoder in the network architecture through a zero convolution layer.

[0037] In some embodiments, the result of the conditional control encoder is the same as the encoder structure.

[0038] In some embodiments, the zero convolution layer is a 1×1 convolution layer whose weights and biases are initialized to zero. This structure can ensure that the unconditional diffusion model that introduces the conditional control encoder maintains a stable output in the initial stage of training, and the initial output value of the unconditional diffusion model will not be significantly changed due to the zero convolution structure. As the iterations in the training process proceed, the zero convolution structure will progressively fit the data set and gradually improve the performance of the unconditional diffusion model that introduces the conditional control encoder.

[0039] S160, freezing the weights of the unconditional diffusion model, and training the conditional control encoder according to the conditional control image in the conditional control dataset, so that the conditional control encoder maps the conditional control image to the unconditional diffusion model network architecture through a zero convolution layer.

[0040] In some embodiments, during the training of the conditional control encoder, the loss function used is the same as the loss function used during the training of the unconditional diffusion model to ensure the consistency and stability of the training process.

[0041] In some embodiments, the conditional control data set is a data set describing palmprint detail image features, such as a hand-drawn palmprint image data set, a multi-spectral palmprint image data set, and the like.

[0042] S170, when the loss function corresponding to the conditional control encoder converges, unlock the weights of the unconditional diffusion model, use the palm print image and the conditional control image as the second training samples, and train the unconditional diffusion model introducing the conditional control encoder according to the defined forward process.

[0043] In some embodiments, the method for training the unconditional diffusion model introducing the conditional control encoder is consistent with the training method of the unconditional diffusion model; By training the unconditional diffusion model with the conditional control encoder, the unconditional diffusion model with the conditional control encoder can generate palmprint sample images that meet the characteristics of the conditional control image based on the palmprint images in the palmprint database and the conditional control images in the conditional control data set, so that the generated palmprint samples are controllable, further improving the generation effect and achieving better quality output.

[0044] S180, performing reverse sampling using the trained unconditional diffusion model with the conditional control encoder introduced according to the defined reverse sampling process to generate a conditional palmprint sample.

[0045] In some embodiments, the method of performing reverse sampling using a trained unconditional diffusion model introducing a conditional control encoder is the same as the method of performing reverse sampling using a trained unconditional diffusion model.

[0046] In some embodiments, Figure 3 The conditional palmprint samples generated by the trained unconditional diffusion model with conditional control encoder are shown; Among them, the conditional control data set is a hand-drawn palmprint image data set, so Figure 3 The leftmost picture in the figure represents the image features of a hand-drawn palmprint image in the hand-drawn palmprint image dataset. The trained unconditional diffusion model with conditional control encoder can generate palmprint sample images that meet the image features according to the image features. Therefore, the generation of palmprint sample images is controllable and is also conducive to generating more types of palmprint sample images.

[0047] According to an embodiment of the present disclosure, an unconditional diffusion model network architecture is constructed based on a Unet structure; a forward process and a reverse sampling process are defined based on a stochastic differential equation; a palmprint image in a palmprint database is used as a first training sample, and an unconditional diffusion model is trained based on the network architecture according to a defined forward process to obtain a noise image; the noise image is reversely sampled using the trained unconditional diffusion model according to a defined reverse sampling process to generate an unconditional palmprint sample; a conditional control encoder is introduced into the trained unconditional diffusion model; wherein the conditional control encoder is connected to a decoder in the network architecture through a zero convolution layer; the weight of the unconditional diffusion model is frozen, and the conditional control encoder is trained according to the conditional control image in the conditional control data set so that the conditional control encoder maps the conditional control image to the unconditional diffusion model network architecture through a zero convolution layer; when the loss function corresponding to the conditional control encoder converges, the weight of the unconditional diffusion model is unlocked, the palmprint image and the conditional control image are used as second training samples, and the unconditional diffusion model introducing the conditional control encoder is trained according to a defined forward process; reverse sampling is performed using the trained unconditional diffusion model introducing the conditional control encoder according to a defined reverse sampling process to generate a conditional palmprint sample. In this way, the authenticity and variety of the generated palmprint samples can be improved, and the generation of palmprint samples can be made more controllable by introducing a conditional control encoder.

[0048] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0049] The above is an introduction to the method embodiment. The following is a further explanation of the scheme disclosed in the present invention through an apparatus embodiment.

[0050] Figure 4 The structure diagram of a controllable palmprint sample generation device based on stochastic differential equations provided by an embodiment of the present disclosure is shown. The device 400 includes: The network architecture building module 410 is used to build an unconditional diffusion model network architecture based on the Unet structure.

[0051] The forward / reverse process definition module 420 is used to define the forward process and the reverse sampling process based on the stochastic differential equation.

[0052] In some embodiments, module 420 is specifically configured to: The forward process is defined as follows: ; in, , ; Represents the state of the palmprint image over time The amount of change, For palmprint image status With time The function of represents the time variation, represents the time-dependent noise function, represents the random noise term.

[0053] In some embodiments, module 420 is further configured to: The reverse sampling process is defined as follows: ; in, is the score function, Represents a noisy image Relative to palm print image The standard deviation of .

[0054] The unconditional diffusion model training module 430 is used to use the palmprint image in the palmprint database as the first training sample, and train the unconditional diffusion model according to the defined forward process based on the network architecture to obtain a noise image.

[0055] In some embodiments, module 430 is specifically configured to: The unconditional diffusion model is trained to obtain a noisy image according to the defined forward process, including: The palmprint image in the palmprint database is denoised using an image denoising formula to generate a corresponding noise image; wherein the image denoising formula is: ; express Palm print image at all times Noise image after adding random noise, represents a random noise variable that follows a Gaussian distribution, Represents a Gaussian distribution.

[0056] The unconditional palmprint sample generating module 440 is used to perform reverse sampling on the noise image using the trained unconditional diffusion model according to the defined reverse sampling process to generate unconditional palmprint samples.

[0057] In some embodiments, module 440 is specifically configured to: The reverse sampling process is defined, and the trained unconditional diffusion model is used to reversely sample the noise image to generate an unconditional palmprint sample, including: According to the defined reverse sampling process, an initial solution of the reverse sampling process is obtained by using a numerical solution method; The noise image at the i-th sampling point Substitute the denoising function in the initial solution and calculate the noise image at the i-th sampling point The noise value of According to the noise image at the i-th sampling point The noise value generates a noisy image The corresponding unconditional palmprint samples; Repeatedly obtain the unconditional palmprint sample corresponding to the noise image at the next sampling point until the sampling time step is reached; The weights of the unconditional diffusion model are calculated based on the standard deviation of the noise image relative to the corresponding palmprint image; Calculate the loss function based on the weights of the denoising function and the unconditional diffusion model; The loss function is used to evaluate the gap between the generated unconditional palmprint sample and the corresponding real palmprint image, and the Adam optimizer is used to execute the gradient descent strategy to train the loss function until the loss function converges.

[0058] In some embodiments, module 440 is further configured to: The initial solution is expressed as: ; in, is the denoising function.

[0059] In some embodiments, module 440 is further configured to: The denoising function is expressed as: ; in, is the residual scaling factor, is the output transform scaling factor, is the input transform scaling factor, is the noise level factor, is a function of the input transform scaling factor and the noise level factor.

[0060] The conditional control encoder introduction module 450 is used to introduce a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to the decoder in the network architecture through a zero convolution layer.

[0061] The conditional control encoder training module 460 is used to freeze the weights of the unconditional diffusion model and train the conditional control encoder according to the conditional control images in the conditional control dataset so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through a zero convolution layer.

[0062] The model joint training module 470 is used to unlock the weights of the unconditional diffusion model when the loss function corresponding to the conditional control encoder converges, use the palm print image and the conditional control image as the second training samples, and train the unconditional diffusion model introduced with the conditional control encoder according to the defined forward process.

[0063] The conditional palmprint sample generating module 480 is used to perform reverse sampling using the trained unconditional diffusion model with the conditional control encoder introduced according to the defined reverse sampling process to generate conditional palmprint samples.

[0064] Understandably, Figure 4 Each module / unit in the illustrated device 400 has the function of implementing each step in the method 100 provided in the embodiment of the present disclosure and can achieve its corresponding technical effect, which will not be described in detail here for the sake of brevity.

[0065] Figure 5 A block diagram of an exemplary electronic device capable of implementing an embodiment of the present disclosure is shown. Electronic device 500 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 500 may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0066] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An I / O interface 505 is also connected to the bus 504.

[0067] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0068] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as method 100. For example, in some embodiments, the method 100 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the method 100 described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the method 100 in any other appropriate manner (e.g., by means of firmware).

[0069] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0070] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0071] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0072] It should be noted that the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute method 100 and achieve the corresponding technical effect achieved by executing the method in the embodiment of the present disclosure. For the sake of concise description, they are not repeated here.

[0073] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0074] The systems and techniques described herein can be implemented in a computing system that includes a backend component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a frontend component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with the systems and techniques described herein). The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0075] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0076] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0077] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A controllable palmprint sample generation method based on stochastic differential equations, characterized in that: include: Construct an unconditional diffusion model network architecture based on the Unet structure; Define the forward process and the reverse sampling process based on stochastic differential equations; Using the palmprint image in the palmprint database as the first training sample, training the unconditional diffusion model according to the defined forward process based on the network architecture to obtain a noise image; According to the defined reverse sampling process, the trained unconditional diffusion model is used to reversely sample the noise image to generate an unconditional palmprint sample; Introducing a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to a decoder in the network architecture through a zero convolution layer; Freeze the weights of the unconditional diffusion model, and train the conditional control encoder based on the conditional control images in the conditional control dataset so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through zero convolutional layers; When the loss function corresponding to the conditional control encoder converges, the weight of the unconditional diffusion model is unlocked, the palm print image and the conditional control image are used as the second training sample, and the unconditional diffusion model with the conditional control encoder introduced is trained according to the defined forward process; According to the defined reverse sampling process, the trained unconditional diffusion model with conditional control encoder is used for reverse sampling to generate conditional palmprint samples.

2. The method according to claim 1, characterized in that The forward process is defined as follows: ； in, , ; Represents the state of the palmprint image over time The amount of change, For palmprint image status With time The function of represents the time variation, represents the time-dependent noise function, represents the random noise term.

3. The method according to claim 1, characterized in that The forward pass according to the definition The unconditional diffusion model is trained to obtain a noisy image, including: The palmprint image in the palmprint database is denoised using an image denoising formula to generate a corresponding noise image; wherein the image denoising formula is: ； express Palm print image at all times Noise image after adding random noise, represents a random noise variable that follows a Gaussian distribution, Represents a Gaussian distribution.

4. The method according to claim 3, characterized in that The reverse sampling process The meaning is as follows: ； in, is the score function, Represents a noisy image Relative to palm print image The standard deviation of .

5. The method according to claim 4, characterized in that The reverse acquisition according to the definition The sampling process uses the trained unconditional diffusion model to reversely sample the noise image to generate unconditional palmprint samples, including: According to the defined reverse sampling process, an initial solution of the reverse sampling process is obtained by using a numerical solution method; The noise image at the i-th sampling point Substitute the denoising function in the initial solution and calculate the noise image at the i-th sampling point The noise value of According to the noise image at the i-th sampling point The noise value generates a noisy image The corresponding unconditional palmprint samples; Repeatedly obtain the unconditional palmprint sample corresponding to the noise image at the next sampling point until the sampling time step is reached; The weights of the unconditional diffusion model are calculated based on the standard deviation of the noise image relative to the corresponding palmprint image; Calculate the loss function based on the weights of the denoising function and the unconditional diffusion model; The loss function is used to evaluate the gap between the generated unconditional palmprint sample and the corresponding real palmprint image, and the Adam optimizer is used to execute the gradient descent strategy to train the loss function until the loss function converges.

6. The method according to claim 5, characterized in that The initial solution is expressed as: ； in, is the denoising function.

7. The method according to claim 6, characterized in that The denoising function is expressed as: ； in, is the residual scaling factor, is the output transform scaling factor, is the input transform scaling factor, is the noise level factor, is a function of the input transform scaling factor and the noise level factor.

8. A controllable palmprint sample generation device based on stochastic differential equations, characterized in that: include: Network architecture building module, used to build the unconditional diffusion model network architecture based on the Unet structure; The forward / reverse process definition module is used to define the forward process and reverse sampling process based on stochastic differential equations; An unconditional diffusion model training module is used to use the palmprint image in the palmprint database as a first training sample, and train the unconditional diffusion model according to the defined forward process based on the network architecture to obtain a noise image; An unconditional palmprint sample generation module, used to perform reverse sampling on the noise image using a trained unconditional diffusion model according to a defined reverse sampling process to generate an unconditional palmprint sample; A conditional control encoder introduction module, used to introduce a conditional control encoder into the trained unconditional diffusion model; wherein the conditional control encoder is connected to the decoder in the network architecture through a zero convolution layer; A conditional control encoder training module, used to freeze the weights of the unconditional diffusion model, and train the conditional control encoder according to the conditional control images in the conditional control dataset, so that the conditional control encoder maps the conditional control images to the unconditional diffusion model network architecture through a zero convolution layer; A model joint training module is used to unlock the weights of the unconditional diffusion model when the loss function corresponding to the conditional control encoder converges, use the palm print image and the conditional control image as the second training sample, and train the unconditional diffusion model introduced with the conditional control encoder according to the defined forward process; The conditional palmprint sample generation module is used to perform reverse sampling using the trained unconditional diffusion model with the conditional control encoder introduced according to the defined reverse sampling process to generate conditional palmprint samples.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

Images