Facial image processing model construction method and device and computer equipment

Abstract

The invention relates to a face image processing model construction method and device and computer equipment, and can be applied to the technical field of artificial intelligence. The method comprises the following steps: acquiring a plurality of groups of facial image pairs, and constructing an initial facial image processing model by a generative network and an adversarial network; the face image pair comprises a first-resolution face image and a second-resolution face image which corresponds to the first-resolution face image and is higher in resolution; inputting each first-resolution face image into a generative network to obtain a face simulation image; obtaining a generative adversarial network loss value, a target pixel loss value and a target total variation loss value based on an adversarial sample pair formed by the second resolution facial image and the facial simulation image; and carrying out weighted summation on the obtained loss values, and adjusting the initial face image processing model based on a model loss value obtained by summation to obtain a target face image processing model. When the target face image processing model constructed by the method is used for processing the face image, the face recognition success rate can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment, storage medium and computer program product for constructing a facial image processing model and a facial image processing method. Background Art

[0002] Currently, if you need to handle business at a financial institution, users usually need to authenticate their identity through facial recognition at the terminal, and can only handle business after passing the identity authentication. With the popularization of application software of financial institutions, users do not need to use the offline terminals of financial institutions, and can perform facial image acquisition, facial recognition, and identity authentication at personal terminals.

[0003] In traditional technology, when a user performs online identity authentication, the server of a financial institution generally performs face recognition directly based on the user's facial image collected by the terminal. However, the quality of the facial image collected by the terminal is easily affected by environmental factors and terminal configuration parameters. If the resolution of the facial image collected by the terminal is not high, it is easy to cause face recognition failure, affecting the user's business. Therefore, in traditional technology, the method of directly performing face recognition on users based on the collected facial image will result in a low success rate of face recognition. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for constructing a facial image processing model that can improve the success rate of face recognition, as well as a facial image processing method, device, computer equipment, computer-readable storage medium and computer program product in response to the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a method for constructing a facial image processing model. The method comprises:

[0006] Acquire a training data set and an initial facial image processing model constructed based on a generative network and an adversarial network; the training data set includes a plurality of facial image pairs; the facial image pairs include a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image; and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image;

[0007] Inputting each first-resolution facial image into a generation network respectively to obtain a facial simulation image corresponding to each first-resolution facial image;

[0008] The second resolution facial image and the facial simulation image corresponding to each first resolution facial image are respectively used as adversarial sample pairs, input into the adversarial network, obtain the generative adversarial network loss value, and obtain the target pixel loss value and the target total variational loss value based on each adversarial sample pair;

[0009] The loss value of the generated adversarial network, the target pixel loss value and the target total variational loss value are weighted summed to obtain the model loss value;

[0010] Based on the model loss value, the model parameters of the initial facial image processing model are adjusted to obtain the target facial image processing model.

[0011] In one embodiment, the generation network includes multiple feature extractors, at least one convolutional layer, and at least one fully connected layer; the feature extractor includes a multi-head attention network and a feedforward neural network.

[0012] In one embodiment, the target pixel loss value is obtained by:

[0013] For each of the adversarial sample pairs, a pixel-by-pixel comparison is performed between the second-resolution facial image and the facial simulation image in the adversarial sample pair to obtain a pixel loss value corresponding to the adversarial sample pair;

[0014] Based on the corresponding pixel loss values ​​of each adversarial sample pair, the target pixel loss value is obtained.

[0015] In one embodiment, the target total variational loss value is obtained by:

[0016] For each of the adversarial example pairs, determining the total variation corresponding to the second resolution facial image and the facial simulation image in the adversarial example pair;

[0017] Based on the total variation of the second-resolution facial image and the facial simulation image in the targeted adversarial sample pair, respectively, obtaining a total variation loss value corresponding to the targeted adversarial sample pair;

[0018] Based on the corresponding total variational loss values ​​of each adversarial sample pair, the target total variational loss value is obtained.

[0019] In one embodiment, based on the model loss value, adjusting the model parameters of the initial facial image processing model to obtain the target facial image processing model includes:

[0020] Based on the model loss value, adjusting the model parameters of the initial facial image processing model to obtain an updated facial image processing model;

[0021] Inputting the training data set into the updated facial image processing model to obtain an updated loss value corresponding to the updated facial image processing model;

[0022] Based on the updated loss value, adjusting the model parameters of the updated facial image processing model, and using the adjusted model as a new updated facial image processing model;

[0023] Jump to the step of inputting the training data set into the updated facial image processing model to obtain the updated loss value corresponding to the updated facial image processing model, until the latest updated loss value meets the model stop training condition, and obtains the target facial image processing model.

[0024] In a second aspect, the present application also provides a device for constructing a facial image processing model. The device comprises:

[0025] A model acquisition module is used to acquire a training data set and an initial facial image processing model constructed based on a generative network and an adversarial network; the training data set includes a plurality of facial image pairs; the facial image pairs include a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image; and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image;

[0026] A simulated image acquisition module, used for inputting each first-resolution facial image into a generation network to obtain a facial simulated image corresponding to each first-resolution facial image;

[0027] A loss value acquisition module is used to input the second resolution facial image and the facial simulation image corresponding to each first resolution facial image as adversarial sample pairs into the adversarial network to obtain a generative adversarial network loss value, and based on each adversarial sample pair, obtain a target pixel loss value and a target total variational loss value;

[0028] The loss value weighting module is used to perform weighted summation on the generative adversarial network loss value, the target pixel loss value and the target total variational loss value to obtain the model loss value;

[0029] The model parameter adjustment module is used to adjust the model parameters of the initial facial image processing model based on the model loss value to obtain the target facial image processing model.

[0030] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0031] Acquire a training data set and an initial facial image processing model constructed based on a generative network and an adversarial network; the training data set includes a plurality of facial image pairs; the facial image pairs include a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image; and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image;

[0032] Inputting each first-resolution facial image into a generation network respectively to obtain a facial simulation image corresponding to each first-resolution facial image;

[0033] The second resolution facial image and the facial simulation image corresponding to each first resolution facial image are respectively used as adversarial sample pairs, input into the adversarial network, obtain the generative adversarial network loss value, and obtain the target pixel loss value and the target total variational loss value based on each adversarial sample pair;

[0034] The loss value of the generated adversarial network, the target pixel loss value and the target total variational loss value are weighted summed to obtain the model loss value;

[0035] Based on the model loss value, the model parameters of the initial facial image processing model are adjusted to obtain the target facial image processing model.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0037] Acquire a training data set and an initial facial image processing model constructed based on a generative network and an adversarial network; the training data set includes a plurality of facial image pairs; the facial image pairs include a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image; and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image;

[0038] Inputting each first-resolution facial image into a generation network respectively to obtain a facial simulation image corresponding to each first-resolution facial image;

[0039] The second resolution facial image and the facial simulation image corresponding to each first resolution facial image are respectively used as adversarial sample pairs, input into the adversarial network, obtain the generative adversarial network loss value, and obtain the target pixel loss value and the target total variational loss value based on each adversarial sample pair;

[0040] The loss value of the generated adversarial network, the target pixel loss value and the target total variational loss value are weighted summed to obtain the model loss value;

[0041] Based on the model loss value, the model parameters of the initial facial image processing model are adjusted to obtain the target facial image processing model.

[0042] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0043] Acquire a training data set and an initial facial image processing model constructed based on a generative network and an adversarial network; the training data set includes a plurality of facial image pairs; the facial image pairs include a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image; and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image;

[0044] Inputting each first-resolution facial image into a generation network respectively to obtain a facial simulation image corresponding to each first-resolution facial image;

[0045] The second resolution facial image and the facial simulation image corresponding to each first resolution facial image are respectively used as adversarial sample pairs, input into the adversarial network, obtain the generative adversarial network loss value, and obtain the target pixel loss value and the target total variational loss value based on each adversarial sample pair;

[0046] The loss value of the generated adversarial network, the target pixel loss value and the target total variational loss value are weighted summed to obtain the model loss value;

[0047] Based on the model loss value, the model parameters of the initial facial image processing model are adjusted to obtain the target facial image processing model.

[0048] The method, apparatus, computer device, storage medium and computer program product for constructing the facial image processing model described above first obtain a training data set including a plurality of facial image pairs and an initial facial image processing model constructed based on a generative network and an adversarial network, wherein the facial image pairs consist of a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image, and the resolution of the second-resolution facial image is higher than that of the first-resolution facial image, and then each first-resolution facial image is respectively input into the generative network to obtain a facial simulation image corresponding to each first-resolution facial image, and further, the second-resolution facial image and the facial simulation image corresponding to each first-resolution facial image are respectively input into the generative network to obtain a facial simulation image corresponding to each first-resolution facial image, and further, the second-resolution facial image and the facial simulation image corresponding to each first-resolution facial image are respectively input into the generative network to obtain a facial simulation image corresponding to each first-resolution facial image, and further, the facial simulation image and the facial simulation ... Different images are used as adversarial sample pairs and input into the adversarial network to obtain the generative adversarial network loss value, and based on each adversarial sample pair, the target pixel loss value and the target total variational loss value are obtained, thereby weighted summing the generative adversarial network loss value, the target pixel loss value and the target total variational loss value to obtain the model loss value, and based on the model loss value, the three dimensions of adversarial network loss, pixel loss and total variational loss are comprehensively considered to adjust the model parameters of the initial facial image processing model to obtain the target facial image processing model that can convert low-resolution images into high-resolution images, so that before face recognition, the resolution of the collected facial image can be improved based on the target facial image processing model to improve the success rate of face recognition.

[0049] In a sixth aspect, the present application also provides a facial image processing method. The method comprises:

[0050] In response to an identity authentication request from the subject, acquiring a first facial image of the subject;

[0051] In the case where the resolution of the first facial image does not meet the facial recognition condition, the first facial image is input into a target facial image processing model to obtain a second facial image of the object; the resolution of the second facial image is higher than the resolution of the first facial image; the target facial image processing model is constructed by the above-mentioned facial image processing model construction method;

[0052] Based on the second facial image, the subject is authenticated.

[0053] In a seventh aspect, the present application also provides a facial image processing device. The device comprises:

[0054] A first facial image acquisition module, configured to acquire a first facial image of the object in response to an identity authentication request of the object;

[0055] A second facial image acquisition module is used to input the first facial image into a target facial image processing model to obtain a second facial image of the object when the resolution of the first facial image does not meet the facial recognition condition; the resolution of the second facial image is higher than the resolution of the first facial image; the target facial image processing model is constructed by the above-mentioned facial image processing model construction method;

[0056] The identity authentication module is used to authenticate the identity of the object based on the second facial image.

[0057] In an eighth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0058] In response to an identity authentication request from the subject, acquiring a first facial image of the subject;

[0059] In the case where the resolution of the first facial image does not meet the facial recognition condition, the first facial image is input into a target facial image processing model to obtain a second facial image of the object; the resolution of the second facial image is higher than the resolution of the first facial image; the target facial image processing model is constructed by the above-mentioned facial image processing model construction method;

[0060] Based on the second facial image, the subject is authenticated.

[0061] In a ninth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0062] In response to an identity authentication request from the subject, acquiring a first facial image of the subject;

[0063] In the case where the resolution of the first facial image does not meet the facial recognition condition, the first facial image is input into a target facial image processing model to obtain a second facial image of the object; the resolution of the second facial image is higher than the resolution of the first facial image; the target facial image processing model is constructed by the above-mentioned facial image processing model construction method;

[0064] Based on the second facial image, the subject is authenticated.

[0065] In a tenth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0066] In response to an identity authentication request from the subject, acquiring a first facial image of the subject;

[0067] In the case where the resolution of the first facial image does not meet the facial recognition condition, the first facial image is input into a target facial image processing model to obtain a second facial image of the object; the resolution of the second facial image is higher than the resolution of the first facial image; the target facial image processing model is constructed by the above-mentioned facial image processing model construction method;

[0068] Based on the second facial image, the subject is authenticated.

[0069] The above-mentioned facial image processing method, apparatus, computer equipment, storage medium and computer program product can obtain a first facial image of the object in response to an identity authentication request of the object. When the resolution of the first facial image does not meet the facial recognition conditions, the first facial image is input into the target facial image processing model to obtain a second facial image with a higher resolution than the first facial image. Therefore, based on the second facial image with higher resolution, the face of the object can be recognized more accurately during the identity authentication process, which can improve the success rate of face recognition and avoid face recognition failure due to the low resolution of the collected facial image. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 FIG. 1 is an application environment diagram of a method for constructing a facial image processing model in one embodiment;

[0071] Figure 2 is a flow chart of a method for constructing a facial image processing model in one embodiment;

[0072] Figure 3 A schematic diagram of a process for training a facial image processing model in one embodiment;

[0073] Figure 4 is a flowchart of a facial image processing method in one embodiment;

[0074] Figure 5 is a structural block diagram of a device for constructing a facial image processing model in one embodiment;

[0075] Figure 6 is a structural block diagram of a facial image processing device in one embodiment;

[0076] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0078] The method for constructing a facial image processing model provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The server 104 of the financial institution may first obtain a training data set and an initial facial image processing model constructed based on a generative network and an adversarial network, wherein the training data set includes multiple groups of facial image pairs, the facial image pairs include a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image, and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image, and each first-resolution facial image is input into the generative network to obtain a facial simulation image corresponding to each first-resolution facial image. Then, the server 104 may use the second-resolution facial image and the facial simulation image corresponding to each first-resolution facial image as adversarial sample pairs, respectively, and input them into the adversarial network to obtain a generative adversarial network loss value, and based on each adversarial sample pair, obtain a target pixel loss value and a target total variational loss value, thereby obtaining a model loss value by weighted summing the generative adversarial network loss value, the target pixel loss value and the target total variational loss value, and then based on the model loss value, adjust the model parameters of the initial facial image processing model to obtain a target facial image processing model.

[0079] Furthermore, if the server 104 of the financial institution receives an identity authentication request initiated by the terminal 102 used by the user, the server 104 can process the facial image of the user collected by the terminal 102 based on the obtained target facial image processing model to improve the success rate of face recognition in the identity authentication process. Among them, the financial institution can specifically be an institution engaged in the financial industry. The terminal 102 used by the user can be a terminal deployed offline by the financial institution, or it can be a terminal held by the user personally. The terminal 102 includes but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0080] In one embodiment, Figure 2 As shown, a method for constructing a facial image processing model is provided, and the method is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0081] Step 202, obtaining a training data set and an initial facial image processing model constructed based on a generative network and an adversarial network; the training data set includes multiple groups of facial image pairs; the facial image pairs include a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image; and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image.

[0082] The image resolution threshold can be set according to the actual application scenario, and the image with a resolution lower than the resolution threshold is used as the first resolution image, and the image with a resolution higher than the resolution threshold is used as the second resolution image. For each facial image pair, the first resolution facial image and the second resolution facial image in the facial image pair have the same image content, and only the resolution is different.

[0083] The initial facial image processing model can be specifically a model built based on a generative adversarial network (GAN). The generative adversarial network is a deep learning model that can generate images through a generative network (G, Generator) and an adversarial network (D, ) produces fairly good outputs through mutual game learning. The generative network can be used to generate data, such as pictures, while the adversarial network can be used to determine the authenticity of the data generated by the generative network.

[0084] Optionally, the server can construct a training data set based on shared facial images published on the network, or based on facial images stored in a data storage system of a financial institution, and store the constructed training data set in the data storage system for subsequent acquisition. Furthermore, the server can construct an initial facial image processing model based on a generative network and an adversarial network, and store the constructed initial facial image processing model in the data storage system for subsequent acquisition.

[0085] Exemplarily, taking the example of a server constructing a training data set based on shared facial images published on the network, the server can divide the acquired facial images into first-resolution facial images and second-resolution facial images according to a set resolution threshold. Furthermore, the server can construct facial image pairs based on first-resolution facial images and second-resolution facial images that have the same image content but differ only in resolution, and obtain multiple groups of facial image pairs, thereby obtaining a training data set. Similarly, the server can construct multiple groups of facial image pairs based on facial images stored in the data storage system of a financial institution, thereby obtaining a training data set.

[0086] Step 204: Input each first-resolution facial image into a generation network to obtain a facial simulation image corresponding to each first-resolution facial image.

[0087] Among them, the facial simulation image corresponding to each first-resolution facial image can specifically refer to: a facial simulation image whose image content is the same as the first-resolution facial image, whose resolution generated by the generation network is higher than the resolution of the first-resolution facial image, and whose resolution is higher than the set resolution threshold.

[0088] Optionally, the server may input each first-resolution facial image into the generation network respectively, and obtain facial simulation images corresponding to each first-resolution facial image generated by the generation network according to a set image generation method. The image generation method is set on the premise that the resolution of the facial simulation image generated by the generation network is higher than the first-resolution facial image, and the image content in the first-resolution facial image is not damaged or changed.

[0089] In step 206, the second resolution facial image and the facial simulation image corresponding to each first resolution facial image are respectively used as adversarial sample pairs, and input into the adversarial network to obtain a generative adversarial network loss value, and based on each adversarial sample pair, a target pixel loss value and a target total variational loss value are obtained.

[0090] Among them, the loss value of the generated adversarial network can be specifically based on the original loss function of the generated adversarial network Calculated, Representation generation network, Representation adversarial network. The smaller the loss value of the generative adversarial network, the better the parameters of the representation generation network.

[0091] For each adversarial sample, the pixel loss value is obtained by comparing the second-resolution facial image and the facial simulation image pixel by pixel, and taking the absolute value of the difference between the pixels of the second-resolution facial image and the facial simulation image. For example, for each adversarial sample pair, the sum of the pixels of the second-resolution facial image is subtracted from the sum of the pixels of the facial simulation image, and the absolute value of the obtained difference is taken to obtain the pixel loss value corresponding to the adversarial sample pair.

[0092] Among them, for each adversarial sample, the total variational loss value is calculated based on the total variational loss function. The calculation result of the total variational loss function of an image represents: in an image, the sum of the differences between each pixel and other adjacent pixels. The total variational loss value between two images is obtained by taking the absolute value of the difference between the calculation results of the total variational loss function of the two images. For example, for each adversarial sample pair, the calculation result of the total variational loss function of the second-resolution facial image is subtracted from the calculation result of the total variational loss function of the facial simulation image, and the absolute value of the difference is taken to obtain the total variational loss value corresponding to the adversarial sample pair. For each adversarial sample, the smaller the total variational loss value, the closer the second-resolution facial image and the facial simulation image corresponding to the first-resolution facial image in the adversarial sample are, and the better the parameters of the generated network are.

[0093] Optionally, the server can use the second resolution facial images and facial simulation images corresponding to each first resolution facial image as adversarial sample pairs, input them into the adversarial network, and obtain the generative adversarial network loss value based on the original loss function of the generative adversarial network. For each adversarial sample pair, the server can calculate the pixel loss value and the target total variational loss value corresponding to each adversarial sample pair based on pixel-by-pixel comparison and the total variational loss function, so as to obtain the target pixel loss value and the target total variational loss value.

[0094] Step 208, performing weighted summation on the generative adversarial network loss value, the target pixel loss value, and the target total variational loss value to obtain the model loss value.

[0095] Optionally, the server may perform weighted summation of the generative adversarial network loss value, the target pixel loss value, and the target total variational loss value according to the weights set for each loss value to obtain the model loss value. The weights corresponding to the generative adversarial network loss value, the target pixel loss value, and the target total variational loss value may be flexibly configured according to actual needs.

[0096] Step 210, based on the model loss value, adjust the model parameters of the initial facial image processing model to obtain the target facial image processing model.

[0097] The target facial image processing model can convert a facial image with a lower resolution into a facial image with a higher resolution, which can provide more facial details and features, and the resolution can meet the face recognition conditions. The face recognition conditions can be: the resolution of the image is higher than the set resolution threshold, and face recognition will not fail due to the image resolution being too low.

[0098] Optionally, the server may feed back the model loss value to the generative network of the initial facial image processing model through back propagation to adjust the model parameters of the generative network, thereby adjusting the initial facial image processing model, and cyclically adjust the model parameters of the generative network until the target facial image processing model is obtained.

[0099] In the method for constructing the facial image processing model, a training data set including a plurality of facial image pairs and an initial facial image processing model constructed based on a generative network and an adversarial network are first obtained, wherein the facial image pair consists of a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image, and the resolution of the second-resolution facial image is higher than that of the first-resolution facial image. Then, each first-resolution facial image is respectively input into the generative network to obtain a facial simulation image corresponding to each first-resolution facial image. Furthermore, the second-resolution facial image and the facial simulation image corresponding to each first-resolution facial image are respectively used as adversarial sample pairs and input into the generative network. The adversarial network is input to obtain the generative adversarial network loss value, and based on each adversarial sample pair, the target pixel loss value and the target total variational loss value are obtained, thereby weighted summing the generative adversarial network loss value, and obtaining the model loss value. Based on the model loss value, the three dimensions of adversarial network loss, pixel loss and total variational loss are comprehensively considered to adjust the model parameters of the initial facial image processing model, and obtain the target facial image processing model that can convert low-resolution images into high-resolution images, so that before face recognition, the resolution of the collected facial image can be improved based on the target facial image processing model, thereby improving the success rate of face recognition.

[0100] In one embodiment, the generation network includes multiple feature extractors, at least one convolutional layer, and at least one fully connected layer; the feature extractor includes a multi-head attention network and a feedforward neural network.

[0101] Among them, the multi-head attention network is a network built based on the multi-head self-attention mechanism (MHSA), which is an extended form of the attention mechanism (SA). It can divide the input data into multiple heads, calculate the attention representation of each head in parallel, and linearly superimpose them to obtain the final multi-head attention representation. That is, the multi-head attention network can pay attention to the characteristics of the input data in multiple aspects, thereby improving the generalization ability of the model. The feedforward neural network can be regarded as a function, which realizes the complex mapping from the input space to the output space through multiple compounding of simple nonlinear functions. The function of the convolutional layer is to extract features from the input data. The fully connected layer is used to integrate the features extracted by the previous layer (such as the convolutional layer).

[0102] Optionally, the server can first construct multiple feature extractors based on the multi-head attention network and the feedforward neural network to more comprehensively extract features in the image based on the multiple feature extractors. Furthermore, the server can use the outputs of the multiple feature extractors as the input of the first convolutional layer, and after processing based on at least one convolutional layer, use the output of the last convolutional layer as the input of the fully connected layer, and use the output of the fully connected layer as the output of the generating network.

[0103] Exemplarily, the server may input each first-resolution facial image into a generation network. For each first-resolution facial image, the server may extract features of the first-resolution facial image based on the generation network, and based on the extracted features, construct a second-resolution facial image having the same image content as the first-resolution facial image but with a higher resolution.

[0104] In this embodiment, a generative network is constructed by fusing a multi-head attention network with a feedforward neural network, so that the generative network can more comprehensively extract features in the first-resolution facial image, thereby ensuring that the features in the generated second-resolution facial image are also relatively comprehensive and closer to the content in the first-resolution facial image, without affecting the image content.

[0105] In one embodiment, the target pixel loss value is obtained by:

[0106] For each of the adversarial sample pairs, a pixel-by-pixel comparison is performed between the second-resolution facial image and the facial simulation image in the adversarial sample pair to obtain a pixel loss value corresponding to the adversarial sample pair;

[0107] Based on the corresponding pixel loss values ​​of each adversarial sample pair, the target pixel loss value is obtained.

[0108] Among them, the pixel value is the value assigned by the computer when the image is digitized, representing the average brightness information of a pixel in the image.

[0109] Optionally, for each of the adversarial sample pairs, the server may perform a pixel-by-pixel comparison between the second-resolution facial image and the facial simulation image in the adversarial sample pair to obtain the difference in pixel values ​​corresponding to each pixel position in the second-resolution facial image and the facial simulation image, thereby obtaining the sum of the differences in pixel values ​​at all pixel positions, and using the obtained sum as the corresponding pixel loss value of the adversarial sample pair. Furthermore, the server may average the corresponding pixel loss values ​​of each adversarial sample pair to obtain the target pixel loss value.

[0110] Exemplarily, for a certain adversarial sample pair, assuming that for pixel coordinate X, the pixel value in the second-resolution facial image is Y, and the pixel value in the facial simulation image is Z, then |YZ| represents the difference in pixel values ​​corresponding to pixel position X, and the sum of the differences in pixel values ​​corresponding to all pixel coordinates of the second-resolution facial image and the facial simulation image is the pixel loss value corresponding to the adversarial sample pair.

[0111] In this embodiment, the concept of pixel loss value is introduced so that the parameters of the generation network can be optimized based on the target pixel loss value in the future, so that the pixel value of the facial simulation image generated by the generation network after the final training can be close to the pixel value of the second resolution facial image, so that the target pixel loss value is less than the set target pixel loss value threshold, or the target pixel loss value converges.

[0112] In one embodiment, the target total variational loss value is obtained by:

[0113] For each of the adversarial example pairs, determining the total variation corresponding to the second resolution facial image and the facial simulation image in the adversarial example pair;

[0114] Based on the total variation of the second-resolution facial image and the facial simulation image in the targeted adversarial sample pair, respectively, obtaining a total variation loss value corresponding to the targeted adversarial sample pair;

[0115] Based on the corresponding total variational loss values ​​of each adversarial sample pair, the target total variational loss value is obtained.

[0116] Optionally, for each of the adversarial sample pairs, the server can determine the corresponding total variations of the second-resolution facial image and the facial simulation image in the adversarial sample pair based on the total variational loss function, and thus obtain the corresponding total variational loss value of the adversarial sample pair based on the absolute value of the difference between the corresponding total variations of the second-resolution facial image and the facial simulation image in the adversarial sample pair, and then average the corresponding total variational loss values ​​of each adversarial sample pair to obtain the target total variational loss value.

[0117] In this embodiment, the concept of total variational loss value is introduced so that the parameters of the generative network can be optimized based on the target total variational loss value in the future, so that the total variation of the facial simulation image generated by the generative network after the final training can be close to the total variation of the second-resolution facial image, so that the target total variational loss value is less than the set target total variational loss value threshold, or the target total variational loss value converges.

[0118] In one embodiment, based on the model loss value, adjusting the model parameters of the initial facial image processing model to obtain the target facial image processing model includes:

[0119] Based on the model loss value, adjusting the model parameters of the initial facial image processing model to obtain an updated facial image processing model;

[0120] Inputting the training data set into the updated facial image processing model to obtain an updated loss value corresponding to the updated facial image processing model;

[0121] Based on the updated loss value, adjusting the model parameters of the updated facial image processing model, and using the adjusted model as a new updated facial image processing model;

[0122] Jump to the step of inputting the training data set into the updated facial image processing model to obtain the updated loss value corresponding to the updated facial image processing model, until the latest updated loss value meets the model stop training condition, and obtains the target facial image processing model.

[0123] The model training stop condition may specifically be: the update loss value is less than the set value, or the update loss value converges.

[0124] Optionally, after the server inputs the training data set into the initial facial image processing model and obtains the model loss value, the server can adjust the model parameters of the initial facial image processing model based on the model loss value to obtain an updated facial image processing model, and then input the training data set into the updated facial image processing model to obtain an updated loss value corresponding to the updated facial image processing model, thereby adjusting the model parameters of the generated network in the updated facial image processing model based on the updated loss value, and using the adjusted model as the new updated facial image processing model, and then jumping to the step of inputting the training data set into the updated facial image processing model to obtain an updated loss value corresponding to the updated facial image processing model, until the latest updated loss value meets the model training stop condition to obtain the target facial image processing model.

[0125] For example, Figure 3As shown, after the server executes step 302, inputs the training data set into the initial facial image processing model, and obtains the model loss value through step 304, the server can execute step 306, adjust the model parameters of the initial facial image processing model based on the model loss value, and obtain the adjusted updated facial image processing model through step 308. Further, the server can execute step 310, input the training data set into the updated facial image processing model, and obtain the updated loss value through step 312, and determine whether the updated loss value meets the model stop training condition. If it does, execute step 314 to use the updated facial image processing model as the target facial image processing model. Otherwise, execute step 316 to adjust the model parameters of the updated facial image processing model based on the updated loss value, and execute step 318 to use the adjusted model as the new updated facial image processing model, and then jump to step 310 until the updated loss value of the new updated facial image processing model meets the model stop training condition to obtain the target facial image processing model.

[0126] In this embodiment, the model loss value obtained by weighted summing the generative adversarial network loss value, the target pixel loss value and the target total variational loss value is used to set the model training stop condition. The three dimensions of the generative adversarial network loss value, the target pixel loss value and the target total variational loss value can be comprehensively considered to achieve a more comprehensive optimization of the model. Ultimately, the weighted sum of the adversarial network loss value, the target pixel loss value and the target total variational loss value of the target facial image processing model finally trained is less than the set value, or converges, that is, the facial simulation graphics output by the target facial image processing model are close to the real high-resolution facial image, which can have a good effect on improving the resolution of low-resolution facial images without destroying the image content, which is beneficial to improving the accuracy and success rate of face recognition.

[0127] The facial image processing method provided in the embodiment of the present application can also be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 of the financial institution can respond to the identity authentication request initiated by the object at the terminal 102, obtain the first facial image of the object, and if the resolution of the first facial image does not meet the facial recognition condition, input the first facial image into the target facial image processing model to obtain the second facial image of the object, and then authenticate the object based on the second facial image.

[0128] In one embodiment, Figure 4As shown, a facial image processing method is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0129] Step 402: In response to an identity authentication request of the subject, a first facial image of the subject is acquired.

[0130] Optionally, the server may communicate with the terminal of the object, respond to the identity authentication request of the object, and control the terminal of the object to capture the first facial image of the object, thereby acquiring the first facial image of the object.

[0131] Step 404, when the resolution of the first facial image does not meet the facial recognition condition, the first facial image is input into a target facial image processing model to obtain a second facial image of the object; the resolution of the second facial image is higher than the resolution of the first facial image; the target facial image processing model is constructed by the above-mentioned facial image processing model construction method.

[0132] Optionally, the server may determine the resolution of the first facial image, and if the resolution of the first facial image does not meet the facial recognition condition, input the first facial image into a target facial image processing model to obtain a second facial image of the object.

[0133] Step 406: Authenticate the subject based on the second facial image.

[0134] Optionally, the server may perform face recognition on the second facial image based on the second facial image, thereby performing identity authentication on the object based on the face recognition result.

[0135] In the above-mentioned facial image processing method, a first facial image of the object can be obtained in response to an identity authentication request of the object. When the resolution of the first facial image does not meet the facial recognition conditions, the first facial image is input into the target facial image processing model to obtain a second facial image with a higher resolution than the first facial image. Therefore, based on the second facial image with higher resolution, the face of the object can be recognized more accurately during the identity authentication process, which can improve the success rate of face recognition and avoid face recognition failure due to the low resolution of the collected facial image.

[0136] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0137] Based on the same inventive concept, the embodiment of the present application also provides a facial image processing model construction device for implementing the facial image processing model construction method involved above, and a facial image processing device of the facial image processing method. The implementation solutions to the problems provided by the facial image processing model construction device and the facial image processing device are similar to the implementation solutions described in the facial image processing model construction method and the facial image processing method described above, so the specific limitations in the embodiments of one or more facial image processing model construction devices and facial image processing devices provided below can refer to the limitations of the facial image processing model construction method and the facial image processing method described above, and will not be repeated here.

[0138] In one embodiment, Figure 5 As shown, a device for constructing a facial image processing model is provided, comprising: a model acquisition module 502, a simulated image acquisition module 504, a loss value acquisition module 506, a loss value weighting module 508 and a model parameter adjustment module 510, wherein:

[0139] A model acquisition module is used to acquire a training data set and an initial facial image processing model constructed based on a generative network and an adversarial network; the training data set includes a plurality of facial image pairs; the facial image pairs include a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image; and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image;

[0140] A simulated image acquisition module, used for inputting each first-resolution facial image into a generation network to obtain a facial simulated image corresponding to each first-resolution facial image;

[0141] A loss value acquisition module is used to input the second resolution facial image and the facial simulation image corresponding to each first resolution facial image as adversarial sample pairs into the adversarial network to obtain a generative adversarial network loss value, and based on each adversarial sample pair, obtain a target pixel loss value and a target total variational loss value;

[0142] The loss value weighting module is used to perform weighted summation on the generative adversarial network loss value, the target pixel loss value and the target total variational loss value to obtain the model loss value;

[0143] The model parameter adjustment module is used to adjust the model parameters of the initial facial image processing model based on the model loss value to obtain the target facial image processing model.

[0144] In the above-mentioned facial image processing model construction device, a training data set including multiple groups of facial image pairs and an initial facial image processing model constructed based on a generative network and an adversarial network are first obtained, wherein the facial image pair consists of a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image, and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image. Then, each first-resolution facial image is respectively input into the generative network to obtain a facial simulation image corresponding to each first-resolution facial image. Furthermore, the second-resolution facial image and the facial simulation image corresponding to each first-resolution facial image are respectively used as adversarial sample pairs and input into the generative network. The adversarial network is input to obtain the generative adversarial network loss value, and based on each adversarial sample pair, the target pixel loss value and the target total variational loss value are obtained, thereby weighted summing the generative adversarial network loss value, and obtaining the model loss value. Based on the model loss value, the three dimensions of adversarial network loss, pixel loss and total variational loss are comprehensively considered to adjust the model parameters of the initial facial image processing model, and obtain the target facial image processing model that can convert low-resolution images into high-resolution images, so that before face recognition, the resolution of the collected facial image can be improved based on the target facial image processing model, thereby improving the success rate of face recognition.

[0145] In one embodiment, the generation network includes multiple feature extractors, at least one convolutional layer, and at least one fully connected layer; the feature extractor includes a multi-head attention network and a feedforward neural network.

[0146] In one of the embodiments, the construction of the facial image processing model also includes a target pixel loss value acquisition module, which is used to perform a pixel-by-pixel comparison between the second resolution facial image and the facial simulation image in each adversarial sample pair, and obtain the corresponding pixel loss value of the adversarial sample pair, thereby obtaining the target pixel loss value based on the corresponding pixel loss value of each adversarial sample pair.

[0147] In one of the embodiments, the construction of the facial image processing model also includes a target total variational loss value acquisition module, which is used to determine the corresponding total variation of the second resolution facial image and the facial simulation image in the targeted adversarial sample pair for each of the adversarial sample pairs, and obtain the corresponding total variational loss value of the targeted adversarial sample pair based on the corresponding total variation of the second resolution facial image and the facial simulation image in the targeted adversarial sample pair, thereby obtaining the target total variational loss value based on the corresponding total variational loss value of each adversarial sample pair.

[0148] In one embodiment, the model parameter adjustment module is also used to adjust the model parameters of the initial facial image processing model based on the model loss value to obtain an updated facial image processing model, and then input the training data set into the updated facial image processing model to obtain an updated loss value corresponding to the updated facial image processing model, thereby adjusting the model parameters of the updated facial image processing model based on the updated loss value, and using the adjusted model as the new updated facial image processing model, and then jumping to the step of inputting the training data set into the updated facial image processing model to obtain an updated loss value corresponding to the updated facial image processing model, until the latest updated loss value meets the model training stop condition to obtain the target facial image processing model.

[0149] In one embodiment, Figure 6 As shown, a facial image processing device is provided, comprising: a first facial image acquisition module 602, a second facial image acquisition module 604 and an identity authentication module 606, wherein:

[0150] A first facial image acquisition module, configured to acquire a first facial image of the object in response to an identity authentication request of the object;

[0151] A second facial image acquisition module is used to input the first facial image into a target facial image processing model to obtain a second facial image of the object when the resolution of the first facial image does not meet the facial recognition condition; the resolution of the second facial image is higher than the resolution of the first facial image; the target facial image processing model is constructed by the above-mentioned facial image processing model construction method;

[0152] The identity authentication module is used to authenticate the identity of the object based on the second facial image.

[0153] In the above-mentioned facial image processing device, a first facial image of the object can be obtained in response to an identity authentication request of the object. When the resolution of the first facial image does not meet the facial recognition condition, the first facial image is input into the target facial image processing model to obtain a second facial image with a higher resolution than the first facial image. Therefore, based on the second facial image with higher resolution, the face of the object can be recognized more accurately during the identity authentication process, which can improve the success rate of face recognition and avoid face recognition failure due to the low resolution of the collected facial image.

[0154] The above-mentioned facial image processing model construction device and each module in the facial image processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0155] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the construction of a facial image processing model and facial image processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for constructing a facial image processing model and a facial image processing method are implemented.

[0156] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0157] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method for constructing the facial image processing model and the steps in the facial image processing method embodiment are implemented.

[0158] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for constructing the facial image processing model and the steps in the facial image processing method embodiment are implemented.

[0159] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the method for constructing the facial image processing model and the steps in the facial image processing method embodiment.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0162] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for constructing a facial image processing model, characterized in that: The method comprises: Acquire a training data set and an initial facial image processing model constructed based on a generative network and an adversarial network; the training data set includes a plurality of facial image pairs; the facial image pairs include a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image; and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image; Inputting each of the first-resolution facial images into the generation network respectively to obtain a facial simulation image corresponding to each of the first-resolution facial images; The second resolution facial image and the facial simulation image corresponding to each of the first resolution facial images are respectively used as adversarial sample pairs, and are input into the adversarial network to obtain a generative adversarial network loss value, and based on each adversarial sample pair, a target pixel loss value and a target total variational loss value are obtained; Performing a weighted summation on the generative adversarial network loss value, the target pixel loss value, and the target total variational loss value to obtain a model loss value; Based on the model loss value, the model parameters of the initial facial image processing model are adjusted to obtain a target facial image processing model.

2. The method according to claim 1, characterized in that The generation network includes multiple feature extractors, at least one convolutional layer and at least one fully connected layer; the feature extractor includes a multi-head attention network and a feedforward neural network.

3. The method according to claim 1, characterized in that: The target pixel loss value is obtained by: For each of the adversarial sample pairs, a pixel-by-pixel comparison is performed between the second-resolution facial image and the facial simulation image in the adversarial sample pair to obtain a pixel loss value corresponding to the adversarial sample pair; Based on the corresponding pixel loss values ​​of each adversarial sample pair, a target pixel loss value is obtained.

4. The method according to claim 1, characterized in that: The target total variational loss value is obtained by: For each of the adversarial sample pairs, determining the total variation corresponding to the second-resolution facial image and the facial simulation image in the adversarial sample pair; Obtaining a total variation loss value corresponding to the adversarial sample pair based on the total variation corresponding to the second resolution facial image and the facial simulation image in the adversarial sample pair; Based on the corresponding total variational loss values ​​of each pair of adversarial examples, a target total variational loss value is obtained.

5. The method according to claim 1, characterized in that: The step of adjusting the model parameters of the initial facial image processing model based on the model loss value to obtain a target facial image processing model comprises: Based on the model loss value, adjusting the model parameters of the initial facial image processing model to obtain an updated facial image processing model; Inputting the training data set into the updated facial image processing model to obtain an updated loss value corresponding to the updated facial image processing model; Based on the update loss value, adjusting the model parameters of the updated facial image processing model, and using the adjusted model as a new updated facial image processing model; Jump to the step of inputting the training data set into the updated facial image processing model to obtain the updated loss value corresponding to the updated facial image processing model, until the latest updated loss value meets the model stop training condition, and obtains the target facial image processing model.

6. A facial image processing method, characterized in that: The method comprises: In response to an identity authentication request of a subject, acquiring a first facial image of the subject; In the case where the resolution of the first facial image does not meet the facial recognition condition, the first facial image is input into a target facial image processing model to obtain a second facial image of the object; the resolution of the second facial image is higher than the resolution of the first facial image; the target facial image processing model is constructed by the method described in any one of claims 1 to 5; Based on the second facial image, identity authentication is performed on the object.

7. A device for constructing a facial image processing model, characterized in that: The device comprises: A model acquisition module, used to acquire a training data set and an initial facial image processing model constructed based on a generative network and an adversarial network; the training data set includes a plurality of facial image pairs; the facial image pairs include a first-resolution facial image and a second-resolution facial image corresponding to the first-resolution facial image; and the resolution of the second-resolution facial image is higher than the resolution of the first-resolution facial image; A simulated image acquisition module, used for inputting each of the first-resolution facial images into the generation network to obtain a facial simulated image corresponding to each of the first-resolution facial images; A loss value acquisition module, configured to input the second resolution facial image and the facial simulation image corresponding to each of the first resolution facial images as adversarial sample pairs into the adversarial network to obtain a generative adversarial network loss value, and obtain a target pixel loss value and a target total variational loss value based on each adversarial sample pair; A loss value weighting module, used to perform weighted summation on the generative adversarial network loss value, the target pixel loss value and the target total variational loss value to obtain a model loss value; A model parameter adjustment module is used to adjust the model parameters of the initial facial image processing model based on the model loss value to obtain a target facial image processing model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images