Training method and apparatus for image processing model, image processing method and apparatus, device, storage medium and program product

Abstract

Disclosed in the present application are a training method and apparatus for an image processing model, an image processing method and apparatus, a device, a storage medium and a program product. The training method comprises: acquiring a body part sample image, and inputting the body part sample image into an image processing model to execute the following processing: extracting a sample identity feature and a sample attribute feature of the body part sample image, the sample identity feature being used for indicating a user identity of the body part sample image, and the sample attribute feature being used for representing an attribute unrelated to the user identity; decoding the sample identity feature and the sample attribute feature to obtain a body part prediction image, wherein the body part prediction image is a body part image having the sample attribute feature and being used for indicating the user identity; and, with the training objective of reducing the error between the body part sample image and the body part prediction image, updating at least one model parameter of the image processing model.

Description

Image processing model training method, image processing method, device, equipment, storage medium and program product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on February 4, 2024, with application number 202410163236.0 and application name “Training method of image processing model, image processing method and device”. Technical Field

[0002] The present application relates to the field of artificial intelligence, and in particular to an image processing model training method, image processing method, device, storage medium and program product.

[0003] Background of the Invention

[0004] Biometrics technology uses biological characteristics (such as fingerprints, palm prints, and faces) and behavioral traits (such as voice and gait) to authenticate and identify individuals. For example, palm prints are the lines on the palm of your hand running from the base of your fingers to your wrist. Palm print recognition technology has been applied in a variety of applications, including biometrics, identity verification, and mobile payments.

[0005] In related art, palm images contain palmprint information, and palmprint features are directly learned from the palmprint information to generate a new palmprint prediction image based on the palmprint features. However, the identity information in the palmprint prediction image generated directly using the above palmprint information is not significant enough. Summary of the Invention

[0006] The present application provides an image processing model training method, image processing method, device, storage medium, and program product. The technical solution is as follows:

[0007] In one aspect, an embodiment of the present application provides a method for training an image processing model, performed by an electronic device, the method comprising:

[0008] Obtain a sample image of a body part, input the sample image of the body part into the image processing model, and perform the following processing:

[0009] Extracting a sample identity feature and a sample attribute feature of the body part sample image, wherein the sample identity feature is used to indicate the user identity of the body part sample image, and the sample attribute feature is used to characterize an attribute unrelated to the user identity;

[0010] Decoding the sample identity feature and the sample attribute feature to obtain a body part prediction image, wherein the body part prediction image is a body part image having the sample attribute feature and used to indicate the identity of the user; and

[0011] At least one model parameter of the image processing model is updated with reducing the error between the body part sample image and the body part prediction image as a training goal.

[0012] On the other hand, an embodiment of the present application provides an image processing method, performed by an electronic device, the method comprising:

[0013] Acquire a first body part image, and acquire a second body part image, wherein the first body part image is a body part image having a first identity feature and a first attribute feature, and the second body part image is a body part image having a second identity feature and a second attribute feature, wherein the first identity feature is used to indicate the identity of a user of the first body part image, and the second identity feature is used to indicate the identity of a user of the second body part image;

[0014] Inputting the first body part image into an image processing model to extract the first identity feature of the first body part image, and inputting the second body part image into the image processing model to extract the second attribute feature of the second body part image, wherein the second attribute feature is used to represent an attribute that is unrelated to the user identity of the second body part image; and

[0015] The first identity feature and the second attribute feature are decoded to obtain a body part target image, where the body part target image is a body part image having the first identity feature and the second attribute feature.

[0016] On the other hand, an embodiment of the present application provides a training device for an image processing model, comprising:

[0017] An acquisition module, used for acquiring sample images of body parts;

[0018] a processing module, configured to input the body part sample image into the image processing model, and extract a sample identity feature and a sample attribute feature of the body part sample image, wherein the sample identity feature is used to indicate the user identity of the body part sample image, and the sample attribute feature is used to represent an attribute unrelated to the user identity;

[0019] a decoding module, configured to decode the sample identity feature and the sample attribute feature based on the image processing model to obtain a body part prediction image, wherein the body part prediction image is a body part image having the sample attribute feature and used to indicate the identity of the user; and

[0020] A training module is configured to update at least one model parameter of the image processing model by taking reducing the error between the body part sample image and the body part prediction image as a training goal.

[0021] On the other hand, an embodiment of the present application provides an image processing device, including:

[0022] an acquisition module, configured to acquire a first body part image and a second body part image, wherein the first body part image is a body part image having a first identity feature and a first attribute feature, and the second body part image is a body part image having a second identity feature and a second attribute feature, wherein the first identity feature is used to indicate an identity of a user of the first body part image, and the second identity feature is used to indicate an identity of a user of the second body part image;

[0023] a processing module, configured to input the first body part image into an image processing model and extract the first identity feature of the first body part image, and input the second body part image into the image processing model and extract the second attribute feature of the second body part image, wherein the second attribute feature is used to represent an attribute unrelated to the user identity of the second body part image; and

[0024] A decoding module is used to decode the first identity feature and the second attribute feature to obtain a body part target image, where the body part target image is a body part image having the first identity feature and the second attribute feature.

[0025] On the other hand, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the training method of the image processing model as described above, or to implement the image processing method as described above.

[0026] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is loaded and executed by a processor to implement the training method of the image processing model as described above, or to implement the image processing method as described above.

[0027] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. A processor obtains the computer instructions from the computer-readable storage medium, so that the processor loads and executes them to implement the training method of the image processing model as described above, or to implement the image processing method as described above.

[0028] BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0030] FIG1 shows a block diagram of a computer system according to an exemplary embodiment of the present invention;

[0031] FIG2 is a schematic diagram showing a training method for an image processing model provided by an exemplary embodiment of the present application;

[0032] FIG3 is a schematic diagram showing an image processing method provided by an exemplary embodiment of the present application;

[0033] FIG4 shows a schematic diagram of obtaining a first body part image provided by an exemplary embodiment of the present application;

[0034] FIG5 shows a schematic diagram of obtaining a second body part image provided by an exemplary embodiment of the present application;

[0035] FIG6 is a schematic diagram showing a method for training an image processing model according to an exemplary embodiment of the present application;

[0036] FIG7 shows a schematic diagram of an image processing method provided by an exemplary embodiment of the present application;

[0037] FIG8 shows a schematic diagram of an image processing model provided by an exemplary embodiment of the present application;

[0038] FIG9 shows a schematic diagram of an image processing model provided by an exemplary embodiment of the present application;

[0039] FIG10 shows a schematic diagram of an image processing model provided by an exemplary embodiment of the present application;

[0040] FIG11 shows a flow chart of a method for training an image processing model provided by an exemplary embodiment of the present application;

[0041] FIG12 shows a flow chart of a method for training an image processing model provided by an exemplary embodiment of the present application;

[0042] FIG13 shows a flow chart of a method for training an image processing model provided by an exemplary embodiment of the present application;

[0043] FIG14 shows a flowchart of a method for training an image processing model provided by an exemplary embodiment of the present application;

[0044] FIG15 shows a flowchart of an image processing method provided by an exemplary embodiment of the present application;

[0045] FIG16 is a schematic diagram showing a method for training an image processing model according to an exemplary embodiment of the present application;

[0046] FIG17 is a schematic diagram showing an image processing method provided by an exemplary embodiment of the present application;

[0047] FIG18 shows a block diagram of a training device for an image processing model provided by an exemplary embodiment of the present application;

[0048] FIG19 shows a block diagram of an image processing apparatus provided by an exemplary embodiment of the present application;

[0049] FIG20 shows a structural block diagram of an electronic device provided by an exemplary embodiment of the present application.

[0050] Implementation Method

[0051] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0052] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0053] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any or all possible combinations of at least one of the associated listed items.

[0054] It should be understood that although the terms first, second, etc. may be used in this application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0055] It should be noted that this application can display a prompt interface, pop-up window or output voice prompt information before collecting user-related data (such as: user identity, sample user identity, body part sample image, first body part image, second body part image, palm sample image, first palm image, second palm image, face sample image, first face image, second face image) and during the process of collecting user-related data. The prompt interface, pop-up window or voice prompt information is used to remind the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining user-related data after obtaining the user's confirmation operation on the prompt interface or pop-up window. Otherwise (that is, when the user's confirmation operation on the prompt interface or pop-up window is not obtained), the relevant steps of obtaining user-related data are terminated, that is, the user's relevant data is not obtained. In other words, all user data collected by this application are collected with the user's consent and authorization, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0056] First, the terms used in all embodiments of the present application are briefly described.

[0057] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to achieve the best results.

[0058] The image processing model is a neural network model provided in embodiments of the present application, and is used to generate new body part images based on known body part images. For example, during the training phase of the image processing model, the image processing model is used to generate a predicted body part image based on sample body part images. During the use phase of the image processing model, the image processing model is used to generate a target body part image based on a first body part image and a second body part image.

[0059] When the application scenario is palmprint recognition, this image processing model is also called a palmprint image processing model. During the training phase of the image processing model, it is used to obtain a predicted palmprint image based on the sample palm image. During the use phase of the image processing model, it is used to obtain a target palmprint image based on the first palm image and the second palm image.

[0060] When used for face recognition, this image processing model is also called a face image processing model. During the training phase, the model is used to generate a predicted face image based on a sample face image. During the operational phase, the model is used to generate a target face image based on a first face image and a second face image.

[0061] Recognition model (Rec_model): is a pre-trained recognition model for extracting identity features. When the application scenario is a palmprint recognition scenario, the recognition model is used to extract palmprint features. When the application scenario is a face recognition scenario, the recognition model is used to extract facial features. In some embodiments, the recognition model can be pre-trained on sample images of body parts using a mobile face network (MobileFaceNet) as the basic model structure and an Additive Angular Margin Loss (ArcFace Loss) as the loss function. Among them, MobileFaceNet is a network model running on a mobile device. The size of a single network model is only 4M, and by adopting the above-mentioned ArcFace loss function, the accuracy of the recognition model reaches more than 99%.

[0062] Body part sample images: are body part images in the training sample set used in the training phase of the image processing model. In this embodiment, the electronic device can process at least one body part sample image or each body part sample image in the training sample set. When the application scenario is a palmprint recognition scenario, the body part is the hand, specifically the palm, and the body part sample image is a palm sample image. When the application scenario is a face recognition scenario, the body part is the face, and the body part sample image is a face sample image. Specifically, the body part sample image refers to the body part image of the sample user, which can be obtained by using a shooting device to shoot the body parts of different sample users in different shooting environments. In some embodiments, different shooting environments include indoor environments, outdoor environments, sunlight environments, light environments, and so on.

[0063] The sample body part images have sample identity features that are used to indicate the identity of the sample user. When the application scenario is palmprint recognition, the sample identity features are sample palmprint features. When the application scenario is face recognition, the sample identity features are sample face features.

[0064] The body part sample images also have sample attribute features, which are used to characterize attributes unrelated to the user's identity. Exemplarily, the attributes include at least one of a photographic attribute and a muscle attribute, and the sample attribute features include at least one of a sample photographic attribute feature and a sample muscle attribute feature.

[0065] In some embodiments, the sample shooting attributes refer to attributes related to the captured sample images of body parts, including at least one of the pixels, resolution, light and shadow, size, color, bit depth, hue, saturation, and brightness of the sample images of body parts, and the sample shooting attribute features include at least one of the sample pixel features, sample resolution features, sample light and shadow features, sample size features, sample color features, sample bit depth features, sample hue features, sample saturation features, and sample brightness features.

[0066] A pixel, also known as a picture element, is the smallest unit in an image. A pixel is a small square in an image, each with a specific position and assigned color value. The position and color value of each square determine the appearance of the image. Image resolution determines the fineness of image detail; higher image resolution indicates clearer images. Light and shadow refer to light and shadow. Light refers to the light produced by a light source (at least one of the sun and a light bulb), while shadow refers to the cast shadow on a background or object. Light and shadow can be used to describe at least one of depth, texture, and texture in an image. Size refers to the size of an image, including its length and width. Color refers to the color of an image, including at least one of color and grayscale. Bit depth refers to the number of binary bits used to describe the color of an image. Hue refers to the relative brightness or darkness of an image, manifesting as color in color images. Saturation, also known as purity, refers to the vividness of a color. Brightness refers to the brightness, which can be represented by the pixel value of the image. The larger the pixel value, the brighter the pixel point corresponding to the pixel value in the image, otherwise the darker it is.

[0067] In some embodiments, the sample muscle attribute refers to the attribute of the muscle itself in the body part in the body part sample image, including at least one of muscle direction, muscle shape, muscle texture, muscle protrusion height, and muscle sink depth. The sample muscle attribute feature includes at least one of sample muscle direction feature, sample muscle shape feature, sample muscle texture feature, sample muscle protrusion height feature, and sample muscle sink depth feature.

[0068] The muscle direction includes at least one of horizontal, longitudinal, oblique from the inside to the outside of the body, and oblique from the outside to the inside of the body. Muscle shape includes at least one of spindle, annular, flaky, triangular, single-pinnate, bi-pinnate, and multi-pinnate. Muscle texture includes at least one of horizontal, vertical, and diagonal. Muscle protrusion height refers to the protrusion height of a muscle in a region relative to the surrounding muscles. Muscle depression depth refers to the depression depth of a muscle in a region relative to the surrounding muscles.

[0069] Palm sample images: These are palm images from the training sample set used during the training phase of the image processing model when the application scenario is palmprint recognition. Specifically, palm sample images refer to palm images of sample users and can be obtained by photographing the palms of different sample users using a camera under different shooting environments. In some embodiments, the different shooting environments include indoor environments, outdoor environments, sunlight environments, light environments, and so on.

[0070] The palm sample image has sample palmprint features, which are used to indicate the user identity of the sample user. In some embodiments, a palmprint includes at least one of main lines, wrinkles, fine textures, ridge ends, and bifurcation points. The sample palmprint features include at least one of main line features, wrinkle features, texture features, ridge end features, bifurcation point features, length features, vein features, and overall shape distribution features.

[0071] The palm sample image also has a sample attribute feature, which is used to characterize an attribute unrelated to the user's identity. Exemplarily, the attribute includes at least one of a photographic attribute and a muscle attribute, and the sample attribute feature includes at least one of a sample photographic attribute feature and a sample muscle attribute feature.

[0072] In some embodiments, the shooting attributes refer to the attributes of the palm sample image itself, including at least one of the pixels, resolution, light and shadow, size, color, bit depth, hue, saturation, and brightness of the palm sample image, and the sample shooting attribute features include at least one of the sample pixel features, sample resolution features, sample light and shadow features, sample size features, sample color features, sample bit depth features, sample hue features, sample saturation features, and sample brightness features.

[0073] In some embodiments, muscle attributes refer to the attributes of the palm muscles themselves in the palm sample image, including at least one of muscle direction, muscle shape, muscle texture, muscle protrusion height, and muscle sink depth. The sample muscle attribute features include at least one of sample muscle direction features, sample muscle shape features, sample muscle texture features, sample muscle protrusion height features, and sample muscle sink depth features.

[0074] Sample face images: These are facial images from a training sample set used during the training phase of an image processing model when the application scenario is face recognition. Specifically, sample face images refer to facial images of sample users, which can be obtained by photographing the faces of different sample users using a camera in different shooting environments. In some embodiments, the different shooting environments include indoor environments, outdoor environments, sunlight environments, lighting environments, and so on.

[0075] The sample face image has sample face features, which are used to indicate the user identity of the sample user. In some embodiments, the face includes at least one of skin, eyes, forehead, ears, nose, and mouth. The sample face features include at least one of sample skin features, sample eye features, sample forehead features, sample ear features, sample nose features, sample mouth features, sample face shape features, and sample facial features overall distribution features. Sample skin features include at least one of sample skin color features and sample skin smoothness features. Sample eye features include at least one of sample eye size features, sample eye spacing features, and sample eye shape features. Sample forehead features include at least one of sample forehead size features and sample forehead shape features. Sample ear features include at least one of sample ear size features and sample ear shape features. Sample nose features include at least one of sample nose size features and sample nose shape features. Sample mouth features include at least one of sample mouth size features and sample mouth shape features.

[0076] The sample face image also has a sample attribute feature, which is used to characterize an attribute unrelated to the user's identity. Exemplarily, the attribute includes at least one of a photographic attribute and a muscle attribute, and the sample attribute feature includes at least one of a sample photographic attribute feature and a sample muscle attribute feature.

[0077] In some embodiments, the shooting attributes refer to the attributes of the image itself of the face sample image, including at least one of the pixels, resolution, light and shadow, size, color, bit depth, hue, saturation, and brightness of the face sample image, and the sample shooting attribute characteristics include at least one of the sample pixel characteristics, sample resolution characteristics, sample light and shadow characteristics, sample size characteristics, sample color characteristics, sample bit depth characteristics, sample hue characteristics, sample saturation characteristics, and sample brightness characteristics.

[0078] In some embodiments, muscle attributes refer to the attributes of the facial muscles themselves in the facial sample image, including at least one of muscle direction, muscle shape, muscle texture, muscle protrusion height, and muscle sink depth. The sample muscle attribute features include at least one of sample muscle direction features, sample muscle shape features, sample muscle texture features, sample muscle protrusion height features, and sample muscle sink depth features.

[0079] Predicted body part image: This is an image generated during the training phase of the image processing model based on the input body part sample image. This predicted body part image is a realistic representation of the sample body part image, possessing the sample attribute characteristics and used to indicate the user's identity. Theoretically, this predicted body part image should be the same as the user's identity indicated by the sample body part image. In some embodiments, this predicted body part image can also be understood as a body part image obtained by simulating a photograph of the user's body part under a photographic environment corresponding to the sample attribute characteristics.

[0080] A predicted palmprint image is a palmprint image obtained during the training phase of the image processing model based on an input palm sample image when the application scenario is palmprint recognition. This predicted palmprint image is a realistic image of the palm sample image, possessing sample attribute characteristics and used to indicate the user's identity. Theoretically, this predicted palmprint image should be the same as the user's identity indicated by the palm sample image. In some embodiments, this predicted palmprint image can also be understood as a palmprint image obtained by simulating a photograph of the user's palm under a shooting environment corresponding to the sample attribute characteristics.

[0081] Predicted face image: When the application scenario is a face recognition scenario, during the training phase of the image processing model, the face image is obtained based on the input face sample image. The predicted face image is a realistic image of the face sample image. The predicted face image is a face image with sample attribute characteristics and is used to indicate the user's identity. In theory, the predicted face image should be the same as the user identity indicated by the face sample image. In some embodiments, it can also be understood that the predicted face image is a face image obtained by simulating the user's face under the shooting environment corresponding to the sample attribute characteristics.

[0082] The first body part image is a body part image used during the use phase of the image processing model. When the application scenario is a palmprint recognition scenario, the first body part image is a first palm image. When the application scenario is a face recognition scenario, the first body part image is a first face image. Specifically, the first body part image is a body part image obtained by photographing a body part of the first user using a camera in a first shooting environment. In some embodiments, the first shooting environment includes an indoor environment, an outdoor environment, a sunlight environment, a light environment, etc.

[0083] The first body part image has a first identity feature and a first attribute feature, the first identity feature is used to indicate the user identity of the first user, and the first attribute feature is used to characterize an attribute unrelated to the user identity, the attribute including at least one of a first shooting attribute and a first muscle attribute.

[0084] A first palm image is a palm image used during the image processing model's application phase when the application scenario is palmprint recognition. Specifically, the first palm image is obtained by photographing the palm of a first user using a camera under a first shooting environment. The first palm image has a first palmprint feature and a first attribute feature. The first palmprint feature indicates the identity of the first user, and the first attribute feature represents an attribute unrelated to the user's identity, including at least one of a first shooting attribute and a first muscle attribute.

[0085] In some embodiments, the first palm print feature includes at least one of the following: a main line feature of the first user's palm print, a wrinkle feature, a texture feature, a ridge tip feature, a bifurcation point feature, a length feature, a vein feature, and an overall shape distribution feature of the palm print. The first shooting attribute feature includes at least one of a pixel feature, a resolution feature, a light and shadow feature, a size feature, a color feature, a bit depth feature, a hue feature, a saturation feature, and a brightness feature of the first palm image. The first muscle attribute feature includes at least one of a muscle direction feature, a muscle shape feature, a muscle texture feature, a muscle bulge height feature, and a muscle depression depth feature of the first palm image.

[0086] A first facial image is a facial image used during the image processing model's usage phase when the application scenario is a facial recognition scenario. Specifically, the first facial image is a facial image captured by a camera device under a first shooting environment and obtained by capturing the face of the first user. The first facial image has a first facial feature and a first attribute feature. The first facial feature is used to indicate the identity of the first user, and the first attribute feature is used to represent an attribute unrelated to the user's identity, including at least one of a first shooting attribute and a first muscle attribute.

[0087] In some embodiments, the first facial features include at least one of the first user's skin features, eye features, forehead features, ear features, nose features, mouth features, facial shape features, and overall facial features distribution features. Skin features include at least one of skin color features and skin smoothness features. Eye features include at least one of eye size features, eye spacing features, and eye shape features. Forehead features include at least one of forehead size features and forehead shape features. Ear features include at least one of ear size features and ear shape features. Nose features include at least one of nose size features and nose shape features. Mouth features include at least one of mouth size features and mouth shape features. The first shooting attribute features include at least one of pixel features, resolution features, lighting features, size features, color features, bit depth features, hue features, saturation features, and brightness features of the first facial image. The first muscle attribute features include at least one of muscle orientation features, muscle shape features, muscle texture features, muscle protrusion height features, and muscle depression depth features of the first facial image.

[0088] The second body part image is a body part image used during the use phase of the image processing model. When the application scenario is a palmprint recognition scenario, the second body part image is a second palm image. When the application scenario is a face recognition scenario, the second body part image is a second face image. Specifically, the second body part image is a body part image obtained by photographing a body part of the second user using a camera in a second shooting environment. In some embodiments, the second shooting environment includes an indoor environment, an outdoor environment, a sunlight environment, a light lighting environment, and the like.

[0089] The second body part image has a second identity feature and a second attribute feature, the second identity feature is used to indicate the user identity of the second user, and the second attribute feature is used to characterize an attribute unrelated to the user identity, which includes at least one of a second shooting attribute and a second muscle attribute.

[0090] The second palm image is used during the image processing model's application phase when the application scenario is palmprint recognition. Specifically, the second palm image is obtained by photographing the second user's palm using a camera under a second shooting environment. The second palm image has second palmprint features and second attribute features. The second palmprint features are used to indicate the second user's identity, and the second attribute features are used to represent attributes unrelated to the user's identity, including at least one of the second shooting attributes and the second muscle attributes.

[0091] In some embodiments, the second palm print features include at least one of the following: palm print main line features, palm print wrinkle features, palm print texture features, palm print ridge tip features, palm print bifurcation point features, palm print length features, palm vein features, and palm print overall shape distribution features. The second shooting attribute features include at least one of the following: pixel features, resolution features, lighting features, size features, color features, bit depth features, hue features, saturation features, and brightness features of the second palm image. The second muscle attribute features include at least one of the following: muscle direction features, muscle shape features, muscle texture features, muscle protrusion height features, and muscle depression depth features of the second palm image.

[0092] The second facial image is a facial image used during the image processing model's usage phase when the application scenario is facial recognition. Specifically, the second facial image is a facial image obtained by photographing the second user's face using a camera under a second shooting environment. The second facial image has second facial features and second attribute features. The second facial features are used to indicate the second user's identity, and the second attribute features are used to represent attributes unrelated to the user's identity, including at least one of the second shooting attribute and the second muscle attribute.

[0093] In some embodiments, the second facial features include at least one of the second user's skin features, eye features, forehead features, ear features, nose features, mouth features, facial shape features, and overall facial features distribution features. Skin features include at least one of skin color features and skin smoothness features. Eye features include at least one of eye size features, eye spacing features, and eye shape features. Forehead features include at least one of forehead size features and forehead shape features. Ear features include at least one of ear size features and ear shape features. Nose features include at least one of nose size features and nose shape features. Mouth features include at least one of mouth size features and mouth shape features. Second shooting attribute features include at least one of pixel features, resolution features, lighting features, size features, color features, bit depth features, hue features, saturation features, and brightness features of the second facial image. Second muscle attribute features include at least one of muscle orientation features, muscle shape features, muscle texture features, muscle bulge height features, and muscle depression depth features of the facial muscles in the second facial image.

[0094] Body Part Target Image: This is a body part image obtained during the image processing model's usage phase based on the first and second body part images. When the application scenario is palmprint recognition, the body part target image is the palmprint target image. When the application scenario is face recognition, the body part target image is the face target image.

[0095] The body part target image has a first identity feature and a second attribute feature, wherein the first identity feature is used to indicate the user identity of the first user. That is, the embodiment of the present application can generate a body part target image with controllable identity features and controllable attribute features according to actual technical needs.

[0096] A palmprint target image is a palmprint image obtained from the first and second palm images during the use phase of the image processing model when the application scenario is palmprint recognition. The palmprint target image has a first palmprint feature and a second attribute feature, where the first palmprint feature indicates the user identity of the first user. In other words, embodiments of the present application can generate a palmprint target image with controllable palmprint features and controllable attribute features based on actual technical needs.

[0097] A target facial image is a facial image generated during the image processing model's usage phase based on the first and second facial images when the application scenario is face recognition. The target facial image has a first facial feature and a second attribute feature, where the first facial feature indicates the identity of the first user. In other words, embodiments of the present application can generate a target facial image with controllable facial features and attribute features based on actual technical needs.

[0098] FIG1 shows a block diagram of a computer system 100 provided by an exemplary embodiment of the present application. The computer system 100 can be implemented as a system architecture for a training method for an image processing model and / or an image processing method. The computer system 100 includes a terminal 120 and a server 140.

[0099] The terminal 120 can be an electronic device such as a mobile phone, a tablet computer, a vehicle-mounted terminal (vehicle computer), a wearable device, a PC (Personal Computer), an unmanned reservation terminal, etc. The terminal 120 can be installed with a client that runs a target application. The target application can be a training application that is used to train the model parameters of the image processing model, or the application can be an inference application that is used to run the image processing model to obtain a target image of a body part. This application does not limit this. This application does not limit the form of the target application, including but not limited to App (Application) installed in the terminal 120, applet, etc., and can also be in the form of a web page.

[0100] Server 140 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud computing services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Server 140 can be the backend server of the target application described above, used to provide backend services to the client of the target application.

[0101] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to enable data computing, storage, processing, and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology, all based on the cloud computing business model. It can form a resource pool that can be used on demand with flexibility and convenience. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identification mark and will need to be transmitted to backend systems for logical processing. Data of varying levels will be processed separately, and data from all industries will require a strong system backend, which can only be achieved through cloud computing.

[0102] In some embodiments, the server 140 can also be implemented as a node in a blockchain system. Blockchain is a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity of the information (for anti-counterfeiting) and generate the next block. Blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0103] The terminal 120 and the server 140 may communicate with each other via a network, such as a wired or wireless network.

[0104] The training method and / or image processing method of the image processing model provided in the embodiment of the present application, the execution subject of each step can be an electronic device, and the electronic device refers to an electronic device with data calculation, processing and storage capabilities. Taking the solution implementation environment shown in Figure 1 as an example, the training method and / or image processing method of the image processing model can be executed by the terminal 120 (such as the client of the target application installed and running in the terminal 120 executes the training method and / or image processing method of the image processing model), or the training method and / or image processing method of the image processing model can be executed by the server 140, or the terminal 120 and the server 140 can interact and cooperate to execute, and this application does not limit this.

[0105] Those skilled in the art will appreciate that the number of the terminals 120 may be greater or less. For example, there may be only one terminal 120, or there may be dozens, hundreds, or even more terminals 120. The embodiment of the present application does not limit the number or device type of the terminals 120.

[0106] Take palmprint recognition as an example. Palmprints refer to the lines on the palm from the base of the fingers to the wrist. They include at least one of the following: main lines, wrinkles, fine lines, ridge endings, and bifurcation points. Palmprint recognition technology has been applied in various application scenarios, including biometrics, identity verification, and mobile payments.

[0107] Related technologies primarily directly learn palmprint information from palm images to obtain palmprint features, and then generate new predicted palmprint images based on these features. Because related technologies fail to decouple various types of information coupled to palmprint features, the identity information in the extracted palmprint features is insufficiently significant. Consequently, the identity information in the generated predicted palmprint images is insufficiently significant, and the predicted palmprint images may not clearly indicate the user's identity.

[0108] Figure 2 shows a schematic diagram of a training method for an image processing model provided by an exemplary embodiment of the present application. The method is performed by an electronic device, which may be the terminal 120 and / or the server 140 shown in Figure 1. Figure 2 illustrates an example of an electronic device being the server 140.

[0109] In some embodiments, server 140 stores an image processing model 10 (see FIG2 ). Image processing model 10 includes an encoder 11 and a decoder 13. The output of encoder 11 is connected to the input of decoder 13. Encoder 11 may specifically include an attribute decoupling encoder 111 and an identity encoder 112. Attribute decoupling encoder 111 and identity encoder 112 are parallel encoders. The output of attribute decoupling encoder 111 is connected to the input of decoder 13, and the output of identity encoder 112 is connected to the input of decoder 13.

[0110] In some embodiments, the image processing model 10 further includes a splicing layer 12, and thus the image processing model 10 includes an encoder 11, a splicing layer 12, and a decoder 13. The output of the encoder 11 is connected to the input of the splicing layer 12, and the output of the splicing layer 12 is connected to the input of the decoder 13. The encoder 11 may specifically include an attribute decoupling encoder 111 and an identity encoder 112. The output of the attribute decoupling encoder 111 is connected to the input of the splicing layer 12, the output of the identity encoder 112 is connected to the input of the splicing layer 12, and the output of the splicing layer 12 is connected to the input of the decoder 13.

[0111] Exemplarily, the steps of the training method of the image processing model 10 are briefly described as follows:

[0112] Step 1. Obtain a body part sample image; the body part sample image has a sample identity feature, and the sample identity feature is used to indicate the user identity of the body part sample image, and each user has a user identity.

[0113] In some embodiments, the training sample set includes sample images of body parts. The training sample set may be a public dataset obtained from a public database. Alternatively, the training sample set may be a dataset obtained by photographing body parts of sample users using a camera. In this embodiment, the electronic device may perform processing on at least one or each sample image of a body part in the training sample set.

[0114] Step 2. Input the body part sample images into the image processing model, extract the sample identity features of the body part sample images, and extract the sample attribute features of the body part sample images; the sample attribute features are used to represent attributes that are not related to the user identity.

[0115] In some embodiments, the sample identity features of the body part sample images are extracted by the identity encoder 112 , and the sample attribute features of the body part sample images are extracted by the attribute decoupling encoder 111 .

[0116] The sample identity feature is a feature used to indicate the user identity of the sample user. Each sample user's sample identity feature is unique. The sample identity feature can be represented as a feature vector, an embedding vector, or a feature matrix. The feature dimension of the sample identity feature can be 512. The feature elements in the sample identity feature can be represented as floating-point numbers.

[0117] The sample attribute feature is used to represent an attribute that is unrelated to the user identity. The attribute includes at least one of a shooting attribute and a muscle attribute. The sample attribute feature includes at least one of a sample shooting attribute feature and a sample muscle attribute feature.

[0118] It should be noted that the sample attribute features here are not related to the user identity. It can be understood that the correlation between the sample attribute features and the user identity features is very small, less than a certain threshold, or even completely unrelated; therefore, step 2 may include: when extracting the sample attribute features and the user identity features, performing correlation processing to reduce the correlation between the sample attribute features and the user identity features.

[0119] In some embodiments, the sample attribute feature can be represented as one of a feature vector, an embedding vector, and a feature matrix. The feature dimension of the sample attribute feature can be 512. Feature elements in the sample attribute feature can be represented by floating-point numbers.

[0120] Step 3. Decode the sample identity features and the sample attribute features to obtain a body part prediction image; the body part prediction image is a body part image with sample attribute features and used to indicate the user's identity.

[0121] In some embodiments, the sample identity features and the sample attribute features are decoded by the decoder 13 to obtain a body part prediction image; the body part prediction image is also used to indicate the user identity corresponding to the body part sample image.

[0122] In some embodiments, the sample identity features and the sample attribute features are spliced ​​together in row dimensions by the splicing layer 12 to obtain a sample splicing feature, and the sample splicing feature is decoded by the decoder 13 to obtain a body part prediction image.

[0123] In some embodiments, the image processing model 10 further includes a pooling layer. Before the sample identity features and the sample attribute features are spliced ​​together by the splicing layer 12 to obtain the sample splicing features, the sample attribute features can be first pooled by the pooling layer to obtain the pooled sample attribute features, so that the feature dimension of the pooled sample attribute features is reduced, redundancy is removed, and the sample attribute features and the sample identity features are further decoupled, and then the sample identity features and the pooled sample attribute features are spliced ​​together. Alternatively, after the sample identity features and the sample attribute features are spliced ​​together by the splicing layer 12 to obtain the sample splicing features, the sample splicing features can be pooled by the pooling layer to obtain the pooled sample splicing features. The feature dimension of the pooled sample splicing features is reduced, redundancy is removed, and the sample attribute features and the sample identity features are further decoupled.

[0124] Step 4. Update at least one model parameter of the image processing model 10 with the reduction of the error between the body part sample image and the body part prediction image as the training goal.

[0125] In some embodiments, the at least one model parameter refers to all or part of the model parameters in the image processing model 10. The at least one model parameter includes all or part of the model parameters corresponding to the encoder 11 and the model parameters corresponding to the decoder 13.

[0126] In some embodiments, the at least one model parameter specifically includes a model parameter corresponding to the attribute decoupling encoder 111 in the encoder 11, and a model parameter corresponding to the decoder 13. In this case, the identity encoder can be considered to be an existing encoder or a pre-trained encoder that can accurately extract sample identity features indicating the user's identity, but the attribute decoupling encoder requires training to improve encoding accuracy.

[0127] In summary, during the training phase of the image processing model, sample identity features and sample attribute features are extracted from sample body part images, enabling the image processing model to extract sample identity features and sample attribute features that have a correlation below a threshold, or even completely unrelated. Since the sample attribute features are used to indicate attributes unrelated to the sample user's identity, that is, the user's identity is indicated solely through the sample identity features, the salience of the identity information of the sample identity features can be increased, thereby increasing the salience of the identity information in the predicted body part image corresponding to the sample body part image.

[0128] Palmprint recognition scenario

[0129] Figure 2 is a schematic diagram of a training method for an image processing model provided by an exemplary embodiment of the present application. Figure 3 is a schematic diagram of an image processing method provided by an exemplary embodiment of the present application. The method is performed by an electronic device, which may be the terminal 120 and / or the server 140 shown in Figure 1. Figures 2 and 3 illustrate the example of the electronic device being the server 140.

[0130] As shown in Figure 2, during the training stage of the image processing model 10, the electronic device inputs a palm sample image 161 into the image processing model 10, the attribute decoupling encoder 111 is used to extract sample attribute features 162 of the palm sample image 161, and the identity encoder 112 is used to extract sample palmprint features 163 of the palm sample image 161; the splicing layer 12 is used to splice the sample attribute features 162 and the sample palmprint features 163; and the decoder 13 is used to generate a palmprint prediction image 165.

[0131] As shown in Figure 3, during the use stage of the image processing model 10, the electronic device inputs the second palm image 171 into the image processing model 10, and the attribute decoupling encoder 111 is used to extract the second attribute feature 173 of the second palm image 171; the electronic device inputs the first palm image 172 into the image processing model 10, and the identity encoder 112 is used to extract the first palmprint feature 174 of the first palm image 172; the splicing layer 12 is used to splice the second attribute feature 173 and the first palmprint feature 174; the decoder 13 is used to generate a palmprint target image 176.

[0132] Exemplarily, as shown in FIG2 , the steps of the training method of the image processing model are briefly described as follows:

[0133] Step 1: Acquire a palm sample image 161 . The palm sample image 161 has a sample palm print feature 163 . The sample palm print feature 163 is used to indicate the identity of the user of the palm sample image 161 .

[0134] In some embodiments, the training sample set includes multiple palm sample images 161. The training sample set can be a public dataset obtained from a public database, such as a multispectral palm print dataset and a palm vein dataset. In this embodiment, the electronic device can perform processing on at least one or each palm sample image 161 in the training sample set.

[0135] Step 2: Input the palm sample image 161 into the image processing model 10, extract the sample palmprint features 163 of the palm sample image 161, and extract the sample attribute features 162 of the palm sample image 161; the sample attribute features 162 are used to represent attributes unrelated to the user identity.

[0136] In some embodiments, the identity encoder 112 extracts sample palmprint features 163 from the sample palm image 161, and the attribute decoupling encoder 111 extracts sample attribute features 162 from the sample palm image 161. The definitions of the sample palmprint features 163 and the sample attribute features 162 are as described above and will not be repeated here.

[0137] Step 3. Decode the sample palmprint features 163 and the sample attribute features 162 to obtain a palmprint prediction image 165; the palmprint prediction image 165 is a palmprint image having the sample attribute features 162 and used to indicate the identity of the user.

[0138] In some embodiments, the decoder 13 decodes the sample palmprint features 163 and the sample attribute features 162 to obtain a palmprint prediction image 165. The palmprint prediction image 165 is also used to indicate the user identity corresponding to the palm sample image 161.

[0139] In some embodiments, the sample palmprint features 163 and the sample attribute features 162 are spliced ​​together according to the row dimension by the splicing layer 12 to obtain a sample splicing feature 164. The sample splicing feature 164 is decoded by the decoder 13 to obtain a palmprint prediction image 165.

[0140] Step 4. Taking reducing the error between the palm sample image 161 and the palmprint prediction image 165 as the training goal, update at least one model parameter of the image processing model 10.

[0141] In some embodiments, the at least one model parameter refers to all or part of the model parameters in the image processing model 10. The at least one model parameter includes all or part of the model parameters corresponding to the encoder 11 and the model parameters corresponding to the decoder 13. In some embodiments, the at least one model parameter includes the model parameters corresponding to the attribute decoupling encoder 111 in the encoder 11 and the model parameters corresponding to the decoder 13.

[0142] In summary, during the training phase of the image processing model, by extracting sample palmprint features and sample attribute features from the sample palm images, the image processing model is able to extract sample palmprint features and sample attribute features with a correlation below a threshold, or even completely unrelated. Because the sample attribute features are used to indicate attributes unrelated to the sample user's identity, that is, the sample user's identity is indicated solely through the sample palmprint features, the salience of the identity information of the sample palmprint features can be increased, thereby increasing the salience of the identity information in the palmprint prediction image corresponding to the palm sample image 164.

[0143] As shown in Figure 3, the steps of the image processing method are briefly described as follows:

[0144] Step 1. Obtain a first palm image 172 and a second palm image 171 ; the first palm image 172 is a palm image having a first palm print feature 174 and a first attribute feature, and the second palm image 171 is a palm image having a second palm print feature and a second attribute feature 173 .

[0145] In some embodiments, the first palm image 172 and the second palm image 171 may be palm images obtained by photographing the palms of different real users in different shooting environments. That is, the first palm image 172 and the second palm image 171 may have different shooting environments, different palm print features, different shooting attribute features, and different muscle attribute features.

[0146] For example, referring to FIG4 , assuming that the first shooting environment is an outdoor bright light environment 113 and the shooting attribute is brightness, the brightness of the outdoor bright light environment 113 is a first brightness, and the illumination type is sunlight, then the first shooting attribute feature includes a first brightness feature. A first user 114 enters a house using a smart door lock 115 . The smart door lock 115 is equipped with a camera 1151 . The camera 1151 captures the palm of the first user 114 to obtain a first palm image 172 . The first palm print feature 174 is used to indicate the user identity of the first user 114 .

[0147] Referring to Figure 5 , for example, the second shooting environment is indoor lighting environment 116, and the shooting attribute is brightness. Indoor lighting environment 116 has a second brightness, and the lighting type is light illumination. The second shooting attribute characteristic includes a second brightness characteristic, which is different from the first brightness characteristic. A second user 117 pays for items in a shopping cart using a storage table 118. Storage table 118 is equipped with a camera 1181. Camera 1181 captures the palm of second user 117, generating a second palm image 171. The second palmprint characteristic is used to indicate the user identity of second user 117.

[0148] Step 2. Input the first palm image 172 into the image processing model 10 to extract the first palmprint feature 174 of the first palm image 172 , and input the second palm image 171 into the image processing model 10 to extract the second attribute feature 173 of the second palm image 171 .

[0149] In some embodiments, the identity encoder 112 extracts the first palm print feature 174 of the first palm image 172, and the attribute decoupling encoder 111 extracts the second attribute feature 173 of the second palm image 171. The definitions of the first palm print feature 174 and the second attribute feature 173 are as described above and will not be repeated here.

[0150] Step 3. Decode the first palmprint feature 174 and the second attribute feature 173 to obtain a palmprint target image 176 ; the palmprint target image 176 is a palmprint image having the first palmprint feature 174 and the second attribute feature 173 .

[0151] In some embodiments, the decoder 13 decodes the first palmprint feature 174 and the second attribute feature 173 to obtain a palmprint target image 176. The palmprint target image 176 has the first palmprint feature 174, and the first palmprint feature 174 is used to indicate the user identity of the first user.

[0152] In some embodiments, the first palmprint feature 174 and the second attribute feature 173 are spliced ​​together by the splicing layer 12 to obtain a target splicing feature 175. The target splicing feature 175 is decoded by the decoder 13 to obtain a palmprint target image 176.

[0153] To sum up, during the usage stage of the image processing model, the image processing model can extract the first palmprint feature and the second attribute feature whose correlation is lower than the threshold or even completely unrelated. Since the second attribute feature also does not include the second palmprint feature for indicating the user identity of the second user, the second attribute feature is only used to indicate the attribute that is not related to the user identity of the second user; in this way, the user identity of the palmprint target image can be controlled by the first palmprint feature, thereby improving the saliency of the identity information in the palmprint target image and the accuracy of indicating the user identity. The attribute of the palmprint target image can also be controlled by the second attribute feature, so that the image processing model can generate a new palmprint target image with controllable palmprint features and controllable attribute features, thereby improving the generation effect of the palmprint target image, and the palmprint target image can more accurately indicate the user identity.

[0154] Face recognition scenario

[0155] Figure 6 shows a schematic diagram of a training method for an image processing model provided by an exemplary embodiment of the present application. Figure 7 shows a schematic diagram of an image processing method provided by an exemplary embodiment of the present application. The method is performed by an electronic device, which may be the terminal 120 and / or the server 140 shown in Figure 1. Figures 6 and 7 illustrate the example of the electronic device being the server 140.

[0156] As shown in Figure 6, during the training stage of the image processing model 10, the electronic device inputs the face sample image 261 into the image processing model 10, the attribute decoupling encoder 111 is used to extract the sample attribute features 262 of the face sample image 261, and the identity encoder 112 is used to extract the sample face features 263 of the face sample image 261; the splicing layer 12 is used to splice the sample attribute features 262 and the sample face features 263; the decoder 13 is used to generate a face prediction image 265.

[0157] As shown in Figure 7, during the use stage of the image processing model 10, the electronic device inputs the second face image 271 into the image processing model 10, and the attribute decoupling encoder 111 is used to extract the second attribute feature 273 of the second face image 271; the electronic device inputs the first face image 272 into the image processing model 10, and the identity encoder 112 is used to extract the first face feature 274 of the first face image 272; the splicing layer 12 is used to splice the second attribute feature 273 and the first face feature 274; the decoder 13 is used to generate a face target image 276.

[0158] Exemplarily, as shown in FIG6 , the steps of the training method of the image processing model are briefly described as follows:

[0159] Step 1. Acquire a sample face image 261 , wherein the sample face image 261 has a sample face feature 263 ; the sample face feature 263 is used to indicate the user identity of the sample face image 261 .

[0160] In some embodiments, the training sample set includes multiple face sample images 261, which may be a public dataset obtained from a public database. In this embodiment, the electronic device may perform processing on at least one or each face sample image 261 in the training sample set.

[0161] Step 2. Input the face sample image 261 into the image processing model 10, extract the sample face features 263 of the face sample image 261, and extract the sample attribute features 262 of the face sample image 261; the sample attribute features 262 are used to represent attributes unrelated to the user identity.

[0162] In some embodiments, the identity encoder 112 extracts sample facial features 263 from the sample facial image 261, and the attribute decoupling encoder 111 extracts sample attribute features 262 from the sample facial image 261. The definitions of the sample facial features 263 and the sample attribute features 262 are as described above and will not be repeated here.

[0163] Step 3. Decode the sample facial features 263 and the sample attribute features 262 to obtain a predicted face image 265; the predicted face image 265 is a facial image having the sample attribute features 262 and used to indicate the identity of the user.

[0164] In some embodiments, the sample facial features 263 and the sample attribute features 262 are decoded by the decoder 13 to obtain a predicted face image 265. The predicted face image 265 is also used to indicate the user identity corresponding to the sample face image 261.

[0165] In some embodiments, the sample face features 263 and the sample attribute features 262 are spliced ​​together in row dimensions by the splicing layer 12 to obtain a sample spliced ​​feature 264. The sample spliced ​​feature 264 is decoded by the decoder 13 to obtain a predicted face image 265.

[0166] Step 4. Update at least one model parameter of the image processing model 10 with the goal of reducing the error between the face sample image 261 and the face prediction image 265 as the training objective.

[0167] In some embodiments, the at least one model parameter refers to all or part of the model parameters in the image processing model 10. The at least one model parameter includes all or part of the model parameters corresponding to the encoder 11 and the model parameters corresponding to the decoder 13. In some embodiments, the at least one model parameter includes the model parameters corresponding to the attribute decoupling encoder 111 in the encoder 11 and the model parameters corresponding to the decoder 13.

[0168] In summary, during the training phase of the image processing model, by extracting sample facial features from the sample face images and extracting sample attribute features from the sample face images, the image processing model is able to extract sample facial features and sample attribute features that have a correlation below a threshold, or are even completely unrelated. Since the sample attribute features are used to indicate attributes that are unrelated to the user identity of the sample user, that is, the user identity of the sample user is indicated solely through the sample facial features, the salience of the identity information of the sample facial features can be increased, thereby increasing the salience of the identity information in the predicted face image corresponding to the sample face image 261.

[0169] Exemplarily, as shown in FIG7 , the steps of the image processing method are briefly described as follows:

[0170] Step 1. Obtain a first facial image 272 and a second facial image 271 ; the first facial image 272 is a facial image having a first facial feature 274 and a first attribute feature, and the second facial image 271 is a facial image having a second facial feature and a second attribute feature 273 .

[0171] In some embodiments, first facial image 272 and second facial image 271 may be facial images of different real users captured in different shooting environments. That is, first facial image 272 and second facial image 271 may have different shooting environments, facial features, shooting attributes, and muscle attributes. This is as described above with respect to Figures 4 and 5 and will not be further elaborated here.

[0172] Step 2. Input the first facial image 272 into the image processing model 10 to extract the first facial features 274 of the first facial image 272 , and input the second facial image 271 into the image processing model 10 to extract the second attribute features 273 of the second facial image 271 .

[0173] In some embodiments, the identity encoder 112 extracts first facial features 274 from the first facial image 272, and the attribute decoupling encoder 111 extracts second attribute features 273 from the second facial image 271. The definitions of the first facial features 274 and the second attribute features 273 are as described above and will not be repeated here.

[0174] Step 3. Decode the first facial feature 274 and the second attribute feature 273 to obtain a facial target image 276; the facial target image 276 is a facial image having the first facial feature 274 and the second attribute feature 273.

[0175] In some embodiments, the decoder 13 decodes the first facial feature 274 and the second attribute feature 273 to obtain a target facial image 276. The target facial image 276 has the first facial feature 274, which is used to indicate the user identity of the first user.

[0176] In some embodiments, the first face feature 274 and the second attribute feature 273 are spliced ​​together by the splicing layer 12 to obtain a target splicing feature 175. The target splicing feature 175 is decoded by the decoder 13 to obtain a target face image 276.

[0177] To sum up, during the usage stage of the image processing model, the correlation between the second attribute feature extracted by the image processing model and the second facial feature is lower than the threshold, or even completely unrelated. The second attribute feature is only used to indicate attributes that are unrelated to the user identity of the second user. In this way, the user identity of the facial target image can be controlled by the first facial feature, thereby improving the saliency of the identity information in the facial target image and the accuracy of indicating the user identity. The attributes of the palm print target image can also be controlled by the second attribute feature, so that the image processing model can generate a new facial target image with controllable facial features and controllable attribute features, thereby improving the generation effect of the facial target image, and the facial target image can more accurately indicate the user identity.

[0178] Figures 8, 9 and 10 are schematic diagrams of an image processing model provided by an exemplary embodiment of the present application. The following describes the model structure of the image processing model in the embodiment.

[0179] In some embodiments, referring to FIG8 , image processing model 10 includes an encoder 11, a concatenation layer 12, and a decoder 13. Encoder 11 includes an identity encoder 112 and an attribute decoupling encoder 111. The specific definitions of each module are as described above in FIG2 , FIG3 , FIG6 , and FIG7 and are not repeated here.

[0180] In some embodiments, referring to FIG9 , the attribute decoupling encoder 111 includes an attribute encoder 111-1 and an orthogonal projection module 111-2, wherein the output of the attribute encoder 111-1 is connected to the input of the orthogonal projection module 111-2. During the training phase of the image processing model 10, the attribute encoder 111-1 is used to extract coupled attribute features from sample images of body parts; the orthogonal projection module 111-2 is used to orthogonally project the coupled attribute features into the feature space of the sample identity features to determine the sample projection features. The output of the attribute encoder 111-1 is connected to the input of the orthogonal projection module 111-2, and the output of the identity encoder 112 is also connected to the input of the orthogonal projection module 111-2, so that the orthogonal projection module 111-2 can obtain the sample identity features.

[0181] In some embodiments, the attribute encoder 111-1 can be implemented using a three-layer residual connection neural network block (Block), each block consisting of a convolution layer, a batch normalization layer, and an activation function (Linear Rectification Function, ReLU) layer. The orthogonal projection module 111-2 can be implemented using a three-layer residual connection block, each block consisting of a convolution layer, a batch normalization layer, and a ReLU layer.

[0182] The identity encoder 112 is implemented using a pre-trained recognition model, wherein the recognition model can use a mobile face network (MobileFaceNet) as the basic model structure and remove the fully connected layer in MobileFaceNet that finally maps the features to the identity ID, that is, the recognition model is a pre-trained mobile face network. The recognition model can use the Additive Angular Margin Loss (ArcFace Loss) as a loss function and be pre-trained on sample images of body parts. In the image processing model 10 of the embodiment of the present application, the identity encoder 112 retains the model parameters of the feature extraction part of the recognition model, and these model parameters are not updated during the training phase of the image processing model 10.

[0183] In some embodiments, the concatenation layer 12 is used to concatenate two or more features. During the training phase of the image processing model 10, the concatenation layer 12 is used to concatenate the sample identity feature and the sample attribute feature. During the operational phase of the image processing model 10, the concatenation layer 12 is used to concatenate the first identity feature and the second attribute feature. In some embodiments, the concatenation layer 12 is implemented as a concatenation layer.

[0184] In some embodiments, decoder 13 is used to generate a new body part prediction image. During the training phase of image processing model 10, decoder 13 is used to decode the sample identity features and sample attribute features to obtain the body part prediction image. During the use phase of image processing model 10, decoder 13 is used to decode the first identity features and the second attribute features to obtain the body part target image. In some embodiments, decoder 13 can be implemented using a deconvolution network. The deconvolution network can be a three-layer deconvolution network.

[0185] In some embodiments, referring to FIG10 , the image processing model 10 further includes a pooling layer 14, that is, the image processing model 10 includes an encoder 11, a splicing layer 12, a pooling layer 14, and a decoder 13. The output of the encoder 11 is connected to the input of the splicing layer 12, the output of the splicing layer 12 is connected to the input of the pooling layer 14, and the output of the pooling layer 14 is connected to the input of the decoder 13.

[0186] In some embodiments, the encoder 11 includes an identity encoder 112 and an attribute decoupling encoder 111. The attribute decoupling encoder 111 and the identity encoder 112 are parallel encoders. The output end of the attribute decoupling encoder 111 is connected to the input end of the splicing layer 12, the output end of the identity encoder 112 is connected to the input end of the splicing layer 12, the output end of the splicing layer 12 is connected to the input end of the pooling layer 14, and the output end of the pooling layer 14 is connected to the input end of the decoder 13.

[0187] In some embodiments, the attribute decoupling encoder 111 includes an attribute encoder 111-1 and an orthogonal projection module 111-2. The output of the attribute encoder 111-1 is connected to the input of the orthogonal projection module 111-2, the output of the orthogonal projection module 111-2 is connected to the input of the splicing layer 12, the output of the identity encoder 112 is connected to the input of the splicing layer 12, the output of the splicing layer 12 is connected to the input of the pooling layer 14, and the output of the pooling layer 14 is connected to the input of the decoder 13. In some embodiments, the pooling layer 14 is implemented using a pooling layer.

[0188] In the above embodiment, the image processing model 10 first uses the splicing layer 12 to splice the sample identity features and the sample attribute features to obtain the sample splicing features, and then uses the pooling layer 14 to pool the sample splicing features to obtain the pooled sample splicing features.

[0189] In other embodiments, the image processing model 10 may first use the pooling layer 14 to pool the sample attribute features to obtain the pooled sample attribute features, and then use the splicing layer 12 to splice the sample identity features with the pooled sample attribute features to obtain the sample splicing features. In this case, the output end of the encoder 11 of the image processing model 10 is connected to the input end of the pooling layer 14, the output end of the pooling layer 14 is connected to the input end of the splicing layer 12, and the output end of the splicing layer 12 is connected to the input end of the decoder 13. Specifically, the output end of the attribute decoupling encoder 111 is connected to the input end of the pooling layer 14.

[0190] FIG11 shows a flowchart of a method for training an image processing model provided by an exemplary embodiment of the present application. The method is executed by an electronic device storing an image processing model, which may be the terminal 120 and / or the server 140 shown in FIG1 . The method includes the following steps:

[0191] Step 210 , obtaining a sample image of a body part, inputting the sample image of the body part into the image processing model, and executing steps 220 , 230 , and 240 .

[0192] Body parts refer to parts of the human body, including at least one of the face, hands, chest, head, and legs. The features corresponding to each user's body part are unique and can be used to characterize the user's identity. This application's embodiments primarily illustrate the face and palm of a hand.

[0193] The training sample set is the dataset used in the training phase of the image processing model.

[0194] The body part images in the training sample set are referred to as body part sample images. The body part sample images refer to body part images of sample users and can be obtained by photographing body parts of different sample users under different shooting environments using a camera. The body part sample images include sample identity features, which are used to indicate the user identity of the sample users. In this embodiment, the electronic device may perform processing on at least one or each body part sample image in the training sample set.

[0195] In some embodiments, the training sample set may be a public data set obtained from a public database. Alternatively, the body part sample images in the training sample set may be obtained by pre-photographing body parts of a large number of sample users (more than a thousand people).

[0196] In some embodiments, the clarity of the body part sample images is required to be greater than or equal to a preset threshold in order to obtain body part sample images with higher clarity and ensure the accuracy of the image processing model.

[0197] Exemplarily, the electronic device obtains a sample image of a body part, where the sample image of the body part has a sample identity feature, and the sample identity feature is used to indicate the identity of the user of the sample image of the body part.

[0198] An image processing model is a neural network model to be trained to generate new body part images.

[0199] Step 220 , extracting sample identity features and sample attribute features of the body part sample images, wherein the sample identity features are used to indicate the user identity of the body part sample images, and the sample attribute features are used to characterize attributes unrelated to the user identity.

[0200] In some embodiments, the attributes unrelated to the user identity include at least one of a shooting attribute and a muscle attribute.

[0201] Shooting attributes refer to attributes related to shooting and unrelated to the user identity of the sample user when the body parts of the sample user are photographed under different shooting environments using a shooting device, or it can be understood that the shooting attributes are attributes of the image itself of the sample image of the body part.

[0202] In some embodiments, the capture attributes include at least one of pixels, resolution, lighting, size, color, bit depth, hue, saturation, and brightness of the sample body part image. The sample capture attribute features include at least one of sample pixel features, sample resolution features, sample lighting, sample size features, sample color features, sample bit depth features, sample hue features, sample saturation features, and sample brightness features.

[0203] Muscles are soft, elastic tissues attached to bones that have the ability to contract. Since every user's body part has muscles attached to it, and muscles are part of the posture presented by that body part itself, muscle attributes are also unrelated to the user identity of the sample user. In other words, muscle attributes refer to the properties of the muscles of the body part itself in the sample body part image.

[0204] In some embodiments, muscle attributes include at least one of muscle direction, muscle shape, muscle texture, muscle protrusion height, and muscle depression depth. Sample muscle attribute characteristics include at least one of sample muscle direction characteristics, sample muscle shape characteristics, sample muscle texture characteristics, sample muscle protrusion height characteristics, and sample muscle depression depth characteristics. Among them, muscle direction includes at least one of horizontal, longitudinal, oblique from the inside of the body to the outside of the body, and oblique from the outside of the body to the inside of the body. Muscle shape includes at least one of spindle, annular, flaky, triangular, single-pinnate, double-pinnate, and multi-pinnate. Muscle texture includes at least one of horizontal stripes, vertical stripes, and oblique stripes. Muscle protrusion height refers to the protrusion height of muscles in one area relative to muscles in the surrounding area. Muscle depression depth refers to the depression depth of muscles in one area relative to muscles in the surrounding area.

[0205] When the image processing model has high accuracy, the sample attribute features extracted by the image processing model may contain very few sample identity features, or even no sample identity features at all. It is understandable that in actual applications, due to the limitations on the number of sample images of body parts used for training, the training level of the image processing model, or the computing power of the electronic device on which the image processing model resides, the sample attribute features extracted by the image processing model may not completely exclude any sample identity features. This embodiment expects that the correlation between the sample attribute features and the sample identity features is less than a certain threshold.

[0206] For example, if the attribute is light and shadow, the image processing model needs to extract the sample attribute features of the sample body part image, namely the light and shadow features. Light and shadow features and sample identity features should be independent of each other, with a correlation below a threshold, or even completely unrelated. However, because light and shadow are often affected by the distribution of organs and the direction of muscles in the body, resulting in some shadows or occlusions, the image processing model may mix the extracted light and shadow features with some sample identity features when extracting light and shadow features. In this case, the light and shadow features can also be called coupled light and shadow features.

[0207] In order to separate the identity information of the sample user from the attribute information, the image processing model of this embodiment is used to decouple and extract sample attribute features of the sample images of body parts. Decoupling refers to removing the coupling between the attribute information and the identity information of the sample user. The sample attribute features are used to represent attributes that are unrelated to the identity of the sample user. It should be noted that the sample attribute features here are unrelated to the user's identity. It can be understood that the correlation between the sample attribute features and the sample identity features is very small, less than a certain threshold, or even completely unrelated. That is, this embodiment expects that the sample identity features and sample attribute features extracted by the image processing model are completely independent of each other, have a correlation below a threshold, or are even completely unrelated decoupled features. It is understandable that as the training phase of the image processing model continues, the degree of coupling or correlation between the sample identity features and the sample attribute features extracted by the image processing model will become lower and lower, and it may even be possible to extract sample identity features and sample attribute features that are completely independent of each other and completely unrelated to each other.

[0208] Exemplarily, the electronic device inputs the sample images of body parts into an image processing model, and extracts sample identity features of the sample images of body parts through the image processing model, as well as sample attribute features of the sample images of body parts; the sample attribute features are used to characterize attributes that are unrelated to the user's identity.

[0209] In some embodiments, the image processing model includes an encoder, and a sample image of a body part is input into the image processing model, and the sample identity feature is extracted (encoded) by the encoder, and the sample attribute feature is extracted (encoded) by the encoder.

[0210] Encoding refers to the process of encoding features related to the user identity in the sample images of body parts into sample identity features, or encoding refers to the process of encoding attributes unrelated to the user identity in the sample images of body parts into sample attribute features.

[0211] In some embodiments, the encoder includes an identity encoder and an attribute decoupling encoder. The identity encoder is used to encode the sample identity features, and the attribute decoupling encoder is used to decouple the sample attribute features. In some embodiments, the attribute decoupling encoder includes an attribute encoder and an orthogonal projection module, as described above in Figure 9 and will not be repeated here.

[0212] In some embodiments, embeddings are used orth Represents sample attribute features using embeddings id Represents the sample identity feature.

[0213] Step 230 : Decode the sample identity feature and the sample attribute feature to obtain a body part prediction image, wherein the body part prediction image is a body part image having the sample attribute feature and used to indicate the user's identity.

[0214] Decoding refers to the process of decoding and outputting body part prediction images based on sample identity features and sample attribute features.

[0215] A predicted body part image is an image generated by decoding the sample identity features and sample attribute features. This predicted body part image is a realistic representation of the sample body part image. It also has sample attribute features and is used to indicate the user's identity. In theory, the predicted body part image and the sample body part image contain the same sample attribute features and should indicate the same user identity as the sample body part image.

[0216] Exemplarily, the electronic device decodes the sample identity features and the sample attribute features through an image processing model to obtain a body part prediction image.

[0217] In some embodiments, the image processing model includes a decoder, which can be implemented using a deconvolutional network. The electronic device can decode the sample identity feature and the sample attribute feature through the decoder of the image processing model to obtain a body part prediction image.

[0218] Step 240 , updating at least one model parameter of the image processing model with reducing the error between the body part sample image and the body part prediction image as a training goal.

[0219] The training objective can be specifically a loss function, that is, a loss function is constructed based on the error between the body part sample image and the body part prediction image, and then, in multiple iterative training, the loss function is reduced in each iteration.

[0220] The at least one model parameter refers to all or part of the model parameters in the image processing model. In some embodiments, the at least one model parameter includes at least one of a model parameter corresponding to the encoder and a model parameter corresponding to the decoder.

[0221] In some embodiments, the at least one model parameter includes a model parameter corresponding to an attribute decoupling encoder in the encoder and a model parameter corresponding to the decoder. In some embodiments, the attribute decoupling encoder includes an attribute encoder and an orthogonal projection module, and the model parameter corresponding to the attribute decoupling encoder can specifically be a model parameter of the attribute encoder.

[0222] Exemplarily, the attribute encoder can be implemented using a three-layer residual connection neural network block (Block), where each Block is composed of a convolutional layer, a batch normalization layer, and an activation function layer. The at least one model parameter includes the model parameters of the attribute encoder, specifically including at least one model parameter of at least one or each Block. The decoder can be implemented using a deconvolutional network, and the at least one model parameter includes the model parameters of the decoder, specifically at least one model parameter of at least one layer or each neural network layer of the deconvolutional network.

[0223] Exemplarily, the electronic device updates at least one model parameter of the image processing model with reducing the error between the body part sample images and the body part prediction images as a training objective. Ideally, there is no error between the body part sample images and the body part prediction images. In some embodiments, the electronic device updates at least one model parameter of the image processing model with minimizing the error between the body part sample images and the body part prediction images as a training objective.

[0224] In some embodiments, the error between the body part sample image and the body part prediction image can be calculated using a cross entropy loss. When the error between the body part sample image and the body part prediction image is minimized, the model parameters of the image processing model are updated, and a trained image processing model is obtained.

[0225] In summary, the training method of the image processing model provided in the embodiment of the present application is executed by an electronic device. During the training stage of the image processing model, the sample identity features of the body part sample images and the sample attribute features of the body part sample images are first extracted by the image processing model. The sample identity features are used to indicate the user identity of the body part sample images, while the sample attribute features are irrelevant to the user identity. The sample identity features and the sample attribute features are then used to obtain the predicted body part sample images. Finally, the image processing model is trained using the difference between the predicted body part image and the body part sample image, so that the image processing model can accurately extract sample identity features and sample attribute features whose correlation is lower than a threshold or even completely unrelated, so that the user identity can be accurately indicated by the sample identity features, thereby improving the significance of the identity information of the sample identity features.

[0226] Thus, during the use phase of the image processing model, the model can extract identity features with significant identity information, as well as attribute features with low correlation with the identity features, and generate a body part target image based on the identity features and attribute features. Because identity features are significant, they can be used to determine the user identity of the body part target image, improving the salience of the identity information and the accuracy of the body part target image in indicating the user's identity. Furthermore, attribute features can be used to determine the attributes of the body part target image, thereby generating a new body part target image with controllable identity features and attribute features, thereby improving the accuracy and quality of the body part target image.

[0227] FIG12 is a flowchart of a training method for an image processing model provided by an exemplary embodiment of the present application. The portion of extracting the sample attribute features of the sample image of the body part in the above step 220 performed by the electronic device can be implemented as steps 320 and 340:

[0228] Step 320 : extracting coupled attribute features of the body part sample images based on the image processing model, wherein the coupled attribute features refer to at least one attribute feature coupled with the sample identity feature.

[0229] A coupled attribute feature refers to at least one attribute feature of a body part sample image that is coupled with a sample identity feature. The coupled attribute feature of this embodiment is an attribute feature whose main part is a sample attribute feature and is mixed with all or part of the sample identity feature. The coupled attribute feature can be considered as a mixed feature of a sample attribute feature and at least one sample identity feature. Taking the attribute being light and shadow as an example, the sample attribute feature of the body part sample image that the image processing model needs to extract is the light and shadow feature. Since light and shadow are usually affected by the organs distributed on the body part and the direction of the muscles of the body part, some shadows or occlusions are generated, so that the light and shadow feature may be mixed with a part of the sample identity feature. The light and shadow feature at this time can also be called a coupled light and shadow feature. As the training stage of the image processing model continues, the degree of coupling between the sample identity feature and the sample attribute feature extracted by the image processing model will become lower and lower, and it may even be possible to extract sample identity features and sample attribute features that are completely independent of each other and completely unrelated to each other.

[0230] Step 340: decouple the coupled attribute features from the sample identity features to obtain the sample attribute features.

[0231] Decoupling refers to removing the coupling relationship between the sample identity feature and the coupled attribute feature, separating the sample identity feature from the coupled attribute feature to obtain the sample attribute feature. This makes the correlation between the sample attribute feature and the sample identity feature less than a threshold, or even completely unrelated. Ultimately, the sample identity feature and the sample attribute feature are independent of each other and do not affect each other.

[0232] The sample attribute features are used to characterize attributes that are independent of the user identity of the body part sample images.

[0233] Exemplarily, the electronic device decouples the coupled attribute features from the sample identity features through an image processing model to obtain the sample attribute features.

[0234] In this embodiment, the coupled attribute features can be decoupled from the sample identity features, so that the image processing model can extract sample attribute features whose correlation with the sample identity features is less than a threshold, or even completely unrelated. During the use stage of the image processing model, the identity features and attribute features can be made independent of each other and controllable.

[0235] FIG13 shows a flow chart of a method for training an image processing model according to an exemplary embodiment of the present application. The above step 340 performed by the electronic device can be specifically implemented as steps 342 and 344.

[0236] Step 342 : orthogonally project the coupling attribute features into the feature space of the sample identity features to obtain the sample projection features.

[0237] The feature space is a vector space, which refers to the space where features exist. For example, the space where sample identity features exist is called the feature space of sample identity features. For example, if the sample identity features are represented as feature vectors, then the feature space of the sample identity features is an N-dimensional vector space, and each dimension of the feature space corresponds to a sample identity feature. It should be noted that, taking the example of both coupling attribute features and sample identity features being represented as feature vectors, since the coupling attribute features describe attribute information and the sample identity features describe identity information, the semantic information of the two is different, and the direction and distribution of the two feature vectors may also be different. Therefore, the coupling attribute features and the sample identity features correspond to different feature spaces.

[0238] Since the coupled attribute feature refers to at least one attribute feature coupled with the sample identity feature, each element in the coupled attribute feature can be orthogonally projected (Orthogonal Projection) to the feature space of the sample identity feature to obtain the representation of the coupled attribute feature in the feature space of the sample identity feature, which is the sample projection feature.

[0239] Specifically, step 342 can be implemented as step 342a and step 342b:

[0240] Step 342a: dot-product the coupled attribute features and the sample identity features to obtain the sample dot-product features.

[0241] In all embodiments of the present application, features are represented by vectors, and each feature vector includes multiple elements. Operations on features also refer to operations on the corresponding vectors.

[0242] Dot product, also known as inner product or scalar product, refers to multiplying the elements at corresponding positions in the coupled attribute features and the sample identity features.

[0243] The sample point product feature refers to the feature obtained by multiplying the coupled attribute feature and the sample identity feature.

[0244] Exemplarily, the electronic device obtains the sample dot product feature by coupling the attribute feature and the sample identity feature through the dot product of the image processing model.

[0245] In some embodiments, the feature dimension of the coupling attribute feature is the same as the feature dimension of the sample identity feature, so that the coupling attribute feature can be dot-producted with the sample identity feature to obtain the sample dot-product feature.

[0246] In other embodiments, when the feature dimension of the coupling attribute feature is different from the feature dimension of the sample identity feature, the electronic device can change the feature dimension of the coupling attribute feature or the sample identity feature so that the feature dimension of the changed coupling attribute feature is the same as the feature dimension of the sample identity feature. For example, taking the example of the feature dimension of the coupling attribute feature being greater than the feature dimension of the sample identity feature, the electronic device can change the feature dimension of the coupling attribute feature through a reshape function so that the feature dimension of the changed coupling attribute feature is the same as the feature dimension of the sample identity feature. Alternatively, the electronic device can change the feature dimension of the sample identity feature through a reshape function so that the feature dimension of the changed sample identity feature is the same as the feature dimension of the coupling attribute feature.

[0247] In step 342b, the sample dot product feature is divided by the modulus of the sample identity feature to obtain the sample projection feature.

[0248] The modulus can be a measure of a sample's identity, specifically the L2 norm, also known as the Euclidean norm. The L2 norm is calculated as the square root of the sum of the squares of the elements in the sample's identity and can be used to measure the length or size of the sample's identity.

[0249] For example, the electronic device divides the sample point product feature by the modulus of the sample identity feature through the image processing model to obtain the sample projection feature. The expression of the orthogonal projection is as follows:

[0250] Among them, embeddings proj Represents sample projection features, embeddings attr Represents coupling attribute features, embeddings id Indicates sample identity features, embeddings attr embeddings id Represents sample point multiplication features, |embeddings id | 2 The L2 norm of the sample identity feature.

[0251] In this embodiment, the sample dot product feature is obtained by dot multiplication of the coupled attribute feature and the sample identity feature, and the sample projection feature is determined by dividing the sample dot product feature by the modulus of the sample identity feature. The sample projection feature can be obtained accurately and quickly. When the coupled attribute feature is subtracted from the sample projection feature, the accuracy of the obtained sample attribute feature can be improved.

[0252] Step 344 : Subtract the sample projection feature from the coupling attribute feature to obtain the sample attribute feature.

[0253] Exemplarily, the electronic device uses an image processing model to subtract the sample projection feature from the coupling attribute feature to obtain a sample attribute feature. This sample attribute feature is also a component of the coupling attribute feature that is orthogonal to the sample identity feature. This component only retains attributes that are unrelated to the sample user's identity. It is expressed as follows:

[0254] embeddings orth =embeddings attr -embeddings proj (2)

[0255] Among them, embeddings orth Represents sample attribute features, embeddings proj Represents sample projection features, embeddings attr Represents coupling property characteristics.

[0256] Through the above-described embodiments, coupled attribute features can be decoupled, allowing the image processing model to extract sample attribute features whose correlation with the sample identity features is lower than a threshold, or even completely unrelated, making the sample attribute features more independent and not containing relevant identity information used to indicate the user identity of the sample user. Thus, the user identity of the sample user is indicated only by the sample identity features, making the identity information represented by the sample identity features in the body part prediction image more prominent. Because the identity information represented by the sample identity features in the body part prediction image is more prominent, the body part prediction image can more accurately and precisely indicate the user identity of the sample user.

[0257] In some embodiments, step 230 may be implemented as steps 230a and 230b:

[0258] Step 230a: Concatenate the sample identity feature and the sample attribute feature to obtain a sample concatenated feature.

[0259] Splicing refers to splicing the elements of two features according to rows or columns.

[0260] Since a body part sample image includes sample identity features for characterizing the user's identity and sample attribute features for characterizing attributes, after obtaining the sample splicing features, the image processing model can generate a new body part prediction image based on the sample splicing features.

[0261] In some embodiments, the sample concatenation feature is represented as follows: Concat(embeddings orth ,embeddings id ). Among them, embeddings orth Represents sample attribute features, embeddings id Represents the sample identity feature, and Concat represents the concatenation function.

[0262] In some embodiments, before step 230a, the following step may be included: pooling the sample attribute features to obtain the pooled sample attribute features. Then, step 230a may be implemented as: splicing the sample identity features and the pooled sample attribute features to obtain the sample splicing features.

[0263] In some embodiments, pooling is used to reduce the feature dimension of sample attribute features. In this embodiment, since the feature dimension of the sample identity feature remains unchanged while the feature dimension of the sample attribute feature is reduced during the process of pooling the sample attribute features, the electronic device can, by pooling the sample attribute features, firstly reduce the dimensionality of the sample attribute features, thereby reducing the complexity of the sample attribute features and the running time of the image processing model; secondly, in the case where the sample attribute features may still be coupled with a portion of the sample identity features, further decoupling between the sample attribute features and the sample identity features can be achieved to further reduce the correlation between the sample attribute features and the sample identity features.

[0264] In some embodiments, after step 230a, the following steps may be performed: pooling the sample splicing features to obtain the pooled sample splicing features, and step 230b may be implemented as: decoding the pooled sample splicing features to obtain the body part prediction image.

[0265] In some embodiments, pooling is used to reduce the feature dimensionality of sample splicing features. In this embodiment, by reducing the dimensionality of sample splicing features, the electronic device can reduce the complexity of the sample splicing features and shorten the runtime of the image processing model. This can also further decouple sample attribute features from sample identity features, further reducing the correlation between the two.

[0266] For example, the sample concatenation features after pooling are expressed as follows:

[0267] embeddings decom =Pooling(Concat(embeddings orth ,embeddingsid)) (3)

[0268] Among them, embeddings decom Represents the sample splicing features after pooling, Pooling represents pooling, Concat represents the splicing function, embeddings orth Represents sample attribute features, embeddings id Represents the sample identity feature.

[0269] Step 230b: decode the sample splicing features to obtain a body part prediction image.

[0270] The body part prediction image is a realistic image of the body part sample image. The body part prediction image is a body part image with sample attribute characteristics and is used to indicate the user identity. The body part prediction image also needs to indicate the user identity indicated by the body part sample image.

[0271] In some embodiments, the image processing model includes a decoder, which can be implemented using a deconvolutional network. The electronic device can decode the sample splicing features through the decoder of the image processing model to obtain a predicted body part image.

[0272] In this embodiment, the electronic device first splices the sample identity features and the sample attribute features to obtain the sample splicing features, takes the sample identity features and the sample attribute features as a whole, and then decodes the sample splicing features to obtain the body part prediction image. The body part prediction image can be obtained quickly, thereby improving the data processing speed.

[0273] FIG14 shows a flow chart of a method for training an image processing model provided by an exemplary embodiment of the present application. The above step 240 performed by the electronic device can be implemented as steps 360 and 380:

[0274] Step 360: Input the predicted body part image into the recognition model to extract the predicted identity features.

[0275] In some embodiments, the recognition model is a pre-trained recognition model for extracting identity features. The recognition model may be a MobileFaceNet, i.e., the recognition model is a pre-trained MobileFaceNet, and the recognition model may be pre-trained on sample body part images using an Additive Angular Margin Loss (ArcFace Loss) as a loss function.

[0276] In the embodiment of the present application, the recognition model can be specifically the identity encoder in the above-mentioned image processing model. Specifically, the identity encoder can also be implemented by the above-mentioned MobileFaceNet. Therefore, the recognition model and the identity encoder can be the same model.

[0277] In other embodiments, the recognition model may also be different from the identity encoder, which is not limited in this embodiment.

[0278] In some embodiments, after the pre-training is completed and the above-mentioned recognition model is obtained, a small batch of incremental training sets can be continued to be used to perform incremental training on the above-mentioned recognition model after the pre-training is completed. The incremental training set includes at least two incremental body part sample images. The at least two incremental body part sample images are images obtained by photographing the body parts of the same sample user under at least two shooting environments. Then, the at least two incremental body part sample images include the same incremental sample identity features and different incremental sample attribute features. The incremental sample identity features are used to indicate the user identity of the above-mentioned same sample user. Using the method of this embodiment to perform incremental training on the above-mentioned recognition model after the pre-training is completed can further enhance the recognition model's effect on extracting identity features and reduce the influence of different incremental sample attribute features on the recognition model to a certain extent.

[0279] In some embodiments, the model parameters of the above-mentioned recognition model are greater than the model parameters of other structures of the image processing model in the embodiment of the present application. Due to the large number of model parameters of the above-mentioned recognition model, the above-mentioned recognition model can have higher accuracy after the pre-training is completed. Inputting sample images of body parts into the recognition model can enable the recognition model to more accurately extract the sample identity features, thereby improving the accuracy of the image processing model in the embodiment of the present application.

[0280] Step 380 , updating at least one model parameter of the image processing model with reducing the error between the sample identity feature and the predicted identity feature as a training goal, wherein the recognition model is a pre-trained recognition model for extracting identity features.

[0281] In some embodiments, the electronic device updates at least one model parameter of the image processing model by minimizing the error between the sample identity feature and the predicted identity feature as a training objective.

[0282] In some embodiments, the error between the sample identity feature and the predicted identity feature can be characterized by at least one of a distance and a similarity between the sample identity feature and the predicted identity feature, wherein the similarity includes cosine similarity, which is determined by calculating the cosine value of the angle between the sample identity feature and the predicted identity feature, and is expressed as follows:

[0283] POS sim =cosine(feature x′,feature x) (4)

[0284] Among them, feature x′ represents the predicted identity feature, feature x represents the sample identity feature, pos sim The similarity between the sample identity features and the predicted identity features of the same user identity is the cosine similarity.

[0285] Furthermore, the training loss is calculated using the error, and then, in multiple iterations of training, the training loss is reduced in each iteration.

[0286] For example, compare 1 with the similarity pos sim The difference is taken as the training loss, expressed as: Loss id =(1-pos sim ).

[0287] Through the training method of this embodiment, the error between the sample identity features and the predicted identity features can be reduced, so that the body part prediction image generated by the image processing model is closer to the body part sample image, which can improve the accuracy of the image processing model.

[0288] In some embodiments, since the sample identity features and the predicted identity features should be identity features indicating the same user identity, and the identity features should also be the same for the same user identity, in order to improve the identity consistency of different identity features of the same user identity and improve the accuracy of body part images, this embodiment regards each user identity as a class, and the body part sample images and body part predicted images in each class constitute positive samples. Constraints are added to each class to constrain the body part images of the same user identity and ensure identity consistency between different body part images of the same user identity. Accordingly, these constraints can also be called intra-class consistency constraints.

[0289] Exemplarily, step 380 may specifically include step 382:

[0290] Step 382: Generate a constraint condition for the user identity based on the similarity between the sample identity feature and the predicted identity feature; and update at least one model parameter of the image processing model based on the constraint condition.

[0291] In this step, at least one model parameter of the image processing model is updated by taking improving the similarity between the sample identity feature and the predicted identity feature of the same user identity as a constraint condition of the user identity.

[0292] In this embodiment, 1 is combined with the similarity pos sim Difference: 1-pos sim , as a constraint to constrain the similarity between the sample identity features and the predicted identity features of the same user identity, the training loss of the image processing model can be expressed as follows:

[0293] Loss id =a*(1-pos sim ) (5)

[0294] Among them, Lossid Represents the training loss in the image processing model, a is a learnable weight parameter, pos sim It refers to the similarity between the sample identity features and the predicted identity features of the same user identity.

[0295] In this embodiment, by adding constraints, the identity consistency of different body part features of the same user identity can be improved, so that the body part images of the same user identity can accurately indicate the same user identity, thereby improving the accuracy of the body part images.

[0296] FIG15 shows a flowchart of an image processing method provided by an exemplary embodiment of the present application. The method is executed by an electronic device, which may be the terminal 120 and / or the server 140 shown in FIG1 , and includes steps 410, 420, and 430:

[0297] Step 410, obtain a first body part image, and obtain a second body part image; the first body part image is a body part image with a first identity feature and a first attribute feature, and the second body part image is a body part image with a second identity feature and a second attribute feature, the first identity feature is used to indicate the user identity of the first body part image, and the second identity feature is used to indicate the user identity of the second body part image.

[0298] In some embodiments, when a target body part image of a user identity needs to be generated for the user identity, the body part image used to control the user identity is the first body part image, i.e., the first body part image is also referred to as the body part image used to control the identity feature. There is no restriction on the first attribute feature in the first body part image and it can be any attribute. The body part image used to control the attribute feature is the second body part image, i.e., the second body part image is also referred to as the body part image used to control the attribute feature. There is no restriction on the second identity feature in the second body part image and it can be any user identity.

[0299] In some embodiments, the electronic device can use a shooting device to shoot a body part of a first user in a first shooting environment to obtain a first body part image, and obtain any second body part image from a public dataset. The second body part image can be obtained by shooting a body part of the second user in a second shooting environment using the shooting device.

[0300] It is understandable that, for the first body part image and the second body part image, the first identity feature and the second identity feature may be the same or different, and the first attribute feature and the second attribute feature may be the same or different. Where the first attribute feature and the second attribute feature are different, it may be that the first shooting attribute feature and the second shooting attribute feature are the same, and the first muscle attribute feature and the second muscle attribute feature are different; or, the first shooting attribute feature and the second shooting attribute feature are different, and the first muscle attribute feature and the second muscle attribute feature are the same; or, the first shooting attribute feature and the second shooting attribute feature are different, and the first muscle attribute feature and the second muscle attribute feature are different.

[0301] In step 420, the first body part image is input into the image processing model to extract the first identity feature of the first body part image, and the second body part image is input into the image processing model to extract the second attribute feature of the second body part image, where the second attribute feature is used to represent an attribute that is unrelated to the user identity of the second body part image.

[0302] It can be understood that, for the first body part image and the second body part image, the first identity feature and the second attribute feature should be independent of each other, with a correlation lower than a threshold, or even completely unrelated features. For the second body part image, the second attribute feature and the second identity feature should be independent of each other, with a correlation lower than a threshold, or even completely unrelated features. When the model accuracy of the image processing model is accurate enough, the second attribute feature is not coupled with any second identity feature, nor is it coupled with any first identity feature. Then, the first identity feature and the second attribute feature extracted by the image processing model should be independent of each other, with a correlation lower than a threshold, or even completely unrelated features, and only include the first identity feature, which is used to indicate the user identity of the first user.

[0303] Step 430 : Decode the first identity feature and the second attribute feature to obtain a body part target image, wherein the body part target image is a body part image having the first identity feature and the second attribute feature.

[0304] The user identity indicated by the body part target image is the user identity of the first user indicated by the first identity feature.

[0305] It can be understood that the processing steps and beneficial effects of the image processing model in the image processing method of this embodiment can be specifically referred to the processing steps and beneficial effects of the image processing model in the training method of the image processing model, and will not be repeated here.

[0306] In summary, the embodiment of the present application provides an image processing method, which is executed by an electronic device. During the use stage of the image processing model, since the correlation between the first attribute feature extracted by the image processing model and the first identity feature is lower than a threshold value, or even completely unrelated, when the body part target image is subsequently obtained by decoding the first identity feature and the second attribute feature, the user identity can be accurately indicated by the first identity feature, thereby improving the identity information significance of the first identity feature. In this way, the user identity of the body part target image can be determined by the first identity feature, thereby improving the identity information significance of the body part target image and the accuracy of indicating the user identity. Furthermore, the attributes of the body part target image can be determined by the second attribute feature, so that the image processing model can generate a new body part target image with controllable identity features and controllable attribute features, thereby improving the accuracy and quality of the body part target image, and the body part target image can more accurately indicate the user identity.

[0307] In some embodiments, step 420 inputs the second body part image into the image processing model to extract the second attribute feature of the second body part image, which can be implemented as steps 422 and 424:

[0308] Step 422: Input the second body part image into the image processing model to extract the coupled attribute features of the second body part image; the coupled attribute features are at least one attribute feature coupled with the second identity feature.

[0309] When the accuracy of the image processing model is high, the degree of coupling between the second identity feature and the second attribute feature is very small, or even no coupling exists at all. That is, when the accuracy of the image processing model is high, the coupled attribute feature contains very few second identity features, or even no second identity features at all.

[0310] Step 424 : Decouple the coupled attribute feature from the second identity feature to obtain the second attribute feature.

[0311] In this embodiment, the electronic device can decouple the coupled attribute feature from the second identity feature, so that the image processing model extracts the second attribute feature whose correlation with the second identity feature is lower than a threshold or even completely unrelated, making the second attribute feature more accurate.

[0312] In some embodiments, step 422 may include steps 520 and 540:

[0313] Step 520 , orthogonally project the coupling attribute feature into the feature space of the second identity feature to obtain a projected feature.

[0314] It should be noted that the second identity feature can be extracted by the identity encoder of the image processing model during the use phase of the image processing model. If the second body part image is also a sample body part image used in the training phase of the image processing model, the second identity feature can be already stored in the electronic device and extracted during the training phase.

[0315] Specifically, step 520 can be implemented as step 520a and step 520b:

[0316] Step 520a: dot-product the coupled attribute feature and the second identity feature to obtain a dot-product feature.

[0317] In step 520b, the dot product feature is divided by the modulus of the second identity feature to obtain a projection feature.

[0318] Exemplarily, the electronic device divides the dot product feature by the L2 norm of the second identity feature through an image processing model to obtain a projection feature.

[0319] Step 540: Subtract the projection feature from the coupling attribute feature to obtain a second attribute feature.

[0320] The second attribute feature is also a component in the coupled attribute feature that is orthogonal to the second identity feature, and this component only retains attributes that are irrelevant to the user identity of the second user.

[0321] Through the above-mentioned embodiment, the coupled attribute features can be decoupled, so that the image processing model can extract the second attribute feature whose correlation with the second identity feature is lower than a threshold or even completely unrelated, so that the second attribute feature can be more independent and does not contain identity information for indicating the user identity of the second user, so that the user identity of the first user is indicated only by the first identity feature, so that the identity information represented by the first identity feature in the body part target image is more prominent, and the body part target image can more accurately and precisely indicate the user identity of the first user.

[0322] In some embodiments, step 430 may be implemented as steps 432 and 434:

[0323] Step 432: Concatenate the first identity feature and the second attribute feature to obtain a target concatenated feature.

[0324] Since a body part image is composed of identity features for representing the user's identity and attribute features for representing attributes, a new body part image can be generated after obtaining the target splicing features.

[0325] In some embodiments, before step 432, the following step may be included: pooling the second attribute feature to obtain the pooled second attribute feature. Then, step 432 may be implemented as: splicing the first identity feature and the pooled second attribute feature to obtain the target spliced ​​feature.

[0326] In some embodiments, pooling is used to reduce the feature dimension of the second attribute feature. In this embodiment, since the feature dimensions of the first identity feature and the second identity feature remain unchanged during the process of pooling the second attribute feature, while the feature dimension of the second attribute feature is reduced, the electronic device can reduce the dimension of the second attribute feature by pooling the second attribute feature, thereby reducing the complexity of the second attribute feature and the running time of the image processing model; and secondly, in the case where the second attribute feature may still be coupled with a part of the second identity feature, further decoupling between the second attribute feature and the second identity feature can be achieved to further reduce the correlation between the second attribute feature and the second identity feature.

[0327] In some embodiments, after step 430, the following steps may be performed: pooling the target splicing features to obtain the pooled target splicing features, and step 434 may be implemented as: decoding the pooled target splicing features to obtain the body part prediction image.

[0328] In some embodiments, pooling is used to reduce the feature dimensionality of the target splicing features. In this embodiment, the electronic device can reduce the complexity of the target splicing features and shorten the running time of the image processing model by reducing the dimensionality of the target splicing features.

[0329] Step 434: decode the target splicing features to obtain the target image of the body part.

[0330] In this embodiment, the electronic device first splices the first identity feature and the second attribute feature to obtain a target splicing feature, takes the first identity feature and the second attribute feature as a whole, and then decodes the target splicing feature to obtain a target image of the body part. This can quickly obtain the target image of the body part and improve the data processing speed.

[0331] For the above steps, please refer to the corresponding steps in the training method of the image processing model, which will not be repeated here.

[0332] Palmprint recognition scenario

[0333] In some embodiments, in a palmprint recognition scenario, steps 210, 220, 230, and 240 performed by the electronic device may be implemented as steps 210-1, 220-1, 230-1, and 240-1:

[0334] Step 210 - 1 , obtaining a sample palm image, inputting the sample palm image into the image processing model, and executing steps 220 - 1 , 230 - 1 , and 240 - 1 .

[0335] Step 220 - 1 , extracting sample palmprint features and sample attribute features of the palm sample image, wherein the sample palmprint features are used to indicate the user identity of the palm sample image, and the sample attribute features are used to represent attributes unrelated to the user identity.

[0336] Step 230 - 1 , decoding the sample palmprint features and the sample attribute features to obtain a palmprint prediction image; the palmprint prediction image is a palmprint image having the sample attribute features and used to indicate the user's identity.

[0337] Step 240 - 1 , taking reducing the error between the palm sample image and the palmprint prediction image as a training goal, and updating at least one model parameter of the image processing model.

[0338] In some embodiments, the above step 220-1 performed by the electronic device may be implemented as steps 320-1 and 340-1:

[0339] Step 320 - 1 : extracting coupled attribute features of the palm sample image based on the image processing model; the coupled attribute features refer to at least one attribute feature coupled with the sample palmprint feature.

[0340] Step 340 - 1 : decouple the coupled attribute features from the sample palmprint features to obtain the sample attribute features.

[0341] In some embodiments, the above step 340 - 1 performed by the electronic device may be specifically implemented as step 342 - 1 and step 344 - 1 .

[0342] Step 342 - 1 , orthogonally project the coupling attribute feature into the feature space of the sample palmprint feature to obtain the sample projection feature.

[0343] Specifically, step 342-1 may be implemented as step 342a-1 and step 342b-1:

[0344] Step 342a-1: dot-product the coupled attribute feature and the sample palmprint feature to obtain the sample dot-product feature.

[0345] Step 342b-1: Divide the sample dot product feature by the modulus of the sample palmprint feature to obtain the sample projection feature.

[0346] Step 344 - 1 : Subtract the sample projection feature from the coupling attribute feature to obtain the sample attribute feature.

[0347] In some embodiments, step 230-1 may be implemented as step 230a-1 and step 230b-1:

[0348] Step 230a-1: Splice the sample palmprint features and the sample attribute features to obtain a sample splicing feature.

[0349] Step 230b-1: decode the sample splicing features to obtain a palmprint prediction image.

[0350] In some embodiments, the above step 240-1 performed by the electronic device may be implemented as steps 360-1 and 380-1:

[0351] Step 360 - 1 : Inputting the palm sample image into the recognition model to extract the sample palmprint features, and inputting the palmprint prediction image into the recognition model to extract the predicted palmprint features.

[0352] Step 380-1, taking reducing the error between the sample palmprint features and the predicted palmprint features as the training goal, updating at least one model parameter of the image processing model; wherein the recognition model is a pre-trained recognition model for extracting palmprint features.

[0353] Exemplarily, after step 360-1 or after step 380-1, the method may further include step 382-1:

[0354] Step 382-1: Generate a constraint condition for the user identity based on the similarity between the sample palmprint feature and the predicted palmprint feature; and update at least one model parameter of the image processing model based on the constraint condition.

[0355] In some embodiments, the above steps 410, 420, and 430 performed by the electronic device may be implemented as steps 410-1, 420-1, and 430-1:

[0356] Step 410-1, obtain a first palm image, and obtain a second palm image; the first palm image is a palm image having a first palm print feature and a first attribute feature, and the second palm image is a palm image having a second palm print feature and a second attribute feature, the first palm print feature is used to indicate the user identity of the first palm image, and the second palm print feature is used to indicate the user identity of the second palm image.

[0357] Step 420-1: Input the first palm image into the image processing model to extract the first palm print feature of the first palm image, and input the second palm image into the image processing model to extract the second attribute feature of the second palm image, where the second attribute feature is used to represent an attribute that is unrelated to the user identity of the second palm image.

[0358] Step 430 - 1 , decoding the first palmprint feature and the second attribute feature to obtain a palmprint target image; the palmprint target image is a palmprint image having the first palmprint feature and the second attribute feature.

[0359] It can be understood that the processing steps and beneficial effects of the image processing model in the image processing method of this embodiment can be specifically referred to the processing steps and beneficial effects of the image processing model in the training method of the image processing model, and will not be repeated here.

[0360] In summary, the embodiment of the present application provides an image processing method, which is executed by an electronic device. During the use stage of the image processing model, the image processing model can extract the first palmprint feature and the second attribute feature whose correlation is lower than a threshold or even completely unrelated, thereby improving the significance of the identity information of the first palmprint feature, so that the image processing model can generate a new palmprint target image with controllable palmprint features and controllable attribute features, thereby improving the accuracy and quality of the palmprint target image, and the palmprint target image can more accurately indicate the user's identity.

[0361] In some embodiments, step 420-1 inputs the second palm image into the image processing model to extract the second attribute feature of the second palm image, which can be implemented as steps 422-1 and 424-1:

[0362] Step 422-1: Input the second palm image into the image processing model to extract the coupled attribute features of the second palm image; the coupled attribute features are at least one attribute feature coupled with the second palmprint feature.

[0363] Step 424 - 1 , decoupling the coupled attribute feature from the second palmprint feature to obtain the second attribute feature.

[0364] In some embodiments, step 422-1 may include steps 520-1 and 540-1:

[0365] Step 520 - 1 , orthogonally project the coupling attribute feature into the feature space of the second palmprint feature to obtain a projection feature.

[0366] Specifically, step 520-1 can be implemented as step 520a-1 and step 520b-1:

[0367] Step 520a-1: dot-product the coupled attribute feature and the second palmprint feature to obtain a dot-product feature.

[0368] Step 520b-1: Divide the dot product feature by the L2 norm of the second palmprint feature to obtain a projection feature.

[0369] Step 540 - 1 : Subtract the projection feature from the coupling attribute feature to obtain a second attribute feature.

[0370] In some embodiments, step 430-1 may be implemented as steps 432-1 and 434-1:

[0371] Step 432-1: Splice the first palmprint feature and the second attribute feature to obtain a target spliced ​​feature.

[0372] In some embodiments, after step 430 - 1 , the following step may be further included: pooling the target splicing features to obtain the pooled target splicing features, and step 434 - 1 may be implemented as: decoding the pooled target splicing features to obtain the palmprint prediction image.

[0373] In some embodiments, pooling is used to reduce the feature dimensionality of the target splicing features. In this embodiment, the electronic device can reduce the complexity of the target splicing features and shorten the running time of the image processing model by reducing the dimensionality of the target splicing features.

[0374] Step 434 - 1 , decoding the target splicing features to obtain the palmprint target image.

[0375] In this embodiment, the electronic device first splices the first palmprint feature and the second attribute feature to obtain a target splicing feature, takes the first palmprint feature and the second attribute feature as a whole, and then decodes the target splicing feature to obtain a palmprint target image. This can quickly obtain the palmprint target image and improve the data processing speed.

[0376] In order to more clearly understand the training method of the image processing model and the image processing method provided in the embodiments of the present application, the following is an illustration with reference to a schematic diagram.

[0377] Figure 16 shows a schematic diagram of a training method for an image processing model provided by an exemplary embodiment of the present application. Exemplarily, the image processing model includes an attribute encoder 21, an identity encoder 22, an orthogonal projection module 24, a splicing layer 26, a pooling layer 27 and a decoder 23, wherein the attribute encoder 21 and the orthogonal projection module 24 can be collectively referred to as an attribute decoupling encoder. The attribute encoder 21 and the identity encoder 22 are parallel encoder units, the output end of the attribute encoder 21 is connected to the input end of the orthogonal projection module 24, and the output end of the attribute encoder 21 is also connected to the input end of the splicing layer 26; the output end of the identity encoder 22 is connected to the input end of the orthogonal projection module 24, and the output end of the identity encoder 22 is also connected to the input end of the splicing layer 26; the output end of the splicing layer 26 is connected to the input end of the pooling layer 27, and the output end of the pooling layer 27 is connected to the input end of the decoder 23.

[0378] The attribute encoder 21 (Encoder_attr) is implemented using a three-layer residual connection neural network block (Block), and each Block is composed of a convolution layer, a batch normalization layer, and an activation function (Linear Rectification Function, Relu) layer. The identity encoder 22 (Encoder_id) uses the mobile face network (MobileFaceNet) as the basic model structure, and removes the fully connected layer in MobileFaceNet that finally maps the features to the identity ID. MobileFaceNet uses the Additive Angular Margin Loss (ArcFace Loss) as the loss function and is pre-trained on the palm sample image. In the image processing model of the embodiment of the present application, the identity encoder retains the model parameters of the feature extraction part of the MobileFaceNet, and this part of the model parameters is not updated during the training phase of the image processing model. The orthogonal projection module 24 is implemented using a three-layer residual connection Block, and each Block is composed of a convolution layer, a Batch Normalization layer, and a Relu layer. The concatenation layer 26 is implemented by a concatenation layer, and the pooling layer 27 is implemented by a pooling layer. The decoder 23 can be implemented by a deconvolution network.

[0379] With reference to FIG16 , the steps of the image processing model training method provided in the embodiment of the present application are as follows:

[0380] Step 1:

[0381] A palm sample image 661 is obtained from the training sample set. The palm sample image 661 can also be represented as a palm sample image x. The attribute encoder (Encoder_attr) 21 in the image processing model is used to extract the coupled attribute features (embeddings attr )662, the coupling attribute feature 662 includes light and shadow information of the palm sample image 661.

[0382] Coupling attribute features (embeddings attr )662 means the following:

[0383] embeddings attr =Encoder_attr(x) (6)

[0384] Step 2:

[0385] The identity encoder (Encoder_id) 22 in the image processing model is used to extract the sample palmprint features (embeddings id ) 663, the sample palmprint feature 663 is used to indicate the sample user identity (sample identity id). It can be understood that the above steps 2 and 3 can be performed in parallel or in sequence, and this embodiment does not limit this.

[0386] Sample palmprint features (embeddings id )663 means the following:

[0387] embeddings id =Encoder_id(x) (7)

[0388] Step 3:

[0389] The sample palmprint feature 663 and the coupled attribute feature 662 are input into the orthogonal projection module 24. The orthogonal projection module 24 is used to orthogonally project the coupled attribute feature 662 onto the feature space of the sample palmprint feature 663. This is equivalent to combining the sample palmprint feature 663 and the coupled attribute feature 662 while maintaining their respective characteristics. The orthogonal projection expression in the orthogonal projection module 24 is shown in the above formula (1).

[0390] Step 4:

[0391] Coupling attribute features (embeddings attr )662 and sample mapping features (embeddings proj ) minus 25 to get the coupling attribute features (embeddings attr )662 orthogonal to the sample palmprint features (embeddings id )663, which is the sample attribute feature (embeddings orth ), this component only retains attributes that are irrelevant to the sample identity, as shown in formula (2) above.

[0392] Step 5:

[0393] The sample attribute features (embeddings orth ) and sample palmprint features (embeddings id )663 is concatenated by the concatenation layer 26 according to the row dimension to obtain the sample splicing features, and the sample splicing features are pooled by the pooling layer 26 to achieve dimensionality reduction aggregation to obtain the final pooled sample splicing features (embeddings decom ), as shown in formula (3) above.

[0394] Step 6:

[0395] The pooled sample splicing features (embeddings decom ) is input into the decoder 23 to obtain a palmprint prediction image 664, which is also represented as a palmprint prediction image x'. The sample user identity indicated by the palmprint prediction image 664 is consistent with the palm sample image 661.

[0396] The palmprint prediction image 664 is represented as follows: x' = Decoder (embeddings decom ) (8)

[0397] Step 7:

[0398] The palm print prediction image 664 is input into the recognition model Rec_model 665 pre-trained with the palm sample image to obtain the predicted palm print feature (feature_x'). The Rec_model uses MobileFaceNet as the basic model structure. At the same time, the palm sample image 661 is input into the recognition model Rec_model 665 to obtain the sample palm print feature (feature_x), wherein the feature_x is consistent with the embeddings id Can be the same.

[0399] Step 8:

[0400] Since the palm sample image 661 and the palm print prediction image 664 indicate the same sample user identity, the two constitute a positive sample. This embodiment also calculates the cosine similarity pos_sim between the predicted palm print feature (feature_x') and the sample palm print feature (feature_x), and uses 1-pos_sim to constrain the similarity between the predicted palm print feature (feature_x') and the sample palm print feature (feature_x).

[0401] The calculation method of similarity can be referred to as shown in the above formula (4); the calculation method of training loss of recognition model 665 can be referred to as shown in the above formula (5).

[0402] In this embodiment, when the training loss reaches the minimum, the image processing model training ends.

[0403] In summary, during the training stage of the image processing model, by extracting sample palmprint features from palm sample images and decoupling the extraction of sample attribute features from palm sample images, the image processing model can extract sample palmprint features and sample attribute features that are independent of each other, have a correlation below a threshold, or are even completely unrelated, thereby improving the saliency of the identity information of the sample palmprint features.

[0404] FIG17 is a schematic diagram of an image processing method provided by an exemplary embodiment of the present application. After the training phase of the image processing model is completed, the trained identity encoder is represented as identity encoder 32, the trained attribute decoupling encoder (including the attribute encoder and the orthogonal projection module) is represented as attribute decoupling encoder 31, the trained splicing layer is represented as splicing layer 33, the trained pooling layer is represented as pooling layer 34, and the trained decoder is represented as decoder 35.

[0405] With reference to FIG11 , the steps of the image processing method provided in this embodiment are as follows:

[0406] Step 1:

[0407] Obtain a first palm image 672, and obtain a second palm image 671; the first palm image 672 is a palm image having a first palm print feature 674 and a first attribute feature, and the second palm image 671 is a palm image having a second palm print feature and a second attribute feature 673, the first palm print feature 674 is used to indicate the user identity of the first palm image 672, and the second palm print feature is used to indicate the user identity of the second palm image 671.

[0408] Step 2:

[0409] The first palm image 672 is input into the image processing model, and the identity encoder 32 extracts the first palm print feature 674 of the first palm image 672. Furthermore, the second palm image 671 is input into the image processing model, and the attribute decoupling encoder 31 extracts the second attribute feature 673 of the second palm image 671. The second attribute feature 673 is used to represent attributes unrelated to the identity of the second palm image 671.

[0410] Step 3:

[0411] The first palmprint feature 674 and the second attribute feature 673 are spliced ​​together through the splicing layer 33 to obtain a target splicing feature.

[0412] Step 4:

[0413] The target splicing features are decoded by the decoder 35 to generate a palmprint target image 675 ; the palmprint target image 675 is a palmprint image having a first palmprint feature 674 and a second attribute feature 673 , and the user identity indicated by the palmprint target image 675 is the user identity indicated by the first palm image 672 .

[0414] In summary, during the use phase of the image processing model, the correlation between the second attribute feature extracted by the image processing model and the second palmprint feature is lower than the threshold, or even completely uncorrelated. This increases the salience of the identity information of the first palmprint feature, allowing the image processing model to generate a new palmprint target image with controllable palmprint features and attribute features, improving the generation effect of the palmprint target image, and the palmprint target image can more accurately indicate the user's identity.

[0415] FIG18 shows a block diagram of an image processing model training device 800 provided by an exemplary embodiment of the present application. The image processing model training device 800 includes an acquisition module 810, a processing module 820, a decoding module 830, and a training module 840:

[0416] An acquisition module 810 is configured to acquire a sample image of a body part;

[0417] a processing module 820 configured to input the body part sample image into the image processing model and extract a sample identity feature and a sample attribute feature of the body part sample image, wherein the sample identity feature is used to indicate the user identity of the body part sample image, and the sample attribute feature is used to represent an attribute unrelated to the user identity;

[0418] A decoding module 830 is configured to decode the sample identity feature and the sample attribute feature based on the image processing model to obtain a body part prediction image, wherein the body part prediction image is a body part image having the sample attribute feature and used to indicate the identity of the user; and

[0419] The training module 840 is configured to update at least one model parameter of the image processing model by taking reducing the error between the body part sample image and the body part prediction image as a training goal.

[0420] In some embodiments, the processing module 820 is used to extract the coupled attribute features of the body part sample image based on the image processing model, where the coupled attribute features refer to at least one attribute feature coupled with the sample identity feature; and decouple the coupled attribute features from the sample identity feature to obtain the sample attribute features.

[0421] In some embodiments, the processing module 820 is configured to orthogonally project the coupling attribute feature into the feature space of the sample identity feature to obtain a sample projection feature; and subtract the sample projection feature from the coupling attribute feature to obtain the sample attribute feature.

[0422] In some embodiments, the processing module 820 is configured to perform a dot product of the coupling attribute feature and the sample identity feature to obtain a sample dot product feature; and divide the sample dot product feature by the modulus of the sample identity feature to obtain the sample projection feature.

[0423] In some embodiments, the decoding module 830 is configured to concatenate the sample identity feature and the sample attribute feature to obtain a sample concatenated feature; and decode the sample concatenated feature to obtain the body part prediction image.

[0424] In some embodiments, the training module 840 is used to input the predicted body part image into a recognition model to extract predicted identity features, wherein the recognition model is a pre-trained recognition model for extracting identity features; and to update at least one model parameter of the image processing model with the training goal of reducing the error between the sample identity features and the predicted identity features.

[0425] In some embodiments, the training module 840 is used to generate constraints on the user identity based on the similarity between the sample identity feature and the predicted identity feature; and update the at least one model parameter of the image processing model based on the constraints.

[0426] FIG19 shows a block diagram of an image processing apparatus 900 provided by an exemplary embodiment of the present application. The image processing apparatus 900 includes an acquisition module 910, a processing module 920, and a decoding module 930:

[0427] Acquisition module 910 is used to acquire a first body part image and a second body part image; the first body part image is a body part image having a first identity feature and a first attribute feature, and the second body part image is a body part image having a second identity feature and a second attribute feature, the first identity feature is used to indicate the user identity of the first body part image, and the second identity feature is used to indicate the user identity of the second body part image.

[0428] Processing module 920 is used to input the first body part image into the image processing model to extract the first identity feature of the first body part image, and to input the second body part image into the image processing model to extract the second attribute feature of the second body part image, where the second attribute feature is used to represent an attribute that is unrelated to the user identity of the second body part image.

[0429] The decoding module 930 is configured to decode the first identity feature and the second attribute feature to obtain a body part target image, where the body part target image is a body part image having the first identity feature and the second attribute feature.

[0430] In some embodiments, the processing module 920 is used to input the second body part image into the image processing model to extract the coupled attribute features of the second body part image; the coupled attribute features are at least one attribute feature coupled with the second identity feature; the coupled attribute features are decoupled from the second identity feature to obtain the second attribute features.

[0431] In some embodiments, the processing module 920 is configured to perform a dot product of the coupling attribute feature and the second identity feature to obtain a dot product feature; and divide the dot product feature by the modulus of the second identity feature to obtain the projection feature.

[0432] In some embodiments, the decoding module 930 is configured to concatenate the first identity feature and the second attribute feature to obtain a target concatenated feature;

[0433] Decode the target splicing features and obtain the target image of the body part.

[0434] It should be noted that the specific limitations in the embodiments of the at least one image processing model training device 800 provided above can refer to the limitations on the image processing model training method provided above, and the specific limitations in the embodiments of the at least one image processing device 900 provided above can refer to the limitations on the image processing method provided above, and will not be repeated here. The image processing model training device 800 and the image processing device 900 can also be applied to palmprint recognition scenarios and face recognition scenarios, respectively. The modules of the above-mentioned devices can be fully or partially implemented by software, hardware, and a combination thereof. Each module can be embedded in the processor of the electronic device in the form of hardware or be independent of it, or it can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0435] An embodiment of the present application also provides an electronic device, which includes: a processor and a memory, wherein a computer program is stored in the memory; the processor is used to execute the computer program in the memory to implement the image processing model training method and / or image processing method provided by the above-mentioned method embodiments.

[0436] 20 is a block diagram of an electronic device 1000 according to an exemplary embodiment of the present application. In some embodiments, the electronic device 1000 is a server 1000. The server 1000 includes a processor 1001 and a memory 1002.

[0437] The processor 1001 may include at least one processing core, such as a 4-core processor, an 8-core processor, etc. The processor 1001 may be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 1001 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1001 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1001 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0438] The memory 1002 may include at least one computer-readable storage medium, which may be non-transitory. The memory 1002 may also include a high-speed random access memory and a non-volatile memory, such as at least one disk storage device and a flash memory storage device. In some embodiments, the non-transitory computer-readable storage medium in the memory 1002 is used to store at least one instruction, which is used to be executed by the processor 1001 to implement the training method and / or image processing method of the image processing model provided in the method embodiment of the present application.

[0439] In some embodiments, the server 1000 may further include: an input interface 1003 and an output interface 1004. The processor 1001, the memory 1002, and the input interface 1003 and the output interface 1004 may be connected via a bus or a signal line. Each peripheral device may be connected to the input interface 1003 and the output interface 1004 via a bus, a signal line, or a circuit board. The input interface 1003 and the output interface 1004 may be used to connect at least one peripheral device related to input / output (I / O) to the processor 1001 and the memory 1002. In some embodiments, the processor 1001, the memory 1002, and the input interface 1003 and the output interface 1004 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1001, the memory 1002, and the input interface 1003 and the output interface 1004 may be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0440] Those skilled in the art will understand that the structure shown in FIG. 20 does not constitute a limitation on the electronic device 1000 , and may include more or fewer components than shown, or combine certain components, or adopt a different component arrangement.

[0441] In an exemplary embodiment, the present application provides a chip, which includes a programmable logic circuit and / or program instructions. When the chip runs on an electronic device, it is used to implement the training method and / or image processing method of the image processing model provided by the above method embodiment.

[0442] The present application provides a computer-readable storage medium, which stores a computer program. The computer program is loaded and executed by a processor to implement the image processing model training method and / or image processing method provided by the above method embodiment.

[0443] The present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the processor of the electronic device to load and execute the computer instructions to implement the image processing model training method and / or image processing method provided in the above method embodiments.

[0444] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0445] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0446] Those skilled in the art will appreciate that, in at least one of the examples above, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as at least one instruction or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, and communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0447] The above description is only a possible embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for training an image processing model, performed by an electronic device, the method comprising: Obtain a sample image of a body part, input the sample image of the body part into the image processing model, and perform the following processing: Extracting a sample identity feature and a sample attribute feature of the body part sample image, wherein the sample identity feature is used to indicate the user identity of the body part sample image, and the sample attribute feature is used to characterize an attribute unrelated to the user identity; Decoding the sample identity feature and the sample attribute feature to obtain a body part prediction image, wherein the body part prediction image is a body part image having the sample attribute feature and used to indicate the identity of the user; and At least one model parameter of the image processing model is updated with reducing the error between the body part sample image and the body part prediction image as a training goal.

2. The method according to claim 1, wherein The extracting of sample attribute features of the body part sample images includes: extracting coupled attribute features of the body part sample image based on the image processing model, where the coupled attribute features refer to at least one attribute feature coupled with the sample identity feature; The coupling attribute feature is decoupled from the sample identity feature to obtain the sample attribute feature.

3. The method according to claim 2, wherein: Decoupling the coupled attribute feature from the sample identity feature to obtain the sample attribute feature includes: Orthogonally projecting the coupling attribute feature into the feature space of the sample identity feature to obtain a sample projection feature; The sample property feature is obtained by subtracting the sample projection feature from the coupling property feature.

4. The method according to claim 3, wherein: The orthogonal projection of the coupling attribute feature into the feature space of the sample identity feature to obtain the sample projection feature includes: Dot product the coupling attribute feature and the sample identity feature to obtain a sample dot product feature; The sample point product feature is divided by the modulus of the sample identity feature to obtain the sample projection feature.

5. The method according to any one of claims 2 to 4, wherein: The characteristic dimension of the coupling attribute feature is the same as the characteristic dimension of the sample identity feature.

6. The method according to any one of claims 1 to 5, wherein: The decoding of the sample identity feature and the sample attribute feature to obtain a body part prediction image includes: Splicing the sample identity feature and the sample attribute feature to obtain a sample splicing feature; The sample splicing features are decoded to obtain the body part prediction image.

7. The method according to any one of claims 1 to 6, wherein: The updating of at least one model parameter of the image processing model by taking reducing the error between the body part sample image and the body part prediction image as a training objective comprises: Inputting the predicted body part image into a recognition model to extract predicted identity features, wherein the recognition model is a pre-trained recognition model for extracting identity features; The at least one model parameter of the image processing model is updated with reducing the error between the sample identity feature and the predicted identity feature as the training goal.

8. The method according to claim 7, wherein: The updating of the at least one model parameter of the image processing model by taking reducing the error between the sample identity feature and the predicted identity feature as the training objective includes: generating a constraint condition for the user identity based on the similarity between the sample identity feature and the predicted identity feature; According to the constraint condition, the at least one model parameter of the image processing model is updated.

9. The method according to claim 7 or 8, wherein The image processing model includes an identity encoder, an attribute decoupling encoder, and a decoder, wherein the identity encoder is used to extract the sample identity feature, the attribute decoupling encoder is used to extract the sample attribute feature, and the decoder is used to decode the sample identity feature and the sample attribute feature; The at least one model parameter includes: at least one of a model parameter corresponding to the attribute decoupling encoder and a model parameter corresponding to the decoder.

10. An image processing method, performed by an electronic device, the method comprising: Acquire a first body part image, and acquire a second body part image, wherein the first body part image is a body part image having a first identity feature and a first attribute feature, and the second body part image is a body part image having a second identity feature and a second attribute feature, wherein the first identity feature is used to indicate the identity of a user of the first body part image, and the second identity feature is used to indicate the identity of a user of the second body part image; Inputting the first body part image into an image processing model to extract the first identity feature of the first body part image, and inputting the second body part image into the image processing model to extract the second attribute feature of the second body part image, wherein the second attribute feature is used to represent an attribute that is unrelated to the user identity of the second body part image; and The first identity feature and the second attribute feature are decoded to obtain a body part target image, where the body part target image is a body part image having the first identity feature and the second attribute feature.

11. The method according to claim 10, wherein: Inputting the second body part image into the image processing model and extracting the second attribute feature of the second body part image includes: Inputting the second body part image into the image processing model, and extracting coupled attribute features of the second body part image; the coupled attribute features are at least one attribute feature coupled with the second identity feature; The coupled attribute feature is decoupled from the second identity feature to obtain the second attribute feature.

12. The method according to claim 11, wherein Decoupling the coupled attribute feature from the second identity feature to obtain the second attribute feature includes: orthogonally projecting the coupling attribute feature into the feature space of the second identity feature to obtain a projected feature; The second attribute feature is obtained by subtracting the projection feature from the coupling attribute feature.

13. The method according to claim 12, wherein: The orthogonally projecting the coupling attribute feature into the feature space of the second identity feature to obtain the projected feature includes: Dot product the coupling attribute feature and the second identity feature to obtain a dot product feature; The dot product feature is divided by the modulus of the second identity feature to obtain the projection feature.

14. The method according to any one of claims 11 to 13, wherein: The characteristic dimension of the coupling attribute feature is the same as the characteristic dimension of the second identity feature.

15. The method according to any one of claims 10 to 14, wherein: The decoding of the first identity feature and the second attribute feature to obtain a body part target image includes: Concatenating the first identity feature and the second attribute feature to obtain a target concatenated feature; The target splicing features are decoded to obtain the target image of the body part.

16. A training device for an image processing model, comprising: An acquisition module, used for acquiring sample images of body parts; a processing module, configured to input the body part sample image into the image processing model, and extract a sample identity feature and a sample attribute feature of the body part sample image, wherein the sample identity feature is used to indicate the user identity of the body part sample image, and the sample attribute feature is used to represent an attribute unrelated to the user identity; a decoding module, configured to decode the sample identity feature and the sample attribute feature based on the image processing model to obtain a body part prediction image, wherein the body part prediction image is a body part image having the sample attribute feature and used to indicate the identity of the user; and A training module is configured to update at least one model parameter of the image processing model by taking reducing the error between the body part sample image and the body part prediction image as a training goal.

17. An image processing apparatus, comprising: an acquisition module, configured to acquire a first body part image and a second body part image, wherein the first body part image is a body part image having a first identity feature and a first attribute feature, and the second body part image is a body part image having a second identity feature and a second attribute feature, wherein the first identity feature is used to indicate an identity of a user of the first body part image, and the second identity feature is used to indicate an identity of a user of the second body part image; a processing module, configured to input the first body part image into an image processing model and extract the first identity feature of the first body part image, and input the second body part image into the image processing model and extract the second attribute feature of the second body part image, wherein the second attribute feature is used to represent an attribute unrelated to the user identity of the second body part image; and A decoding module is used to decode the first identity feature and the second attribute feature to obtain a body part target image, where the body part target image is a body part image having the first identity feature and the second attribute feature.

18. An electronic device comprising: A processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the training method of the image processing model as described in any one of claims 1 to 9, or to implement the image processing method as described in any one of claims 10 to 15.

19. A computer-readable storage medium storing a computer program, wherein the computer program is loaded and executed by a processor to implement the training method of the image processing model as described in any one of claims 1 to 9, or to implement the image processing method as described in any one of claims 10 to 15.

20. A computer program product, comprising computer instructions, wherein the computer instructions are stored in a computer-readable storage medium, and a processor obtains the computer instructions from the computer-readable storage medium, so that the processor loads and executes the computer instructions to implement the training method of the image processing model as described in any one of claims 1 to 9, or to implement the image processing method as described in any one of claims 10 to 15.

Images