Image processing method, device, equipment, medium and program product

Abstract

The embodiment of the invention provides an image processing method and device, equipment, a medium and a program product. The method comprises the steps of obtaining a first image containing face information of a first object in response to an expression face changing operation received in a social session process; obtaining an expression template image, wherein the expression template image comprises face information of the first reference object; according to the first image and the expression template image, generating a first expression image, the first expression image including facial features reflected by the facial information of the first object and including the facial information of a target reference object, the target reference object being any one of a first reference object and a second reference object; and displaying the first expression image in the social interface. According to the method, the face in the expression template can be subjected to expression change based on the face of the user, the expression image of the face of the user can be generated in a personalized manner, and the interestingness and personalized requirements of the expression image are met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to the field of image processing, and specifically relates to an image processing method, an image processing device, a computer device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] In various social APPs (Application) run by social clients, in order to facilitate communication and exchange, expression images are usually used for social interaction between social objects. At present, the expression images provided for different social objects in the social APP are usually some expression images uniformly preset by the system, and the expression images obtained by each social object are the same, which cannot meet the individual needs of the social objects to a certain extent. SUMMARY

[0003] The embodiments of the present application provide an image processing method, device, equipment, medium and program product, which can perform expression face changing in an expression template based on a user face, and can generate an expression image of the user face individually, so as to meet the interesting and individual needs of the expression image.

[0004] In one aspect, the embodiments of the present application provide an image processing method, which comprises:

[0005] In response to an expression face changing operation received in a social conversation process, a first image containing face information of a first object is obtained;

[0006] An expression template image is obtained, which contains face information of a first reference object and face information of a second reference object;

[0007] According to the first image and the expression template image, a first expression image is generated, which contains face features reflected by the face information of the first object and contains face information of a target reference object, the target reference object being any one of the first reference object and the second reference object; and

[0008] The first expression image is displayed in a social interface.

[0009] In one aspect, the embodiments of the present application provide an image processing method, which comprises:

[0010] An image processing request sent by a social client is received, the image processing request being generated after an expression face changing operation is received in a social conversation process of the social client, the image processing request containing a first image and an expression template image, the first image containing face information of a first object, and the expression template image containing face information of a first reference object and face information of a second reference object;

[0011] generate a first expression image according to the first image and the expression template image; the first expression image has a style feature of the expression template image, and contains a face feature reflected by the face information of the first object and face information of a target reference object, the target reference object being any one of the first reference object and the second reference object;

[0012] return the first expression image to the social client, so that the social client displays the first expression image in the social interface.

[0013] In an aspect, an embodiment of the present application provides an image processing apparatus, which comprises:

[0014] an acquisition unit configured to acquire a first image containing face information of a first object in response to an expression face changing operation received in a social conversation process;

[0015] the acquisition unit is further configured to acquire an expression template image containing face information of a first reference object and face information of a second reference object;

[0016] a processing unit configured to generate a first expression image according to the first image and the expression template image, the first expression image containing a face feature reflected by the face information of the first object and face information of a target reference object, the target reference object being any one of the first reference object and the second reference object;

[0017] a display unit configured to display the first expression image in a social interface.

[0018] In a possible implementation, the acquisition unit acquires the first image, and is configured to perform the following operations:

[0019] display an image uploading entrance, the image uploading entrance being configured to trigger acquisition of the first image to be processed;

[0020] when it is detected that the image uploading entrance is selected, acquire any image containing face information of the first object from an image library as the first image, or call a shooting device to collect face information of the first object to generate the first image.

[0021] In a possible implementation, the acquisition unit acquires the expression template image, and is configured to perform the following operations:

[0022] display an expression package template list, the expression package template list having at least one expression package template displayed therein, any expression package template including one or more template images;

[0023] acquire a selected expression template image in response to a selection operation performed in the expression package template list;

[0024] The expression template image is any one of the sticker template list, or the expression template image is any one of the template images under any one of the sticker template list.

[0025] In a possible implementation, the processing unit is further configured to perform the following operation:

[0026] Obtain the social permission corresponding to the first object in the social conversation process;

[0027] Display the sticker template list to be selected by the first object according to the social permission;

[0028] If the social permission is the first permission, the sticker template list is the first template list; if the social permission is the second permission, the sticker template list is the second template list; the first permission is different from the second permission, and the first template list is different from the second template list.

[0029] In a possible implementation, the expression template image is a target sticker template in the sticker template list, the target sticker template includes K template images, K is a positive integer; the processing unit is further configured to perform the following operation:

[0030] Generate K expression images according to the first image and the K template images;

[0031] Any one of the expression images is generated based on the first image and any one of the K template images; and any one of the expression images has the style characteristics of the corresponding template image and the facial characteristics reflected by the facial information of the first object.

[0032] In a possible implementation, the processing unit generates the first expression image according to the first image and the expression template image, and is configured to perform the following operation:

[0033] Perform expression face changing processing on the first object in the first image and the first reference object in the expression template image to generate the first expression image, the first expression image including the facial characteristics of the first object and the facial characteristics of the first reference object; or

[0034] Perform expression face changing processing on the first object in the first image and the second reference object in the expression template image to generate the first expression image, the first expression image including the facial characteristics of the first object and the facial characteristics of the second reference object.

[0035] The reference object subjected to the expression face changing processing is an object with the same attribute as the first object.

[0036] In a possible implementation, the first image includes face information of the first object and face information of the second object; and the processing unit is further configured to perform the following operation:

[0037] performing expression face swapping processing on the first object in the first image and the first reference object in the expression template image, and performing expression face swapping processing on the second object in the first image and the second reference object in the expression template image, to obtain a second expression image;

[0038] The second expression image has the style feature of the expression template image, and includes face features reflected by the face information of the first object and face features reflected by the face information of the second object.

[0039] In a possible implementation, the expression template image includes face information of the first reference object and face information of the second reference object, and the first expression image includes face features of the first object and face features of the second reference object; and the processing unit is further configured to perform the following operation:

[0040] in response to a selection operation on the first expression image, displaying a face swapping invitation entry;

[0041] when it is detected that the face swapping invitation entry is triggered, generating a face swapping invitation message;

[0042] sending the face swapping invitation message to the second object; the face swapping invitation message is used to trigger obtaining a second image including face information of the second object, and generating a second expression image based on the second image and the first expression image; the second expression image has the style feature of the expression template image, and includes face features reflected by the face information of the first object and face features reflected by the face information of the second object.

[0043] In a possible implementation, the second object for receiving the face swapping invitation message includes any one of the following:

[0044] any social object in a one-on-one social conversation with the first object; or

[0045] a reference social object in a group social conversation to which the first object belongs; the reference social object includes any one of the following: any social object in the group social conversation, a social object with the highest interaction frequency with the first object in the group social conversation, a social object with the most recent interaction operation time with the first object in the group social conversation, and a social object publishing the latest conversation message in the group social conversation.

[0046] In a possible implementation, the processing unit is further configured to perform the following operation:

[0047] Display a prompt message in a social interface, the prompt message being used to prompt that a second image containing face information of a second object has been acquired;

[0048] In response to an update operation triggered for the first expression image displayed in the social interface, update and display the first expression image as a second expression image in the social interface;

[0049] The second expression image is generated based on an update processing of the first expression image according to the second image; and the second expression image includes face features of the first object and face features of the second object.

[0050] In a possible implementation, the display unit displays the first expression image in the social interface, for performing any one of the following operations:

[0051] Display the first expression image in an expression panel of the social interface; or,

[0052] Display the first expression image in a single chat session interface of the first object and the second object; or,

[0053] Display the first expression image in any group chat social session interface to which the first object belongs; or,

[0054] Display the first expression image in an image identifier of the first object; the image identifier includes any one of the following: a head portrait, a personal homepage, and a background image.

[0055] In a possible implementation, the expression panel of the social interface displays at least one system expression image; and the processing unit is further configured to perform the following operation:

[0056] Distinguishively display the first expression image and each system expression image in the expression panel of the social interface; and the distinguishive display manner includes any one of the following: animation, highlighting, and prompting.

[0057] In a possible implementation, the social session process refers to a process of a group chat social session in which the first object is located, and the group chat social session includes the first object and N social objects, N being a positive integer; the expression template image includes face information of a first reference object and face information of a second reference object; and the processing unit is further configured to perform the following operation:

[0058] Display a reference expression image in an expression panel of the group chat social session; the reference expression image is generated according to the first image, the expression template image, and a reference image containing face information of a reference social object.

[0059] The reference social object includes any one of the following: a social object that has the highest interaction frequency with the first object among the N social objects, a social object that has the most recent interaction operation time with the first object among the N social objects, a social object that publishes the latest conversation message among the N social objects, and a social object that has the highest activity among the N social objects.

[0060] In a possible implementation, the processing unit is further configured to perform the following operation:

[0061] In response to the selection operation on the reference expression image, the reference expression image is displayed in a conversation window of the group chat social conversation; and

[0062] The interaction message for the reference social object is displayed in the conversation window of the group chat social conversation.

[0063] The display position of the interaction message includes any one of the following: an arbitrary position in the conversation window, a specified position in the conversation window, and an associated position of the image identifier of the reference social object.

[0064] In a possible implementation, the processing unit is further configured to perform the following operation:

[0065] In response to an expression update operation detected in the expression panel of the group chat social conversation, an object selection prompt box is output; and the object selection prompt box displays object identifiers of N social objects to be selected in the group chat social conversation.

[0066] In response to a selection operation on the object identifier of the target social object among the N social objects, the reference expression image is updated and displayed as a third expression image in the sticker panel.

[0067] The third expression image has the style characteristics of the expression template image, and includes face features reflected by the face information of the first object and face features reflected by the face information of the target social object.

[0068] In a possible implementation, the expression face changing operation is generated in any one of the following ways:

[0069] When it is detected that a face changing function entry set in the social interface is triggered, the expression face changing operation is generated, and the face changing function entry includes any one of the following: a control and an option; or

[0070] When it is detected that a specific touch operation exists in an operation area of the social interface, the expression face changing operation is generated, and the specific touch operation includes any one of the following: a single-click operation, a double-click operation, a gesture operation, and a hovering gesture; or

[0071] When it is detected that a face changing invitation message displayed in a social interface is selected, a facial expression changing operation is generated, the face changing invitation message being sent by a second object to a first object during a social conversation process; wherein the format of the face changing invitation message includes any one of a card, a link, a website, and a structured message.

[0072] In one aspect, an embodiment of the present application provides an image processing device, which comprises:

[0073] A receiving unit is configured to receive an image processing request sent by a social client, the image processing request being generated after a facial expression changing operation is received during a social conversation process of the social client, the image processing request comprising a first image and an expression template image, the first image containing face information of a first object, and the expression template image containing face information of a first reference object and face information of a second reference object;

[0074] A processing unit is configured to generate a first expression image according to the first image and the expression template image, the first expression image having style features of the expression template image, and containing face features reflected by the face information of the first object and face information of a target reference object, the target reference object being any one of the first reference object and the second reference object;

[0075] A sending unit is configured to return the first expression image to the social client, so that the social client displays the first expression image in a social interface.

[0076] In one possible implementation, the processing unit generates the first expression image according to the first image and the expression template image, and is configured to perform the following operations:

[0077] perform region segmentation processing on the first image to obtain i first regions, and perform feature extraction processing on each first region to obtain i first region feature vectors corresponding to the i first regions, i being a positive integer; and

[0078] perform region segmentation processing on the expression template image to obtain j second regions, and perform feature extraction processing on each second region to obtain j second region feature vectors corresponding to the j second regions, j being a positive integer;

[0079] perform facial expression changing processing based on the i first region feature vectors and the j second region feature vectors to generate the first expression image.

[0080] In one possible implementation, the i first region feature vectors include face features reflected by the face information of the first object, and the j second region feature vectors include face features reflected by the face information of the first reference object and style features of the expression template image.

[0081] The processing unit performs expression face replacement processing based on the i first regional feature vectors and the j second regional feature vectors, and generates a first expression image, for performing the following operations:

[0082] The i first regional feature vectors and the j second regional feature vectors are encoded by using an attention mechanism to obtain encoded feature vectors;

[0083] The encoded feature vectors are decoded, and the decoded feature vectors are synthesized to generate the first expression image;

[0084] The synthesis processing is used to indicate that the i first regional feature vectors and the j second regional feature vectors are fused, and the facial features of the first object and the facial features of the first reference object are replaced.

[0085] In one aspect, the embodiments of the present application provide a computer device, which includes a processor, an input device, an output device and a memory; the memory stores a computer program; and the computer program is executed by the processor to perform the image processing method.

[0086] In one aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program; and the computer program is executed by a processor to perform the image processing method.

[0087] In one aspect, the embodiments of the present application provide a computer program product, which includes a computer program; and the computer program is executed by a processor to perform the image processing method.

[0088] In the embodiments of the present application, in response to the received facial expression changing operation in the social conversation process, a first image containing face information of a first object is obtained; an expression template image containing face information of a first reference object and face information of a second reference object is obtained; a first expression image is generated according to the first image and the expression template image, the first expression image containing face features reflected by the face information of the first object and containing face information of a target reference object, the target reference object being any one of the first reference object and the second reference object; and the first expression image is displayed in a social interface. As can be seen, the present application can perform facial expression changing processing on the face of the first object in the first image and the face of any one reference object (for example, the first reference object) in the expression template image, so that the first expression image generated after face changing not only has the style features of the expression template image, but also reflects the face features of the first object after face changing, and also has the face features of the target reference object (for example, the second reference object) that has not been changed. That is, the first expression image is an image personalized by the face features of the first object. Then, different expression images can be generated by different face features of objects for the same expression template image, so as to meet the personalized needs of users for expression images, and make the expression images generated by the present application more personalized and interesting. BRIEF DESCRIPTION OF DRAWINGS

[0089] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0090] Figure 1a is a structural schematic diagram of an image processing system provided by the embodiments of the present application;

[0091] Figure 1b is an interactive flowchart of an image processing method provided by the embodiments of the present application;

[0092] Figure 2 is a flowchart of an image processing method provided by the embodiments of the present application;

[0093] Figure 3a is an interface schematic diagram of generating an expression changing operation provided by the embodiments of the present application;

[0094] Figure 3b is another interface schematic diagram of generating an expression changing operation provided by the embodiments of the present application;

[0095] Figure 3cis another interface schematic diagram provided by an embodiment of the present application for generating an expression face changing operation;

[0096] Figure 4a is an interface schematic diagram provided by an embodiment of the present application for acquiring a first image;

[0097] Figure 4b is another interface schematic diagram provided by an embodiment of the present application for acquiring a first image;

[0098] Figure 5a is an interface schematic diagram provided by an embodiment of the present application for acquiring an expression template image;

[0099] Figure 5b is an interface schematic diagram provided by an embodiment of the present application for displaying an expression package template list;

[0100] Figure 6a is a flow schematic diagram provided by an embodiment of the present application for single-person expression face changing processing;

[0101] Figure 6b is a flow schematic diagram provided by an embodiment of the present application for double-person expression face changing processing;

[0102] Figure 6c is another flow schematic diagram provided by an embodiment of the present application for double-person expression face changing processing;

[0103] Figure 7a is a flow schematic diagram provided by an embodiment of the present application for one-key generation of multiple expression images;

[0104] Figure 7b is a flow schematic diagram provided by an embodiment of the present application for one-key update of multiple expression images;

[0105] Figure 7c is a flow schematic diagram provided by an embodiment of the present application for inviting a social object to cooperate in face changing;

[0106] Figure 7d is an interface schematic diagram provided by an embodiment of the present application for inviting a second object to cooperate in face changing;

[0107] Figure 7e is another interface schematic diagram provided by an embodiment of the present application for inviting a second object to cooperate in face changing;

[0108] Figure 7f is another flow schematic diagram provided by an embodiment of the present application for acquiring a first image;

[0109] Figure 8 is an interface schematic diagram provided by an embodiment of the present application for displaying a first expression image;

[0110] Figure 9a is a flowchart of updating an expression image provided by an embodiment of the present application;

[0111] Figure 9b is an interface diagram provided by an embodiment of the present application for distinguishing display of a first expression image;

[0112] Figure 10a is an application interface diagram of a reference expression image provided by an embodiment of the present application;

[0113] Figure 10b is a flowchart of generating a third expression image provided by an embodiment of the present application;

[0114] Figure 11 is a flowchart of another image processing method provided by an embodiment of the present application;

[0115] Figure 12 is a flowchart of generating a first expression image provided by an embodiment of the present application;

[0116] Figure 13 is a structural diagram of an image processing device provided by an embodiment of the present application;

[0117] Figure 14 is a structural diagram of another image processing device provided by an embodiment of the present application;

[0118] Figure 15 is a structural diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0119] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0120] The present application provides an image processing scheme, which mainly relates to personalized generation of expression images reflecting facial features of a user based on a face image of the user. Different users can generate different expression images based on the same expression template image, thereby meeting the interesting and personalized needs of expression images. Specifically, the implementation principle of the image processing scheme is roughly as follows:

[0121] (1) In response to an expression face changing operation received in a social conversation process, a first image containing face information of a first object is obtained. The first image can be an image uploaded by the first object himself / herself, or an image uploaded by another object (not the first object).

[0122] (2) Obtain an expression template image, which contains face information of a first reference object and face information of a second reference object; here, the first reference object or the second reference object refers to a virtual object contained in the expression template image, which is usually an AI object of an animation type, and is generated by a background system by default.

[0123] (3) Generate a first expression image according to the first image and the expression template image; wherein the first expression image contains face features reflected by the face information of the first object, and contains face information of a target reference object, which refers to any one of the first reference object and the second reference object; and display the first expression image in a social interface. When performing expression face replacement, the first object can be face replaced with the first reference object; or the first object can be face replaced with the second reference object, and the target reference object to be face replaced can be an object with the same attribute as the first object and the first reference object and the second reference object. For example, the expression template image is an image that expresses a "happy" expression together through the first reference object and the second reference object, and the generated first expression image can be an image that reflects the "happy" expression together through the face features of the first object and the face features of the second reference object; or the first expression image can also be an image that reflects the "happy" expression together through the face features of the first object and the face features of the first reference object. That is, the present application realizes expression face replacement of the first object with any reference object in the expression template image, so that the generated first expression image is a personalized expression image that expresses facial expression through the unique first object.

[0124] As can be seen from the above, the present application can perform expression face replacement of the face of the first object in the first image and the face of the reference reference object in the expression template image, so that the generated first expression image after face replacement can have the style features of the expression template image, can reflect the face features of the first object, and can contain the face information of the target reference object that is not executed face replacement, that is, the first expression image is an image that is personalized generated through the face features of the first object; then, different expression images can be generated through the face features of different objects for the same expression template image, so as to meet the personalized needs of users for expression images, so that the expression images generated by the present application are more personalized and interesting.

[0125] The technical terms related to the embodiments of the present application will be introduced as follows:

[0126] I. Social client:

[0127] The image processing scheme provided in the application can be executed by a social client, and specifically can be implemented by an application program providing social capabilities deployed in the social client. The application program can refer to a computer program for completing one or more specific tasks. According to different dimensions (such as the running mode and function of the application program), the application program can be classified to obtain the type of the same application program in different dimensions, wherein: ① according to the running mode of the application program, the application program can include but is not limited to: a client installed in a terminal, a small program that can be used without downloading and installing, a web application program opened through a browser, and the like. ② According to the function type of the application program, the application program can include but is not limited to: an IM (Instant Messaging, instant messaging) application program, a content interaction application program, and the like; wherein the instant messaging application program refers to an application program based on instant messaging and social interaction on the Internet, and the instant messaging application program can include but is not limited to: a social application program containing a communication function, a map application program containing a social interaction function, a game application program, and the like. The content interaction application program refers to an application program capable of realizing content interaction, for example, can be an online banking application program, a sharing platform application program, a personal space application program, a news application program, and the like. The embodiments of the present application mainly relate to an application program providing social capabilities (which can be referred to as a social application program), which is used to support a social conversation.

[0128] II. Social Conversation

[0129] The social client supports a social session, which can include a one-on-one social session and a group social session. The one-on-one social session refers to a social session in which two social objects (e.g., a first object and a second object) participate, for information exchange between the two social objects. The group social session refers to a social session in which multiple (more than two) social objects participate, for information exchange between the multiple social objects. It should be understood that the social session can include a management object and a member object, where the management object refers to a user who has a management right of the social session, for example, the management object can be a creator or an administrator in a certain group social session; the member object refers to a user who does not have the management right of the social session. The management right of the social session can be divided into a right of managing a social object in the social session and a right of managing information in the social session, where the right of managing the social object in the social session includes but is not limited to: ① a right of controlling a number of users in the social session, for example, adding a new member object or deleting an existing member object in the social session; ② a right of managing a user right of a social object, for example, assigning a management right to a certain member object in the social session, so that the member object changes to a management object. The right of managing information in the social session includes but is not limited to: ③ a right of auditing information in the social session, for example, auditing whether the format of information in the social session meets the format requirement, and whether the content meets the theme requirement, and the like; ④ a right of modifying information in the social session, for example, deleting a piece of information in the social session when it is found that the piece of information does not meet the requirement; or changing a piece of information in a social channel to another social channel when it is found that the content of the piece of information does not meet the theme requirement of the social channel.

[0130] III. Social interface

[0131] The social interface refers to an interface for displaying social messages generated in a social session. For example, when the social session is an instant messaging session group, the social interface of the social session can be an All In One (AIO) page of the instant messaging session group, which is used to display a message stream generated by social exchange of each social object in the instant messaging session group. For another example, when the social session is a game social session, a map social session, or the like, the social interface of the social session can be a message dynamic page, which is used to display a message dynamic published by a social object in the content interaction group.

[0132] IV. First image, expression template image, and first expression image

[0133] The first image refers to an image containing face information of the first object, that is, the first image is an image uploaded by the user containing the face of the real user, and the user uploading the first image can be the first object himself or other objects other than the first object. In addition, the first image can be any image containing face information of the first object uploaded from an image library, or an image generated by real-time collection of face information of the first object during a social conversation, which is not limited.

[0134] The expression template image, as the name implies, is a template image used to depict the user's expression. The expression template image can contain face information of the first reference object and face information of the second reference object. The first reference object and the second reference object herein are both virtual objects, which are usually AI characters of an animation type. The expression template image can include one or more reference objects, such as only the first reference object, or the first reference object and the second reference object, and so on. In addition, the expression template image has different types of style features, which include but are not limited to: couple daily life, early eight people collapse daily life, magic school, baby bus, and so on.

[0135] The first expression image is an image generated based on the first image and the template expression image. Specifically, it is an expression image generated by performing expression face replacement processing on the first object in the first image and any reference object (such as the first reference object or the second reference object with the same attribute as the first object) in the template expression image. Therefore, the first expression image has the style features of the expression template image, the face features reflected by the face information of the first object, and the face information of the target reference object that has not been replaced. The face features include but are not limited to: eyelid type (such as single eyelid or double eyelid), eyebrow type (such as crescent eyebrow, straight eyebrow, thick eyebrow, or thin eyebrow), high nose bridge or flat nose bridge, lip type (thin lips or thick lips), whether wearing glasses, and hair features (such as hair color, hairstyle, hair length, etc.).

[0136] Five, artificial intelligence:

[0137] Artificial Intelligence (AI) is the use of digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence; artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-training model technology, operation / interaction system, mechatronics, etc.; artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. several major directions.

[0138] In this application, in the process of generating the first expression image according to the first image and the expression template image, many image processing processes are involved, such as: expression face changing processing, face feature extraction processing, and image segmentation processing, etc. Optionally, a machine learning technology can be used to train an AI diffusion model, so that the AI diffusion model has an expression face changing function, that is, calling the trained AI diffusion model can realize single face changing function or multi-face changing function based on the expression template image and the first image, thereby realizing the synthesis of personalized expression images, while maintaining high fidelity, supporting the generation of various different style expression images. Subsequently, the expression image generated by the present application can be used for social interaction in the process of social conversation, which can improve the interest of social interaction while meeting the personalized needs of users for expression images.

[0139] Six, blockchain:

[0140] Blockchain is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. Blockchain (Block chain), in essence, is a decentralized database, which is a series of data blocks associated using cryptography. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-fake) and generate the next block. The following describes the concepts of blockchain system, blockchain node, and block structure.

[0141] In the present application, a plurality of image data are involved in the image processing process, such as: first image, expression template image, first expression image, etc. These images are the main data involved in the expression face changing process. Optionally, the present application can send the above-mentioned images to the blockchain for storage. Based on the characteristics of the blockchain, such as non-tamperability and traceability, the image data can be avoided to be tampered or leaked, thereby improving the data security and reliability of the image processing process.

[0142] It should be particularly noted that the related data (such as: first image, second image, expression template image, etc.) involved in the image processing process of the present application. When the above embodiments of the present application are applied to specific products or technologies, the permission or consent of the object needs to be obtained, and the related data collection, use and processing process needs to comply with the relevant laws, regulations and standards of the region, and meet the principles of legality, legitimacy and necessity, and does not involve the acquisition of data types prohibited or restricted by laws and regulations. In some optional embodiments, the related data involved in the embodiments of the present application is obtained after the object authorizes separately, and in addition, the object is informed of the purpose of the related data when the object authorizes separately.

[0143] The following will be described in combination with Figure 1a and Figure 1b The image processing system provided by the present application is introduced.

[0144] Please refer to Figure 1a , Figure 1a is a structural schematic diagram of an image processing system provided by an embodiment of the present application. As Figure 1a indicated, the image processing system can include a plurality of terminal devices (such as a first terminal device 1001, a second terminal device 1002, etc.) and a server 1003, and the number of terminal devices and servers is not limited by the present application. For example, the first terminal device 1001 is a terminal device used by a first object, and the second terminal device 1002 is a terminal device used by a second object; and any terminal device runs a social client, which is used to provide a social conversation capability for a social object through the social client. It should be noted that any terminal device (such as the first terminal device 1001) and the server 1003 can be directly or indirectly connected through wired or wireless communication, and the first terminal device 1001 can interact with other terminal devices (such as the second terminal device 1002) connected through the server 1003.

[0145] It should be understood that, Figure 1aAny terminal device in the image processing system shown can include, but is not limited to: smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, in-vehicle terminals, smart wearable devices, etc. The client often has a display device, which can be a monitor, display screen, touchscreen, etc., and the touchscreen can be a touchscreen, touch panel, etc. Furthermore, Figure 1a The server 1003 in the image processing system shown can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0146] The following example illustrates the interaction process between the first terminal device 1001, the second terminal device 1002, and the server 1003.

[0147] (1) The first terminal device 1001 responds to the face-swapping operation received during the social conversation and acquires a first image, which contains facial information of the first object; wherein, the first image may be selected by the first object from an image library, or the first image may be acquired in real time by the first object through the shooting device (such as a camera) of the first terminal device 1001.

[0148] (2) The first terminal device 1001 acquires an expression template image, which contains facial information of the first reference object and facial information of the second reference object.

[0149] (3) The first terminal device 1001 sends a face-swapping invitation message to the second terminal device 1002. The face-swapping invitation message is used to trigger the acquisition of a second image containing the facial information of the second object.

[0150] (4) The second terminal device 1002 acquires the second image, which includes the facial information of the second object; similarly, the second image may be selected by the second object from the image library, or it may be acquired in real time by the second object through the shooting device (such as a camera) of the second terminal device 1002.

[0151] (5) The first terminal device 1001 sends the first image and the expression template image to the server 1003; the second terminal device 1002 sends the second image to the server 1003.

[0152] (6) The server 1003 generates a second expression image based on the first image, the second image, and the expression template image. The second expression image has the style characteristics of the expression template image and contains facial features reflected by the facial information of the first subject and facial features reflected by the facial information of the second subject.

[0153] (7) The second expression image is displayed in the social interface of the first terminal device 1001 or the social interface of the second terminal device 1002.

[0154] In the above process, the first subject and the second subject can realize expression cooperation face changing, that is, the first reference subject and the first subject can be expression face changed, and the second reference subject and the second subject can be expression face changed, so as to generate a second expression image after double face changing, thereby enriching the interestingness of the expression face changing process.

[0155] In a possible implementation, taking the first terminal device 1001 as an example, a social client runs in the first terminal device 1001, and a social conversation can run in the social client, and a social interface (or a front-end interface) is displayed to the user in the social conversation process; a server runs in the server 1003. Please refer to Figure 1b , Figure 1b is an interactive flowchart of an image processing method provided by an embodiment of the present application. As shown in Figure 1b , the interactive flowchart of the image processing method mainly involves the interaction among a social client, a front-end interface, and a server, and the specific interaction process includes the following:

[0156] (1) The user (such as the first subject) generates an expression face changing operation in the social conversation process of the social client to enter a front-end interface for selecting an expression template image. For example, the first subject can trigger the expression face changing operation by clicking a face changing function entry, which includes any one of a control, an option, or the first subject can generate the expression face changing operation after performing a specific touch operation in an operation area of a social interface displayed by the social client, where the specific touch operation includes any one of a single-click operation, a double-click operation, a gesture operation, or a hovering gesture.

[0157] (2) The front-end interface displays an expression package template list. The expression package template list displays at least one expression package template, and each expression package template includes one or more template images.

[0158] (3) The first subject selects a corresponding expression template image from the expression package template list. The expression template image selected by the first subject can be any expression package template in the expression package template list, or the expression template image can be any template image under any expression package template in the expression package template list.

[0159] (4) The front-end interface calls the social client's ability to select pictures through jsAPI, and pulls up a picture selector. The picture selector is used to display an image library including multiple pictures collected, and the first object can select any picture from the image library as the first image to be uploaded. Optionally, the first object can also call the shooting device of the social client to collect the first image in real time.

[0160] (5) The social client uploads the expression template image and the first image to the server. The first image includes the face information of the first object, and the expression template image includes the face information of the first reference object.

[0161] (6) The server saves the expression template image and the first image, and performs face information detection and security-related detection on the expression template image and the first image, and returns related information, such as the image address and image information of the expression template image (and the first image).

[0162] (7) The social client returns the related information (i.e., the image address and image information of the first image, and the image address and image information of the expression template image, etc.) sent by the server to the front-end interface.

[0163] (8) The first object saves the picture in the social client.

[0164] (9) The front-end interface sends the first image and the expression template image to the server, and requests the server to generate a face expression image.

[0165] (10) The server generates a first expression image based on the first image and the expression template image. Specifically, the server can extract the face features of the first object, and thus perform expression face changing processing on the first object and the first reference object to generate a face expression (i.e., the first expression image) exclusive to the first object.

[0166] (11) The server returns the first expression image to the front-end interface for display.

[0167] (12) The exclusive first expression image is saved to an AIO page, and the exclusive face expression first expression image can be used for social interaction with other social objects in a subsequent social conversation process.

[0168] In response to the received facial expression changing operation in the social conversation process, the image processing system provided by the embodiment of the present application acquires a first image containing face information of a first object; acquires an expression template image containing face information of a first reference object and face information of a second reference object; generates a first expression image according to the first image and the expression template image, the first expression image containing face features reflected by the face information of the first object and containing face information of a target reference object, the target reference object being any one of the first reference object and the second reference object; and displays the first expression image in a social interface. As can be seen, the present application can perform facial expression changing processing on the face of the first object in the first image and the face of any one reference object (for example, the first reference object) in the expression template image, so that the first expression image generated after face changing not only has the style features of the expression template image, but also reflects the face features of the first object after face changing, and also has the face features of the second reference object that has not been changed. That is, the first expression image is an image personalized by the face features of the first object. Then, different expression images can be generated by different face features for the same expression template image, thereby meeting the personalized needs of users for expression images and making the expression images generated by the present application more personalized and interesting.

[0169] Based on the above introduction of the image processing system provided by the embodiment of the present application, the following points need to be explained:

[0170] ① The image processing system mentioned above in the embodiment of the present application Figure 1a The system architecture shown is to more clearly illustrate the technical solutions of the embodiment of the present application, and does not constitute a limitation on the technical solutions provided by the embodiment of the present application. Those skilled in the art can know that, with the evolution of system architecture and the appearance of new business scenarios, the technical solutions provided by the embodiment of the present application are also applicable to similar technical problems. That is, Figure 2 The architecture diagram of the image processing system shown is an example architecture diagram; in actual application, the number and distribution of devices (such as terminals and servers) included in the image processing system can change, and the embodiment of the present application does not limit the architecture diagram of the image processing system.

[0171] ② The image processing scheme provided by the embodiment of the present application is not only deployed in a social client, but also supports existing in the form of a plug-in (such as a ControlNet plug-in). Specifically, the plug-in can be a system-level program, so that the plug-in can be directly deployed in the first terminal device 1001 to implement the image processing scheme provided by the embodiment of the present application; or the plug-in can also be an application-level program running in a social client, and the image processing scheme provided by the embodiment of the present application can be implemented by calling the plug-in through the social client.

[0172] ③The relevant data collection and processing in the embodiments of the present application should strictly comply with the requirements of relevant laws and regulations. The personal information should be obtained with the knowledge or consent of the personal subject (or with the legal basis for obtaining information), and the subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject. For example, when the embodiments of the present application are applied to specific products or technologies, such as obtaining a first image containing face information of a first object, the permission or consent of the first object is required, and the collection, use and processing of relevant data (such as generating a first expression image according to the first image and an expression template image) should comply with relevant laws and regulations and standards in the relevant region.

[0173] The image processing method provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0174] Please refer to Figure 2 , Figure 2 is a flowchart of an image processing method provided by the embodiments of the present application. The image processing method can be executed by a computer device, which can be any terminal device as shown in Figure 1a . As shown in Figure 2 , the image processing method can include the following steps S201-S204:

[0175] S201: In response to an expression face changing operation received in a social conversation process, a first image containing face information of a first object is obtained.

[0176] Wherein, a social interface is displayed in the social conversation process, and if an expression face changing operation is detected in the social interface, it can be considered that an expression face changing operation is received in the social conversation process. Several different ways of generating expression face changing operation are illustrated as follows:

[0177] (1) The expression face changing operation is generated through a face changing function entry.

[0178] Specifically, the expression face changing operation is generated after detecting that the face changing function entry set in the social interface is triggered, and the face changing function entry includes any one of a control and an option. Wherein, the face changing function entry can be set at a specified position (such as in an expression panel) or at any position in the social interface, and the face changing function entry can be a first-level entry or a second-level entry. The first-level entry refers to an entry directly set in the social interface and directly displayed in the current social interface, and the second-level entry refers to an entry indirectly set in the social interface and not directly displayed in the social interface (i.e. the second-level entry needs to be triggered by other controls before it can be displayed).

[0179] Please refer to Figure 3a , Figure 3ais a schematic diagram of an interface provided by an embodiment of the present application for generating an expression face changing operation. As shown in Figure 3a The current social conversation process is a one-on-one social conversation process between a first object and a second object, and the current social interface is a one-on-one conversation interface S301 between the first object and the second object (e.g., user A), in which an expression panel is displayed, and the expression panel is provided with a search control 3011. The first object clicks the search control 3011, and a face changing function entry, e.g., an "expression laboratory" control 3012, is displayed. If the first object clicks (e.g., single-clicks, double-clicks, or long-presses, etc.) the "expression laboratory" control 3012, it is considered that an expression face changing operation is generated. Subsequently, a display interface S302 is triggered, and the first image is acquired in the interface S302.

[0180] (2) Generating an expression face changing operation through a specific touch operation.

[0181] Specifically, when a specific touch operation is detected in the operation area of the social interface, an expression face changing operation is generated. The specific touch operation includes any one of a single-click operation, a double-click operation, a gesture operation, and a hovering gesture. Please refer to Figure 3b , Figure 3b is another schematic diagram of an interface provided by an embodiment of the present application for generating an expression face changing operation. As shown in Figure 3b An operation area 3021 is provided in a one-on-one conversation interface between a first object and a second object (e.g., user A), allowing any user (e.g., the first object) to perform a specific touch operation in the operation area 3021. When a specific touch operation is detected in the operation area 3021, it is considered that an expression face changing operation is generated. The specific touch operation can be used to indicate a gesture of a specific shape (e.g., "S", "V", or any other shape). For example, the first object can draw a "V" character in the operation area 3021, and it is considered that an expression face changing operation is detected.

[0182] (3) Generating an expression face changing operation through an invitation from another object.

[0183] Specifically, when a face changing invitation message displayed in the social interface is selected, an expression face changing operation is generated. The face changing invitation message is sent by the second object to the first object in the social conversation process. The format of the face changing invitation message includes any one of a card, a link, a website, and a structured message (e.g., an ark message). Please refer to Figure 3c , Figure 3c is another schematic diagram of an interface provided by an embodiment of the present application for generating an expression face changing operation. As shown in Figure 3cAs shown, in the single chat session interface between the first object and the second object (such as user A), a face-changing invitation message 3031 of the second object is displayed, which can be in any format of message such as a card, or a link, or a website, or an ARK message; and the face-changing invitation message 3031 is provided with an invitation control 3032; if the first object clicks the invitation control 3032, an expression face-changing operation is triggered.

[0184] Further, after detecting the expression face-changing operation in the social conversation process, the first image is acquired to trigger the execution of the expression face-changing, and the first image includes the face information of the first object. In a possible implementation, in response to the expression face-changing operation received in the social conversation process, an image uploading entrance is displayed, the image uploading entrance is used to trigger the acquisition of the first image to be processed; when it is detected that the image uploading entrance is selected, any image is acquired from the image library as the first image; or a shooting device is called to collect the face information of the first object to generate the first image. The first image can be a photo uploaded by the first object himself, and the first image can also be a photo uploaded by other objects (not the first object). For example, Figure 3a As shown, after the first object clicks the face-changing function entrance 3012, an interface S302 is displayed, and the interface S302 is provided with an image uploading entrance 3013, which is used to trigger the acquisition of the first image to be processed.

[0185] The following describes two ways of how to acquire the first image.

[0186] (1) The first image is selected from the image library.

[0187] Please refer to Figure 4a , Figure 4a is an interface schematic diagram provided by an embodiment of the present application for acquiring the first image. As shown in Figure 4a , after the first object clicks the image uploading entrance 3013, an image library is displayed, and the image library provides at least one collected image; then, the first object can select any image (such as 3014) containing the face information of the first object in the image library as the first image. In this implementation, the first image does not need to be collected in real time, and the efficiency of acquiring the first image can be improved.

[0188] (2) The first image is obtained by the first object in real time.

[0189] Please refer to Figure 4b , Figure 4b is another interface schematic diagram provided by an embodiment of the present application for acquiring the first image. As shown in Figure 4aAs shown, after the first object clicks the image uploading entrance 3013, a shooting interface is displayed, and the first object can call a shooting device (such as a camera of a mobile phone) to collect face information of the first object in the shooting interface to shoot a first image. In this implementation manner, the first image is a real-time collected image, which can guarantee the real-time performance of the first image and improve the accuracy of image processing.

[0190] In S202, an expression template image is acquired, and the expression template image includes face information of the first reference object and face information of the second reference object.

[0191] In a possible implementation manner, the process of acquiring the expression template image is as follows: displaying an expression package template list, the expression package template list including at least one expression package template, and each expression package template including one or more template images; and acquiring a selected expression template image in response to a selection operation performed in the expression package template list. The acquired expression template image is any expression package template in the expression package template list, or the expression template image is any template image under any expression package template in the expression package template list. It should be noted that one expression package template has one style feature, for example, any type of style feature such as a lover daily style, a worker work style, a student school style, and the like; and different template images under the same expression package template are used to reflect different types of expressions in the same style feature, for example, the expression package template of the "lover daily style" includes three template images, for example, image 1, image 2, and image 3. The image 1 is an expression image used to reflect a lover quarrel, the image 2 is an expression image used to reflect a lover showing love, and the image 3 is an expression image used to reflect a lover eating.

[0192] Please refer to Figure 5a , Figure 5a is an interface schematic diagram provided by an embodiment of the present application for acquiring an expression template image. As shown in Figure 5aAs shown, after the first object clicks the image uploading entrance 3013, an emoticon package template list 501 can be displayed, and the emoticon package template list 501 displays at least one emoticon package template. The emoticon package template here can include a single-person emoticon package template and a double-person emoticon package template. The single-person emoticon package template refers to a template image including face information of one reference object (such as the first reference object), such as the single-person emoticon package template 5011 and the single-person emoticon package template 5012. The double-person emoticon package template refers to a template image including face information of two reference objects (such as the first reference object and the second reference object), such as the double-person emoticon package template 5021 and the double-person emoticon package template 5022. Further, any emoticon package template (single-person emoticon package template or double-person emoticon package template) can include one or more template images. For example, for the single-person emoticon package template 5011, three single-person template images are included, and the template image 5013 is one of the single-person template images. For example, for the double-person emoticon package template 5021, two double-person template images are included, and the template image 5023 is one of the double-person template images.

[0193] In a possible implementation, the emoticon package template list to be selected can be displayed for the first object according to a social permission. Specifically, a social permission corresponding to the first object in a social session is obtained; and the emoticon package template list to be selected by the first object is displayed according to the social permission. If the social permission is a first permission, the emoticon package template list is a first template list; if the social permission is a second permission, the emoticon package template list is a second template list; the first permission is different from the second permission, and the first template list is different from the second template list. ① The social permission here can be a permission of the first object in the social session. For example, if the social session is a group chat social session, the social permission includes a management permission and a normal permission. If the first object is a management object in the group chat social session, the first object has the management permission. If the first object is a member object in the group chat social session, the first object has the normal permission. The management permission is higher than the normal permission. ② The social permission can also be a permission of the first object in the social client. For example, the first object can be an SVIP user, a VIP user, and a normal user in the social client. The social permission of the SVIP user > the social permission of the VIP user > the social permission of the normal user.

[0194] Specifically, if the first permission is higher than the second permission, the number of the sticker templates included in the first template list is greater than the number of the sticker templates included in the second template list, and the types of the sticker templates included in the first template list are different from the types of the sticker templates included in the second template list. For example, the first object with higher social permission can have the selection permission of the sticker templates of specified types, while the second object with lower social permission is allowed to select some general sticker templates. Please refer to Figure 5b , Figure 5b FIG. 1 is a schematic diagram of an interface for displaying a sticker template list according to an embodiment of the present application. As shown in FIG. 1, Figure 5b Figure 5b FIG. 1(a) is a schematic diagram of an interface of a first template list corresponding to a display under a first permission. The first template list displays three single-person sticker templates (e.g., template 1, template 2, and template 3) and two double-person sticker templates (e.g., template 1 and template 2); Figure 5b FIG. 1(b) is a schematic diagram of an interface of a second template list corresponding to a display under a second permission. Assuming that the first permission is higher than the second permission, the second template list displays two single-person sticker templates (e.g., template 1 and template 2) and one double-person sticker template (e.g., template 3). As can be seen, the number and types of the sticker templates allowed to be selected by users with different social permissions are different. For example, the number of the sticker templates allowed to be selected by a user with lower social permission is smaller, and the types of the sticker templates allowed to be selected by a user with lower social permission are different from the types of the sticker templates allowed to be selected by a user with higher social permission.

[0195] It should be noted that in the embodiments of the present application, the first image can be acquired first and then the sticker template image is acquired, or the sticker template image can be acquired first and then the first image is acquired. The present application does not make a specific limitation on the execution order between step S201 and step S202.

[0196] S203: generating a first sticker image according to the first image and the sticker template image; the first sticker image contains facial features reflected by facial information of the first object and facial information of a target reference object, the target reference object being any one of the first reference object and the second reference object.

[0197] ​It should be noted that, ① the first expression image can be consistent with all style features of the expression template image, that is, the first expression image has all the style features possessed by the expression template image, and the style features here refer to the features reflected by the expression template image, for example, the expression template image is an image expressing the daily style features of a couple through the first reference image and the second reference image, and the first expression image is an expression image expressing the daily style features of a couple through the first object and the target reference object, that is, the first expression image is consistent with the style features of the expression template image, and the style features here include image style and image parameters (such as undercoat, frame, color, etc.), that is, the first expression image is an expression image generated by performing face replacement on any reference object in the expression template image. ② The first expression image can be consistent with part of the style features of the expression template image, that is, the first expression image has part of the style features possessed by the expression template image, for example, the expression template image is an image expressing the first style features (such as daily style features of a couple) through the first reference image and the second reference image, and the first expression image generated by the present application can be an expression image expressing the second style features (such as funny style) through the first object and the target reference object, and the first expression image maintains the same undercoat and frame and other image parameters as the first expression image, that is, the first expression image maintains part of the consistent style features with the expression template image, that is, the image style of the first expression image is inconsistent with the image style of the expression template image, but the image parameters of the first expression image are consistent with the image parameters of the expression template image.

[0198] Among them, the first image and the expression template image can be single-person images or multi-person images (such as double-person images). ① If the first image and the expression template image are both single-person images, the first image contains face information of the first object, and the expression template image contains face information of the first reference object. ② If the first image and the expression template image are both multi-person images (such as double-person images), the first image contains face information of the first object and face information of the second object, and the expression template image contains face information of the first reference object and face information of the second reference object. ③ The first image is a single-person image, and the expression template image is a double-person image; or, the first image is a double-person image, and the expression template image is a single-person image. The number of objects contained in the first image and the expression template image is not limited in the embodiments of the present application.

[0199] In a possible implementation, generating the first expression image according to the first image and the expression template image can include the following two cases:

[0200] (1) Single-person face replacement of single-person expression template image.

[0201] Specifically, if the facial information of the first reference object is contained in the expression template image, the expression face changing is performed on the first object in the first image and the first reference object in the expression template image to generate a first expression image. Please refer to Figure 6a , Figure 6a is a flowchart of a single-person expression face changing process provided by an embodiment of the present application. As shown in Figure 6a , the facial information of the first object is contained in the first image, and the facial information of the first reference object is contained in the expression template image, then the expression face changing is directly performed on the first object and the first reference object to generate a first expression image. The first expression image has both the style characteristics (for example, the style of poking with fingers) of the expression template image and the facial characteristics of the first object.

[0202] (2) Single-person face changing of a double-person expression template image.

[0203] Specifically, if the facial information of the first reference object and the facial information of the second reference object are contained in the expression template image, the expression face changing is performed on the first object in the first image and the target reference object in the expression template image to generate a first expression image; the first expression image includes the facial characteristics of the first object and the facial characteristics of the target reference object; the target reference object refers to a reference object that is not subjected to the expression face changing, and thus the target reference object can include any one of the following: the object that is different from the first object in attributes (for example, gender, age, type, and the like) among the first reference object and the second reference object, or any one of the first reference object and the second reference object, or the reference object that is not selected by the user (that is, the present application supports the user to select the reference object to be subjected to the expression face changing by himself / herself; if the user selects the first reference object to perform the expression face changing with the first object, the target reference object is the second reference object; otherwise, if the user selects the second reference object to perform the expression face changing with the first object, the target reference object is the first reference object). Please refer to Figure 6b , Figure 6b is a flowchart of a double-person expression face changing process provided by an embodiment of the present application. As shown in Figure 6b , the facial information of the first object is contained in the first image, but the facial information of the first reference object and the facial information of the second reference object are contained in the expression template image, then the expression face changing is performed on the first reference object (that is, the reference object with the same gender as the first object) in the expression template image and the first object to generate a first expression image. In the first expression image, only one object (that is, the first reference object) is subjected to the expression face changing, and the other object (the second reference object) retains the original facial characteristics in the expression template image; since the first expression image has both the style characteristics of the expression template image and the facial characteristics of the first object, the first expression image has the same style characteristics as the expression template image and the same facial characteristics as the first object. Figure 6bThe first expression image shown can be used to reflect the expression style of the first object poking the second reference object with a finger.

[0204] (3) Double-person face swapping of double-person expression template image.

[0205] Optionally, the embodiments of the present application can also obtain a second image uploaded by the second object, and generate a first expression image based on the first image, the second image, and the expression template image. Specifically, if the expression template image contains face information of the first reference object and face information of the second reference object, and the first image contains face information of the first object, and the second image contains face information of the second object. Then, performing expression face swapping processing on the first object in the first image and the first reference object in the expression template image to obtain a first expression image, and performing expression face swapping processing on the second object in the second image and the second reference object in the first expression image, that is, a second expression image can be generated. Wherein, the gender of the first object is the same as that of the first reference object, and the gender of the second object is the same as that of the second reference object, and finally the generated second expression image includes face features of the first object and face features of the second object. Please refer to Figure 6c , Figure 6c is another flowchart of double-person expression face swapping processing provided by the embodiments of the present application. As shown in Figure 6c , first, the first reference object in the expression template image can be subjected to expression face swapping processing based on the first image; then the second reference object in the expression template image can be subjected to expression face swapping processing based on the second image, so that the generated second expression image has both the style features of the expression template image and the face features of the first object and the face features of the second object.

[0206] The above takes the single-person expression template image and the double-person expression template image as examples for illustration, it should be understood that the number of reference objects included in the expression template image can also be three or more, and the present application does not specifically limit the number of reference objects in the expression template image; and the objects in the expression template image can include: persons, animals, etc., and the present application does not specifically limit this. If the expression template image includes: a first reference object (person), a second reference object (person), and a third reference object (animal), then in the expression face swapping, the system can automatically perform expression face swapping with the reference object of the same type as the first object, for example, the first object can perform face swapping with the first reference object or the second reference object; if the first object self-defines to select to perform expression face swapping with the third reference object, then the first object and the third reference object can be subjected to expression face swapping, thereby improving the interest of the face swapping process.

[0207] In a possible implementation, the embodiment of the present application supports one key to generate a plurality of expression images (i.e., a group of expression packages). Specifically, when the expression template image is a target expression package template in the expression package template list, the target expression package template includes K template images, and K is a positive integer. Then, K expression images can be generated according to the first image and the K template images; wherein any expression image is generated based on the first image and any template image in the K template images; and any expression image has the style characteristics of the corresponding template image and the facial characteristics reflected by the facial information of the first object. It should be understood that the generation manner of any expression image can refer to the generation process of the first expression image in Figure 6a or Figure 6b , which will not be described here again. Please refer to Figure 7a , Figure 7a is a flowchart of one key to generate a plurality of expression images provided by the embodiment of the present application. As shown in Figure 7a , after the first object uploads the first image, one template expression template can be randomly selected in the expression package template list. Assuming that the selected target expression package template includes four template images, then four expression images after expression face changing can be generated based on the first image by one key. Since only the first object has uploaded the photo at present, part of the multi-person expression images in the generated expression images display that the synthesis is not completed; further, the generated first expression image can be saved to the expression panel in the social interface for the user to use in the social conversation process.

[0208] In addition, when the expression template image is updated, the embodiment of the present application supports one key to update each expression image. Please refer to Figure 7b , Figure 7b is a flowchart of one key to update a plurality of expression images provided by the embodiment of the present application. As shown in Figure 7b , a group of expression images under a plurality of expression package templates are displayed in the expression panel; when it is detected that there is a new expression package template (for example, expression package template 7111) in the expression panel, since the expression package template 7111 is a new expression package template, the update control 7112 is arranged at the expression package template 7111, and the user can click the to-be-updated expression package template 7112, so as to one key to update a plurality of expression images 7113 under the expression package template based on the first image of the first object and the second image of the second object, that is, each expression image 7113 is generated based on the first image and the second image and the template image under the corresponding expression package template 7111 after expression face changing. This manner can one key to update a plurality of expression images under the expression package template, which is more convenient, thereby improving the efficiency of generating the expression template image.

[0209] Further, if the expression template image is a multi-person expression template image, other objects can be invited to cooperate in the expression face changing process. The process of multi-person cooperation face changing is exemplified below.

[0210] In a possible implementation, if the expression template image contains face information of the first reference object and face information of the second reference object, and the first expression image includes face features of the first object and face features of the second reference object, the computer device can further perform the following operation: in response to the selection operation on the first expression image, display a face changing invitation entry; when it is detected that the face changing invitation entry is triggered, generate a face changing invitation message; and send the face changing invitation message to the second object; wherein the face changing invitation message is used to trigger obtaining a second image containing face information of the second object, and generating a second expression image based on the second image and the first expression image; the second expression image includes face features of the first object and face features of the second object.

[0211] Please refer to Figure 7c , Figure 7c is a flowchart of inviting a social object to cooperate in face changing provided by an embodiment of the present application. As shown in Figure 7c , the user can save each expression image after expression face changing (such as each expression image displayed in interface S701) to the expression panel 702 for the user to use. During a social conversation, when the user clicks on an un-synthesized multi-person face changing expression image, a prompt window 703 can be displayed, which is used to prompt that the currently selected expression image is an un-synthesized multi-person expression image, and the prompt window is provided with a face changing invitation entry 7031, which is used to trigger inviting other social objects to cooperate in expression face changing. Further, when the user clicks on the face changing invitation entry 7031, it can be considered that a face changing invitation message is generated, and the face changing invitation message can be in any format of message in a card, or a link, or a website, or an ark message, which is not limited here.

[0212] ①The second object for receiving the face changing invitation message refers to any social object in a single chat social conversation with the first object. Please refer to Figure 7d , Figure 7d is an interface diagram of inviting a second object to cooperate in face changing provided by an embodiment of the present application, as Figure 7dAs shown, if the current social conversation process is a one-on-one social conversation process between a first object (e.g., user A) and a second object (e.g., user B), user A can click the face swap invitation entry for the non-synthesized multi-person expression image (e.g., the first expression image), and a face swap invitation message 7041 can be generated and sent to user B, e.g., displayed in the conversation window of the first object (user A) and the second object (user B).

[0213] ② The second object for receiving the face swap invitation message refers to a reference social object in the group chat social conversation to which the first object belongs. The reference social object includes any one of the following: any social object in the group chat social conversation, a social object with the highest interaction frequency with the first object in the group chat social conversation, a social object with the most recent interaction operation time with the first object in the group chat social conversation, and a social object that publishes the latest conversation message in the group chat social conversation.

[0214] Please refer to Figure 7e , Figure 7e is another interface schematic diagram provided by the embodiment of the present application for inviting a second object to cooperate in face swapping, as shown in Figure 7e As shown, if the current social conversation process is a group chat social conversation process of a group chat A to which the first object belongs, user A can click the face swap invitation entry for the non-synthesized multi-person expression image (e.g., the first expression image), and an object selection list can be displayed, which displays the identifiers of at least one social object (e.g., users 1-6) in the group chat A. The display of the identifiers of each social object in the object selection list can be random or arranged in a preset order. The preset order can be based on the interaction frequency between each social object and the first object from high to low. Alternatively, the preset order can be based on the activity level of each social object in the group chat social conversation from high to low. The embodiment of the present application does not limit the arrangement order of each social object in the object selection list. The first object is allowed to randomly select a social object in the object selection list to initiate a cooperation invitation to generate a face swap invitation message, which will be sent to the selected social object (e.g., user 1). Subsequently, user 1 can upload a second image based on the face swap invitation message.

[0215] In addition, in the process of generating the expression image by personalized face replacement in the embodiment of the present application, each time the expression face replacement operation is performed (i.e., each time the expression image is generated), one image generation number possessed by the user is consumed. Here, the image generation number refers to the number of times that the user can generate the expression image for free. Different users can have different image generation numbers. Optionally, the image generation number corresponding to the first object can be determined according to the social permission of the first object. For example, the higher the social permission of the object, the higher the image generation number corresponding to the object. For example, if the first object is a common user, the image generation number of the expression image that the first object can generate for free is a first preset number (e.g., 6 times). For another example, if the first object is a VIP user, the image generation number of the expression image that the first object can generate for free is a second preset number (e.g., 100 times). For yet another example, if the first object is an SVIP user, the image generation number of the expression image that the first object can generate for free is unlimited, i.e., the image generation number of the first object is not limited.

[0216] Please refer to Figure 7f , Figure 7f is another flowchart for acquiring the first image provided by the embodiment of the present application. As shown in Figure 7f , if the first object is in the process of performing expression face replacement, the image generation number of the expression image that the first object can generate for free is less than 1, i.e., the first object does not have a free generation number to use, then when the first object clicks the image upload portal, a prompt pop-up window 7010 can be displayed. The prompt pop-up window 7010 displays a prompt message: Your free generation number has been used up. Please confirm whether to recharge! That is, the prompt pop-up window 7010 is used to prompt that the image generation number needs to be recharged to generate the expression image. The prompt pop-up window 7010 is provided with a recharge portal 7011. The first object can click the recharge portal 7011 to trigger the display of a recharge page. The recharge page displays different recharge packages. Different recharge packages correspond to different image generation numbers. For example, package 1 corresponds to 10 recharge image generation numbers, package 2 corresponds to 20 recharge image generation numbers, and so on. The first object can select the package to be recharged in the recharge page as needed to perform the recharge processing, so that after the recharge is successful, the first object can continue to perform the expression face replacement operation, i.e., the first object after the recharge is successful can upload the first image through the image upload portal and perform the expression face replacement processing with the selected expression template image to generate the first expression image. In this implementation manner, the generation of the expression image is controlled by setting different image generation numbers for social objects with different social permissions. In this way, the cost in the expression face replacement process can be saved, so that more rich social interaction modes can be introduced in the social conversation process.

[0217] S204: Display the first expression image in the social interface.

[0218] Specifically, the display manner of the first expression image includes any one of the following:

[0219] (1) Display the first expression image in an expression panel of a social interface.

[0220] (2) Display the first expression image in a single chat session interface between the first object and a second object.

[0221] (3) Display the first expression image in any group chat social session interface to which the first object belongs.

[0222] (4) Display the first expression image in a visual identifier of the first object; the visual identifier includes any one of the following: a head portrait, a personal homepage, a background image.

[0223] Please refer to Figure 8 , Figure 8 is an interface schematic diagram provided by an embodiment of the present application for displaying a first expression image. As shown in FIG. (a) of Figure 8 , the generated first expression image can be saved to a surface panel 801 for use by a user in a social session process, thereby improving the interest of social interaction; as shown in FIG. (b) of Figure 8 , the generated first expression image can also serve as a head portrait of the first object, so that the head portrait 802 of the first object is displayed according to the first expression image in a social session process between the first object and a second object; as shown in FIG. (c) of Figure 8 , the generated first expression image can also serve as a picture 803 of a background image of a personal homepage of the first object. As can be seen, the first expression image can be displayed at multiple different positions in a social interface, thereby enriching the application scenarios of the first expression image.

[0224] Further, if the second object updates the first expression image, the first object can update the first expression image. Specifically, a prompt message is displayed in a social interface, the prompt message being used to prompt that a second image containing face information of the second object has been acquired; in response to an update operation triggered for the first expression image displayed in the social interface, the first expression image is updated and displayed as a second expression image in the social interface. For example, the user can directly click the first expression image to be updated, where the click operation can be: single click, double click, or long press, etc., thereby generating an update operation for the first expression image; for another example, an update control is provided in the social interface, and if the user clicks the update control, each expression image to be updated can be displayed, and the user can select the first expression image as the expression image to be updated from the expression images to be updated. The second expression image is an image generated by updating the first expression image based on the second image; the second expression image includes face features of the first object and face features of the second object.

[0225] Please refer to Figure 9a , Figure 9a is a flowchart of updating an expression image provided by an embodiment of the present application. As shown in Figure 9a , a first object can display an expression panel in a process of a social conversation with a second object (such as user A), and the expression panel displays a first expression image 901. Since the first expression image is generated based on an expression face replacement of a first image of the first object, and the expression template image includes face information of a first reference object and face information of a second reference object, when the first object clicks the first expression image, a prompt message can be displayed in the expression panel, and the prompt message can be: the other party (such as the second object) has updated the photo, click to update the expression. The first object can click the prompt message to trigger the display of an update control 902, and then the first expression image can be updated and displayed as a second expression image 903 in the social interface, where the second expression image is generated based on an update processing of a second image uploaded by the second object and the first expression image, and the second expression image includes face features of the first object and face features of the second object. Based on this, when the friend (i.e. the second object) updates the photo, the first object can update and display the first expression image as the second expression image in the expression panel.

[0226] Optionally, if the expression panel of the social interface displays at least one system expression image. Then the first expression image and each system expression image can be distinguished and displayed in the expression panel of the social interface; where the distinguishing display mode includes any one of the following: animation, highlighting, and prompt. Please refer to Figure 9b , Figure 9b is an interface diagram for distinguishing and displaying a first expression image provided by an embodiment of the present application. As shown in Figure 9b , the expression panel of the social interface displays a plurality of expression images (including system expression images 9013 and 9014, a first expression image 9012, and other expression images after face replacement 9011), and the first expression image 9012 can be distinguished and displayed from the system expression images 9013 and 9014, where the distinguishing display includes highlighting the first expression image or flashing the first expression image. Further, if a user clicks the first expression image 9012, the first expression image can also be animated and displayed in the social interface, thereby improving the interest of displaying the first expression image and facilitating the experience of social interaction.

[0227] It should be understood that the first expression image generated by the embodiments of the present application can be applied to social interaction between various social objects in a social conversation. The social conversation can include a single chat social conversation and a group chat social conversation. For details of the specific process of performing expression face replacement in a single chat social conversation, please refer to the above content. The following takes a group chat social conversation as an example to illustrate the expression face replacement process.

[0228] (1) The system automatically selects a reference social object for the first object to cooperate in face swapping.

[0229] In a possible implementation, the social conversation process refers to a process of a group chat social conversation in which the first object and N social objects are included, N being a positive integer; and the expression template image includes face information of a first reference object and face information of a second reference object. Then, a reference expression image can be displayed in an expression panel of the group chat social conversation, the reference expression image being generated according to the first image, the expression template image, and a reference image containing face information of a reference social object; wherein the reference social object includes any one of the following: a social object that has the highest interaction frequency with the first object among the N social objects, a social object that has the most recent interaction operation time with the first object among the N social objects, a social object that publishes the latest conversation message among the N social objects, and a social object that has the highest activity level among the N social objects (for example, publishes the most conversation messages in the group chat social conversation, or has the longest active time in the group chat social conversation). It should be understood that the reference social object selected by default is a social object that has provided its own photo for performing expression face swapping, and the reference expression image synthesized by the first object and the reference social object can be used in the group chat social conversation for interaction by each social object.

[0230] Optionally, in response to a selection operation on the reference expression image, the reference expression image is displayed in a conversation window of the group chat social conversation; and an interaction message for the reference social object is displayed in the conversation window of the group chat social conversation; wherein the display position of the interaction message includes any one of the following: an arbitrary position in the conversation window, a specified position in the conversation window, and an associated position of an avatar of the reference social object. Please refer to Figure 10a , Figure 10a is an application interface schematic diagram of the reference expression image provided by the embodiments of the present application. As Figure 10aAs shown, the first object (e.g., user A) can select a reference expression image in the expression panel and send it to the group chat social conversation, and then the reference expression image sent by user A can be displayed in the group chat social conversation. Further, since the reference expression image is a face-swapped expression image synthesized based on the first object (user A) and the second object (e.g., user B), a prompt message for user B can be displayed in the group chat social conversation. For example, the prompt message displayed for user B in the conversation window of the group chat social conversation can be: you have received an expression image synthesized with user A. It should be noted that the format and display position of the prompt message in the embodiments of the present application are not limited, and the prompt message can be displayed as a new conversation message in the conversation window of the group chat social conversation, or directly displayed as a separate prompt message in the conversation window. Moreover, the display position of the prompt message can be at any position in the conversation window, such as the associated position below the reference expression image, or a floating window in the conversation window, or the top or bottom of the conversation window, etc.

[0231] (2) The group chat social conversation supports the first object to define and select a social object for cooperative face swapping.

[0232] In a possible implementation, the group chat social conversation supports the first object to define and select a social object for cooperative face swapping. Specifically, in response to the expression update operation detected in the expression panel of the group chat social conversation, an object selection prompt box is output, wherein the object selection prompt box displays the object identifiers of N social objects to be selected in the group chat social conversation; in response to the selection operation on the object identifier of the target social object in the N social objects, the reference expression image is updated and displayed as a third expression image in the sticker panel; wherein the third expression image has the style characteristics of the expression template image, and contains the facial features reflected by the facial information of the first object and the facial features reflected by the facial information of the target social object.

[0233] Please refer to Figure 10b , Figure 10b is a flowchart of generating a third expression image provided by the embodiments of the present application. As Figure 10bAs shown, a reference expression image 1001 is displayed in the expression panel of the group chat social conversation, the reference expression image being an expression image associated with the first object and a reference social object; optionally, if the user clicks (such as double-clicks or long-presses) the reference expression image 1001, an object selection prompt box can be output, which displays the object identifiers (such as avatars, IDs, etc.) of N social objects in the current group chat social conversation, wherein the N social objects have all uploaded their respective photos for expression face changing; further, the user can customize to select a social object in the object selection prompt box to synthesize a new expression image, for example, the selected social object is user 3 (referred to as a third object). Then a third image containing the face information of the third object can be obtained, and the reference expression image is updated based on the third image. Specifically, the reference social object in the reference expression image is expression face changed with the third object, so that the updated third expression image is an expression image associated with the first object and the third object. Further, the reference expression image 1001 can be updated and displayed as the third expression image 1002 in the expression panel.

[0234] In summary, after receiving the expression face changing operation of the first object in the social conversation process, the embodiments of the present application can obtain the real photo (i.e. the first image) uploaded by the first object itself, and obtain the expression template image selected by the first object. Then, the first expression image can be generated according to the first image and the expression template image, the first expression image containing the face features reflected by the face information of the first object, and the face information of the target reference object which is not face changed. Subsequently, the first expression image can be displayed in the social interface. As can be seen, the present application can perform expression face changing on the face of the first object in the first image and the face of any reference object (such as the first reference object) in the expression template image, so that the first expression image generated after face changing not only has the style features of the expression template image, but also reflects the face features of the first object, i.e. the first expression image is an image personalized by the face features of the first object. Then, different expression images can be generated for the same expression template image by using the face features of different objects, so as to meet the personalized needs of users for expression images, and make the expression images generated by the present application more personalized and interesting.

[0235] The above Figure 2 The embodiments shown mainly introduce the interface implementation process of the image processing method provided by the embodiments of the present application from the product interface perspective. The background implementation manner of the image processing method provided by the embodiments of the present application will be introduced from the background technology perspective in combination with the drawings.

[0236] Please refer to Figure 11 , Figure 11is a flowchart of another image processing method provided by an embodiment of the present application. The image processing method can be executed by a computer device, which can be the server shown in FIG. 1. Specifically, the image processing method can include, but is not limited to, steps S1101-S1104: Figure 1a

[0237] S1101: receiving an image processing request sent by a social client.

[0238] The image processing request is generated after detecting an expression face changing operation in a social session of the social client. The image processing request includes a first image and an expression template image. The first image contains face information of a first object, and the expression template image contains face information of a first reference object and face information of a second reference object. It should be noted that the specific process of how to detect the expression face changing operation in the social session and how to obtain the first image and the expression template image can be referred to the detailed description of the related steps in the embodiments of the present application, which will not be repeated here. Figure 2

[0239] Optionally, the expression template image can contain face information of both the first reference object and the second reference object, that is, the expression template image is a template image containing expressions of multiple people. At this time, the first image can be a single-person image containing face information of the first object. The first image can also be a double-person image containing face information of both the first object and the second object. The number of objects contained in the first image and the expression template image is not limited in the embodiments of the present application.

[0240] ​​In a possible implementation, after receiving the image processing request sent by the social client, the server can acquire identity information of a social object (such as the first object) that sends the image processing request, and perform verification processing on the first object based on the identity information (such as the identifier) of the first object; if the verification processing on the first object is passed, the server triggers to perform the subsequent step S1102, otherwise, the server deletes the image processing request. The verification here includes any one or more of identity verification, security verification, and permission verification. For example, 1) the server pre-stores a permission list, and the permission list here stores identifiers of each social object that has the expression face changing permission, if the identifier of the first object exists in the permission list, it is determined that the permission verification on the first object is passed; 2) the server can acquire login data of the first object logging into the social client, for example, time, frequency, IP and other information, and then perform security verification and legality verification on the login operation of the first object based on the login data of the first object, if the login data shows that the login operation of the first object is normal, the security verification on the first object is passed. In this implementation, the identity of the social object (such as the first object) that sends the image processing request can be verified, the data security in the social conversation process is improved, and the reliability and security of the image processing process are ensured.

[0241] S1102: generating a first expression image according to the first image and the expression template image.

[0242] The first expression image has the style feature of the expression template image and contains the facial feature reflected by the facial information of the first object. It should be understood that 1) the style feature of the expression template image can include any type of style feature such as couple daily style, office worker work style, student school style, etc. The style feature here can be used to reflect the facial expression (such as happy, joyful, crying, sad, etc.), body action (such as standing, walking, lying, running, etc.), environment (such as indoor, outdoor) and other features of the object in the expression template image. 2) The facial feature reflected by the facial information of the first object can include features such as eyelid type (such as single eyelid or double eyelid), eyebrow type (such as crescent eyebrow, horizontal eyebrow, thick eyebrow, or thin eyebrow), high nose bridge or flat nose bridge, lip type (thin lips or thick lips), whether wearing glasses, and hair features (such as hair color, hairstyle, hair length), etc.

[0243] The specific process of how to generate the first expression image is described in detail below.

[0244] In a possible implementation, the server generates the first expression image according to the first image and the expression template image, including the following steps (1)-(2):

[0245] (1) performing region segmentation processing on the first image to obtain i first regions, and performing feature extraction processing on each first region to obtain i first region feature vectors corresponding to the i first regions, i being a positive integer; and performing region segmentation processing on the expression template image to obtain j second regions, and performing feature extraction processing on each second region to obtain j second region feature vectors corresponding to the j second regions, j being a positive integer. The number of regions i included in the first image and the number of regions j included in the expression template image can be the same or different.

[0246] (2) performing expression face changing processing based on the i first region feature vectors and the j second region feature vectors to generate a first expression image.

[0247] In a specific implementation, the i first region feature vectors include facial features reflected by facial information of the first object, and the j second region feature vectors include facial features reflected by facial information of the first reference object and style features of the expression template image. Then, the server performs expression face changing processing based on the i first region feature vectors and the j second region feature vectors to generate the first expression image, including: performing encoding processing on the i first region feature vectors and the j second region feature vectors by using an attention mechanism to obtain encoded feature vectors; performing decoding processing on the encoded feature vectors, and performing synthesis processing on the decoded feature vectors to generate the first expression image; and the synthesis processing is used to indicate that the i first region feature vectors and the j second region feature vectors are fused, and the facial features of the first object and the facial features of the first reference object are replaced.

[0248] Please refer to Figure 12 , Figure 12 is a flowchart of generating a first expression image provided by an embodiment of the present application. As shown in Figure 12 , it is assumed that the first image contains facial information of a first object and facial information of a second object, and the expression template image contains facial information of a first reference object and facial information of a second reference object; then, the first expression image generated based on the first image and the expression template image is a double expression image. Specifically, the specific process of generating the first expression image is as follows:

[0249] (1) FaceID extraction: extracting an object identifier FaceID from an input image, the FaceID being a unique identifier for identifying facial features of an object, used to distinguish different objects, that is, an object has a unique FaceID. Specifically, the FaceID1 of the first object and the FaceID2 of the second object can be extracted from the first image.

[0250] (2) Region segmentation: the face of the first object and the face of the second object are respectively segmented into different regions (region 1, region 2, etc.), for example, the regions can be divided according to the facial features of the object, for example, the first object (or the second object) is divided into: eye region, nose region, mouth region, etc. It should be noted that the region division method of the first object and the second object can be the same or different. In addition, in the region division process, detailed analysis and operation of specific facial regions are allowed, and the attention weight of different regions is combined to control the generation of personalized expression images. For example, a higher weight ratio can be set for regions such as the eye region and the mouth region that directly reflect facial features, and a relatively lower weight ratio can be set for regions such as the nose region.

[0251] (3) Region feature vector generation: for each segmented region, the corresponding region feature vector is extracted. The region feature vector is used to reflect the basic features corresponding to the region, which can include: color, texture, shape and other related attribute features. Optionally, a feature extraction model can be used to extract the region feature vector of each region. The feature extraction model can include but is not limited to any one of the following models: CNN (Convolutional Neural Networks, Convolutional Neural Networks) model, VGG (Visual Geometry Group, Deep Convolutional Neural Network) model, and other neural network models with image feature extraction capability. The model structure and type of the feature extraction model are not limited in the present application.

[0252] (4) Attention mechanism: a set of encoders (V, K, Q) is used to encode the extracted region feature vectors, and an attention mechanism is used for the encoded region feature vectors, so that the model can fuse the corresponding FaceID and the face features of the corresponding object in different regions. In the process of using the attention mechanism, the personalized face features of the corresponding characters (such as the first object and the second object) need to be fused on the feature map corresponding to the expression template image, so as to generate an expression image with personalized face features. This step is helpful to accurately create a personalized expression image.

[0253] (5) Decoding and synthesis: the region feature vectors processed by the attention mechanism can be decoded and synthesized by a diffusion model to form an expression image (such as the first expression image) containing new face features. This makes the synthesized first expression image not only have the style features reflected by the expression template image (such as the couple daily style), but also maintain the unique face features of the first object and the second object.

[0254] (6) output generation: output the first expression image synthesized by the diffusion model, which has the style characteristics (such as the couple daily style) of the expression template image and contains the facial features of the first object and the facial features of the second object. Subsequently, the generated first expression image can be used for social interaction of each social object in the process of social conversation.

[0255] In the generation process of the first expression image shown in steps (1)-(6) above, on the one hand, the semantic face information is captured by the FaceID embedding method to ensure high identity fidelity; on the other hand, the region feature vectors of each image region are obtained after the image is regionally segmented, and then the attention mechanism is used to process the image according to different weight ratios for different face regions, and through the lightweight adaptive module of decoupled cross attention, the face image can be used as a visual prompt, which can improve the accuracy of the image processing process; on the other hand, a variety of plug-ins (such as ControlNet plug-in) can be used to replace the function of the diffusion model to more accurately control the image generation details.

[0256] S1103: return the first expression image to the social client, so that the social client displays the first expression image in the social interface.

[0257] The server can return the generated first expression image to the social client, and the social client can display the first expression image in the social interface. For example, the first expression image is displayed in the expression panel of the social interface; or the first expression image is displayed in the single chat conversation interface of the first object and the second object; or the first expression image is displayed in any group chat social conversation interface to which the first object belongs; or the first expression image is displayed in the image identification of the first object; the image identification includes any one of the following: avatar, personal homepage, background image. It should be noted that the display mode of the first expression image in the social interface can be referred to Figure 2 The detailed steps in the embodiments are not described here.

[0258] In summary, the embodiments of the present application provide a mechanism for generating a first expression image based on an expression template image and a first image containing real face information of a first object. Specifically, the first object in the first image and the first reference object or the second reference object in the expression template image can be processed for expression face replacement, while the original style characteristics of the expression template image are retained, that is, only the face is replaced in the expression face replacement process, so that the generated first expression image has both the style characteristics of the expression template image and the facial features of the first object. This can generate a first expression image that is personalized for the first object, thereby improving the interest of the expression image in the process of social conversation and meeting the personalized needs of users.

[0259] The following describes the related device of the image processing scheme provided by the embodiments of the present application. In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the functions of the module or unit.

[0260] Please refer to Figure 13 , Figure 13 is a structural schematic diagram of an image processing device provided by the embodiments of the present application. As Figure 13 indicated, the image processing device 1300 can be applied to the computer device (for example, a terminal device) mentioned in the foregoing embodiments. Specifically, the image processing device 1300 can be a computer program (including program code) running in a blockchain node, for example, the image processing device 1300 is an application software; the image processing device 1300 can be used to execute the corresponding steps in the image processing method provided by the embodiments of the present application. Specifically, the image processing device 1300 can specifically include:

[0261] The acquisition unit 1301 is configured to acquire a first image containing face information of a first object in response to an expression face changing operation received in a social conversation process;

[0262] The acquisition unit 1301 is further configured to acquire an expression template image containing face information of a first reference object and face information of a second reference object;

[0263] The processing unit 1302 is configured to generate a first expression image containing face features reflected by the face information of the first object and containing face information of a target reference object according to the first image and the expression template image, the target reference object being any one of the first reference object and the second reference object;

[0264] The display unit 1303 is configured to display the first expression image in a social interface.

[0265] In a possible implementation, the acquisition unit 1301 acquires the first image, which is used to perform the following operations:

[0266] Display an image upload portal, the image upload portal being used to trigger acquisition of a first image to be processed;

[0267] When it is detected that the image uploading entrance is selected, any image containing face information of the first object in the image library is acquired as the first image; or, a photographing device is called to collect face information of the first object to generate the first image.

[0268] In a possible implementation, the obtaining unit 1301 obtains an expression template image, for performing the following operations:

[0269] displaying an expression package template list, the expression package template list displaying at least one expression package template, any expression package template including one or more template images;

[0270] in response to a selection operation performed in the expression package template list, obtaining a selected expression template image;

[0271] The expression template image is any expression package template in the expression package template list, or the expression template image is any template image under any expression package template in the expression package template list.

[0272] In a possible implementation, the processing unit 1302 is further configured to perform the following operations:

[0273] obtaining a social permission corresponding to the first object in the social conversation process;

[0274] displaying an expression package template list to be selected by the first object according to the social permission;

[0275] If the social permission is a first permission, the expression package template list is a first template list; if the social permission is a second permission, the expression package template list is a second template list; the first permission is different from the second permission, and the first template list is different from the second template list.

[0276] In a possible implementation, the expression template image is a target expression package template in the expression package template list, the target expression package template including K template images, K being a positive integer; the processing unit 1302 is further configured to perform the following operations:

[0277] generating K expression images according to the first image and the K template images;

[0278] Any expression image is generated based on the first image and any template image of the K template images; and any expression image has a style feature of the corresponding template image and a face feature reflected by the face information of the first object.

[0279] In a possible implementation, the processing unit 1302 generates a first expression image according to the first image and the expression template image, for performing the following operations:

[0280] performing expression face changing processing on the first object in the first image and a first reference object in the expression template image to generate a first expression image, the first expression image including facial features of the first object and facial features of the first reference object; or

[0281] performing expression face changing processing on the first object in the first image and a second reference object in the expression template image to generate a first expression image, the first expression image including facial features of the first object and facial features of the second reference object.

[0282] In an implementation, the reference object on which the expression face changing processing is performed is an object with the same attribute as the first object.

[0283] In an implementation, the first image includes facial information of the first object and facial information of a second object; and the processing unit 1302 is further configured to perform the following operation:

[0284] performing expression face changing processing on the first object in the first image and a first reference object in the expression template image, and performing expression face changing processing on the second object in the first image and a second reference object in the expression template image to obtain a second expression image.

[0285] In an implementation, the second expression image has the style characteristics of the expression template image, and includes facial features reflected by the facial information of the first object and facial features reflected by the facial information of the second object.

[0286] In an implementation, the expression template image includes facial information of the first reference object and facial information of a second reference object, and the first expression image includes facial features of the first object and facial features of the second reference object; and the processing unit 1302 is further configured to perform the following operation:

[0287] In response to a selection operation on the first expression image, display a face changing invitation entry.

[0288] When it is detected that the face changing invitation entry is triggered, generate a face changing invitation message.

[0289] send the face changing invitation message to the second object; the face changing invitation message is used to trigger obtaining a second image including facial information of the second object, and generating a second expression image based on the second image and the first expression image; the second expression image has the style characteristics of the expression template image, and includes facial features reflected by the facial information of the first object and facial features reflected by the facial information of the second object.

[0290] In an implementation, the second object for receiving the face changing invitation message includes any one of the following:

[0291] any social object in the single-chat social session with the first object; or

[0292] a reference social object in a group-chat social session to which the first object belongs; the reference social object includes any of the following: any social object in the group-chat social session, a social object that has the highest interaction frequency with the first object in the group-chat social session, a social object that has the most recent interaction operation time with the first object in the group-chat social session, a social object that publishes the latest conversation message in the group-chat social session.

[0293] In a possible implementation, the processing unit 1302 is further configured to perform the following operation:

[0294] displaying a prompt message in the social interface, the prompt message being used to prompt that the second image containing the face information of the second object has been acquired;

[0295] updating the first expression image to a second expression image in the social interface in response to an update operation triggered for the displayed first expression image in the social interface;

[0296] The second expression image is an image generated by updating the first expression image based on the second image; the second expression image includes the face feature of the first object and the face feature of the second object.

[0297] In a possible implementation, the display unit 1303 displays the first expression image in the social interface, and is configured to perform any of the following operations:

[0298] displaying the first expression image in an expression panel of the social interface; or

[0299] displaying the first expression image in a single-chat session interface of the first object and the second object; or

[0300] displaying the first expression image in any group-chat social session interface to which the first object belongs; or

[0301] displaying the first expression image in an image identifier of the first object; the image identifier includes any of the following: a head portrait, a personal homepage, a background image.

[0302] In a possible implementation, the expression panel of the social interface displays at least one system expression image; the processing unit 1302 is further configured to perform the following operation:

[0303] distinguishing and displaying the first expression image and each system expression image in the expression panel of the social interface; the distinguishing and displaying manner includes any of the following: animation, highlighting, and prompting.

[0304] In a possible implementation, the social conversation process refers to a process of a group chat social conversation in which the first object is located, and the group chat social conversation includes the first object and N social objects, N being a positive integer; the expression template image includes face information of a first reference object and face information of a second reference object; and the processing unit 1302 is further configured to perform the following operations:

[0305] displaying a reference expression image in an expression panel of the group chat social conversation; the reference expression image is generated according to the first image, the expression template image, and a reference image containing face information of a reference social object;

[0306] The reference social object includes any one of the following: a social object that interacts with the first object most frequently among the N social objects, a social object that performs an interaction operation with the first object most recently among the N social objects, a social object that publishes a latest conversation message among the N social objects, and a social object that is most active among the N social objects.

[0307] In a possible implementation, the processing unit 1302 is further configured to perform the following operations:

[0308] in response to a selection operation on the reference expression image, displaying the reference expression image in a conversation window of the group chat social conversation; and

[0309] displaying an interaction message for the reference social object in the conversation window of the group chat social conversation;

[0310] The display position of the interaction message includes any one of the following: an arbitrary position in the conversation window, a specified position in the conversation window, and an associated position of an avatar of the reference social object.

[0311] In a possible implementation, the processing unit 1302 is further configured to perform the following operations:

[0312] in response to an expression updating operation detected in the expression panel of the group chat social conversation, outputting an object selection prompt box; and

[0313] in response to a selection operation on an object identifier of a target social object among the N social objects, updating the reference expression image to a third expression image and displaying the third expression image in the sticker panel;

[0314] The third expression image has the style characteristics of the expression template image, and contains face features reflected by the face information of the first object and face features reflected by the face information of the target social object.

[0315] In a possible implementation, the manner of generating the facial expression changing operation includes any one of the following:

[0316] The facial expression changing operation is generated when it is detected that a facial changing function entry provided in the social interface is triggered, and the facial changing function entry includes any one of the following: a control, an option, or the like.

[0317] The facial expression changing operation is generated when it is detected that a specific touch operation exists in an operation area of the social interface, and the specific touch operation includes any one of the following: a single-click operation, a double-click operation, a gesture operation, and a hovering gesture.

[0318] The facial expression changing operation is generated when it is detected that a facial changing invitation message displayed in the social interface is selected, and the facial changing invitation message is sent by a second object to a first object in a social conversation process, and a format of the facial changing invitation message includes any one of the following: a card, a link, a website, and a structured message.

[0319] In the embodiments of the present application, in response to the facial expression changing operation received in the social conversation process, a first image containing face information of the first object is obtained; an expression template image containing face information of a first reference object and face information of a second reference object is obtained; a first expression image is generated according to the first image and the expression template image, the first expression image contains face features reflected by the face information of the first object and contains face information of a target reference object, and the target reference object refers to any one of the first reference object and the second reference object; and the first expression image is displayed in the social interface. As can be seen, the present application can perform facial expression changing processing on the face of the first object in the first image and the face of any one of the reference objects (for example, the first reference object) in the expression template image, so that the first expression image generated after the facial expression changing processing can have the style features of the expression template image, can reflect the face features of the first object after the facial expression changing processing, and can have the face features of the target reference object (for example, the second reference object) that has not been changed, that is, the first expression image is an image generated by personalization of the face features of the first object. Then, different expression images can be generated by using the face features of different objects for the same expression template image, so as to meet the personalization demand of the user for the expression image, and make the expression image generated by the present application more personalized and interesting.

[0320] Please refer to Figure 14 , Figure 14 is another structure schematic diagram of an image processing device provided by the embodiments of the present application. As Figure 14As shown, the image processing apparatus 1400 can be applied to the computer device (for example, a server) mentioned in the foregoing embodiments. Specifically, the image processing apparatus 1400 can be a computer program (including program code) running in a blockchain node, for example, the image processing apparatus 1400 is an application software; the image processing apparatus 1400 can be used to execute the corresponding steps in the image processing method provided in the embodiments of the present application. Specifically, the image processing apparatus 1400 can specifically include:

[0321] The receiving unit 1401 is configured to receive an image processing request sent by a social client, the image processing request being generated after receiving an expression face changing operation in a social session of the social client, and the image processing request comprising a first image and an expression template image, the first image containing face information of a first object, and the expression template image containing face information of a first reference object and face information of a second reference object;

[0322] The processing unit 1402 is configured to generate a first expression image according to the first image and the expression template image; the first expression image has style features of the expression template image, and contains face features reflected by the face information of the first object and face information of a target reference object, the target reference object being any one of the first reference object and the second reference object.

[0323] The sending unit 1403 is configured to return the first expression image to the social client, so that the social client displays the first expression image in a social interface.

[0324] In a possible implementation, the processing unit 1402 generates the first expression image according to the first image and the expression template image, for performing the following operations:

[0325] performing region segmentation processing on the first image to obtain i first regions, and performing feature extraction processing on each first region to obtain i first region feature vectors corresponding to the i first regions, i being a positive integer; and

[0326] performing region segmentation processing on the expression template image to obtain j second regions, and performing feature extraction processing on each second region to obtain j second region feature vectors corresponding to the j second regions, j being a positive integer;

[0327] performing expression face changing processing based on the i first region feature vectors and the j second region feature vectors to generate the first expression image.

[0328] In a possible implementation, the i first region feature vectors include face features reflected by the face information of the first object, and the j second region feature vectors include face features reflected by the face information of the first reference object and style features of the expression template image.

[0329] The processing unit 1402 performs expression face replacement processing based on the i first regional feature vectors and the j second regional feature vectors, and generates a first expression image, for performing the following operations:

[0330] The i first regional feature vectors and the j second regional feature vectors are encoded by using an attention mechanism to obtain an encoded feature vector;

[0331] The encoded feature vector is decoded, and the decoded feature vector is synthesized to generate the first expression image;

[0332] The synthesis processing is used to indicate that the i first regional feature vectors and the j second regional feature vectors are fused, and the facial features of the first object are replaced with the facial features of the first reference object.

[0333] The embodiment of the present application provides a mechanism for generating a first expression image based on an expression template image and a first image containing real facial information of a first object. Specifically, the first object in the first image and the first reference object or the second reference object in the expression template image can be subjected to expression face replacement processing, while the original style features of the expression template image are retained. That is, only the face is replaced during the expression face replacement process, so that the generated first expression image has both the style features of the expression template image and the facial features of the first object. In this way, the expression image of the first object can be personalized, thereby improving the interestingness of the expression image in the social conversation process and meeting the personalized needs of users.

[0334] Please refer to Figure 15 , Figure 15 is a structural schematic diagram of a computer device provided by the embodiment of the present application. The computer device 1500 is used to execute the related steps executed by the terminal device or the server in the foregoing method embodiment. The computer device 1500 can include a stand-alone device (for example, one or more of servers, nodes, terminals, and the like) or a component (for example, a chip, a software module, or a hardware module) inside the stand-alone device. The computer device can include at least one processor 1501 and a communication interface 1502. Further, the computer device can further include at least one memory 1503 and a bus 1504. In addition, the processor 1501, the communication interface 1502, and the memory 1503 are connected through the bus 1504. Wherein:

[0335] The processor 1501 is a module for performing arithmetic operations and / or logical operations, and can be one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), a co-processor (assisting the central processor to complete the corresponding processing and application), a micro controller unit (MCU), and the like.

[0336] The communication interface 1502 can be used to provide information input or output for the at least one processor 1501. And / or, the communication interface 1502 can be used to receive externally transmitted data and / or transmit data to the outside, which can be a wired link interface including an Ethernet cable, etc., or a wireless link (Wi-Fi, Bluetooth, universal wireless transmission, vehicle-mounted short-range communication technology, and other short-range wireless communication technologies, etc.) interface. The communication interface 1502 can serve as a network interface.

[0337] The memory 1503 is used to provide a storage space, in which operating systems and computer programs and other data can be stored. The memory 1503 can be one or a combination of a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a compact disc read-only memory (CD-ROM), etc.

[0338] (1) When the computer device is a terminal device, the processor 1501 invokes program instructions stored in the memory 1503 to perform the following operations:

[0339] In response to the received facial expression changing operation in the social conversation process, a first image containing face information of a first object is obtained;

[0340] obtain an expression template image, the expression template image containing face information of the first reference object and face information of the second reference object;

[0341] generate a first expression image according to the first image and the expression template image, the first expression image containing face features reflected by the face information of the first object and containing face information of a target reference object, the target reference object being any one of the first reference object and the second reference object;

[0342] display the first expression image in the social interface.

[0343] In a possible implementation, the processor 1501 obtains the first image, for performing the following operations:

[0344] display an image upload portal, the image upload portal being used to trigger obtaining the first image to be processed;

[0345] obtain any image containing face information of the first object from an image library as the first image when it is detected that the image upload portal is selected; or, call a photographing device to collect face information of the first object to generate the first image.

[0346] In a possible implementation, the processor 1501 obtains the expression template image, for performing the following operations:

[0347] display an expression package template list, the expression package template list displaying at least one expression package template, any expression package template including one or more template images;

[0348] obtain a selected expression template image in response to a selection operation performed in the expression package template list;

[0349] The expression template image is any expression package template in the expression package template list, or the expression template image is any template image under any expression package template in the expression package template list.

[0350] In a possible implementation, the processor 1501 is further configured to perform the following operations:

[0351] obtain a social permission corresponding to the first object in a social conversation process;

[0352] display an expression package template list to be selected by the first object according to the social permission;

[0353] The expression package template list is a first template list if the social permission is a first permission, and the expression package template list is a second template list if the social permission is a second permission; the first permission is different from the second permission, and the first template list is different from the second template list.

[0354] In a possible implementation, the expression template image is a target sticker template in a sticker template list, the target sticker template includes K template images, K is a positive integer; the processor 1501 is further configured to perform the following operation:

[0355] generate K expression images according to the first image and the K template images;

[0356] wherein any one expression image is generated based on the first image and any one of the K template images; and any one expression image has a style feature of the corresponding template image and a face feature reflected by the face information of the first object.

[0357] In a possible implementation, the processor 1501 generates a first expression image according to the first image and the expression template image, and is configured to perform the following operation:

[0358] perform expression face changing processing on the first object in the first image and a first reference object in the expression template image to generate the first expression image, the first expression image including the face feature of the first object and the face feature of the first reference object; or

[0359] perform expression face changing processing on the first object in the first image and a second reference object in the expression template image to generate the first expression image, the first expression image including the face feature of the first object and the face feature of the second reference object.

[0360] wherein the reference object on which the expression face changing processing is performed is an object with the same attribute as the first object.

[0361] In a possible implementation, the first image includes face information of the first object and face information of a second object; the processing unit is further configured to perform the following operation:

[0362] perform expression face changing processing on the first object in the first image and a first reference object in the expression template image, and perform expression face changing processing on the second object in the first image and a second reference object in the expression template image to obtain a second expression image;

[0363] wherein the second expression image has a style feature of the expression template image, and includes a face feature reflected by the face information of the first object and a face feature reflected by the face information of the second object.

[0364] In a possible implementation, the expression template image includes face information of the first reference object and face information of a second reference object, and the first expression image includes the face feature of the first object and the face feature of the second reference object; the processor 1501 is further configured to perform the following operation:

[0365] In response to the selection operation on the first expression image, display a face swapping invitation entry;

[0366] When it is detected that the face swapping invitation entry is triggered, generate a face swapping invitation message;

[0367] Send the face swapping invitation message to the second object; wherein the face swapping invitation message is used to trigger obtaining a second image containing face information of the second object, and generating a second expression image based on the second image and the first expression image; the second expression image has the style features of the expression template image, and contains face features reflected by the face information of the first object and face features reflected by the face information of the second object.

[0368] In a possible implementation, the second object for receiving the face swapping invitation message includes any one of the following:

[0369] Any social object in a one-on-one social conversation with the first object; or,

[0370] A reference social object in a group chat social conversation to which the first object belongs; the reference social object includes any one of the following: any social object in the group chat social conversation, a social object with the highest interaction frequency with the first object in the group chat social conversation, a social object with the most recent interaction operation time with the first object in the group chat social conversation, and a social object publishing the latest conversation message in the group chat social conversation.

[0371] In a possible implementation, the processor 1501 is further configured to perform the following operations:

[0372] Display a prompt message in the social interface, the prompt message being used to prompt that the second image containing the face information of the second object has been obtained;

[0373] In response to an update operation triggered on the first expression image displayed in the social interface, update and display the first expression image as a second expression image in the social interface;

[0374] The second expression image is an image generated by updating the first expression image based on the second image; the second expression image includes the face features of the first object and the face features of the second object.

[0375] In a possible implementation, the processor 1501 displays the first expression image in the social interface, and is configured to perform any one of the following operations:

[0376] Display the first expression image in an expression panel of the social interface; or,

[0377] Display the first expression image in a one-on-one conversation interface of the first object and the second object; or,

[0378] display the first expression image in any group chat social conversation interface to which the first object belongs; or

[0379] display the first expression image in an image identifier of the first object; the image identifier includes any of the following: a head portrait, a personal homepage, a background image.

[0380] In a possible implementation, at least one system expression image is displayed in an expression panel of the social interface; the processor 1501 is further configured to perform the following operation:

[0381] The first expression image and each system expression image are displayed differently in the expression panel of the social interface; wherein the different display manner includes any of the following: animation, highlighting, and prompt.

[0382] In a possible implementation, the social conversation process refers to a process of a group chat social conversation in which the first object is located, the group chat social conversation includes the first object and N social objects, N is a positive integer; the expression template image includes face information of a first reference object and face information of a second reference object; the processor 1501 is further configured to perform the following operation:

[0383] display a reference expression image in an expression panel of the group chat social conversation; the reference expression image is generated according to the first image, the expression template image, and a reference image, the reference image contains face information of a reference social object;

[0384] The reference social object includes any of the following: a social object that interacts with the first object most frequently among the N social objects, a social object that performs an interaction operation with the first object most recently among the N social objects, a social object that publishes a latest conversation message among the N social objects, and a social object that is most active among the N social objects.

[0385] In a possible implementation, the processor 1501 is further configured to perform the following operation:

[0386] In response to a selection operation on the reference expression image, display the reference expression image in a conversation window of the group chat social conversation; and,

[0387] display an interaction message for the reference social object in the conversation window of the group chat social conversation;

[0388] The display position of the interaction message includes any of the following: any position in the conversation window, a specified position in the conversation window, and an associated position of an image identifier of the reference social object.

[0389] In a possible implementation, the processor 1501 is further configured to perform the following operation:

[0390] In response to the detected expression update operation in the expression panel of the group chat social session, output an object selection prompt box; wherein the object selection prompt box displays object identifiers of N social objects to be selected in the group chat social session;

[0391] In response to a selection operation on the object identifier of the target social object in the N social objects, update and display the reference expression image as a third expression image in the sticker panel;

[0392] The third expression image has the style characteristics of the expression template image, and includes facial features reflected by the facial information of the first object and facial features reflected by the facial information of the target social object.

[0393] In a possible implementation, the expression face changing operation is generated in the following ways:

[0394] When it is detected that a face changing function entry set in the social interface is triggered, the expression face changing operation is generated, and the face changing function entry includes any one of the following: a control, an option, or the like; or

[0395] When it is detected that a specific touch operation exists in the operation area of the social interface, the expression face changing operation is generated, and the specific touch operation includes any one of the following: a single-click operation, a double-click operation, a gesture operation, and a hovering gesture; or

[0396] When it is detected that a face changing invitation message displayed in the social interface is selected, the expression face changing operation is generated, and the face changing invitation message is sent by a second object to a first object during a social session; wherein the format of the face changing invitation message includes any one of the following: a card, a link, a website, and a structured message.

[0397] (2) When the computer device is a server, the processor 1501 invokes program instructions stored in the memory 1503 to perform the following operations:

[0398] Receive an image processing request sent by a social client, the image processing request being generated after receiving an expression face changing operation during a social session of the social client, and the image processing request including a first image and an expression template image, the first image including facial information of a first object, the expression template image including facial information of a first reference object and facial information of a second reference object;

[0399] Generate a first expression image according to the first image and the expression template image; the first expression image has the style characteristics of the expression template image, and includes facial features reflected by the facial information of the first object and facial information of a target reference object, the target reference object being any one of the first reference object and the second reference object;

[0400] return the first expression image to the social client to cause the social client to display the first expression image in the social interface.

[0401] In a possible implementation, the processor 1501 generates the first expression image according to the first image and the expression template image, to perform the following operations:

[0402] perform region segmentation processing on the first image to obtain i first regions, and perform feature extraction processing on each first region to obtain i first region feature vectors corresponding to the i first regions, i being a positive integer; and

[0403] perform region segmentation processing on the expression template image to obtain j second regions, and perform feature extraction processing on each second region to obtain j second region feature vectors corresponding to the j second regions, j being a positive integer;

[0404] perform expression face changing processing based on the i first region feature vectors and the j second region feature vectors to generate the first expression image.

[0405] In a possible implementation, the i first region feature vectors include facial features reflected by facial information of the first object, and the j second region feature vectors include facial features reflected by facial information of the first reference object and style features of the expression template image.

[0406] The processor 1501 performs expression face changing processing based on the i first region feature vectors and the j second region feature vectors to generate the first expression image, to perform the following operations:

[0407] perform encoding processing on the i first region feature vectors and the j second region feature vectors by using an attention mechanism to obtain encoded feature vectors;

[0408] perform decoding processing on the encoded feature vectors, and perform synthesis processing on the decoded feature vectors to generate the first expression image;

[0409] The synthesis processing is configured to indicate that the i first region feature vectors and the j second region feature vectors are fused, and the facial features of the first object and the facial features of the first reference object are replaced.

[0410] In summary, on one hand, the computer device (such as a terminal device) can perform expression face replacement processing on the face of the first object in the first image and the face of any reference object (such as the first reference object) in the expression template image, so that the first expression image generated after face replacement can not only have the style features of the expression template image, but also reflect the face features of the first object, and further include the face information of the target reference object that is not replaced. That is, the first expression image is an image personalized by the face features of the first object. Then, different expression images can be generated for the same expression template image by using the face features of different objects, so as to meet the personalized needs of users for expression images, and make the expression images generated by the present application more personalized and interesting. On the other hand, the computer device (such as a server) can perform expression face replacement processing on the first object in the first image and any reference object (such as the first expression image) in the expression template image, while retaining the original style features of the expression template image. That is, only the face is replaced in the expression face replacement process, so that the generated first expression image not only has the style features of the expression template image, but also has the face features of the first object, and further includes the face information of the target reference object that is not replaced. In this way, the expression image personalized for the first object can be generated, so as to improve the interestingness of the expression image in the social conversation process, and meet the personalized needs of users.

[0411] In addition, it should be noted that the embodiments of the present application also provide a computer storage medium, and the computer storage medium stores a computer program, and the computer program includes program instructions. When a processor executes the program instructions, the method in the foregoing embodiments can be performed, and thus, details will not be described herein. For technical details that are not disclosed in the embodiments of the computer storage medium, please refer to the description of the method embodiments of the present application. For example, the program instructions can be deployed on one computer device, or executed on multiple computer devices located in one place, or executed on multiple computer devices distributed in multiple places and interconnected through a communication network.

[0412] According to an aspect of the present application, the embodiments of the present application also provide a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device can perform the method in the foregoing embodiments, and thus, details will not be described herein.

[0413] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in or transmitted by a computer readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (for example, coaxial cable, optical fiber, digital line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data processing device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk (SSD)) and the like.

[0414] The above disclosure is only the preferred embodiment of the present application, and of course cannot limit the scope of the right of the present application, so the equivalent changes made according to the claims of the present application still fall within the scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises: in response to an expression face changing operation received in a social conversation process, obtaining a first image containing face information of a first object; obtaining an expression template image containing face information of a first reference object and face information of a second reference object; generating a first expression image according to the first image and the expression template image; the first expression image contains face features reflected by the face information of the first object and contains face information of a target reference object, the target reference object being any one of the first reference object and the second reference object; and displaying the first expression image in a social interface. The method comprises:

2. The method of claim 1, wherein, displaying an image upload portal for triggering the acquisition of a first image to be processed; when detecting that the image upload portal is selected, acquiring any image containing face information of the first object from an image library as the first image; or, calling a shooting device to collect face information of the first object to generate the first image. The method comprises:

3. The method of claim 1, wherein, displaying an expression package template list, the expression package template list displaying at least one expression package template, any of the expression package templates including one or more template images; in response to a selection operation performed in the expression package template list, obtaining a selected expression template image; wherein the expression template image is any expression package template in the expression package template list, or the expression template image is any template image under any expression package template in the expression package template list. The method further comprises:

4. The method of claim 3, wherein, acquiring a social permission corresponding to the first object in a social conversation process; displaying an expression package template list to be selected by the first object according to the social permission; wherein, if the social permission is a first permission, the expression package template list is a first template list; if the social permission is a second permission, the expression package template list is a second template list; the first permission is different from the second permission, and the first template list is different from the second template list. The expression template image is a target expression package template in the expression package template list, the target expression package template including K template images, K being a positive integer; the method further comprises:

5. The method of claim 3 or 4, wherein, generating K expression images according to the first image and the K template images; wherein, any expression image is generated based on the first image and any template image of the K template images; and any expression image has the style characteristics of the corresponding template image and contains face features reflected by the face information of the first object. The method comprises:

6. The method of claim 1, wherein, performing expression face changing processing on the first object in the first image and the first reference object in the expression template image to generate a first expression image, the first expression image including face features of the first object and face features of the first reference object; or ​ performing expression face changing processing on a first object in the first image and a second reference object in the expression template image to generate a first expression image, the first expression image including facial features of the first object and facial features of the second reference object; wherein the reference object on which the expression face changing processing is performed is an object with the same attribute as the first object.

7. The method of claim 6, wherein, The first image includes facial information of a first object and facial information of a second object; the method further includes: performing expression face changing processing on a first object in the first image and a first reference object in the expression template image, and performing expression face changing processing on a second object in the first image and a second reference object in the expression template image to obtain a second expression image; wherein the second expression image has style characteristics of the expression template image, and includes facial features reflected by the facial information of the first object and facial features reflected by the facial information of the second object.

8. The method of claim 1, wherein, The method further includes: in response to a selection operation on the first expression image, displaying a face changing invitation portal; when it is detected that the face changing invitation portal is triggered, generating a face changing invitation message; sending the face changing invitation message to the second object; wherein the face changing invitation message is used to trigger obtaining a second image including facial information of the second object, and generating a second expression image based on the second image and the first expression image; the second expression image has style characteristics of the expression template image, and includes facial features reflected by the facial information of the first object and facial features reflected by the facial information of the second object.

9. The method of claim 8, wherein, The second object for receiving the face changing invitation message includes any one of the following: any social object in a one-on-one social conversation with the first object; or, a reference social object in a group social conversation to which the first object belongs; the reference social object includes any one of the following: any social object in the group social conversation, a social object with the highest interaction frequency with the first object in the group social conversation, a social object with the most recent interaction operation time with the first object in the group social conversation, and a social object that publishes the latest conversation message in the group social conversation.

10. The method of claim 8 or 9, wherein, The method further includes: displaying a prompt message in a social interface, the prompt message being used to prompt that a second image including facial information of the second object has been obtained; in response to an update operation triggered on the first expression image displayed in the social interface, updating and displaying the first expression image as a second expression image in the social interface; wherein the second expression image is generated by updating the first expression image based on the second image; the second expression image includes facial features of the first object and facial features of the second object.

11. The method of claim 1, wherein, The displaying of the first expression image in the social interface includes any one of the following: displaying the first expression image in an expression panel of the social interface; or, displaying the first expression image in a one-on-one conversation interface of the first object and the second object; or, displaying the first sticker image in any group chat social conversation interface to which the first object belongs; or, displaying the first sticker image in an image identifier of the first object; the image identifier includes any of the following: a head portrait, a personal homepage, a background image.

12. The method of claim 11, wherein, The expression panel of the social interface displays at least one system sticker image; the method further comprises: distinguishing the display of the first sticker image and each system sticker image in the expression panel of the social interface; wherein the distinguishing display mode includes any of the following: animation, highlighting, prompt.

13. The method of claim 11 or 12, wherein, The social conversation process refers to the process of a group chat social conversation in which the first object is located, and the group chat social conversation includes the first object and N social objects, N being a positive integer; The expression template image includes face information of a first reference object and face information of a second reference object; the method further comprises: displaying a reference sticker image in the expression panel of the group chat social conversation; the reference sticker image is generated according to the first image, the expression template image, and a reference image containing face information of a reference social object; Wherein, the reference social object includes any of the following: the social object that interacts with the first object most frequently among the N social objects, the social object that performs an interaction operation with the first object most recently among the N social objects, the social object that publishes the latest conversation message among the N social objects, and the social object that is most active among the N social objects.

14. The method of claim 13, wherein, The method further comprises: in response to a selection operation on the reference sticker image, displaying the reference sticker image in a conversation window of the group chat social conversation; and, displaying an interaction message for the reference social object in the conversation window of the group chat social conversation; Wherein, the display position of the interaction message includes any of the following: any position in the conversation window, a specified position in the conversation window, and an associated position of an image identifier of the reference social object.

15. The method of claim 14, wherein, The method further comprises: in response to an expression update operation detected in the expression panel of the group chat social conversation, outputting an object selection prompt box; wherein the object selection prompt box displays object identifiers of N social objects to be selected in the group chat social conversation; in response to a selection operation on an object identifier of a target social object among the N social objects, updating the display of the reference sticker image to a third sticker image in the sticker panel; Wherein, the third sticker image has the style characteristics of the expression template image, and contains face features reflected by the face information of the first object and face features reflected by the face information of the target social object.

16. The method of claim 1, wherein, The expression face changing operation is generated in any of the following ways: When it is detected that a face changing function entry set in the social interface is triggered, the expression face changing operation is generated, and the face changing function entry includes any of the following: a control, an option; or, When a specific touch operation is detected in the operation area of the social interface, the expression face changing operation is generated, the specific touch operation including any one of a single click operation, a double click operation, a gesture operation, and a hovering gesture; or When a face changing invitation message displayed in the social interface is detected to be selected, the expression face changing operation is generated, the face changing invitation message being sent by a second object to a first object during a social conversation process; and a format of the face changing invitation message includes any one of a card, a link, a website, and a structured message.

17. An image processing method, characterized by, The method comprises: receiving an image processing request sent by a social client, the image processing request being generated after an expression face changing operation is received during a social conversation process of the social client, the image processing request including a first image and an expression template image, the first image containing face information of a first object, and the expression template image containing face information of a first reference object and face information of a second reference object; generating a first expression image according to the first image and the expression template image; the first expression image has style characteristics of the expression template image, and contains face characteristics reflected by the face information of the first object and face information of a target reference object, the target reference object being any one of the first reference object and the second reference object; returning the first expression image to the social client to enable the social client to display the first expression image in a social interface.

18. The method of claim 17, wherein, The method comprises: performing region segmentation processing on the first image to obtain i first regions, and performing feature extraction processing on each first region to obtain i first region feature vectors corresponding to the i first regions, i being a positive integer; and performing region segmentation processing on the expression template image to obtain j second regions, and performing feature extraction processing on each second region to obtain j second region feature vectors corresponding to the j second regions, j being a positive integer; performing expression face changing processing based on the i first region feature vectors and the j second region feature vectors to generate a first expression image.

19. The method of claim 18, wherein, The i first region feature vectors include face characteristics reflected by the face information of the first object, and the j second region feature vectors include face characteristics reflected by the face information of the first reference object and style characteristics of the expression template image; The method comprises: encoding the i first region feature vectors and the j second region feature vectors using an attention mechanism to obtain encoded feature vectors; decoding the encoded feature vectors, and synthesizing the decoded feature vectors to generate the first expression image; and returning the first expression image to the social client to enable the social client to display the first expression image in a social interface. The synthesis processing is used to indicate that the i first regional feature vectors are fused with the j second regional feature vectors, and the face features of the first object are replaced with the face features of the first reference object.

20. An image processing apparatus characterized by comprising: The method comprises the steps of: An acquisition unit is configured to acquire a first image containing face information of a first object in response to a face exchange operation received in a social conversation process. The acquisition unit is further configured to acquire an expression template image containing face information of a first reference object and face information of a second reference object. A processing unit is configured to generate a first expression image according to the first image and the expression template image. The first expression image has a style feature of the expression template image, and contains face features reflected by the face information of the first object and face information of a target reference object, the target reference object being any one of the first reference object and the second reference object. And, A display unit is configured to display the first expression image in a social interface.

21. An image processing apparatus characterized by comprising: The method comprises the steps of: A receiving unit is configured to receive an image processing request sent by a social client, the image processing request being generated after a face exchange operation is received in a social conversation process of the social client, the image processing request containing a first image and an expression template image, the first image containing face information of a first object, and the expression template image containing face information of a first reference object and face information of a second reference object. A processing unit is configured to generate a first expression image according to the first image and the expression template image. The first expression image contains face features reflected by the face information of the first object and face information of a target reference object, the target reference object being any one of the first reference object and the second reference object. A sending unit is configured to return the first expression image to the social client, so that the social client displays the first expression image in a social interface.

22. A computer device, comprising: The method comprises the steps of: A memory and a processor; A memory in which one or more computer programs are stored; The processor is configured to load the one or more computer programs to implement the image processing method according to any one of claims 1-16 or 17-19.

23. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is adapted to be loaded and executed by the processor to implement the image processing method according to any one of claims 1-16 or 17-19.

24. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is adapted to be loaded and executed by the processor to implement the image processing method according to any one of claims 1-16 or 17-19.

Images