Wallpaper interaction method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a wallpaper interaction method and device, equipment and a storage medium, and the method comprises the steps: obtaining the angle information of the face orientation of a user in response to a personalized wallpaper triggering operation; based on the angle information of the face orientation of the user, determining target angle information of the face orientation of the preset wallpaper; obtaining preset wallpaper, and performing angle adjustment on the face orientation of the face information part in the preset wallpaper based on target angle information of the face orientation of the preset wallpaper to obtain target wallpaper; the face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face of the face information part in the target wallpaper is opposite to the face of the user; and displaying the target wallpaper. According to the method, the interactivity of the wallpaper is improved, and personalized interaction experience is provided for the user.

Description

Technical Field

[0001] The present application relates to the field of smart terminal technology, and specifically to a wallpaper interaction method, device, equipment and storage medium. Background Art

[0002] With the rapid development of mobile Internet technology, terminal devices such as mobile phones, watches, and car computers have become indispensable tools in users' daily lives. The wallpaper of a terminal device is the lock screen background image or desktop background image of the terminal device, including static wallpapers and dynamic wallpapers. In some technologies, a static image can be set as a static wallpaper of a terminal device, which can meet the personalized needs of users, but static wallpapers lack a sense of dynamics and interactivity. In some technologies, a recorded video can be set as a dynamic wallpaper of a terminal device. Such a dynamic wallpaper has a sense of vividness, but cannot provide an interactive experience. In some technologies, a dynamic expression video can be generated based on an image containing a human face and an expression video template, and the video can be set as a dynamic wallpaper of a terminal device. This can meet the personalized needs of users, but still cannot provide an interactive experience.

[0003] In some technologies, dynamic wallpapers can respond to trigger information and undergo corresponding changes to provide users with an interactive experience. For example, when it is determined according to the trigger information that the user is looking at the screen, the cartoon character in the wallpaper turns its head. However, these changes rely on preset fixed templates and lack interactivity. For example, the head-turning action of the cartoon character in the above response is a fixed template. Each time it is determined that the user is looking at the screen, the head-turning action of the cartoon character in the dynamic wallpaper is the same, that is, the dynamic wallpaper undergoes the same change. Summary of the invention

[0004] In view of this, the present application provides a wallpaper interaction method, device, equipment and storage medium, so as to solve the problem that the interactive dynamic wallpaper in the prior art relies on preset fixed templates and has insufficient interactivity.

[0005] In a first aspect, an embodiment of the present application provides a wallpaper interaction method, the method comprising:

[0006] In response to the personalized wallpaper triggering operation, obtaining angle information of the user's facial orientation;

[0007] Determining target angle information of the facial orientation of a preset wallpaper based on the angle information of the user's facial orientation;

[0008] Acquire a preset wallpaper, and adjust the facial orientation of the facial information part of the preset wallpaper based on the target angle information of the facial orientation of the preset wallpaper to acquire a target wallpaper; the facial orientation of the facial information part of the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information part of the target wallpaper is opposite to the face of the user;

[0009] The target wallpaper is displayed.

[0010] In a possible implementation of the first aspect, the angle information of the user's facial orientation includes angle information between the user's facial orientation and a first direction of a target plane, and angle information between the user's facial orientation and a second direction of the target plane; wherein the first direction and the second direction are perpendicular to each other, and the target plane is parallel to the plane where the display screen is located.

[0011] In a possible implementation manner of the first aspect, determining target angle information of the facial orientation of the preset wallpaper based on the angle information of the user's facial orientation includes:

[0012] Based on the angle information between the user's facial orientation and the first direction of the target plane and the angle information between the user's facial orientation and the second direction of the target plane, the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane and the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane are determined.

[0013] In a possible implementation manner of the first aspect, the acquiring of the preset wallpaper, adjusting the angle of the facial orientation of the facial information portion in the preset wallpaper based on the target angle information of the facial orientation of the preset wallpaper, and acquiring the target wallpaper includes:

[0014] Obtain a preset wallpaper and determine the initial angle information of the face orientation of the preset wallpaper;

[0015] Based on the target angle information of the face orientation of the preset wallpaper and the initial angle information of the face orientation of the preset wallpaper, a target angle adjustment path is determined in at least one corresponding angle adjustment path; the starting position of the at least one corresponding angle adjustment path is the initial angle of the face orientation of the preset wallpaper, and the ending position of the angle adjustment path is the target angle of the face orientation of the preset wallpaper;

[0016] Get the preset adjustment duration and the preset frame rate, and determine the frame number m of the dynamic wallpaper video; m is an integer greater than 1;

[0017] Based on the target angle adjustment path and the preset adjustment duration, determining the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video; wherein the facial orientation angle information of the facial information part in at least two frames of the wallpaper image in the dynamic wallpaper video is different;

[0018] Based on the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video, a dynamic wallpaper video is generated, wherein the first frame of the wallpaper image of the dynamic wallpaper video is a preset wallpaper with the facial orientation of the facial information part being an initial angle, and the last frame of the wallpaper image of the dynamic wallpaper video is the target wallpaper.

[0019] In a possible implementation manner of the first aspect, determining the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video based on the target angle adjustment path and the preset adjustment duration includes:

[0020] Determining an angle switching speed based on the target angle adjustment path and a preset adjustment duration;

[0021] Based on the target angle adjustment path and the angle switching speed, the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video is determined.

[0022] In a possible implementation manner of the first aspect, the angle switching speed includes a first angle switching speed and a second angle switching speed, and the first angle switching speed is different from the second angle switching speed.

[0023] In a possible implementation manner of the first aspect, generating a dynamic wallpaper video based on the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video includes:

[0024] Extracting a first key point of a facial information portion in the preset wallpaper;

[0025] For the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video, adjust the facial orientation of the first key point of the facial information part in the preset wallpaper according to the angle information of the facial orientation of the facial information part in the frame of the wallpaper image in the dynamic wallpaper video, and obtain a second key point corresponding to the frame of the wallpaper image;

[0026] Based on the preset wallpaper and the second key points corresponding to the m frames of wallpaper images, a dynamic wallpaper video is generated.

[0027] In a possible implementation manner of the first aspect, the facial information portion of n frames of wallpaper images in the dynamic wallpaper video is superimposed with interactive expression information; n is an integer greater than 0.

[0028] In a possible implementation manner of the first aspect, before generating a dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m frames of wallpaper images, the method further includes:

[0029] Determine the start and end time of the interactive expression;

[0030] According to the start time, end time and preset frame rate of the interactive expression, determine the frame number n of the interactive expression image, and obtain n frames of interactive expression image from the preset interactive expression image;

[0031] Determine the facial orientation angle information of the facial information part in the n-frame wallpaper image within the target time period from the facial orientation angle information of the facial information part in the m-frame wallpaper image of the dynamic wallpaper video; wherein the target time period is the time period consisting of the start time and the end time of the interactive expression;

[0032] Extracting the third key point of the facial information part in the n-frame interactive expression image, and adjusting the facial orientation of the third key point of the facial information part in the n-frame interactive expression image according to the facial orientation angle information of the facial information part in the n-frame wallpaper image corresponding to the target time period, to obtain the fourth key point corresponding to the n-frame interactive expression image;

[0033] Determining the second key points corresponding to the n frames of wallpaper images within the target time period among the second key points corresponding to the m frames of wallpaper images;

[0034] Based on the fourth key point corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images, determine the fifth key point corresponding to the n frames of wallpaper images superimposed with interactive expressions within the target time period;

[0035] The generating a dynamic wallpaper video based on the preset wallpaper and the second key point corresponding to the m-frame wallpaper image comprises:

[0036] A dynamic wallpaper video is generated based on the preset wallpaper, the fifth key points corresponding to the wallpaper images of n frames superimposed with interactive expressions within the target time period, and the second key points corresponding to other wallpaper images; wherein the second key points corresponding to other wallpaper images refer to the second key points corresponding to the m frames of wallpaper images, except the second key points corresponding to the n frames of wallpaper images within the target time period.

[0037] In a possible implementation of the first aspect, determining the start time and end time of the interactive expression includes: obtaining the first time of the interactive expression; obtaining the length of the target angle adjustment path corresponding to the interactive expression; determining the second time of the interactive expression based on the length of the target angle adjustment path corresponding to the interactive expression and the first time of the interactive expression; the first time of the interactive expression includes the start time or end time of the interactive expression; the second time of the interactive expression includes the end time or start time of the interactive expression.

[0038] In a possible implementation manner of the first aspect, determining, based on the fourth key point corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images, the fifth key point corresponding to the n frames of wallpaper images superimposed with interactive expressions within the target time period includes:

[0039] Based on the fourth key points corresponding to the n frames of interactive expression images, determining the position offset of the corresponding key points between each two adjacent frames between the fourth key points corresponding to the n frames of interactive expression images;

[0040] Based on the position offset of the corresponding key points between each two adjacent frames between the fourth key points corresponding to the n frames of interactive expression images, and the second key points corresponding to the n frames of wallpaper images, determine the fifth key points corresponding to the n frames of wallpaper images with superimposed interactive expressions within the target time period.

[0041] In a possible implementation of the first aspect, in response to a personalized wallpaper triggering operation, obtaining angle information of a user's facial orientation includes:

[0042] In response to the user looking at the electronic device, angle information of the user's facial orientation is obtained.

[0043] In a second aspect, an embodiment of the present application provides a wallpaper interaction device, including:

[0044] An acquisition unit, configured to acquire angle information of the user's facial orientation in response to a personalized wallpaper trigger operation;

[0045] A determination unit, configured to determine target angle information of the facial orientation of a preset wallpaper based on the angle information of the user's facial orientation;

[0046] a processing unit, configured to obtain a preset wallpaper, and adjust the facial orientation of a facial information portion in the preset wallpaper based on target angle information of the facial orientation of the preset wallpaper to obtain a target wallpaper; the facial orientation of the facial information portion in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information portion in the target wallpaper is opposite to the face of the user;

[0047] The display unit is used to display the target wallpaper.

[0048] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute any one of the methods described in the first aspect above.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any method described in the first aspect.

[0050] According to the scheme provided in the embodiment of the present application, in response to the personalized wallpaper trigger operation, the angle information of the user's facial orientation is obtained; based on the angle information of the user's facial orientation, the target angle information of the facial orientation of the preset wallpaper is determined; the preset wallpaper is obtained, and the facial orientation of the facial information part in the preset wallpaper is adjusted according to the target angle information of the facial orientation of the preset wallpaper to obtain the target wallpaper; the facial orientation of the facial information part in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information part in the target wallpaper is opposite to the face of the user; the target wallpaper is displayed. In the embodiment of the present application, when the personalized wallpaper trigger operation is detected, the target wallpaper is generated and displayed based on the angle information of the user's facial orientation and the preset wallpaper. The facial orientation of the facial information part in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information part in the target wallpaper is opposite to the face of the user, so as to achieve the effect that the facial information part in the target wallpaper looks at the face of the user. In this way, the interactivity of the wallpaper is improved, and the target wallpaper is generated based on the angle information of the user's facial orientation, not relying on the preset fixed template, and the generated target wallpaper is different for different angle information of the user's facial orientation, providing a personalized interactive experience for the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0052] Figure 1 A schematic diagram of a flow chart of a wallpaper interaction method provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of a flow chart of another wallpaper interaction method provided in an embodiment of the present application;

[0054] Figure 3A schematic diagram of a flow chart of another wallpaper interaction method provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of a scene of a wallpaper interaction method provided in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0057] Figure 6 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0058] Figure 7 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0059] Figure 8 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0060] Fig. 9 A schematic diagram of a flow chart of another wallpaper interaction method provided in an embodiment of the present application;

[0061] Fig.10 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0062] Fig.11 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0063] Fig.12 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0064] Fig.13 A schematic diagram of a flow chart of another wallpaper interaction method provided in an embodiment of the present application;

[0065] Fig.14 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0066] Fig.15 A schematic diagram of a flow chart of another wallpaper interaction method provided in an embodiment of the present application;

[0067] Fig.16 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0068] Fig.17 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0069] Fig.18 A schematic diagram of a flow chart of another wallpaper interaction method provided in an embodiment of the present application;

[0070] Fig.19 A schematic diagram of a flow chart of another wallpaper interaction method provided in an embodiment of the present application;

[0071] Fig. 20 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0072] Fig.21 A schematic diagram of a flow chart of another wallpaper interaction method provided in an embodiment of the present application;

[0073] Fig. 22 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0074] Fig.23 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0075] Fig.24 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0076] Fig.25 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0077] Figure 26a-Figure 26e A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0078] Fig. 27 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0079] Fig.28 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0080] Fig.29 A schematic diagram of another scenario of a wallpaper interaction method provided in an embodiment of the present application;

[0081] Fig.30 A schematic diagram of the structure of a wallpaper interaction device provided in an embodiment of the present application;

[0082] Fig.31 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0084] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0085] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0086] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0087] With the rapid development of mobile Internet technology, mobile phones, watches, car computers and other terminal devices have become indispensable tools in users' daily lives. The wallpaper of a terminal device is the lock screen background image or desktop background image of the terminal device, including static wallpaper and dynamic wallpaper. In some technologies, a static image can be set as the static wallpaper of the terminal device, which can meet the personalized needs of users, but static wallpaper lacks dynamic and interactive feeling. Figure 1 As shown, for specific scenarios, professional UX (User experience) designers create wallpapers or users crop and optimize pictures in the mobile phone gallery to generate their own electronic device wallpapers. This solution is widely used in scenarios such as the built-in wallpaper library of mobile phones and the watch face market. The watch face wallpaper selected on the mobile phone is sent to other terminal devices (such as watches) for rendering. Applicable devices include but are not limited to smart watches, sports watches, mobile phone AOD (always on display), small external screens of mobile phones, mobile phone wallpapers, car wallpapers, etc. This technology is mostly used in current electronic device wallpaper systems, relying on the design of professional wallpaper designers or user-defined wallpapers, and cannot meet the user's personalized, diverse, and interactive needs.

[0088] In some technologies, the recorded video can be set as a dynamic wallpaper of the terminal device. Such a dynamic wallpaper has a vivid feeling, but cannot provide an interactive experience.

[0089] In some technologies, a dynamic expression video can be generated based on an image containing a human face and an expression video template, and the video can be set as a dynamic wallpaper of the terminal device. This can meet the personalized needs of users, but still cannot provide an interactive experience. Based on the computational expression technology of generative adversarial networks, users only need to upload a picture containing a human face, and a dynamic expression video will be automatically generated according to the expression changes in the expression video template. This technology is widely used in video editing, wallpaper generation and other scenarios. Figure 2 As shown, the wallpaper generation device in this technology includes: a portrait acquisition and content understanding unit 201, which is used to obtain the user's uploaded image and perform content understanding on the portrait in the image, including detection, segmentation, key feature point acquisition and other operations; an expression template creation unit 202, which is used to design corresponding dial expression actions according to public preferences and record and create or user-defined expression video templates; an expression calculation generation unit 203, which completes the expression migration action with high quality by learning the facial feature differences between the uploaded image and each frame expression template and the mapping relationship between the facial feature difference and the image expression change; a device perception interaction unit 204, which is used to obtain device interaction information, including but not limited to the face angle information collected by the mobile phone camera, the interaction information of tapping the screen on the watch side, gyroscope information, crown rotation information, etc. according to the difference in device usage scenarios; a transmission rendering display unit 205, which is used to send the micro-motion expression video generated in the cloud to the terminal device for rendering and presentation, including but not limited to smart watches, sports watches, mobile phone AOD, small external screens of mobile phones, mobile phone wallpapers, and car wallpapers. This technology is implemented based on the method of expression migration. It can only generate video wallpapers that are completely consistent with the preset template expression movements based on the preset video expression template. In essence, it cannot provide users with a rich and diverse interactive experience.

[0090] In some technologies, dynamic wallpapers can respond to trigger information and undergo corresponding changes to provide users with an interactive experience. For example, when it is determined according to the trigger information that the user is looking at the screen, the cartoon character in the wallpaper turns its head. However, these changes rely on preset fixed templates and lack interactivity. For example, the head-turning action of the cartoon character in the above response is a fixed template. Each time it is determined that the user is looking at the screen, the head-turning action of the cartoon character in the dynamic wallpaper is the same, that is, the dynamic wallpaper undergoes the same change.

[0091] In view of the above problems, the embodiments of the present application provide a wallpaper interaction method, device, equipment and storage medium. In response to a personalized wallpaper trigger operation, the angle information of the user's facial orientation is obtained; based on the angle information of the user's facial orientation, the target angle information of the facial orientation of the preset wallpaper is determined; the preset wallpaper is obtained, and the facial orientation of the facial information part in the preset wallpaper is adjusted according to the target angle information of the facial orientation of the preset wallpaper to obtain the target wallpaper; the facial orientation of the facial information part in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information part in the target wallpaper is opposite to the face of the user; and the target wallpaper is displayed. In the embodiment of the present application, when a personalized wallpaper trigger operation is detected, the target wallpaper is generated and displayed based on the angle information of the user's facial orientation and the preset wallpaper. The facial orientation of the facial information part in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information part in the target wallpaper is opposite to the face of the user, so as to achieve the effect that the facial information part in the target wallpaper looks at the face of the user. In this way, the interactivity of the wallpaper is improved, and the target wallpaper is generated based on the angle information of the user's facial orientation, not relying on a preset fixed template, and the generated target wallpaper is different for different angle information of the user's facial orientation, providing a personalized interactive experience for the user. The following is a detailed description.

[0092] See also Figure 3 , is a flowchart of a wallpaper interaction method provided in an embodiment of the present application. The method is applied to electronic devices, such as Figure 3 As shown, the method includes:

[0093] Step S301: In response to a personalized wallpaper triggering operation, obtain angle information of the user's facial orientation.

[0094] In the embodiment of the present application, the personalized wallpaper triggering operation can trigger the generation and display of the personalized wallpaper. For different angle information of the user's facial orientation, the generated personalized wallpaper is different, that is, the personalized wallpaper is generated based on the angle information of the user's facial orientation, so it is necessary to obtain the angle information of the user's facial orientation.

[0095] In some embodiments, the personalized wallpaper trigger operation may be the user looking at the electronic device, the user making a special expression such as a smile, the display screen being woken up, and other trigger operations, which are not limited in this application.

[0096] In some embodiments, the angle information of the user's facial orientation may be determined based on the user's facial image information. The user's facial image information may be collected by a camera of an electronic device.

[0097] Alternatively, in some embodiments, the angle information of the user's facial orientation may be determined based on sensor information such as gyroscope information.

[0098] As a possible implementation method, the angle information of the user's facial orientation includes the angle information between the user's facial orientation and a first direction of the target plane, and the angle information between the user's facial orientation and a second direction of the target plane; wherein the first direction and the second direction are perpendicular to each other, and the target plane is parallel to the plane where the display screen is located.

[0099] In an embodiment of the present application, the angle information between the user's facial orientation and the first direction of the target plane, and the angle information between the user's facial orientation and the second direction of the target plane can represent the angle information between the user's facial orientation and the plane where the display screen is located.

[0100] The angle information between the user's facial orientation and the first direction of the target plane includes the angle size and angle direction between the user's facial orientation and the first direction of the target plane; the angle information between the user's facial orientation and the second direction of the target plane includes the angle size and angle direction between the user's facial orientation and the second direction of the target plane.

[0101] For example, the first direction may be a direction parallel to the direction in which the data lines in the display screen are arranged, and the second direction may be a direction parallel to the direction in which the scan lines in the display screen are arranged. Figure 4 As shown. For ease of understanding, the present application embodiment takes the first direction as a direction parallel to the direction in which the data lines are arranged in the display screen, and the second direction as a direction parallel to the direction in which the scan lines are arranged in the display screen as an example for explanation. That is, the angle information between the user's face and the first direction of the target plane is the pitch angle information of the user looking at the display screen, and the angle information between the user's face and the second direction of the target plane is the deflection angle information of the user looking at the display screen. For ease of understanding, as Figure 4 As shown, point a1 is to the left of point a2, and point a2 is to the right of point a1; point b1 is above point b2, and point b2 is below point b1.

[0102] For example, Figure 5 As shown, the user's facial orientation is ray D, the angle information between the user's facial orientation and the first direction of the target plane is an elevation angle of 30°, and the angle information between the user's facial orientation and the second direction of the target plane is a left deflection angle of 30°.

[0103] As a possible implementation manner, in response to a personalized wallpaper triggering operation, obtaining the angle information of the user's facial orientation includes: in response to the user looking at the electronic device, obtaining the angle information of the user's facial orientation.

[0104] In an embodiment of the present application, in order to improve the interactivity of the personalized wallpaper, when it is detected that the user is looking at the electronic device, the generation and display of the personalized wallpaper is triggered.

[0105] As a possible implementation manner, in response to a personalized wallpaper triggering operation, obtaining the angle information of the user's facial orientation includes: in response to the user looking at the electronic device and smiling, obtaining the angle information of the user's facial orientation.

[0106] In an embodiment of the present application, when it is detected that a user is looking at the electronic screen and smiling, the generation and display of the target wallpaper can be triggered.

[0107] Step S302: Determine target angle information of the facial orientation of the preset wallpaper based on the angle information of the user's facial orientation.

[0108] In an embodiment of the present application, in order to make the facial orientation of the facial information part in the target wallpaper parallel to the facial orientation of the user, and the face of the facial information part in the target wallpaper is opposite to the user's face, the facial orientation after the facial orientation in the wallpaper is adjusted to be parallel to and opposite to the user's face, and based on the angle information of the user's facial orientation, the target angle information of the facial orientation of the preset wallpaper is determined, that is, the angle information of the facial orientation of the adjusted target wallpaper is determined.

[0109] As a possible implementation method, determining the target angle information of the facial orientation of the preset wallpaper based on the angle information of the user's facial orientation includes: determining the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane, and the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane based on the angle information between the user's facial orientation and the first direction of the target plane and the angle information between the user's facial orientation and the second direction of the target plane.

[0110] In an embodiment of the present application, the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane can be determined based on the angle information between the facial orientation of the user and the first direction of the target plane; the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane can be determined based on the angle information between the facial orientation of the user and the second direction of the target plane.

[0111] In some embodiments, in order to make the facial orientation of the facial information part in the target wallpaper parallel to the facial orientation of the user, and the face of the facial information part in the target wallpaper is opposite to the face of the user, the angle size of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane is the same as the angle size of the angle information between the facial orientation of the user and the first direction of the target plane; the angle direction of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane is opposite to the angle direction of the angle information between the facial orientation of the user and the first direction of the target plane; that is, the target pitch angle information between the facial orientation of the facial information part in the preset wallpaper and the plane where the display screen is located is the same as the pitch angle information when the user looks at the display screen, but in the opposite direction. The angle size of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane is the same as the angle size of the angle information between the facial orientation of the user and the second direction of the target plane; the angle direction of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane is opposite to the angle direction of the angle information between the facial orientation of the user and the second direction of the target plane; that is, the target deflection angle information between the facial orientation of the facial information part in the preset wallpaper and the plane where the display screen is located is the same as the deflection angle information when the user looks at the display screen, but opposite in direction.

[0112] Step S303: obtaining a preset wallpaper, adjusting the angle of the facial orientation of the facial information part in the preset wallpaper based on the target angle information of the facial orientation of the preset wallpaper, and obtaining the target wallpaper.

[0113] The facial orientation of the facial information portion in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information portion in the target wallpaper is opposite to the face of the user.

[0114] In the embodiment of the present application, the preset wallpaper includes facial information, such as real human face information, cartoon face information, animal face information, etc. For example, Figure 6 , which is a schematic diagram of a preset wallpaper containing facial information of a human face.

[0115] In some embodiments, the preset wallpaper is pre-set by the user.

[0116] In some embodiments, in order to improve the privacy security of users, wallpapers containing real human face information uploaded by users can be converted into wallpapers containing animated human face information, and the wallpapers can be used as default wallpapers. Figure 7As shown in the figure, it is a cartoon image generation algorithm based on facial key points, which is essentially a deformation algorithm of CycleGAN (Cycle-Consistent Generative Adversarial Networks). By using facial key point information, the synergy of the global discriminator and the local discriminator can effectively improve the detail richness of the facial features such as eyes, mouth, and teeth. Based on the image containing the real face and its facial key points, the corresponding image containing comic face information is generated through the generator. The key point detection is performed on the image containing the comic face information to obtain the facial key points of the image containing the comic face information. Based on the facial key points of the image containing the real face information, the facial key points of the image containing the comic face information are adjusted so that the facial key points of the two are consistent, and based on this, the facial features of the image containing the comic face information are located, and the image containing the comic face information is updated. The image containing the real face information and its key points, and the image containing the comic face information and its key points are used as the input of the global discriminator and the local discriminator to ensure that the image containing the comic face information is indistinguishable from the real one. The global discriminator is used to ensure that the overall effect of the image containing comic face information is indistinguishable from the real one. Among them, the input of the conditional global discriminator is the image containing real face information and its key points, and the image containing comic face information and its key points. The input of the unconditional global discriminator is the image containing real face information and the image containing comic face information. The local discriminator is used to ensure that the local effect of the image containing comic face information is indistinguishable from the real one. The local discriminator includes an eye discriminator, a nose discriminator, and a mouth discriminator. Among them, the input of the eye discriminator is the eye segment information in the image containing real face information and the eye segment information in the image containing comic face information. The input of the nose discriminator is the nose segment information in the image containing real face information and the nose segment information in the image containing comic face information. The input of the mouth discriminator is the mouth segment information in the image containing real face information and the mouth segment information in the image containing comic face information.

[0117] Relative to the preset wallpaper, the facial orientation of the facial information part of the target wallpaper is adjusted in angle so that the angle information of the facial orientation of the target wallpaper is the target angle information. At this time, the facial orientation of the facial information part in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information part in the target wallpaper is opposite to the face of the user, thereby achieving the effect that the facial information part in the target wallpaper is facing the user's face.

[0118] As in the example above, for the preset wallpaper Figure 6 As shown, the angle information between the user's face orientation and the first direction of the target plane is an elevation angle of 30°, and the angle information between the user's face orientation and the second direction of the target plane is a leftward deflection angle of 30°. Figure 8 , which is a schematic diagram of the facial orientation of the facial information part in the target wallpaper and the facial orientation of the user.

[0119] In some embodiments, Fig. 9 As shown, a preset wallpaper is obtained, and the facial orientation of the facial information part in the preset wallpaper is adjusted based on the target angle information of the facial orientation of the preset wallpaper. Obtaining the target wallpaper includes:

[0120] Step S901: Obtain a preset wallpaper, and extract initial key points of the facial information part in the preset wallpaper.

[0121] In the embodiment of the present application, the key points of the facial information part can represent the facial information part. The initial key points of the facial information part in the preset wallpaper can represent the facial information part in the preset wallpaper. Fig.10 As shown, for human face information, 68 human face key points can represent the human face information. The initial key points of the facial information part in the preset wallpaper are the key points of the facial information part in the preset wallpaper that has not been adjusted.

[0122] For ease of understanding, in the following example, facial information is human face information and key points are 68 human face key points.

[0123] Step S902: According to the target angle information of the facial orientation of the preset wallpaper, the facial orientation of the initial key points of the facial information part of the preset wallpaper is adjusted to obtain the target key points corresponding to the target wallpaper.

[0124] In an embodiment of the present application, the target wallpaper is a preset wallpaper with a face orientation of a target angle, and the key points can represent the facial information part. Therefore, the target key points corresponding to the target wallpaper can be obtained based on the initial key points and the target angle information of the facial information part in the preset wallpaper. In some embodiments, the target key points corresponding to the target wallpaper can be obtained through a neural network, with the first key point of the facial information part in the preset wallpaper and the angle information of the facial orientation of the facial information part in the frame of the wallpaper image as the input of the neural network, and the target key points corresponding to the target wallpaper image as the output of the neural network.

[0125] In some embodiments, the face positioning points corresponding to the target wallpaper can be obtained based on the initial key points of the facial information part in the preset wallpaper and the target angle information. Fig.11 As shown in the figure, the initial key points of the facial information part of the preset wallpaper and the target angle information are used as the input of the neural network structure 1, and the output layer is the face positioning point corresponding to the target wallpaper. The number of neurons in the input layer is 138. Fig.12As shown, the face positioning points include the left eye corner, right eye corner, nose tip, left mouth corner, right mouth corner, and jaw. The output layer neurons of the neural network structure one have a total of 12, which respectively represent the coordinates of the 6 face positioning points. The neural network structure one can be a fully connected layer network structure. After determining the face positioning points corresponding to the target wallpaper, the target key points corresponding to the target wallpaper are obtained based on the initial key points of the facial information part in the preset wallpaper and the face positioning points corresponding to the target wallpaper. Fig.11 As shown, the 6 face positioning points corresponding to the target wallpaper and the initial key points of the facial information part in the preset wallpaper are used as the input of the neural network structure 2, and the output layer is the target key points corresponding to the target wallpaper. The number of neurons in the input layer is 148, and the number of neurons in the output layer is 136. The neural network structure 2 can be a fully connected layer network structure.

[0126] Step S903: Generate a target wallpaper based on the target key points corresponding to the target wallpaper and the preset wallpaper.

[0127] In the embodiment of the present application, the target key points corresponding to the target wallpaper can represent the facial information part of the target wallpaper. The facial orientation of the target wallpaper changes relative to the facial information part of the preset wallpaper. Therefore, the target wallpaper can be generated based on the target key points corresponding to the target wallpaper and the preset wallpaper.

[0128] In some embodiments, based on the target key points corresponding to the target wallpaper and the preset wallpaper, generating the target wallpaper includes:

[0129] Step S1301: perform face regularization processing on the preset wallpaper to obtain the face regularization facial information part of the preset wallpaper.

[0130] In an embodiment of the present application, in order to ensure the consistency of the facial information part of the face input by the motion estimation model and improve the generation effect of the target wallpaper, before generating the target wallpaper through the motion estimation model, it is necessary to first perform face regularization processing on the preset wallpaper.

[0131] like Fig.14 As shown, first refer to the face outline point and eyebrow height point of the initial key point of the preset wallpaper to determine the cropping size, and crop the preset wallpaper according to the cropping size. Based on the first key point of the preset wallpaper, expand the rectangular area formed by 68 face key points to the left and right by 0.3 times the width, expand the rectangular area formed by 68 face key points upward by 0.8 times the length, and expand the rectangular area formed by 68 face key points downward by 0.3 times the length.

[0132] After cropping the preset wallpaper, perform affine transformation on the cropped preset wallpaper to achieve face rotation alignment. The specific steps of affine transformation are as follows: calculate the left eye center coordinates and the right eye center coordinates respectively, calculate the angle θ between the connecting line of the left and right eye center coordinates and the horizontal direction, calculate the overall center coordinates of the two eyes, and use the overall center coordinates of the two eyes as the base point to rotate the cropped preset wallpaper counterclockwise by θ.

[0133] Step S1302: Based on the target key points corresponding to the target wallpaper and the regularized facial information portion of the preset wallpaper, expression calculation is performed through a motion estimation model to obtain the regularized facial information portion of the target wallpaper.

[0134] In the embodiment of the present application, the target key points corresponding to the target wallpaper and the face information part of the preset wallpaper are used as the input of the motion estimation model, and the output of the motion estimation model is the face information part of the target wallpaper that is processed by face regularization. In some embodiments, the motion estimation model can use TPS Motion (Thin-Plate Spline Motion Model). Of course, other motion estimation models can also be used, and the present application does not limit this.

[0135] Step S1303: based on the facial information part of the target wallpaper that has been processed by regularization, post-reply processing is performed to obtain the target wallpaper.

[0136] In the embodiment of the present application, the preset wallpaper includes not only a facial information part but also a non-facial information part. The non-facial information part in the target wallpaper is the same as the non-facial information part in the target wallpaper. In order to ensure the consistency between the target wallpaper and the preset wallpaper, the facial information part of the target wallpaper that has been processed by face regularization is pasted back to the preset wallpaper to generate the target wallpaper. That is, the facial information part of the target wallpaper that has been processed by face regularization is subjected to the inverse transformation of the affine transformation in step S1301, and the facial information part of the target wallpaper that has been inversely transformed is spliced ​​with the non-facial information part of the preset wallpaper to obtain the target wallpaper.

[0137] Step S304: display the target wallpaper.

[0138] In the embodiment of the present application, after the target wallpaper is acquired, the target wallpaper is displayed. The facial orientation of the facial information portion in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information portion in the target wallpaper is opposite to the face of the user.

[0139] In this way, when a personalized wallpaper trigger operation is detected, a target wallpaper is generated and displayed based on the angle information of the user's facial orientation and the preset wallpaper. The facial orientation of the facial information part in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information part in the target wallpaper is opposite to the face of the user, thereby achieving the effect that the facial information part in the target wallpaper and the user's face are facing each other. This method improves the interactivity of the wallpaper, and the target wallpaper is generated based on the angle information of the user's facial orientation, and does not rely on a preset fixed template. For different angle information of the user's facial orientation, the generated target wallpaper is different, providing users with a personalized interactive experience.

[0140] As a possible implementation method, in order to increase interactivity and improve user experience, the target wallpaper can be displayed through a dynamic wallpaper video. Fig.15 As shown, the above step S303 obtains the preset wallpaper, and adjusts the facial orientation of the facial information part in the preset wallpaper based on the target angle information of the facial orientation of the preset wallpaper. Obtaining the target wallpaper includes:

[0141] Step S3031: Obtain a preset wallpaper and determine the initial angle information of the facial orientation of the preset wallpaper.

[0142] In the embodiment of the present application, the initial angle information of the face orientation of the preset wallpaper is the angle information of the face orientation of the preset wallpaper in the initial state. The preset wallpaper is used as the input of the neural network, and the initial angle information of the face orientation of the preset wallpaper is used as the output of the neural network, so as to determine the initial angle information of the face orientation of the preset wallpaper.

[0143] In some embodiments, in order to improve the viewing quality of the preset wallpaper, the initial angle information of the face orientation of the preset wallpaper is 0° with respect to the first direction of the target plane, and 0° with respect to the second direction of the target plane. That is, the face orientation of the preset wallpaper is perpendicular to the target plane. For example, Figure 6 The facial orientation of the facial information portion in the preset wallpaper is shown to be perpendicular to the target plane.

[0144] For ease of understanding, the embodiment of the present application is described by taking the initial angle information of the facial orientation of the preset wallpaper as 0° with the first direction of the target plane and 0° with the second direction of the target plane as an example.

[0145] Step S3032: determining a target angle adjustment path in at least one corresponding angle adjustment path based on the target angle information of the facial orientation of the preset wallpaper and the initial angle information of the facial orientation of the preset wallpaper.

[0146] Among them, the starting position of at least one corresponding angle adjustment path is the initial angle of the facial orientation of the preset wallpaper, and the ending position of the angle adjustment path is the target angle of the facial orientation of the preset wallpaper.

[0147] In an embodiment of the present application, the facial orientation of the preset wallpaper is adjusted from an initial angle to a target angle, and an angle adjustment path for the facial orientation needs to be determined. The initial angle is the angle included in the initial angle information, and the target angle is the angle included in the target angle information. An angle adjustment path can be randomly selected from at least one corresponding angle adjustment path as the target angle adjustment path. The first angle adjustment path can also be selected from at least one corresponding angle adjustment path as the target angle adjustment path. The angle adjustment path with the shortest length can also be selected from at least one corresponding angle adjustment path as the target angle adjustment path. The angle adjustment path with the longest length can also be selected from at least one corresponding angle adjustment path as the target angle adjustment path. Of course, the target angle adjustment path can also be determined based on other methods, and the present application does not limit this.

[0148] In some embodiments, in order to make the angle adjustment more natural, the angle information between the face orientation of the preset wallpaper and the first direction of the target plane in the target angle adjustment path and the angle information between the face orientation of the preset wallpaper and the second direction of the target plane satisfy the single injection property. That is, the angle information between the face orientation of the preset wallpaper and the first direction of the target plane in the target angle adjustment path and the angle information between the face orientation of the preset wallpaper and the second direction of the target plane are in a one-to-one correspondence.

[0149] like Fig.16 As shown, the horizontal axis represents the deflection angle information of the facial orientation of the facial information part in the preset wallpaper. A positive value of the horizontal axis represents that the angle direction of the deflection angle information is deflected to the right, and a negative value of the horizontal axis represents that the angle direction of the deflection angle information is deflected to the left. The vertical axis represents the pitch angle information of the facial orientation of the facial information part in the preset wallpaper. A positive value of the vertical axis represents that the angle direction of the pitch angle information is a depression angle, and a negative value of the vertical axis represents that the angle direction of the pitch angle information is an elevation angle. Assuming that the deflection angle information in the initial angle information of the facial orientation of the preset wallpaper is 0° and the pitch angle information is 0°, the starting position of the angle switching path is (0,0). Assuming that the deflection angle information in the target angle information of the facial orientation of the preset wallpaper is 30° to the right and the pitch angle information is 0°, the ending position of the angle switching path is (30,0). For the angle switching path with a starting position of (0,0) and an ending position of (30,0), only one angle switching path satisfies the single injection property, that is, y=0 (0≤x≤30). Therefore, this angle switching path is determined as the target angle switching path.

[0150] Step S3033: Obtain a preset adjustment duration and a preset frame rate, and determine the number of frames m of the dynamic wallpaper video.

[0151] Here, m is an integer greater than 1.

[0152] In the embodiment of the present application, the preset adjustment duration is the time length of the dynamic wallpaper video, and the preset frame rate is the number of frames played by the dynamic wallpaper video in a unit time. The number of frames m of the dynamic wallpaper video is the product of the preset adjustment duration and the preset frame rate. The preset adjustment duration can be a default value or can be set by the user, and the present application does not limit this. The preset frame rate can be a default value or can be set by the user, and the present application does not limit this.

[0153] In some embodiments, in order to balance video smoothness and operating overhead, the preset frame rate is set to 30 frames per second.

[0154] Exemplarily, assuming that the preset adjustment time is 3 seconds and the preset frame rate is 30 frames per second, the number of frames of the dynamic wallpaper video is determined to be 90 frames, that is, m=90.

[0155] Step S3034: determining the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video based on the target angle adjustment path and the preset adjustment duration.

[0156] Among them, there are at least two frames of wallpaper images in the dynamic wallpaper video, in which the angle information of the facial orientation of the facial information part is different.

[0157] In an embodiment of the present application, within a preset adjustment time, the facial orientation angle information of the m-frame wallpaper image in the dynamic wallpaper video is adjusted from the initial angle to the target angle through the target angle adjustment path. That is, the facial orientation angle information of the facial information part in the m-frame wallpaper image in the dynamic wallpaper video is determined on the target angle adjustment path. The facial orientation angle information of the facial information part in the first frame wallpaper image of the dynamic wallpaper video is the initial angle information. The facial orientation angle information of the facial information part in the last frame wallpaper image of the dynamic wallpaper video is the target angle information.

[0158] In some embodiments, m angle information may be uniformly extracted on the target angle adjustment path as the angle information of the facial orientation of the facial information part in the m frames of wallpaper images in the dynamic wallpaper video. That is, the angle information difference of the facial orientation of the facial information part in the wallpaper images of adjacent frames in the dynamic wallpaper video is the same. Of course, the angle information difference of the facial orientation of the facial information part in the wallpaper images of adjacent frames in the dynamic wallpaper video may also be different, and this application does not limit this.

[0159] As a possible implementation method, determining the angle information of the facial orientation of the facial information part in each frame of wallpaper image in the dynamic wallpaper video based on the target angle adjustment path and the preset adjustment duration includes: determining the angle switching speed based on the target angle adjustment path and the preset adjustment duration; determining the angle information of the facial orientation of the facial information part in each frame of wallpaper image in the dynamic wallpaper video based on the angle switching speed.

[0160] In the embodiment of the present application, within the preset adjustment time, the face orientation angle of the preset wallpaper is adjusted from the initial angle to the target angle through the preset adjustment path. For target angle switching paths of unequal lengths, the preset adjustment time is equal. For target angle switching paths with longer lengths, the angle switching speed is faster; for target angle switching paths with shorter lengths, the angle switching speed is slower. The angle switching speed v satisfies Wherein, T is the preset adjustment time, and s is the path length of the target angle switching path.

[0161] After determining the angle switching speed, the angle change between adjacent frames can be determined based on the angle switching speed, and the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video can be determined based on the angle change between adjacent frames and the target angle switching path.

[0162] As a possible implementation, the angle switching speed is a constant value.

[0163] In the embodiment of the present application, in order to improve the efficiency of generating dynamic wallpaper videos, the angle switching speed is constant on a target angle switching path. The angle switching speed is the ratio of the length of the target angle switching path to the preset adjustment time.

[0164] As a possible implementation, the angle switching speed includes a first angle switching speed and a second angle switching speed, and the first angle switching speed is different from the second angle switching speed.

[0165] In an embodiment of the present application, in order to meet different requirements of dynamic wallpaper videos, the angle switching speed may be variable for a target angle switching path.

[0166] In some embodiments, during the first adjustment time, the angle switching speed is the first angle switching speed. During the second adjustment time, the angle switching speed is the second angle switching speed. During the third adjustment time, the angle switching speed slows down over time. Fig.17 As shown, the angle switching speed is

[0167] Wherein, b is the first angle switching speed, a is the second angle switching speed. The first adjustment time starts at 0 and ends at t1; the third adjustment time starts at t1 and ends at t2; the second adjustment time starts at t2 and ends at t3. t3 is the preset adjustment duration.

[0168] Step S3035: Generate a dynamic wallpaper video based on the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video.

[0169] The first frame wallpaper image of the dynamic wallpaper video is a preset wallpaper with the facial orientation of the facial information part being an initial angle, and the last frame wallpaper image of the dynamic wallpaper video is a target wallpaper.

[0170] In the embodiment of the present application, the angle information of the facial orientation of the facial information part in the dynamic wallpaper video switches from the initial angle to the target angle according to the target angle switching path. The first frame wallpaper image of the dynamic wallpaper is a preset wallpaper with the facial orientation of the facial information part as the initial angle. The angle information of the facial orientation of the facial information part in the last frame wallpaper image of the dynamic wallpaper video is the target angle information, that is, the last frame wallpaper image of the dynamic wallpaper video is the target wallpaper. In this way, in response to the personalized wallpaper triggering operation, a dynamic wallpaper video is generated and displayed. The dynamic wallpaper video shows the process of the angle information of the facial orientation of the facial information part in the preset wallpaper switching from the initial angle to the target angle. The facial orientation of the target wallpaper is parallel to the facial orientation of the user and opposite to the facial information part of the target wallpaper and the user's face.

[0171] As a possible implementation, Fig.18 As shown, based on the facial orientation angle information of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video, generating the dynamic wallpaper video includes:

[0172] Step S1801, extracting the first key point of the facial information part in the preset wallpaper.

[0173] In the embodiment of the present application, the first key point of the facial information part in the preset wallpaper is the key point of the facial information part in the preset wallpaper with the facial orientation being the initial angle. For details, please refer to step S901, which will not be described in detail here.

[0174] Step S1802: for the angle information of the facial orientation of the facial information part in each frame of wallpaper image in the dynamic wallpaper video, adjust the facial orientation of the first key point of the facial information part in the preset wallpaper according to the angle information of the facial orientation of the facial information part in the frame of wallpaper image in the dynamic wallpaper video, and obtain the second key point corresponding to the frame of wallpaper image.

[0175] In the embodiment of the present application, for each frame of wallpaper image in the dynamic wallpaper video, based on the angle information of the facial orientation of the facial information part in the frame of wallpaper image, the first key point of the facial information part in the preset wallpaper is adjusted to obtain the second key point corresponding to the frame of wallpaper image. For details, please refer to step S902, which will not be repeated here.

[0176] Step S1803: Generate a dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m frames of wallpaper images.

[0177] In an embodiment of the present application, the second key point corresponding to the m-frame wallpaper image can represent the facial information part of each frame wallpaper image in the dynamic wallpaper video. Therefore, a dynamic wallpaper video can be generated based on the preset wallpaper and the second key points corresponding to the m-frame wallpaper image.

[0178] As a possible implementation, Fig.19 and Fig. 20 As shown, based on the preset wallpaper and the second key points corresponding to the m-frame wallpaper images, generating a dynamic wallpaper video includes:

[0179] Step S1901: perform face regularization processing on the preset wallpaper to obtain the face regularization facial information part of the preset wallpaper.

[0180] In the embodiment of the present application, in order to ensure the consistency of the facial information part of the face input by the motion estimation model and improve the generation effect of the dynamic wallpaper video, before the dynamic wallpaper video is generated by the motion estimation model, the preset wallpaper needs to be first processed for face regularization. The specific steps can be referred to step S1301, which will not be repeated here.

[0181] Step S1902: Calculate the adjacent frame offsets between the second key points corresponding to the m frames of wallpaper images.

[0182] In an embodiment of the present application, the adjacent frame offsets between the second key points corresponding to the m frames of wallpaper images can represent changes in the facial information of the human face corresponding to the adjacent frames of wallpaper images in the dynamic wallpaper video.

[0183] In some embodiments, for each frame in the m frames of wallpaper image, the coordinates of the second key point corresponding to the frame can be subtracted from the second key point corresponding to the adjacent previous frame of wallpaper image to determine the adjacent frame offset between the second key points corresponding to the frame of wallpaper image.

[0184] Step S1903, based on the adjacent frame offsets between the second key points corresponding to the m frames of wallpaper images and the regularized facial information portion of the preset wallpaper, expression calculation is performed through a motion estimation model to obtain the regularized facial information portion of the face in the dynamic wallpaper video.

[0185] In an embodiment of the present application, the adjacent frame offsets between the second key points corresponding to the m-frame wallpaper images and the face regularization facial information portion of the preset wallpaper are used as inputs to the motion estimation model, and the output of the motion estimation model is the face regularization facial information portion in the dynamic wallpaper video. In some embodiments, the motion estimation model may use TPS Motion (Thin-Plate Spline Motion Model). Of course, other motion estimation models may also be used, and the present application does not limit this.

[0186] Step S1904: based on the facial information part of the regularized face processing in the dynamic wallpaper video, post-reply processing is performed to obtain the dynamic wallpaper video.

[0187] In an embodiment of the present application, the preset wallpaper includes not only a facial information portion but also a non-facial information portion. In order to ensure the consistency between the dynamic wallpaper video and the preset wallpaper, the facial information portion of the dynamic wallpaper video that has been processed by face regularization is posted back to the preset wallpaper to generate a dynamic wallpaper video. That is, the facial information portion of the dynamic wallpaper video that has been processed by face regularization is subjected to the inverse transformation of the affine transformation in step S1901, and the dynamic wallpaper video of the facial information portion after the inverse transformation is spliced ​​with the non-facial information portion of the preset wallpaper to obtain a complete dynamic wallpaper video. In some embodiments, in order to ensure the inter-frame stability of the dynamic wallpaper video, the dynamic wallpaper video may be subjected to inter-frame time domain smoothing filtering.

[0188] As a possible implementation manner, the facial information portion of n frames of wallpaper images in the dynamic wallpaper video is superimposed with interactive expression information; n is an integer greater than 0.

[0189] In the embodiment of the present application, in order to improve the user's interactive experience, interactive expressions are superimposed on a portion or the entirety of the dynamic wallpaper video. For example, the interactive expression may be a smile or a wink.

[0190] As a possible implementation, Fig.21 As shown, for the facial information part of n frames of wallpaper images in the dynamic wallpaper video, the interactive expression information is superimposed, and based on the facial orientation angle information of the facial information part in each frame of wallpaper image in the dynamic wallpaper video, generating the dynamic wallpaper video includes:

[0191] Step S2101, extracting the first key point of the facial information part in the preset wallpaper.

[0192] Please refer to step S1801 for details, which will not be repeated here.

[0193] Step S2102: for the angle information of the facial orientation of the facial information part in each frame of wallpaper image in the dynamic wallpaper video, adjust the facial orientation of the first key point of the facial information part in the preset wallpaper according to the angle information of the facial orientation of the facial information part in the frame of wallpaper image in the dynamic wallpaper video, and obtain the second key point corresponding to the frame of wallpaper image.

[0194] Please refer to step S1802 for details, which will not be repeated here.

[0195] Step S2103: determine the start time and end time of the interactive expression.

[0196] In the embodiment of the present application, the start time and end time of the interactive expression are the start time and end time of the interactive expression in the dynamic wallpaper video, that is, the start time and end time of the interactive expression superimposed on the dynamic wallpaper video. The start time and end time of the interactive expression can be default values, for example, the start time of the interactive expression is the start time of the dynamic wallpaper video, and the end time of the interactive expression is the end time of the dynamic wallpaper video. The start time and end time of the interactive expression can also be generated by adaptive judgment.

[0197] As a possible implementation method, determining the start time and end time of the interactive expression includes: obtaining the first time of the interactive expression; obtaining the length of the target angle adjustment path corresponding to the interactive expression; and determining the second time of the interactive expression according to the length of the target angle adjustment path corresponding to the interactive expression and the first time of the interactive expression. The first time of the interactive expression includes the start time or the end time of the interactive expression; and the second time of the interactive expression includes the end time or the start time of the interactive expression.

[0198] In an embodiment of the present application, the first time of an interactive expression may be a default value or may be set by a user. The length of the target angle adjustment path corresponding to the interactive expression is the angle change corresponding to the start time of the interactive expression to the end time of the interactive expression on the target adjustment path. The length of the target angle adjustment path corresponding to the interactive expression may be a default value or may be set by a user. For an interactive expression in which the first time is the start time of the interactive expression, the end time of the interactive expression is determined based on the length of the target angle adjustment path corresponding to the interactive expression and the start time of the interactive expression. For an interactive expression in which the first time is the end time of the interactive expression, the start time of the interactive expression is determined based on the length of the target angle adjustment path corresponding to the interactive expression and the end time of the interactive expression. For example, Fig. 22 As shown, the end time of the interactive expression is the end time of the dynamic wallpaper video, that is, the end time of the interactive expression is the preset adjustment time T. Assuming that the length of the target angle adjustment path corresponding to the interactive expression is s1, based on Determine the start time t4 of the interactive expression. Where v is the angle switching speed.

[0199] Step S2104: determine the frame number n of the interactive expression image according to the start time, end time and preset frame rate of the interactive expression, and obtain n frames of interactive expression images from the preset interactive expression images.

[0200] In an embodiment of the present application, the time length of the interactive expression is determined according to the start time and the end time of the interactive expression, and the number of frames n of the interactive expression image can be determined based on the time length of the interactive expression and the preset frame rate. For example, if the time length of the interactive expression is 1 second and the preset frame rate is 30 frames per second, the number of frames n of the interactive expression image is 30. 30 frames of interactive expression images are obtained from the preset interactive expression images.

[0201] In some embodiments, n frames of interactive expression images are uniformly extracted from preset interactive expression images.

[0202] Step S2105: Determine the angle information of the facial orientation of the facial information part in the n-frame wallpaper image within the target time period from the angle information of the facial orientation of the facial information part in the m-frame wallpaper image of the dynamic wallpaper video.

[0203] The target time period is the time period consisting of the start time and end time of the interactive expression.

[0204] In an embodiment of the present application, the facial orientation angle information of the facial information part in the n-frame wallpaper image within the target time period is the facial orientation angle information of the facial information part in the n-frame wallpaper image with the interactive expression information superimposed thereon. The target time period is determined according to the start time and the end time of the interactive expression. Among the m frames of the dynamic wallpaper video, n frames whose time is within the target time period are determined. The facial orientation angle information of the facial information part in these frames of wallpaper image is the facial orientation angle information of the facial information part in the n-frame wallpaper image within the target time period.

[0205] Step S2106, extract the third key point of the facial information part in the n-frame interactive expression image, and adjust the facial orientation of the third key point of the facial information part in the n-frame interactive expression image according to the angle information of the facial orientation of the facial information part in the n-frame wallpaper image corresponding to the target time period, to obtain the fourth key point corresponding to the n-frame interactive expression image.

[0206] In the embodiments of the present application, Fig.23As shown, for each frame in the n-frame interactive expression image, a key point of the facial information part is extracted, which is the third key point. The angle information corresponding to the frame is determined from the angle information of the facial orientation of the facial information part in the n-frame wallpaper image corresponding to the target time period, and the facial orientation of the third key point of the facial information part in the frame interactive expression image is adjusted according to the angle information corresponding to the frame to obtain the fourth key point corresponding to the frame interactive expression image.

[0207] Step S2107: Determine the second key points corresponding to the n frames of wallpaper images within the target time period from among the second key points corresponding to the m frames of wallpaper images.

[0208] In an embodiment of the present application, the target time period is determined according to the start time and the end time of the interactive expression. Among the m frames of the dynamic wallpaper video, n frames whose time is within the target time period are determined. The corresponding second key points in these frames of wallpaper image are the second key points corresponding to the n frames of wallpaper image within the target time period.

[0209] Step S2108: Determine the fifth key point corresponding to the n-frame wallpaper image with superimposed interactive expressions within the target time period based on the fourth key point corresponding to the n-frame interactive expression image and the second key point corresponding to the n-frame wallpaper image.

[0210] In the embodiments of the present application, Fig.23 As shown, the fourth key point corresponding to the n-frame interactive expression image is determined by adjusting the third key point corresponding to the n-frame interactive expression image according to the corresponding angle information. The second key point corresponding to the n-frame wallpaper image is determined by adjusting the first key point of the preset wallpaper according to the corresponding angle information. Therefore, based on the above fourth key point and second key point, the fifth key point corresponding to the n-frame wallpaper image with superimposed interactive expressions within the target time period can be determined.

[0211] In some embodiments, in order to ensure the smoothness and naturalness of the segments with superimposed interactive expressions in the dynamic wallpaper video, after determining the fifth key point corresponding to the wallpaper image of n frames with superimposed interactive expressions within the target time period, the optical flow method can be used to update the fifth key point to improve the smoothness of the facial information part between frames.

[0212] As a possible implementation method, based on the fourth key point corresponding to the n-frame interactive expression image, and the second key point corresponding to the n-frame wallpaper image, determining the fifth key point corresponding to the n-frame wallpaper image with superimposed interactive expressions within the target time period includes: determining the position offset of the corresponding key point between each adjacent two frames between the fourth key point corresponding to the n-frame interactive expression image; determining the fifth key point corresponding to the n-frame wallpaper image with superimposed interactive expressions within the target time period based on the position offset of the corresponding key point between each adjacent two frames between the fourth key point corresponding to the n-frame interactive expression image, and the second key point corresponding to the n-frame wallpaper image.

[0213] In an embodiment of the present application, the position offset of the corresponding key points between each two adjacent frames of the fourth key points corresponding to the n-frame interactive expression images can represent the change of the key points of the interactive expression images with n frames superimposed with corresponding angle information.

[0214] Step S2109: Generate a dynamic wallpaper video based on the preset wallpaper, the fifth key points corresponding to the wallpaper image of the n frames superimposed with interactive expressions within the target time period, and the second key points corresponding to other wallpaper images.

[0215] The second key points corresponding to other wallpaper images refer to the second key points corresponding to other frames of wallpaper images among the second key points corresponding to the m frames of wallpaper images except the second key points corresponding to the n frames of wallpaper images within the target time period.

[0216] In an embodiment of the present application, the fifth key point corresponding to the wallpaper image with interactive expressions superimposed on n frames within the target time period can represent the facial information portion in the n-frame wallpaper image with interactive expression information superimposed on it in the dynamic wallpaper video. The second key point corresponding to other wallpaper images can represent the facial information portion in other frames of wallpaper images in the dynamic wallpaper video except for the second key point corresponding to the n-frame wallpaper image within the target time period. In this way, the generated dynamic wallpaper video shows the process of switching the angle information of the facial orientation of the facial information portion in the preset wallpaper from the initial angle to the target angle, and the interactive expressions are superimposed within the target time period, further improving the interactivity.

[0217] Please refer to step S1803 for details, which will not be repeated here.

[0218] It should be noted that, in some embodiments, if the length s of the target angle adjustment path is less than the length s1 of the target angle adjustment path corresponding to the acquired interactive expression, the number of frames n of the interactive expression image is determined, and n frames of interactive expression images are obtained in the preset expression image, where n=m*s1 / s. A front interactive expression video is generated based on a preset wallpaper with a facial orientation at an initial angle, and the first nm frames of interactive expression images in the n frames of interactive expression images. A dynamic wallpaper video is generated based on the angle information and the second key point of the facial orientation of the m frames of wallpaper images in the dynamic wallpaper video, and the last m frames of interactive expression images in the n frames of interactive expression images.

[0219] In some embodiments, after displaying the target wallpaper, the user's expression can be determined based on the user image information, and based on the user's expression information and the target wallpaper, a wallpaper with a superimposed feedback expression is generated and displayed. The feedback expression can be the same as the user's expression. That is, the facial information part in the target wallpaper imitates the user's expression in real time. Alternatively, in response to the user's expression being a special expression, the feedback expression can be the corresponding target feedback expression.

[0220] In some embodiments, after the electronic device stops displaying the target wallpaper, the wallpaper of the electronic device is restored to a preset wallpaper with the facial orientation as the initial angle information.

[0221] In some embodiments, after the target wallpaper is displayed, the initial angle information of the face orientation of the face information part of the preset wallpaper may be updated to the target angle information, that is, the target angle information is determined as the updated initial angle information.

[0222] In some embodiments, because the expression calculation generation based on the motion estimation model is essentially based on GAN (Consistent Generative Adversarial Networks), it requires high computing power. In order to achieve real-time generation and real-time rendering, such as Fig.24 As shown, a solution of end-cloud collaboration can be adopted. Expression calculation and generation are performed in the cloud, and the dynamic wallpaper video generated in the cloud is sent to the corresponding electronic device, and rendered and presented on the corresponding electronic device, so as to achieve different interactive experiences according to the hardware of the electronic device itself. Electronic devices that can apply the method of this application include mobile phones, car computers, large screens, etc.

[0223] like Fig.24 As shown, for electronic devices such as smart watches and car computers, the mobile phone can send dynamic wallpaper videos to the corresponding electronic devices.

[0224] For example, Fig.25 As shown in Figure 1, the mobile phone obtains real-time video streams through the camera, and the sampling rate of user image information is 30 frames per second. Fig.26aAs shown, based on the user's facial image information, a shared feature extraction network is used to extract shared features, and the shared features obtained by the shared feature extraction network are used as inputs to the three sub-networks of face angle discrimination sub-network, face key point detection sub-network, and face smile binary classification sub-network to respectively determine the angle information of the user's facial orientation, the user's face key points, and whether the user is smiling. Figure 26b As shown in the figure, it is a schematic diagram of a shared feature extraction network. The input layer is the face image information, and the output layer is the shared feature. The shared feature extraction network can be the network structure and pre-trained model of Vgg-19, and the output feature layer parameter is 2048. The sub-network can use a fully connected layer network. Fig.26c As shown in , it is a schematic diagram of the face angle discrimination subnetwork. The number of neurons in the output layer of this subnetwork is 2, which represent the deflection angle and pitch angle values ​​respectively. Fig.26d As shown in , it is a schematic diagram of the facial key point detection sub-network, which is used to obtain the 68-point positioning of the user's face. The number of neurons in the output layer is 136, which are the coordinates of the user's 68 facial key points, covering the eyes, nose, mouth and facial contour. Fig.26e As shown, it is a schematic diagram of a face smile binary classification subnetwork, which is essentially a binary classification network, and the number of neurons in the output layer is 1, that is, whether the user is smiling. It should be noted that the shared features can be used as the input layer of any one of the above subnetworks, or as any two of the above subnetworks, or as the input layer of other subnetworks, and this application does not limit this. This example uses the shared features as the input of the three subnetworks: the face angle discrimination subnetwork, the face key point detection subnetwork, and the face smile binary classification subnetwork. Based on the face key point parameters, the relative coordinates of the left pupil in the left eye, the relative coordinates of the right pupil in the right eye, and the distance from the two eyes to the nose can be determined through preprocessing calculations. The above information is used as the input of the neural network, and the classification result of whether the user is looking at the electronic device is used as the output of the neural network to determine whether the user is looking at the electronic device. The neural network is a single hidden layer neural network.

[0225] In response to the user looking at the mobile phone and smiling, the angle information of the user's facial orientation is obtained. Assume that the angle information between the user's facial orientation and the first direction of the target plane is an elevation angle of 30°. The angle information between the user's facial orientation and the second direction of the target plane is a left deflection angle of 30°. The angle size of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane is the same as the angle size of the angle information between the user's facial orientation and the first direction of the target plane, that is, 30°; the angle direction of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane is opposite to the angle direction of the angle information between the user's facial orientation and the first direction of the target plane, that is, the depression angle. That is, the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane is a depression angle of 30°. The angle size of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane is the same as the angle size of the angle information between the facial orientation of the user and the second direction of the target plane, that is, 30°; the angle direction of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane is opposite to the angle direction of the angle information between the facial orientation of the user and the second direction of the target plane, that is, deflected to the right. That is, the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane is a rightward deflection angle of 30°.

[0226] Get the preset wallpaper, assume that the initial angle information of the face orientation of the preset wallpaper is 0° pitch angle and 0° deflection angle, and extract the first key point of the face information part in the preset wallpaper. Figure 6 The following is a schematic diagram of the preset wallpaper in this example.

[0227] like Fig. 27As shown, the horizontal axis represents the deflection angle information of the facial orientation of the facial information part in the preset wallpaper. A positive value of the horizontal axis represents that the angle direction of the deflection angle information is deflected to the right, and a negative value of the horizontal axis represents that the angle direction of the deflection angle information is deflected to the left. The vertical axis represents the pitch angle information of the facial orientation of the facial information part in the preset wallpaper. A positive value of the vertical axis represents that the angle direction of the pitch angle information is a depression angle, and a negative value of the vertical axis represents that the angle direction of the pitch angle information is an elevation angle. The starting position of the angle switching path is the initial angle (0,0), and the ending position of the angle switching path is (30,30). There are countless angle switching paths with a starting position of (0,0) and an ending position of (30,30), including angle adjustment paths r1, r2, r3, r4, ..., and one of them is randomly selected as the target angle adjustment path. Assuming that the target adjustment path is the angle adjustment path r4, the path length of the target angle switching path is determined by calculation to be s. Assuming that the preset adjustment time is T=3 seconds and the preset frame rate is 30 frames per second, the number of frames of the dynamic wallpaper video is determined to be 90. Assuming that the angle switching speed is a constant value, the angle switching speed v=s / T, such as Fig.28 As shown in FIG. 1 , a schematic diagram of the angle switching speed is shown. Based on the target adjustment path and the angle switching speed, the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video is determined. Fig.29 As shown in the figure, it is assumed that the angle information of the facial information part of each frame of the wallpaper image in the dynamic wallpaper video is (x0, y0), (x1, y1), (x2, y2), ..., (x 89 ,y 89 ). For the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video, the angle information of the facial orientation of the facial information part in the wallpaper image of the frame in the dynamic wallpaper video and the first key point of the facial information part in the preset wallpaper are used as the input of the neural network, and the second key point corresponding to the wallpaper image of the frame is used as the output of the neural network, thereby obtaining the second key points of 90 frames of wallpaper images.

[0228] The interactive expression is a smile and wink expression. The end time of the dynamic wallpaper video is 3 seconds, which is determined as the end time of the interactive expression. Assume that the length of the target angle adjustment path corresponding to the interactive expression is s1. Assume that based on The starting time t4 of the interactive expression is determined to be 2 seconds. Fig.28As shown. According to the start time and end time of the interactive expression, the length of the target time period is determined to be 1s, the preset frame rate is 30 frames per second, the number of frames of the interactive expression image is determined to be 30 frames, and 30 frames of smiling and winking expression images are obtained in the preset smiling and winking expression video template. From the angle information of the facial orientation of the facial information part in the 90 frames of wallpaper images of the dynamic wallpaper video, the angle information of the facial orientation of the facial information part in the last 30 frames of wallpaper images is determined. Extract the third key point of the facial information part in 30 frames of smiling and winking expression images, and adjust the facial orientation of the third key point of the facial information part in the 30 frames of smiling and winking expression images according to the angle information of the facial orientation of the facial information part in the last 30 frames of wallpaper images to obtain the fourth key point corresponding to the 30 frames of smiling and winking expression images; determine the second key point corresponding to the last 30 frames of wallpaper images among the second key points corresponding to the 90 frames of wallpaper images; determine the fifth key point corresponding to the last 30 frames of wallpaper images superimposed with smiling and winking expressions based on the fourth key point corresponding to the 30 frames of smiling and winking expression images and the second key point corresponding to the last 30 frames of wallpaper images; generate a dynamic wallpaper video based on the preset wallpaper, the fifth key point corresponding to the last 30 frames of wallpaper images superimposed with smiling and winking expressions, and the second key point corresponding to the first 60 frames of wallpaper images.

[0229] refer to Fig.30 , is a schematic diagram of the structure of a wallpaper interaction device provided in an embodiment of the present application. Fig.30 As shown, the device comprises:

[0230] An acquisition unit 3001 is used to acquire the angle information of the user's facial orientation in response to a personalized wallpaper triggering operation;

[0231] A determination unit 3002, configured to determine target angle information of the facial orientation of a preset wallpaper based on the angle information of the user's facial orientation;

[0232] The processing unit 3003 is used to obtain a preset wallpaper, and adjust the facial orientation of the facial information part in the preset wallpaper based on the target angle information of the facial orientation of the preset wallpaper to obtain a target wallpaper; the facial orientation of the facial information part in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information part in the target wallpaper is opposite to the face of the user;

[0233] The display unit 3004 is used to display the target wallpaper.

[0234] As a possible implementation method, the angle information of the user's facial orientation includes the angle information between the user's facial orientation and a first direction of the target plane, and the angle information between the user's facial orientation and a second direction of the target plane; wherein the first direction and the second direction are perpendicular to each other, and the target plane is parallel to the plane where the display screen is located.

[0235] As a possible implementation method, the determination unit 3002 is specifically used to determine the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane, and the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane based on the angle information between the user's facial orientation and the first direction of the target plane and the angle information between the user's facial orientation and the second direction of the target plane.

[0236] As a possible implementation method, the processing unit 3003 is specifically used to obtain a preset wallpaper and determine the initial angle information of the facial orientation of the preset wallpaper; based on the target angle information of the facial orientation of the preset wallpaper and the initial angle information of the facial orientation of the preset wallpaper, determine a target angle adjustment path in at least one corresponding angle adjustment path; the starting position of at least one corresponding angle adjustment path is the initial angle of the facial orientation of the preset wallpaper, and the ending position of the angle adjustment path is the target angle of the facial orientation of the preset wallpaper; obtain a preset adjustment duration and a preset frame rate, and determine the number of frames m of the dynamic wallpaper video; m is an integer greater than 1; based on the target angle adjustment path and the preset adjustment duration, determine the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video; wherein, there are at least two frames of wallpaper images in the dynamic wallpaper video in which the angle information of the facial orientation of the facial information part is different; based on the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video, generate a dynamic wallpaper video, wherein the first frame of the wallpaper image of the dynamic wallpaper video is the preset wallpaper with the facial orientation of the facial information part as the initial angle, and the last frame of the wallpaper image of the dynamic wallpaper video is the target wallpaper.

[0237] As a possible implementation method, the processing unit 3003 is specifically used to determine the angle switching speed based on the target angle adjustment path and the preset adjustment duration; based on the target angle adjustment path and the angle switching speed, determine the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video.

[0238] As a possible implementation manner, the angle switching speed includes a first angle switching speed and a second angle switching speed, and the first angle switching speed is different from the second angle switching speed.

[0239] As a possible implementation method, the processing unit 3003 is specifically used to extract the first key point of the facial information part in the preset wallpaper; for the angle information of the facial orientation of the facial information part in each frame of wallpaper image in the dynamic wallpaper video, according to the angle information of the facial orientation of the facial information part in the frame of wallpaper image in the dynamic wallpaper video, adjust the facial orientation of the first key point of the facial information part in the preset wallpaper to obtain the second key point corresponding to the frame of wallpaper image; based on the second key points corresponding to the preset wallpaper and m frames of wallpaper images, generate a dynamic wallpaper video.

[0240] As a possible implementation manner, the facial information portion of n frames of wallpaper images in the dynamic wallpaper video is superimposed with interactive expression information; n is an integer greater than 0.

[0241] As a possible implementation method, the processing unit 3003 is specifically used to determine the start time and end time of the interactive expression; determine the frame number n of the interactive expression image according to the start time, end time and preset frame rate of the interactive expression, and obtain n frames of interactive expression images in the preset interactive expression image; determine the angle information of the facial orientation of the facial information part in the n frames of wallpaper images within the target time period from the angle information of the facial orientation of the facial information part in the m frames of wallpaper images of the dynamic wallpaper video; wherein the target time period is a time period formed by the start time and end time of the interactive expression; extract the third key point of the facial information part in the n frames of interactive expression images, and adjust the facial orientation of the third key point of the facial information part in the n frames of interactive expression images according to the angle information of the facial orientation of the facial information part in the n frames of wallpaper images corresponding to the target time period, to obtain the fourth key point corresponding to the n frames of interactive expression images; determine the second key point corresponding to the n frames of wallpaper images within the target time period from the second key points corresponding to the m frames of wallpaper images; based on the fourth key point corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images, determine the fifth key point corresponding to the n frames of wallpaper images superimposed with interactive expressions within the target time period.

[0242] The processing unit 3003 is specifically used to generate a dynamic wallpaper video based on a preset wallpaper, the fifth key point corresponding to the wallpaper image of n frames superimposed with interactive expressions within a target time period, and the second key point corresponding to other wallpaper images; wherein the second key point corresponding to other wallpaper images refers to the second key point corresponding to the other frames of wallpaper images among the second key points corresponding to the m frames of wallpaper images except the second key point corresponding to the n frames of wallpaper images within the target time period.

[0243] As a possible implementation method, the processing unit 3003 is specifically used to obtain the first time of the interactive expression; obtain the length of the target angle adjustment path corresponding to the interactive expression; determine the second time of the interactive expression according to the length of the target angle adjustment path corresponding to the interactive expression and the first time of the interactive expression; the first time of the interactive expression includes the start time or end time of the interactive expression; the second time of the interactive expression includes the end time or start time of the interactive expression.

[0244] As a possible implementation method, the processing unit 3003 is specifically used to determine the position offset of the corresponding key point between each two adjacent frames between the fourth key points corresponding to the n frames of interactive expression images; based on the position offset of the corresponding key point between each two adjacent frames between the fourth key points corresponding to the n frames of interactive expression images, and the second key point corresponding to the n frames of wallpaper image, determine the fifth key point corresponding to the n frames of wallpaper image superimposed with interactive expressions within the target time period.

[0245] As a possible implementation manner, the acquisition unit 3001 is specifically configured to acquire the angle information of the user's facial orientation in response to the user gazing at the electronic device.

[0246] Corresponding to the above embodiments, the present application also provides an electronic device. Fig.31 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, the electronic device 3100 may include: a processor 3101, a memory 3102 and a communication unit 3103. These components communicate through one or more buses, and those skilled in the art can understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiment of the present invention, it can be a bus structure or a star structure, and can also include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0247] The communication unit 3103 is used to establish a communication channel so that the electronic device can communicate with other devices, receive user data sent by other devices or send user data to other devices.

[0248] The processor 3101 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It runs or executes software programs and / or modules stored in the memory 3102, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 3101 can only include a central processing unit (CPU). In an embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0249] The memory 3102 is used to store the execution instructions of the processor 3101. The memory 3102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0250] When the execution instructions in the memory 3102 are executed by the processor 3101, the electronic device 3100 can execute Figure 3 Some or all of the steps in the illustrated embodiments.

[0251] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the wallpaper interaction method provided by the present invention. The storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0252] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.

[0253] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the device embodiment and the terminal embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

Claims

1. A wallpaper interaction method, characterized in that: Applied to electronic equipment, the method comprises: In response to the personalized wallpaper triggering operation, obtaining angle information of the user's facial orientation; Determining target angle information of the facial orientation of the preset wallpaper based on the angle information of the user's facial orientation; Acquire a preset wallpaper, and adjust the facial orientation of the facial information part of the preset wallpaper based on the target angle information of the facial orientation of the preset wallpaper to acquire a target wallpaper; the facial orientation of the facial information part of the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information part of the target wallpaper is opposite to the face of the user; The target wallpaper is displayed.

2. The method according to claim 1, characterized in that The angle information of the user's facial orientation includes the angle information between the user's facial orientation and a first direction of a target plane, and the angle information between the user's facial orientation and a second direction of the target plane; wherein the first direction and the second direction are perpendicular to each other, and the target plane is parallel to the plane where the display screen is located.

3. The method according to claim 2, characterized in that The determining of target angle information of the facial orientation of the preset wallpaper based on the angle information of the user's facial orientation includes: Based on the angle information between the user's facial orientation and the first direction of the target plane and the angle information between the user's facial orientation and the second direction of the target plane, the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane and the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane are determined.

4. The method according to claim 3, characterized in that The step of obtaining the preset wallpaper and adjusting the angle of the facial orientation of the facial information portion of the preset wallpaper based on the target angle information of the facial orientation of the preset wallpaper, wherein obtaining the target wallpaper comprises: Obtain a preset wallpaper and determine the initial angle information of the face orientation of the preset wallpaper; Based on the target angle information of the face orientation of the preset wallpaper and the initial angle information of the face orientation of the preset wallpaper, a target angle adjustment path is determined in at least one corresponding angle adjustment path; the starting position of the at least one corresponding angle adjustment path is the initial angle of the face orientation of the preset wallpaper, and the ending position of the angle adjustment path is the target angle of the face orientation of the preset wallpaper; Get the preset adjustment duration and the preset frame rate, and determine the frame number m of the dynamic wallpaper video; m is an integer greater than 1; Based on the target angle adjustment path and the preset adjustment duration, determining the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video; wherein the facial orientation angle information of the facial information part in at least two frames of the wallpaper image in the dynamic wallpaper video is different; Based on the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video, a dynamic wallpaper video is generated, wherein the first frame of the wallpaper image of the dynamic wallpaper video is a preset wallpaper with the facial orientation of the facial information part being an initial angle, and the last frame of the wallpaper image of the dynamic wallpaper video is the target wallpaper.

5. The method according to claim 4, characterized in that The step of determining the angle information of the facial orientation of the facial information portion in each frame of the wallpaper image in the dynamic wallpaper video based on the target angle adjustment path and the preset adjustment duration includes: Determining an angle switching speed based on the target angle adjustment path and a preset adjustment duration; Based on the target angle adjustment path and the angle switching speed, the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video is determined.

6. The method according to claim 5, characterized in that The angle switching speed includes a first angle switching speed and a second angle switching speed, and the first angle switching speed is different from the second angle switching speed.

7. The method according to any one of claims 4 to 6, characterized in that: The generating of the dynamic wallpaper video based on the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video comprises: Extracting a first key point of a facial information portion in the preset wallpaper; For the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the dynamic wallpaper video, adjust the facial orientation of the first key point of the facial information part in the preset wallpaper according to the angle information of the facial orientation of the facial information part in the frame of the wallpaper image in the dynamic wallpaper video, and obtain a second key point corresponding to the frame of the wallpaper image; Based on the preset wallpaper and the second key points corresponding to the m frames of wallpaper images, a dynamic wallpaper video is generated.

8. The method according to claim 7, characterized in that The facial information portion of n frames of wallpaper images in the dynamic wallpaper video is superimposed with interactive expression information; n is an integer greater than 0.

9. The method according to claim 8, characterized in that Before generating a dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m-frame wallpaper images, the method further includes: Determine the start and end time of the interactive expression; According to the start time, end time and preset frame rate of the interactive expression, determine the frame number n of the interactive expression image, and obtain n frames of interactive expression image from the preset interactive expression image; Determine the facial orientation angle information of the facial information part in the n-frame wallpaper image within the target time period from the facial orientation angle information of the facial information part in the m-frame wallpaper image of the dynamic wallpaper video; wherein the target time period is the time period consisting of the start time and the end time of the interactive expression; Extracting the third key point of the facial information part in the n-frame interactive expression image, and adjusting the facial orientation of the third key point of the facial information part in the n-frame interactive expression image according to the facial orientation angle information of the facial information part in the n-frame wallpaper image corresponding to the target time period, to obtain the fourth key point corresponding to the n-frame interactive expression image; Determining the second key points corresponding to the n frames of wallpaper images within the target time period among the second key points corresponding to the m frames of wallpaper images; Based on the fourth key point corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images, determine the fifth key point corresponding to the n frames of wallpaper images superimposed with interactive expressions within the target time period; The generating a dynamic wallpaper video based on the preset wallpaper and the second key point corresponding to the m-frame wallpaper image comprises: A dynamic wallpaper video is generated based on the preset wallpaper, the fifth key points corresponding to the wallpaper images of n frames superimposed with interactive expressions within the target time period, and the second key points corresponding to other wallpaper images; wherein the second key points corresponding to other wallpaper images refer to the second key points corresponding to the m frames of wallpaper images, except the second key points corresponding to the n frames of wallpaper images within the target time period.

10. The method according to claim 9, characterized in that Determining the start time and end time of the interactive expression includes: Get the first-hand interactive expressions; Get the target angle adjustment path length corresponding to the interactive expression; The length of the path is adjusted according to the target angle corresponding to the interactive expression and the first time of the interactive expression to determine the second time of the interactive expression; the first time of the interactive expression includes the start time or the end time of the interactive expression; the second time of the interactive expression includes the end time or the start time of the interactive expression.

11. The method according to claim 9 or 10, characterized in that: The determining, based on the fourth key point corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images, the fifth key point corresponding to the n frames of wallpaper images superimposed with interactive expressions within the target time period comprises: Based on the fourth key points corresponding to the n frames of interactive expression images, determining the position offset of the corresponding key points between each two adjacent frames between the fourth key points corresponding to the n frames of interactive expression images; Based on the position offset of the corresponding key points between each two adjacent frames between the fourth key points corresponding to the n frames of interactive expression images, and the second key points corresponding to the n frames of wallpaper images, determine the fifth key points corresponding to the n frames of wallpaper images with superimposed interactive expressions within the target time period.

12. The method according to any one of claims 1 to 11, characterized in that: In response to the personalized wallpaper triggering operation, obtaining the angle information of the user's facial orientation includes: In response to the user looking at the electronic device, angle information of the user's facial orientation is obtained.

13. An interactive device, characterized in that: include: An acquisition unit, configured to acquire angle information of the user's facial orientation in response to a personalized wallpaper trigger operation; A determination unit, configured to determine target angle information of the facial orientation of a preset wallpaper based on the angle information of the user's facial orientation; a processing unit, configured to obtain a preset wallpaper, and adjust the facial orientation of a facial information portion in the preset wallpaper based on target angle information of the facial orientation of the preset wallpaper to obtain a target wallpaper; the facial orientation of the facial information portion in the target wallpaper is parallel to the facial orientation of the user, and the face of the facial information portion in the target wallpaper is opposite to the face of the user; The display unit is used to display the target wallpaper.

14. An electronic device, characterized in that: The device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 12.

Images