A wallpaper interaction method, device, equipment and storage medium

Abstract

Embodiments of the present application provide a wallpaper interaction method and device, equipment and storage medium, the method comprises: in response to a personalized wallpaper triggering operation, obtaining angle information of the face orientation of the user; based on the angle information of the face orientation of the user, determining the target angle information of the face orientation of the preset wallpaper; obtaining the preset wallpaper, and adjusting the angle of the face orientation of the face information part in the preset wallpaper based on the target angle information of the face orientation of the preset wallpaper, obtaining the target wallpaper; the face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face of the face information part in the target wallpaper is opposite to the face of the user; display the target wallpaper. The method improves the interactivity of the wallpaper and provides personalized interactive experience for the user.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent terminals, in particular to a wallpaper interaction method and device, equipment and a storage medium. BACKGROUND

[0002] With the rapid development of mobile Internet technology, terminal devices such as mobile phones, watches and car machines have become essential tools in users' daily life. The wallpaper of a terminal device is the lock screen background picture or the desktop background picture of the terminal device, including static wallpaper and dynamic wallpaper. In some technologies, a static image can be set as the static wallpaper of a terminal device, which can meet the personalized needs of users, but the static wallpaper lacks dynamic and interactive feelings. In some technologies, a recorded video can be set as the dynamic wallpaper of a terminal device, which has a lively feeling but cannot provide an interactive experience. In some technologies, a dynamic expression video can be generated based on an image containing a face and an expression video template, and the video can be set as the dynamic wallpaper of a terminal device, which can meet the personalized needs of users, but still cannot provide an interactive experience.

[0003] In some technologies, a dynamic wallpaper can change in response to trigger information to provide an interactive experience for users. For example, according to the trigger information, when it is determined that the user gazes at the screen, an animation character in the wallpaper makes a turning head action. However, these changes depend on a pre-set fixed template and have insufficient interactivity. For example, the turning head action of the animation character in the above response is a fixed template, and each time it is determined that the user gazes at the screen, the turning head action of the animation character in the dynamic wallpaper is the same, i.e., the same change occurs in the dynamic wallpaper. SUMMARY

[0004] Therefore, the present application provides a wallpaper interaction method, device, equipment and storage medium to solve the problem that the interactive dynamic wallpaper in the prior art depends on a pre-set fixed template and has insufficient interactivity.

[0005] In a first aspect, an embodiment of the present application provides a wallpaper interaction method, which comprises:

[0006] In response to a personalized wallpaper trigger operation, obtaining angle information of a face orientation of a user;

[0007] Based on the angle information of the face orientation of the user, determining target angle information of a face orientation of a preset wallpaper;

[0008] obtaining a preset wallpaper, performing angle adjustment on a face orientation of a face information part in the preset wallpaper based on target angle information of the face orientation of the preset wallpaper, and obtaining a target wallpaper; the face orientation of the face information part in the target wallpaper is parallel to a face orientation of the user, and a face of the face information part in the target wallpaper faces the face of the user;

[0009] displaying the target wallpaper.

[0010] In a possible implementation manner of the first aspect, the angle information of the face orientation of the user includes angle information between the face orientation of the user and a first direction of a target plane and angle information between the face orientation of the user and a second direction of the target plane; the first direction is perpendicular to the second direction, and the target plane is parallel to a plane on which the display screen is located.

[0011] In a possible implementation manner of the first aspect, the determining the target angle information of the face orientation of the preset wallpaper based on the angle information of the face orientation of the user includes:

[0012] determining target angle information between the face orientation of the face information part in the preset wallpaper and the first direction of the target plane and target angle information between the face orientation of the face information part in the preset wallpaper and the second direction of the target plane based on the angle information between the face orientation of the user and the first direction of the target plane and the angle information between the face orientation of the user and the second direction of the target plane.

[0013] In a possible implementation manner of the first aspect, the obtaining the target wallpaper by performing angle adjustment on the face orientation of the face information part in the preset wallpaper based on the target angle information of the face orientation of the preset wallpaper includes:

[0014] obtaining the preset wallpaper, and determining initial angle information of the face orientation of the preset wallpaper;

[0015] determining a target angle adjustment path in at least one corresponding angle adjustment path based on the target angle information of the face orientation of the preset wallpaper and the initial angle information of the face orientation of the preset wallpaper; a starting position of the at least one corresponding angle adjustment path is the initial angle of the face orientation of the preset wallpaper, and an ending position of the angle adjustment path is the target angle of the face orientation of the preset wallpaper.

[0016] obtaining a preset adjustment time length and a preset frame rate, and determining a frame number m of the dynamic wallpaper video; m is an integer greater than 1.

[0017] determine angle information of a face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video based on the target angle adjustment path and the preset adjustment time length, wherein the angle information of the face orientation of the face information part in at least two frames of the wallpaper image in the dynamic wallpaper video is different.

[0018] generate the dynamic wallpaper video based on the angle information of the face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video, wherein the first frame of the wallpaper image in the dynamic wallpaper video is the preset wallpaper with the face orientation of the face information part being the initial angle, and the last frame of the wallpaper image in the dynamic wallpaper video is the target wallpaper.

[0019] In a possible implementation manner of the first aspect, the determining the angle information of the face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video based on the target angle adjustment path and the preset adjustment time length comprises:

[0020] determine an angle switching speed based on the target angle adjustment path and the preset adjustment time length.

[0021] determine the angle information of the face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video based on the target angle adjustment path and the angle switching speed.

[0022] In a possible implementation manner of the first aspect, the angle switching speed comprises a first angle switching speed and a second angle switching speed, and the first angle switching speed is different from the second angle switching speed.

[0023] In a possible implementation manner of the first aspect, the generating the dynamic wallpaper video based on the angle information of the face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video comprises:

[0024] extract a first key point of the face information part in the preset wallpaper;

[0025] for the angle information of the face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video, adjust the face orientation of the first key point of the face information part in the preset wallpaper according to the angle information of the face orientation of the face information part in the frame of the wallpaper image in the dynamic wallpaper video, to obtain a second key point corresponding to the frame of the wallpaper image;

[0026] generate the dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m frames of the wallpaper image.

[0027] In a possible implementation manner of the first aspect, the face information part of the n frames of the wallpaper image in the dynamic wallpaper video is superimposed with interactive expression information; n is an integer greater than 0.

[0028] In a possible implementation manner of the first aspect, before the generating the dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m frame wallpaper images, the method further includes:

[0029] determining a start time and an end time of the interactive expression;

[0030] determining a frame number n of the interactive expression image according to the start time, the end time of the interactive expression and a preset frame rate, and obtaining n frame interactive expression images in the preset interactive expression image;

[0031] determining angle information of a face orientation of a face information part in the n frame wallpaper images in the target time period from the angle information of the face orientation of the face information part in the m frame wallpaper images of the dynamic wallpaper video; wherein the target time period is a time period formed by the start time and the end time of the interactive expression;

[0032] extracting third key points of the face information part in the n frame interactive expression images, and adjusting the face orientation of the third key points of the face information part in the n frame interactive expression images according to the angle information of the face orientation of the face information part in the n frame wallpaper images corresponding to the target time period, to obtain fourth key points corresponding to the n frame interactive expression images;

[0033] determining the second key points corresponding to the n frame wallpaper images in the target time period from the second key points corresponding to the m frame wallpaper images;

[0034] determining fifth key points corresponding to the n frame wallpaper images with the interactive expression superimposed in the target time period based on the fourth key points corresponding to the n frame interactive expression images and the second key points corresponding to the n frame wallpaper images;

[0035] the generating the dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m frame wallpaper images includes:

[0036] generating the dynamic wallpaper video based on the preset wallpaper, the fifth key points corresponding to the n frame wallpaper images with the interactive expression superimposed in the target time period and the second key points corresponding to other wallpaper images; wherein the second key points corresponding to other wallpaper images refer to the second key points corresponding to other frame wallpaper images except the second key points corresponding to the n frame wallpaper images in the target time period from the second key points corresponding to the m frame wallpaper images.

[0037] In a possible implementation manner of the first aspect, the determining the start time and the end time of the interactive expression includes: obtaining a first time of the interactive expression; obtaining a length of a target angle adjustment path corresponding to the interactive expression; and determining a second time of the interactive expression according to the length of the target angle adjustment path corresponding to the interactive expression and the first time of the interactive expression; the first time of the interactive expression includes the start time or the end time of the interactive expression; and the second time of the interactive expression includes the end time or the start time of the interactive expression.

[0038] In a possible implementation manner of the first aspect, the determining the fifth key point of the wallpaper image with the interactive expression superimposed in the target time period based on the fourth key point corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images includes:

[0039] determining a position offset of the corresponding key point between each adjacent two frames of the fourth key point corresponding to the n frames of interactive expression images based on the fourth key point corresponding to the n frames of interactive expression images;

[0040] determining the fifth key point of the wallpaper image with the interactive expression superimposed in the target time period based on the position offset of the corresponding key point between each adjacent two frames of the fourth key point corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images.

[0041] In a possible implementation manner of the first aspect, the obtaining the angle information of the face orientation of the user in response to the personalized wallpaper triggering operation includes:

[0042] obtaining the angle information of the face orientation of the user in response to the user gazing at the electronic device.

[0043] In the second aspect, the embodiments of the present application provide a wallpaper interaction device, including:

[0044] an obtaining unit configured to obtain angle information of a face orientation of a user in response to a personalized wallpaper triggering operation;

[0045] a determining unit configured to determine target angle information of a face orientation of a preset wallpaper based on the angle information of the face orientation of the user;

[0046] a processing unit configured to obtain the preset wallpaper, perform angle adjustment on a face orientation of a face information part in the preset wallpaper based on the target angle information of the face orientation of the preset wallpaper, and obtain a target wallpaper; the face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and a face in the face information part in the target wallpaper faces the face of the user;

[0047] a display unit configured to display the target wallpaper.

[0048] In a third aspect, an electronic device is provided, including a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to perform the method of any one of the first aspect.

[0049] In a fourth aspect, a computer readable storage medium is provided, including a stored program, wherein when the program is running, the device where the computer readable storage medium is located is controlled to perform the method of any one of the first aspect.

[0050] According to the scheme provided in the embodiments of the present application, in response to a personalized wallpaper triggering operation, angle information of a face orientation of a user is acquired; based on the angle information of the face orientation of the user, target angle information of a face orientation of a preset wallpaper is determined; the preset wallpaper is acquired, the face orientation of a face information part in the preset wallpaper is adjusted in angle based on the target angle information of the face orientation of the face information part in the preset wallpaper, a target wallpaper is acquired; the face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face in the face information part in the target wallpaper faces the face of the user; and the target wallpaper is displayed. In the embodiments of the present application, when a personalized wallpaper triggering operation is detected, a target wallpaper is generated and displayed based on the angle information of the face orientation of the user and a preset wallpaper. The face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face in the face information part in the target wallpaper faces the face of the user, so that the face information part in the target wallpaper looks at the face of the user. In this way, the interactivity of the wallpaper is improved, and the target wallpaper is generated based on the angle information of the face orientation of the user and does not rely on a preset fixed template. For different angle information of the face orientation of the user, the generated target wallpaper is different, and personalized interactive experience is provided for the user. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 A flowchart of a wallpaper interaction method provided by the embodiments of the present application;

[0053] Figure 2 A flowchart of another wallpaper interaction method provided by the embodiments of the present application;

[0054] Figure 3Another flowchart of a wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 6;

[0055] Figure 4 A scene diagram of a wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 7;

[0056] Figure 5 A scene diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 8;

[0057] Figure 6 A scene diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 9;

[0058] Figure 7 A scene diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 10;

[0059] Figure 8 A scene diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 11;

[0060] Figure 9 A flowchart of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 12;

[0061] Figure 10 A scene diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 13;

[0062] Figure 11 A scene diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 14;

[0063] Figure 12 A scene diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 15;

[0064] Figure 13 A flowchart of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 16;

[0065] Figure 14 A scene diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 17;

[0066] Figure 15 A flowchart of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 18;

[0067] Figure 16 A scene diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 19;

[0068] Figure 17 A scene diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 20;

[0069] Figure 18 A flowchart of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 21;

[0070] Figure 19 A flowchart of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 6;

[0071] Figure 20 A scenario diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 7;

[0072] Figure 21 A flowchart of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 8;

[0073] Figure 22 A scenario diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 9;

[0074] Figure 23 A scenario diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 10;

[0075] Figure 24 A scenario diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 11;

[0076] Figure 25 A scenario diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 12;

[0077] Figures 26a-26e A scenario diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 13;

[0078] Figure 27 A scenario diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 14;

[0079] Figure 28 A scenario diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 15;

[0080] Figure 29 A scenario diagram of another wallpaper interaction method provided by an embodiment of the present application is shown in FIG. 16;

[0081] Figure 30 A structural diagram of a wallpaper interaction device provided by an embodiment of the present application is shown in FIG. 17;

[0082] Figure 31 A structural diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 18. DETAILED DESCRIPTION

[0083] In order to better understand the technical solutions of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0084] It should be noted that the described embodiments are merely some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0085] The terms used in the embodiments of the present application are merely for the purpose of describing the specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0086] It should be understood that the term "and / or" used herein is merely to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are in an "or" relationship.

[0087] With the rapid development of mobile Internet technology, mobile phones, watches, car machines and other terminal devices have become essential tools in users' daily life. The wallpaper of a terminal device is the lock screen background picture or the desktop background picture of the terminal device, including static wallpaper and dynamic wallpaper. In some technologies, a static image can be set as a static wallpaper of a terminal device, which can meet the personalized needs of users, but the static wallpaper lacks dynamic and interactive sense. As shown in Figure 1 For a specific scene, a professional UX (User experience) designer creates a wallpaper or a user optimizes the picture in the mobile phone gallery to generate his own electronic device wallpaper, which is widely used in the built-in wallpaper library of mobile phones, watch face market and other scenes. The watch face wallpaper selected by the mobile phone is sent to other terminal devices (such as watches) for rendering and presentation. The applicable devices include but are not limited to smart watches, sports watches, mobile phone AOD (always on display, screen display / always on display), mobile phone small external screen, mobile phone wallpaper, car machine wallpaper, etc. This technology is mainly used in the current electronic device wallpaper system, which relies on the design of professional wallpaper designers or the user's custom wallpaper, and cannot meet the user's personalized, diverse and interactive needs.

[0088] In some technologies, a recorded video can be set as a dynamic wallpaper of a terminal device, which has a lively sense, but cannot provide an interactive experience.

[0089] Some technologies can generate a dynamic facial expression video based on an image containing a face and an expression video template, and set this video as a live wallpaper on a terminal device. This can meet users' personalized needs, but it still cannot provide an interactive experience. Computational facial expression technology based on generative adversarial networks allows users to simply upload an image containing a face, and a dynamic facial expression video is automatically generated according to the expression changes in the facial expression video template. This technology is widely used in video editing, wallpaper generation, and other scenarios. Figure 2 As shown, the wallpaper generation device in this technology includes: a portrait acquisition and content understanding unit 201, used to acquire user-uploaded images and perform content understanding on the portraits in the images, including detection, segmentation, and key feature point acquisition; an expression template creation unit 202, used to design corresponding watch face expression actions based on public preferences and record and create or user-defined expression video templates; an expression calculation and generation unit 203, which completes expression transfer actions with high quality by learning the differences in facial features between uploaded images and expression templates in each frame, as well as the mapping relationship between facial feature differences and image expression changes; a device perception and interaction unit 204, used to acquire device interaction information, including but not limited to facial angle information collected by the mobile phone camera, interaction information from tapping the screen on the watch, gyroscope information, crown rotation information, etc., depending on the device usage scenario; and a transmission, rendering, and display unit 205, used to send the micro-motion expression video generated in the cloud to the terminal device for rendering and presentation, including but not limited to smartwatches, sports watches, mobile phone AOD, mobile phone small external screens, mobile phone wallpapers, and car wallpapers. This technology is based on facial expression transfer and can only generate video wallpapers that are completely identical to the facial expressions of the preset video templates. In essence, it cannot provide users with a rich and diverse interactive experience.

[0090] In some technologies, live wallpapers can respond to trigger information and change accordingly, providing an interactive experience for users. For example, based on the trigger information, when it is determined that the user is looking at the screen, the anime character in the wallpaper turns its head. However, these changes rely on preset fixed templates, resulting in insufficient interactivity. For instance, in the above response, the anime character's head-turning action is a fixed template; every time it is determined that the user is looking at the screen, the anime character's head-turning action in the live wallpaper is the same, meaning the live wallpaper undergoes the same change.

[0091] To solve the above problems, the embodiment of the present application provides a wallpaper interaction method, device and equipment and storage medium. In response to a personalized wallpaper triggering operation, angle information of a face orientation of a user is acquired. Based on the angle information of the face orientation of the user, target angle information of a face orientation of a preset wallpaper is determined. The preset wallpaper is acquired, and the face orientation of a face information part in the preset wallpaper is adjusted in angle based on the target angle information of the face orientation of the face information part in the preset wallpaper, to acquire a target wallpaper. The face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face of the face information part in the target wallpaper faces the face of the user. The target wallpaper is displayed. In the embodiment of the present application, when the personalized wallpaper triggering operation is detected, the target wallpaper is generated and displayed based on the angle information of the face orientation of the user and the preset wallpaper. The face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face of the face information part in the target wallpaper faces the face of the user, so that the face information part in the target wallpaper is in eye contact with the face of the user. In this way, the wallpaper interactivity is improved, and the target wallpaper is generated based on the angle information of the face orientation of the user and does not depend on a preset fixed template. For different angle information of the face orientation of the user, the generated target wallpaper is different, and personalized interaction experience is provided for the user. The following will be described in detail.

[0092] Referring to Figure 3 A flowchart of a wallpaper interaction method provided by the embodiment of the present application is shown. The method is applied to an electronic device, such as a mobile phone. Figure 3 As shown in the figure, the method comprises the following steps.

[0093] In step S301, in response to a personalized wallpaper triggering operation, angle information of a face orientation of a user is acquired.

[0094] In the embodiment of the present application, the personalized wallpaper triggering operation can trigger the generation and display of the personalized wallpaper. For different angle information of the face orientation of the user, the generated personalized wallpaper is different, that is, the personalized wallpaper is generated based on the angle information of the face orientation of the user, and therefore, the angle information of the face orientation of the user needs to be acquired.

[0095] In some embodiments, the personalized wallpaper triggering operation can be that the user gazes at the electronic device, can be that the user makes a special expression such as smiling, can be that the display screen is woken up, and other triggering operations, which are not limited by the present application.

[0096] In some embodiments, the angle information of the face orientation of the user can be determined based on face image information of the user. The face image information of the user can be collected by a camera of the electronic device.

[0097] Alternatively, in some embodiments, the angle information of the face orientation of the user can be determined based on sensor information such as gyroscope information.

[0098] As a possible implementation manner, the angle information of the face orientation of the user includes angle information between the face orientation of the user and a first direction of a target plane, and angle information between the face orientation of the user and a second direction of the target plane; the first direction is perpendicular to the second direction, and the target plane is parallel to a plane where the display screen is located.

[0099] In the embodiment of the present application, the angle information between the face orientation of the user and the first direction of the target plane, and the angle information between the face orientation of the user and the second direction of the target plane can represent angle information between the face orientation of the user and a plane where the display screen is located.

[0100] The angle information between the face orientation of the user and the first direction of the target plane includes an angle size and an angle direction between the face orientation of the user and the first direction of the target plane; and the angle information between the face orientation of the user and the second direction of the target plane includes an angle size and an angle direction between the face orientation of the user and the second direction of the target plane.

[0101] For example, the first direction can be a direction parallel to a direction in which data lines in the display screen are arranged, and the second direction can be a direction parallel to a direction in which scan lines in the display screen are arranged, as shown in FIG. 1. Figure 4 For the convenience of understanding, the embodiment of the present application takes the first direction as the direction parallel to the direction in which the data lines in the display screen are arranged, and the second direction as the direction parallel to the direction in which the scan lines in the display screen are arranged as an example for description. That is, the angle information between the face orientation of the user and the first direction of the target plane is the pitch angle information of the user looking at the display screen, and the angle information between the face orientation of the user and the second direction of the target plane is the yaw angle information of the user looking at the display screen. For the convenience of understanding, as shown in FIG. 1, point a1 is on the left of point a2, and point a2 is on the right of point a1; point b1 is on the top of point b2, and point b2 is on the bottom of point b1. Figure 4

[0102] For example, as shown in FIG. 1, the face orientation of the user is a ray D, the angle information between the face orientation of the user and the first direction of the target plane is an elevation angle of 30°, and the angle information between the face orientation of the user and the second direction of the target plane is a left yaw angle of 30°. Figure 5 As a possible implementation manner, in response to the personalized wallpaper triggering operation, the angle information of the face orientation of the user includes: in response to the user gazing at the electronic device, the angle information of the face orientation of the user is acquired.

[0103] In the embodiment of the present application, in order to improve the interactivity of the personalized wallpaper, when it is detected that the user gazes at the electronic device, the generation and display of the personalized wallpaper are triggered.

[0104] In the embodiment of the present application, in order to improve the interactivity of the personalized wallpaper, when it is detected that the user gazes at the electronic device, the generation and display of the personalized wallpaper are triggered.​

[0105] As a possible implementation manner, in response to the individual wallpaper triggering operation, the obtaining of the angle information of the face orientation of the user comprises: in response to the user staring at the electronic device and smiling, the angle information of the face orientation of the user is obtained.

[0106] In the embodiment of the present application, the generation and display of the target wallpaper can be triggered when it is detected that the user stares at the electronic screen and smiles.

[0107] In step S302, the target angle information of the face orientation of the preset wallpaper is determined based on the angle information of the face orientation of the user.

[0108] In the embodiment of the present application, in order to make the face orientation of the face information part in the target wallpaper parallel to the face orientation of the user and the face in the face information part of the target wallpaper opposite to the face of the user, the target angle information of the face orientation of the preset wallpaper is determined based on the angle information of the face orientation of the user, that is, the angle information of the adjusted face orientation of the target wallpaper is determined, so that the face orientation of the adjusted face orientation in the wallpaper is parallel to the face orientation of the user and opposite to the face of the user.

[0109] As a possible implementation manner, the determination of the target angle information of the face orientation of the preset wallpaper based on the angle information of the face orientation of the user comprises: based on the angle information between the face orientation of the user and the first direction of the target plane and the angle information between the face orientation of the user and the second direction of the target plane, the target angle information between the face orientation of the face information part in the preset wallpaper and the first direction of the target plane is determined, and the target angle information between the face orientation of the face information part in the preset wallpaper and the second direction of the target plane is determined.

[0110] In the embodiment of the present application, the target angle information between the face orientation of the face information part in the preset wallpaper and the first direction of the target plane can be determined based on the angle information between the face orientation of the user and the first direction of the target plane, and the target angle information between the face orientation of the face information part in the preset wallpaper and the second direction of the target plane can be determined based on the angle information between the face orientation of the user and the second direction of the target plane.

[0111] In some embodiments, in order to make the face direction of the face information part in the target wallpaper parallel to the face direction of the user, and the face of the face information part in the target wallpaper opposite to the face of the user, the angle size of the target angle information between the face direction of the face information part in the preset wallpaper and the first direction of the target plane is the same as the angle size of the angle information between the face direction of the user and the first direction of the target plane, and the angle direction of the target angle information between the face direction of the face information part in the preset wallpaper and the first direction of the target plane is opposite to the angle direction of the angle information between the face direction of the user and the first direction of the target plane, that is, the target pitch angle information between the face direction of the face information part in the preset wallpaper and the plane where the display screen is located is the same in size and opposite in direction to the pitch angle information when the user looks at the display screen. The angle size of the target angle information between the face direction of the face information part in the preset wallpaper and the second direction of the target plane is the same as the angle size of the angle information between the face direction of the user and the second direction of the target plane, and the angle direction of the target angle information between the face direction of the face information part in the preset wallpaper and the second direction of the target plane is opposite to the angle direction of the angle information between the face direction of the user and the second direction of the target plane, that is, the target yaw angle information between the face direction of the face information part in the preset wallpaper and the plane where the display screen is located is the same in size and opposite in direction to the yaw angle information when the user looks at the display screen.

[0112] In step S303, the preset wallpaper is obtained, the face direction of the face information part in the preset wallpaper is adjusted in angle based on the target angle information of the face direction of the preset wallpaper, and the target wallpaper is obtained.

[0113] In the target wallpaper, the face direction of the face information part is parallel to the face direction of the user, and the face of the face information part in the target wallpaper is opposite to the face of the user.

[0114] In the embodiments of the present application, the preset wallpaper contains face information, for example, real face information, cartoon face information, animal face information, etc. For example, as shown in FIG. 1, it is a schematic diagram of a preset wallpaper containing face information. Figure 6

[0115] In some embodiments, the preset wallpaper is set by the user in advance.

[0116] In some embodiments, in order to improve the privacy security of the user, the wallpaper uploaded by the user and containing real face information can be converted into a wallpaper containing cartoon face information, and the wallpaper is used as the preset wallpaper. For example, as shown in FIG. 2, it is a schematic diagram of a preset wallpaper containing cartoon face information. Figure 7 ​As shown, the cartoon image generation algorithm based on face key points is essentially a modified algorithm of CycleGAN (Cycle-Consistent Generative Adversarial Networks), which uses face key point information, and the global discriminator and the local discriminator work together to effectively improve the detail richness of the generated features such as eyes, mouth, teeth, and other facial features. Based on the image containing a real face and its face key points, the generator generates an image containing a cartoon face. The face key points of the image containing the cartoon face information are obtained by key point detection. Based on the face key points of the image containing the real face information, the face key points of the image containing the cartoon face information are adjusted to make them consistent, and based on this, the facial features of the image containing the cartoon face information are positioned to update the image containing the cartoon face information. The image containing the real face information and its key points, and the image containing the cartoon face information and its key points are used as inputs of the global discriminator and the local discriminator to ensure that the image containing the cartoon face information is almost real. The global discriminator is used to ensure that the overall effect of the image containing the cartoon face information is almost real. The input of the conditional global discriminator is the image containing the real face information and its key points, and the image containing the cartoon face information and its key points. The input of the unconditional global discriminator is the image containing the real face information and the image containing the cartoon face information. The local discriminator is used to ensure that the local effect of the image containing the cartoon face information is almost real. The local discriminator includes an eye discriminator, a nose discriminator, and a mouth discriminator. The input of the eye discriminator is the eye segment information in the image containing the real face information and the eye segment information in the image containing the cartoon face information. The input of the nose discriminator is the nose segment information in the image containing the real face information and the nose segment information in the image containing the cartoon face information. The input of the mouth discriminator is the mouth segment information in the image containing the real face information and the mouth segment information in the image containing the cartoon face information.

[0117] The face orientation of the face information part in the target wallpaper is adjusted relative to the preset wallpaper, so that the angle information of the face orientation of the target wallpaper is target angle information. At this time, the face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face in the face information part in the target wallpaper is opposite to the face of the user, thereby achieving the effect that the face information part in the target wallpaper looks at the face of the user.

[0118] As described in the above example, the preset wallpaper is Figure 6 As shown, the angle information between the face orientation of the user and the first direction of the target plane is an elevation angle of 30°, and the angle information between the face orientation of the user and the second direction of the target plane is a left deflection angle of 30°. As shown inFigure 8 As shown in FIG. 6, a schematic diagram of the face orientation of the face information part in the target wallpaper and the face orientation of the user is shown.

[0119] In some embodiments, as shown in FIG. 7, a preset wallpaper is obtained, and the face orientation of the face information part in the preset wallpaper is adjusted based on the target angle information of the face orientation of the preset wallpaper, and the target wallpaper is obtained, including: Figure 9 In step S901, a preset wallpaper is obtained, and initial key points of a face information part in the preset wallpaper are extracted.

[0120] In the embodiments of the present application, the key points of the face information part can represent the face information part. The initial key points of the face information part in the preset wallpaper can represent the face information part in the preset wallpaper. For example, as shown in FIG. 6, for face information, 68 face key points can represent the face information. The initial key points of the face information part in the preset wallpaper are the key points of the face information part in the preset wallpaper without adjustment.

[0121] Figure 10 For the convenience of understanding, in the following examples, the face information is taken as face information, and the key points are taken as 68 face key points.

[0122] For the convenience of understanding, in the following examples, the face information is taken as face information, and the key points are taken as 68 face key points.

[0123] In step S902, the face orientation of the initial key points of the face information part in the preset wallpaper is adjusted according to the target angle information of the face orientation of the preset wallpaper, and target key points corresponding to the target wallpaper are obtained.

[0124] In the embodiments of the present application, the target wallpaper is the preset wallpaper with the target angle of the face orientation, and the key points can represent the face information part. Therefore, the target key points corresponding to the target wallpaper can be obtained based on the initial key points of the face information part in the preset wallpaper and the target angle information. In some embodiments, the target key points corresponding to the target wallpaper can be obtained through a neural network, taking the first key points of the face information part in the preset wallpaper and the angle information of the face orientation of the face information part in the frame wallpaper image as the input of the neural network, and taking the target key points corresponding to the target wallpaper image as the output of the neural network.

[0125] In some embodiments, the face positioning points corresponding to the target wallpaper can be obtained based on the initial key points of the face information part in the preset wallpaper and the target angle information. As shown in FIG. 8, the initial key points of the face information part in the preset wallpaper and the target angle information are taken as the input of the neural network structure one, and the output layer is the face positioning points corresponding to the target wallpaper. The number of input layer neurons is 138. As shown in FIG. 9, the initial key points of the face information part in the preset wallpaper and the target angle information are taken as the input of the neural network structure two, and the output layer is the face positioning points corresponding to the target wallpaper. The number of input layer neurons is 68. Figure 11 Figure 12 ​​As shown, the facial landmarks include six key facial points: the left corner of the eye, the right corner of the eye, the tip of the nose, the left corner of the mouth, the right corner of the mouth, and the jaw. The output layer of neural network structure one has 12 neurons, each representing the coordinates of one of the six facial landmarks. Neural network structure one can be a fully connected layer network. After determining the facial landmarks corresponding to the target wallpaper, the target key points corresponding to the target wallpaper are obtained based on the initial key points of the facial information portion in the preset wallpaper and the facial landmarks corresponding to the target wallpaper. For example... Figure 11 As shown, the six face localization points corresponding to the target wallpaper and the initial key points of the facial information portion in the preset wallpaper are used as inputs to neural network structure two, and the output layer is the target key points corresponding to the target wallpaper. The number of neurons in the input layer is 148, and the number of neurons in the output layer is 136. Neural network structure two can be a fully connected layer network structure.

[0126] Step S903: Generate the target wallpaper based on the target key points and preset wallpapers corresponding to the target wallpaper.

[0127] In this embodiment, the target key points corresponding to the target wallpaper can characterize the facial information portion of the target wallpaper. The facial orientation of the facial information portion of the target wallpaper changes compared to the preset wallpaper. Therefore, a target wallpaper can be generated based on the target key points corresponding to the target wallpaper and the preset wallpaper.

[0128] In some embodiments, generating a target wallpaper based on target key points corresponding to the target wallpaper and a preset wallpaper includes:

[0129] Step S1301: Perform face regularization processing on the preset wallpaper to obtain the face regularization facial information part of the preset wallpaper.

[0130] In this embodiment of the application, in order to ensure the consistency of the facial information input to the motion estimation model and improve the generation effect of the target wallpaper, it is necessary to perform face regularization processing on the preset wallpaper before generating the target wallpaper through the motion estimation model.

[0131] like Figure 14 As shown, first, refer to the face outline points and eyebrow height points of the initial key points of the preset wallpaper to determine the cutting size, and then cut the preset wallpaper according to the cutting size. Based on the first key point of the preset wallpaper, expand the rectangular area formed by 68 face key points to the left and right by 0.3 times the width, expand the rectangular area formed by 68 face key points to the top by 0.8 times the length, and expand the rectangular area formed by 68 face key points to the bottom by 0.3 times the length.

[0132] After the preset wallpaper is cropped, affine transformation is performed on the cropped preset wallpaper to realize face rotation alignment. The specific steps of the affine transformation are as follows: the center coordinates of the left eye and the right eye are calculated respectively, the included angle θ between the line connecting the center coordinates of the left eye and the right eye and the horizontal direction is calculated, the overall center coordinates of the two eyes are calculated, and the cropped preset wallpaper is rotated counterclockwise by θ based on the overall center coordinates of the two eyes.

[0133] In step S1302, the face regularized face information part in the target wallpaper is generated by performing expression calculation based on the target key points corresponding to the target wallpaper and the face regularized face information part of the preset wallpaper through a motion estimation model.

[0134] In the embodiments of the present application, the target key points corresponding to the target wallpaper and the face regularized face information part of the preset wallpaper are used as the input of the motion estimation model, and the output of the motion estimation model is the face regularized face information part in the target wallpaper. In some embodiments, the motion estimation model can adopt TPS Motion (Thin-Plate Spline Motion Model, Thin-Plate Spline Motion Model). Of course, other motion estimation models can also be used, which are not limited in the present application.

[0135] In step S1303, the target wallpaper is obtained by performing post-processing after the reply based on the face regularized face information part in the target wallpaper.

[0136] In the embodiments of the present application, the preset wallpaper includes not only the face information part, but also the non-face information part. The non-face information part in the target wallpaper is the same as the non-face information part in the target wallpaper. In order to ensure the consistency of the target wallpaper and the preset wallpaper, the face regularized face information part in the target wallpaper is replied to the preset wallpaper to generate the target wallpaper. That is, the face regularized face information part in the target wallpaper is inversely transformed by the affine transformation in step S1301, and the inversely transformed face information part in the target wallpaper is spliced with the non-face information part of the preset wallpaper to obtain the target wallpaper.

[0137] In step S304, the target wallpaper is displayed.

[0138] In the embodiments of the present application, after the target wallpaper is obtained, the target wallpaper is displayed. The face in the face information part of the target wallpaper is parallel to the face orientation of the user, and the face in the face information part of the target wallpaper is opposite to the face of the user.

[0139] In this way, when the personalized wallpaper triggering operation is detected, the target wallpaper is generated and displayed based on the angle information of the face orientation of the user and the preset wallpaper. The face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face in the face information part of the target wallpaper is opposite to the face of the user, so as to achieve the effect that the face information part in the target wallpaper looks at the face of the user. This method improves the interactivity of the wallpaper, and the target wallpaper is generated based on the angle information of the face orientation of the user, and does not depend on the preset fixed template. For different angle information of the face orientation of the user, the generated target wallpaper is different, and personalized interactive experience is provided for the user.

[0140] As a possible implementation manner, in order to increase the interactivity and improve the user experience, the target wallpaper can be displayed through a dynamic wallpaper video. As shown in Figure 15 The step S303 acquires the preset wallpaper, adjusts the face orientation of the face information part in the preset wallpaper based on the target angle information of the face orientation of the preset wallpaper, and acquires the target wallpaper, including:

[0141] In step S3031, the preset wallpaper is acquired, and the initial angle information of the face orientation of the preset wallpaper is determined.

[0142] In the embodiment of the present application, the initial angle information of the face orientation of the preset wallpaper is the angle information of the face orientation of the preset wallpaper in the initial state. The preset wallpaper is taken as the input of the neural network, and the initial angle information of the face orientation of the preset wallpaper is taken as the output of the neural network, so as to determine the initial angle information of the face orientation of the preset wallpaper.

[0143] In some embodiments, in order to improve the watchability of the preset wallpaper, the initial angle information of the face orientation of the preset wallpaper is 0° between the first direction of the target plane and 0° between the second direction of the target plane. That is, the face orientation of the preset wallpaper is perpendicular to the target plane. For example, as shown in Figure 6 The face orientation of the face information part in the preset wallpaper is perpendicular to the target plane.

[0144] For the convenience of understanding, the embodiment of the present application takes the initial angle information of the face orientation of the preset wallpaper as an example, which is 0° between the first direction of the target plane and 0° between the second direction of the target plane.

[0145] In step S3032, the target angle adjustment path is determined in at least one corresponding angle adjustment path based on the target angle information of the face orientation of the preset wallpaper and the initial angle information of the face orientation of the preset wallpaper.

[0146] Among them, the starting position of at least one corresponding angle adjustment path is the initial angle of the face orientation of the preset wallpaper, and the ending position of the angle adjustment path is the target angle of the face orientation of the preset wallpaper.

[0147] In this embodiment, adjusting the facial orientation of the preset wallpaper from an initial angle to a target angle requires determining an angle adjustment path. The initial angle is the angle included in the initial angle information, and the target angle is the angle included in the target angle information. An angle adjustment path can be randomly selected from at least one corresponding angle adjustment path as the target angle adjustment path. Alternatively, the first angle adjustment path can be selected from at least one corresponding angle adjustment path as the target angle adjustment path. The shortest angle adjustment path can also be selected from at least one corresponding angle adjustment path as the target angle adjustment path. The longest angle adjustment path can also be selected from at least one corresponding angle adjustment path as the target angle adjustment path. Of course, other methods can also be used to determine the target angle adjustment path, and this application does not impose any limitations on this.

[0148] In some embodiments, to make the angle adjustment more natural, the angle information between the face orientation of the preset wallpaper and the first direction of the target plane in the target angle adjustment path, and the angle information between the face orientation of the preset wallpaper and the second direction of the target plane, satisfy injectivity. That is, there is a one-to-one correspondence between the angle information between the face orientation of the preset wallpaper and the first direction of the target plane in the target angle adjustment path, and the angle information between the face orientation of the preset wallpaper and the second direction of the target plane.

[0149] like Figure 16 As shown, the horizontal axis represents the deflection angle of the facial information portion in the preset wallpaper. A positive horizontal axis value indicates a rightward deflection, and a negative horizontal axis value indicates a leftward deflection. The vertical axis represents the pitch angle of the facial information portion in the preset wallpaper. A positive vertical axis value indicates a downward pitch angle, and a negative vertical axis value indicates an upward pitch angle. Assuming the initial deflection angle and pitch angle of the preset wallpaper's facial information are both 0°, the starting position of the angle switching path is (0,0). Assuming the target deflection angle of the preset wallpaper's facial information is 30° to the right and the pitch angle is 0°, the ending position of the angle switching path is (30,0). For an angle switching path with a starting position of (0,0) and an ending position of (30,0), only one angle switching path satisfies injectivity, i.e., y = 0 (0 ≤ x ≤ 30). Therefore, this angle switching path is determined as the target angle switching path.

[0150] In step S3033, a preset adjustment duration and a preset frame rate are obtained, and the frame number m of the dynamic wallpaper video is determined.

[0151] wherein m is an integer greater than 1.

[0152] In the embodiments of the present application, the preset adjustment duration is the time length of the dynamic wallpaper video, and the preset frame rate is the frame number of the dynamic wallpaper video played in a unit of time. The frame number m of the dynamic wallpaper video is the product of the preset adjustment duration and the preset frame rate. The preset adjustment duration can be a default value or set by the user, which is not limited in the present application. The preset frame rate can be a default value or set by the user, which is not limited in the present application.

[0153] In some embodiments, in order to balance the video smoothness and running overhead, the preset frame rate is set to 30 frames per second.

[0154] For example, assuming that the preset adjustment duration is 3 seconds and the preset frame rate is 30 frames per second, the frame number of the dynamic wallpaper video is determined to be 90 frames, i.e., m = 90.

[0155] In step S3034, the angle information of the face orientation of the face information part in each wallpaper image in the dynamic wallpaper video is determined based on the target angle adjustment path and the preset adjustment duration.

[0156] wherein the angle information of the face orientation of the face information part in at least two wallpaper images in the dynamic wallpaper video is different.

[0157] In the embodiments of the present application, in the preset adjustment duration, the angle information of the face orientation of the m wallpaper images in the dynamic wallpaper video is adjusted from the initial angle to the target angle through the target angle adjustment path. That is, the angle information of the face orientation of the face information part in the m wallpaper images in the dynamic wallpaper video is determined on the target angle adjustment path. The angle information of the face orientation of the face information part in the first wallpaper image of the dynamic wallpaper video is the initial angle information. The angle information of the face orientation of the face information part in the last wallpaper image of the dynamic wallpaper video is the target angle information.

[0158] In some embodiments, the m angle information can be uniformly extracted on the target angle adjustment path as the angle information of the face orientation of the face information part in the m wallpaper images in the dynamic wallpaper video. That is, the angle information of the face orientation of the face information part in the adjacent wallpaper images in the dynamic wallpaper video has the same difference value. Of course, the angle information of the face orientation of the face information part in the adjacent wallpaper images in the dynamic wallpaper video can also have different difference values, which is not limited in the present application.

[0159] As a possible implementation manner, the angle information of the face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video is determined based on the target angle adjustment path and the preset adjustment time length, comprising: determining the angle switching speed based on the target angle adjustment path and the preset adjustment time length; determining the angle information of the face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video based on the angle switching speed.

[0160] In the embodiment of the application, within the preset adjustment time length, the angle of the face orientation of the preset wallpaper is adjusted through the preset adjustment path from the initial angle to the target angle. For the target angle switching path with unequal length, the preset adjustment time length is equal. For the target angle switching path with longer length, the angle switching speed is faster; for the target angle switching path with shorter length, the angle switching speed is slower. The angle switching speed v satisfies wherein T is the preset adjustment time length, and s is the path length of the target angle switching path.

[0161] After the angle switching speed is determined, the angle change amount of adjacent frames can be determined based on the angle switching speed, and the angle information of the face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video can be determined based on the angle change amount of adjacent frames and the target angle switching path.

[0162] As a possible implementation manner, the angle switching speed is a constant value.

[0163] In the embodiment of the application, in order to improve the generation efficiency of the dynamic wallpaper video, the angle switching speed is constant on a target angle switching path. The angle switching speed is the ratio of the length of the target angle switching path to the preset adjustment time length.

[0164] As a possible implementation manner, the angle switching speed comprises a first angle switching speed and a second angle switching speed, and the first angle switching speed is different from the second angle switching speed.

[0165] In the embodiment of the application, in order to meet different needs of the dynamic wallpaper video, for a target angle switching path, the angle switching speed can be variable.

[0166] In some embodiments, within the first adjustment time, the angle switching speed is the first angle switching speed. Within the second adjustment time, the angle switching speed is the second angle switching speed. Within the third adjustment time, the angle switching speed slows down over time. As shown in Figure 17 the angle switching speed is

[0167] Wherein, b is the first angle switching speed, a is the second angle switching speed. The starting time of the first adjustment time is 0, and the ending time is t1; the starting time of the third adjustment time is t1, and the ending time is t2; the starting time of the second adjustment time is t2, and the ending time is t3. t3 is the preset adjustment duration.

[0168] Step S3035, generating the dynamic wallpaper video based on the angle information of the face orientation of the face information part in each frame of wallpaper image in the dynamic wallpaper video.

[0169] Wherein, the first frame of wallpaper image of the dynamic wallpaper video is the preset wallpaper with the face orientation of the face information part being the initial angle, and the last frame of wallpaper image of the dynamic wallpaper video is the target wallpaper.

[0170] In the embodiments of the present application, the angle information of the face orientation of the face information part in the dynamic wallpaper video is switched from the initial angle to the target angle according to the target angle switching path, the first frame of wallpaper image of the dynamic wallpaper is the preset wallpaper with the face orientation of the face information part being the initial angle, and the angle information of the face orientation of the face information part in the last frame of wallpaper image of the dynamic wallpaper video is the target angle information, that is, the last frame of wallpaper image of the dynamic wallpaper video is the target wallpaper. In this way, in response to the personalized wallpaper triggering operation, the dynamic wallpaper video is generated and displayed, and the dynamic wallpaper video shows the process of switching the angle information of the face orientation of the face information part in the preset wallpaper from the initial angle to the target angle, and the face orientation of the target wallpaper is parallel to the face orientation of the user and the face information part of the target wallpaper is opposite to the face of the user.

[0171] As a possible implementation manner, as shown in Figure 18 Based on the angle information of the face orientation of the face information part in each frame of wallpaper image in the dynamic wallpaper video, the dynamic wallpaper video is generated, including:

[0172] Step S1801, extracting the first key point of the face information part in the preset wallpaper.

[0173] In the embodiments of the present application, the first key point of the face information part in the preset wallpaper is the key point of the face information part in the preset wallpaper with the face orientation being the initial angle. For details, refer to step S901, which will not be repeated here.

[0174] Step S1802, for the angle information of the face orientation of the face information part in each frame of wallpaper image in the dynamic wallpaper video, adjusting the face orientation of the first key point of the face information part in the preset wallpaper according to the angle information of the face orientation of the face information part in the frame of wallpaper image in the dynamic wallpaper video, to obtain the second key point corresponding to the frame of wallpaper image.

[0175] In the embodiment of the present application, for each frame of wallpaper image in the dynamic wallpaper video, the first key point of the face information part in the preset wallpaper is adjusted based on the angle information of the face orientation of the face information part in the frame of wallpaper image, and the second key point corresponding to the frame of wallpaper image can be obtained. For details, please refer to step S902, which will not be repeated here.

[0176] Step S1803, generating a dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m frames of wallpaper images.

[0177] In the embodiment of the present application, the second key points corresponding to the m frames of wallpaper images can represent the face information part in the dynamic wallpaper video, so the dynamic wallpaper video can be generated based on the preset wallpaper and the second key points corresponding to the m frames of wallpaper images.

[0178] As a possible implementation manner, as shown in Figure 19 and Figure 20 , generating a dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m frames of wallpaper images includes:

[0179] Step S1901, performing face regularization processing on the preset wallpaper to obtain a face regularized face information part of the preset wallpaper.

[0180] In the embodiment of the present application, in order to ensure the consistency of the face information part input by the motion estimation model and improve the generation effect of the dynamic wallpaper video, the face regularization processing needs to be performed on the preset wallpaper before the dynamic wallpaper video is generated by the motion estimation model. For details, please refer to step S1301, which will not be repeated here.

[0181] Step S1902, calculating the adjacent frame offset between the second key points corresponding to the m frames of wallpaper images.

[0182] In the embodiment of the present application, the adjacent frame offset between the second key points corresponding to the m frames of wallpaper images can represent the change of the face information part corresponding to the adjacent frames of wallpaper images in the dynamic wallpaper video.

[0183] In some embodiments, for each frame of the m frames of wallpaper images, the coordinates of the second key point corresponding to the frame can be subtracted from the second key point corresponding to the adjacent previous frame of wallpaper image of the frame to determine the adjacent frame offset between the second key points corresponding to the frame of wallpaper image.

[0184] Step S1903, performing expression calculation generation by the motion estimation model based on the adjacent frame offset between the second key points corresponding to the m frames of wallpaper images and the face regularized face information part of the preset wallpaper to obtain a face regularized face information part in the dynamic wallpaper video.

[0185] In the embodiments of the present application, the adjacent frame offset between the second key points corresponding to the m frame wallpaper images, and the face information part of the face regularization of the preset wallpaper, are taken as the input of the motion estimation model, and the output of the motion estimation model is the face information part of the face regularization processing in the dynamic wallpaper video. In some embodiments, the motion estimation model can adopt TPS Motion (Thin-Plate Spline Motion Model). Of course, other motion estimation models can also be adopted, which are not limited in the present application.

[0186] Step S1904, based on the face information part of the face regularization processing in the dynamic wallpaper video, post-processing is performed to obtain the dynamic wallpaper video.

[0187] In the embodiments of the present application, in addition to the face information part, the preset wallpaper also includes a non-face information part. In order to ensure the consistency of the dynamic wallpaper video and the preset wallpaper, the face information part of the face regularization processing in the dynamic wallpaper video is posted to the preset wallpaper to generate the dynamic wallpaper video. That is, the face information part of the face regularization processing in the dynamic wallpaper video is inversely transformed by the affine transformation in step S1901, and the dynamic wallpaper video of the inversely transformed face information part is spliced with the non-face information part of the preset wallpaper to obtain the complete dynamic wallpaper video. In some embodiments, in order to ensure the inter-frame stability of the dynamic wallpaper video, inter-frame time domain smoothing filtering can be performed on the dynamic wallpaper video.

[0188] As a possible implementation manner, the face information part of the n frame wallpaper images in the dynamic wallpaper video is superimposed with interactive expression information; n is an integer greater than 0.

[0189] In the embodiments of the present application, in order to improve the interactive experience of the user, the interactive expression is superimposed on part of the segments or the whole of the dynamic wallpaper video. For example, the interactive expression can be a smile and a wink.

[0190] As a possible implementation manner, as shown in Figure 21 For the face information part of the n frame wallpaper images in the dynamic wallpaper video, the interactive expression information is superimposed, and based on the angle information of the face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video, the dynamic wallpaper video is generated, including:

[0191] Step S2101, extracting the first key point of the face information part in the preset wallpaper.

[0192] For details, reference can be made to step S1801, which will not be repeated here.

[0193] Step S2102: For the angle information of the facial orientation of the facial information part in each frame of the wallpaper image in the live wallpaper video, adjust the facial orientation of the first key point of the facial information part in the preset wallpaper according to the angle information of the facial orientation of the facial information part in the frame of the wallpaper image in the live wallpaper video, and obtain the second key point corresponding to the frame of the wallpaper image.

[0194] For details, please refer to step S1802, which will not be repeated here.

[0195] Step S2103: Determine the start and end times of the interactive emoticons.

[0196] In this embodiment, the start and end times of the interactive emoticon are the same as the start and end times of the interactive emoticon within the live wallpaper video, i.e., the start and end times of the interactive emoticon being overlaid within the live wallpaper video. The start and end times of the interactive emoticon can be default values; for example, the start time of the interactive emoticon can be the start time of the live wallpaper video, and the end time of the interactive emoticon can be the end time of the live wallpaper video. Alternatively, the start and end times of the interactive emoticon can be adaptively determined and generated.

[0197] As one possible implementation, determining the start and end times of an interactive emoticon includes: acquiring the first time of the interactive emoticon; acquiring the length of the target angle adjustment path corresponding to the interactive emoticon; and determining the second time of the interactive emoticon based on the length of the target angle adjustment path and the first time of the interactive emoticon. The first time of the interactive emoticon includes either the start or end time of the interactive emoticon; the second time of the interactive emoticon includes either the end or start time of the interactive emoticon.

[0198] In this embodiment, the first time of the interactive emoticon can be a default value or set by the user. The length of the target angle adjustment path corresponding to the interactive emoticon is the angle change from the start time to the end time of the interactive emoticon along the target adjustment path. The length of the target angle adjustment path corresponding to the interactive emoticon can be a default value or set by the user. If the first time of the interactive emoticon is the start time, the end time of the interactive emoticon is determined based on the length of the target angle adjustment path and the start time. If the first time of the interactive emoticon is the end time, the start time of the interactive emoticon is determined based on the length of the target angle adjustment path and the end time. For example, as... Figure 22 As shown, the end time of the interactive emoticon is the end time of the live wallpaper video, that is, the end time of the interactive emoticon is the preset adjustment duration T. Assuming the length of the target angle adjustment path corresponding to the interactive emoticon is s1, based on... The starting time t4 of the interactive expression is determined. Wherein, v is the angle switching speed.

[0199] In step S2104, the frame number n of the interactive expression image is determined according to the starting time, the ending time and the preset frame rate of the interactive expression, and n frames of the interactive expression images are obtained in the preset interactive expression images.

[0200] In the embodiments of the present application, the time length of the interactive expression is determined according to the starting time and the ending time of the interactive expression, and based on the time length of the interactive expression and the preset frame rate, the frame number n of the interactive expression image can be determined. For example, the time length of the interactive expression is 1 second, and the preset frame rate is 30 frames per second, then the frame number n of the interactive expression image is 30. 30 frames of the interactive expression images are obtained in the preset interactive expression images.

[0201] In some embodiments, n frames of the interactive expression images are uniformly extracted in the preset interactive expression images.

[0202] In step S2105, the angle information of the face orientation of the face information part in the n frames of the wallpaper images in the target time period is determined.

[0203] Wherein, the target time period is the time period composed of the starting time and the ending time of the interactive expression.

[0204] In the embodiments of the present application, the angle information of the face orientation of the face information part in the n frames of the wallpaper images in the target time period is the angle information of the face orientation of the face information part in the n frames of the wallpaper images superimposed with the interactive expression information. According to the starting time and the ending time of the interactive expression, the target time period is determined. In the m frames of the dynamic wallpaper video, n frames of the wallpaper images in the target time period are determined. The angle information of the face orientation of the face information part in these frames of the wallpaper images is the angle information of the face orientation of the face information part in the n frames of the wallpaper images in the target time period.

[0205] In step S2106, the third key points of the face information part in the n frames of the interactive expression images are extracted, and the face orientation of the third key points of the face information part in the n frames of the interactive expression images is adjusted according to the angle information of the face orientation of the face information part in the n frames of the wallpaper images corresponding to the target time period, to obtain the fourth key points corresponding to the n frames of the interactive expression images.

[0206] In the embodiments of the present application, as Figure 23As shown, for each frame of the n frames of the interactive expression image, a key point of the face information part is extracted, that is, a third key point. Angle information of the face orientation of the face information part in the n frames of the wallpaper image corresponding to the target time period is determined, and the face orientation of the third key point of the face information part in the frame of the interactive expression image is adjusted according to the angle information corresponding to the frame, to obtain a fourth key point corresponding to the frame of the interactive expression image.

[0207] In step S2107, the second key points corresponding to the n frames of the wallpaper image in the target time period are determined from the second key points corresponding to the m frames of the wallpaper image.

[0208] In the embodiments of the present application, the target time period is determined according to the start time and the end time of the interactive expression. In the m frames of the dynamic wallpaper video, n frames in the target time period are determined. The second key points corresponding to these frames of the wallpaper image are the second key points corresponding to the n frames of the wallpaper image in the target time period.

[0209] In step S2108, the fifth key points corresponding to the n frames of the wallpaper image with the interactive expression superimposed in the target time period are determined based on the fourth key points corresponding to the n frames of the interactive expression image and the second key points corresponding to the n frames of the wallpaper image.

[0210] In the embodiments of the present application, as shown, Figure 23 The fourth key points corresponding to the n frames of the interactive expression image are determined by adjusting the third key points corresponding to the n frames of the interactive expression image according to the corresponding angle information. The second key points corresponding to the n frames of the wallpaper image are determined by adjusting the first key points of the preset wallpaper according to the corresponding angle information. Therefore, the fifth key points corresponding to the n frames of the wallpaper image with the interactive expression superimposed in the target time period can be determined based on the fourth key points and the second key points.

[0211] In some embodiments, to ensure the fluency and naturalness of the segment with the interactive expression superimposed in the dynamic wallpaper video, after the fifth key points corresponding to the n frames of the wallpaper image with the interactive expression superimposed in the target time period are determined, the fifth key points can be updated by using the optical flow method to improve the smoothness of the face information part between frames.

[0212] As a possible implementation manner, the fifth key point corresponding to the n frames of wallpaper images with the interactive expression superimposed in the target time period is determined based on the fourth key point corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images, including: determining a position offset of the corresponding key point between each adjacent two frames among the fourth key points corresponding to the n frames of interactive expression images based on the fourth key points corresponding to the n frames of interactive expression images; and determining the fifth key point corresponding to the n frames of wallpaper images with the interactive expression superimposed in the target time period based on the position offset of the corresponding key point between each adjacent two frames among the fourth key points corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images.

[0213] In the embodiment of the present application, the position offset of the corresponding key point between each adjacent two frames among the fourth key points corresponding to the n frames of interactive expression images can represent the change of the key point of the n frames of interactive expression images with the corresponding angle information superimposed.

[0214] In step S2109, the dynamic wallpaper video is generated based on the preset wallpaper, the fifth key point corresponding to the n frames of wallpaper images with the interactive expression superimposed in the target time period, and the second key point corresponding to the other wallpaper images.

[0215] The second key point corresponding to the other wallpaper images refers to the second key point corresponding to the other frames of wallpaper images except the second key point corresponding to the n frames of wallpaper images in the target time period among the second key points corresponding to the m frames of wallpaper images.

[0216] In the embodiment of the present application, the fifth key point corresponding to the n frames of wallpaper images with the interactive expression superimposed in the target time period can represent the facial information part in the n frames of wallpaper images with the interactive expression information superimposed in the dynamic wallpaper video. The second key point corresponding to the other wallpaper images can represent the facial information part in the other frames of wallpaper images except the second key point corresponding to the n frames of wallpaper images in the target time period among the second key points corresponding to the dynamic wallpaper video. In this way, the generated dynamic wallpaper video displays the process that the angle information of the facial orientation of the facial information part in the preset wallpaper is switched from the initial angle to the target angle, and the interactive expression is superimposed in the target time period, further improving the interactivity.

[0217] For details, refer to step S1803, which will not be repeated here.

[0218] It should be noted that in some embodiments, if the length s of the target angle adjustment path is less than the length s1 of the target angle adjustment path corresponding to the obtained interactive emoticon, the number of frames n of the interactive emoticon image is determined, and n frames of interactive emoticon images are obtained from the preset emoticon images, where n = m * s1 / s. Based on the preset wallpaper with the face orientation as the initial angle, and the first nm frames of interactive emoticon images in the n frames of interactive emoticon images, a preceding interactive emoticon video is generated. Based on the face orientation angle information and second key points of the m frames of wallpaper images in the dynamic wallpaper video, and the last m frames of interactive emoticon images in the n frames of interactive emoticon images, a dynamic wallpaper video is generated.

[0219] In some embodiments, after displaying the target wallpaper, the user's facial expression can be determined based on the user's image information, and a wallpaper with an overlaid feedback expression can be generated and displayed based on the user's facial expression information and the target wallpaper. The feedback expression can be the same as the user's facial expression; that is, the facial information portion of the target wallpaper mimics the user's facial expression in real time. Alternatively, in response to a specific user expression, the feedback expression can be the corresponding target feedback expression.

[0220] In some embodiments, after the electronic device stops displaying the target wallpaper, the wallpaper of the electronic device is restored to a preset wallpaper with the face orientation as the initial angle information.

[0221] In some embodiments, after displaying the target wallpaper, the initial angle information of the facial orientation of the facial information portion of the preset wallpaper can be updated to the target angle information. That is, the target angle information is determined as the updated initial angle information.

[0222] In some embodiments, facial expression calculation and generation based on motion estimation models is essentially based on GANs (Consistent Generative Adversarial Networks), which require high computational power. To achieve real-time generation and rendering, such as... Figure 24 As shown, a cloud-edge collaborative solution can be adopted. Expression calculation and generation are performed in the cloud, and the generated dynamic wallpaper video is sent to the corresponding electronic device for rendering and presentation. Different interactive experiences are achieved based on the hardware of the electronic device. Electronic devices that can apply the method of this application include mobile phones, in-vehicle systems, and large screens.

[0223] like Figure 24 As shown, for electronic devices such as smartwatches and in-vehicle systems, the mobile phone can send the live wallpaper video to the corresponding electronic device.

[0224] For example, such as Figure 25 As shown, the mobile phone acquires a real-time video stream through its camera, with the user's image information sampled at a rate of 30 frames per second. Figure 26aAs shown, based on the facial image information of the user, shared features are extracted through the shared feature extraction network, and the shared features obtained by the shared feature extraction network are taken as inputs of the three sub-networks of the face angle determination sub-network, the face key point detection sub-network, and the face smile binary classification sub-network, to determine the angle information of the face orientation of the user, the face key points of the user, and whether the user smiles, respectively. As shown in Figure 26b As shown, the input layer of the shared feature extraction network is the face image information, and the output layer is the shared features. The shared feature extraction network can be the network structure and pre-trained model of Vgg-19, and the output feature layer parameter is 2048. The sub-network can adopt a fully connected layer network. As shown in Figure 26c As shown, the input layer of the shared feature extraction network is the face image information, and the output layer is the shared features. The shared feature extraction network can be the network structure and pre-trained model of Vgg-19, and the output feature layer parameter is 2048. The sub-network can adopt a fully connected layer network. As shown in Figure 26d As shown, the input layer of the shared feature extraction network is the face image information, and the output layer is the shared features. The shared feature extraction network can be the network structure and pre-trained model of Vgg-19, and the output feature layer parameter is 2048. The sub-network can adopt a fully connected layer network. As shown in Figure 26e As shown, the input layer of the shared feature extraction network is the face image information, and the output layer is the shared features. The shared feature extraction network can be the network structure and pre-trained model of Vgg-19, and the output feature layer parameter is 2048. The sub-network can adopt a fully connected layer network. As shown in It should be noted that the shared features can be taken as the input layer of any one of the above sub-networks, or the input layer of any two of the above sub-networks, or the input layer of other sub-networks, which is not limited in the present application. In this example, the shared features are taken as inputs of the three sub-networks of the face angle determination sub-network, the face key point detection sub-network, and the face smile binary classification sub-network. Based on the face key point parameters, the relative coordinates of the left pupil in the left eye, the relative coordinates of the right pupil in the right eye, and the distance from the two eyes to the nose can be determined through pre-processing calculation. The above information is taken as the input of the neural network, and the classification result of whether the user gazes at the electronic device is taken as the output of the neural network, so as to determine whether the user gazes at the electronic device. The neural network is a single hidden layer neural network.

[0225] In response to the user gazing at the mobile phone and smiling, the angle information of the user's facial orientation is acquired. It is assumed that the angle information between the user's facial orientation and the first direction of the target plane is an elevation angle of 30°. The angle information between the user's facial orientation and the second direction of the target plane is a left deflection angle of 30°. The angle size of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane is the same as the angle size of the angle information between the user's facial orientation and the first direction of the target plane, i.e. 30°; the angle direction of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane is opposite to the angle direction of the angle information between the user's facial orientation and the first direction of the target plane, i.e. a depression angle. That is, the target angle information between the facial orientation of the facial information part in the preset wallpaper and the first direction of the target plane is a depression angle of 30°. The angle size of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane is the same as the angle size of the angle information between the user's facial orientation and the second direction of the target plane, i.e. 30°; the angle direction of the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane is opposite to the angle direction of the angle information between the user's facial orientation and the second direction of the target plane, i.e. a right deflection angle. That is, the target angle information between the facial orientation of the facial information part in the preset wallpaper and the second direction of the target plane is a right deflection angle of 30°.

[0226] The preset wallpaper is acquired, and it is assumed that the initial angle information of the facial orientation of the preset wallpaper is a pitch angle of 0° and a deflection angle of 0°, and the first key point of the facial information part in the preset wallpaper is extracted. As shown in FIG. 4, it is a schematic diagram of the preset wallpaper in this example. Figure 6

[0227] As shown in FIG. 5, it is a schematic diagram of the preset wallpaper in this example. Figure 27 ​As shown, the horizontal axis represents the deflection angle of the facial information portion in the preset wallpaper. A positive horizontal axis value indicates a rightward deflection, while a negative horizontal axis value indicates a leftward deflection. The vertical axis represents the pitch angle of the facial information portion in the preset wallpaper. A positive vertical axis value indicates a downward pitch, while a negative vertical axis value indicates an upward pitch. The starting position of the angle switching path is the initial angle (0,0), and the ending position of the angle switching path is (30,30). There are countless angle switching paths with a starting position of (0,0) and an ending position of (30,30), including angle adjustment paths r1, r2, r3, r4, ... One of these is randomly selected as the target angle adjustment path. Assuming the target adjustment path is angle adjustment path r4, the path length of the target angle switching path is determined by calculation as s. Assuming the preset adjustment duration is T = 3 seconds and the preset frame rate is 30 frames per second, the frame rate of the live wallpaper video is determined to be 90. Assuming the angle switching speed is constant, the angle switching speed v = s / T, as... Figure 28 The diagram illustrates the angle switching speed. Based on the target adjustment path and angle switching speed, the angle information of the facial orientation in each frame of the wallpaper image in the live wallpaper video is determined. Figure 29 As shown, assume that the angle information of the facial information portion in each frame of the live wallpaper video is (x0, y0), (x1, y1), (x2, y2), ..., (x... 89 ,y 89 For each frame of a live wallpaper video, the angle information of the facial orientation of the facial information portion in that frame of the live wallpaper video, along with the first key point of the facial information portion in the preset wallpaper, are used as inputs to a neural network. The second key point corresponding to that frame of the wallpaper image is used as the output of the neural network, thereby obtaining the second key points of 90 frames of wallpaper images.

[0228] The interactive emoticon is a smiling and winking emoji. The end time of the live wallpaper video (3 seconds) is determined as the end time of the interactive emoticon. Assuming the length of the target angle adjustment path corresponding to the interactive emoticon is s1, and assuming it is based on... The start time t4 for interactive emoticons is determined to be 2 seconds. Figure 28The length of the target time period is determined to be 1 second, the preset frame rate is determined to be 30 frames per second, the number of frames of the interactive expression image is determined to be 30 frames, and 30 frames of smile-blinking expression images are obtained from the preset smile-blinking expression video template. In the angle information of the face orientation of the face information part in the 90 frames of wallpaper images of the dynamic wallpaper video, the angle information of the face orientation of the face information part in the last 30 frames of wallpaper images is determined. The third key points of the face information part in the 30 frames of smile-blinking expression images are extracted, and the face orientation of the third key points of the face information part in the 30 frames of smile-blinking expression images is adjusted according to the angle information of the face orientation of the face information part in the last 30 frames of wallpaper images, to obtain fourth key points corresponding to the 30 frames of smile-blinking expression images; the second key points corresponding to the last 30 frames of wallpaper images are determined from the second key points corresponding to the 90 frames of wallpaper images; the fifth key points corresponding to the last 30 frames of wallpaper images with the smile-blinking expression overlaid are determined based on the fourth key points corresponding to the 30 frames of smile-blinking expression images and the second key points corresponding to the last 30 frames of wallpaper images; and the dynamic wallpaper video is generated based on the preset wallpaper, the fifth key points corresponding to the last 30 frames of wallpaper images with the smile-blinking expression overlaid, and the second key points corresponding to the first 60 frames of wallpaper images.

[0229] Reference Figure 30 A structure schematic diagram of a wallpaper interaction device is provided for the embodiments of the present application. As shown in Figure 30 The device comprises:

[0230] The acquisition unit 3001 is configured to acquire angle information of a face orientation of a user in response to a personal wallpaper triggering operation.

[0231] The determination unit 3002 is configured to determine target angle information of a face orientation of a preset wallpaper based on the angle information of the face orientation of the user.

[0232] The processing unit 3003 is configured to acquire the preset wallpaper, perform angle adjustment on the face orientation of a face information part in the preset wallpaper based on the target angle information of the face orientation of the preset wallpaper, and acquire a target wallpaper; the face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face in the face information part in the target wallpaper faces the face of the user.

[0233] The display unit 3004 is configured to display the target wallpaper.

[0234] As a possible implementation manner, the angle information of the face orientation of the user includes angle information between the face orientation of the user and a first direction of a target plane, and angle information between the face orientation of the user and a second direction of the target plane; the first direction and the second direction are perpendicular to each other, and the target plane is parallel to a plane on which the display screen is located.

[0235] As a possible implementation manner, the determining unit 3002 is specifically configured to determine target angle information between the face direction of the face information part in the preset wallpaper and the first direction of the target plane, and target angle information between the face direction of the face information part in the preset wallpaper and the second direction of the target plane based on the angle information between the face direction of the user and the first direction of the target plane and the angle information between the face direction of the user and the second direction of the target plane.

[0236] As a possible implementation manner, the processing unit 3003 is specifically configured to obtain a preset wallpaper, determine initial angle information of the face direction of the preset wallpaper; determine a target angle adjustment path in at least one corresponding angle adjustment path based on the target angle information of the face direction of the preset wallpaper and the initial angle information of the face direction of the preset wallpaper; the starting position of the at least one corresponding angle adjustment path is the initial angle of the face direction of the preset wallpaper, and the ending position of the angle adjustment path is the target angle of the face direction of the preset wallpaper; obtain a preset adjustment time length and a preset frame rate, and determine the number m of frames of the dynamic wallpaper video; m is an integer greater than 1; determine angle information of the face direction of the face information part in each frame of wallpaper image in the dynamic wallpaper video based on the target angle adjustment path and the preset adjustment time length; wherein the angle information of the face direction of the face information part in at least two frames of wallpaper image in the dynamic wallpaper video is different; generate the dynamic wallpaper video based on the angle information of the face direction of the face information part in each frame of wallpaper image in the dynamic wallpaper video, wherein the first frame of wallpaper image of the dynamic wallpaper video is the preset wallpaper with the initial angle of the face direction of the face information part, and the last frame of wallpaper image of the dynamic wallpaper video is the target wallpaper.

[0237] As a possible implementation manner, the processing unit 3003 is specifically configured to determine an angle switching speed based on the target angle adjustment path and the preset adjustment time length; and determine the angle information of the face direction of the face information part in each frame of wallpaper image in the dynamic wallpaper video based on the target angle adjustment path and the angle switching speed.

[0238] As a possible implementation manner, the angle switching speed includes a first angle switching speed and a second angle switching speed, and the first angle switching speed is different from the second angle switching speed.

[0239] As a possible implementation manner, the processing unit 3003 is specifically configured to extract first key points of the face information part in the preset wallpaper; for angle information of a face orientation of the face information part in each frame of wallpaper image in the dynamic wallpaper video, the angle information of the face orientation of the face information part in the frame of wallpaper image in the dynamic wallpaper video is adjusted to adjust the face orientation of the first key points of the face information part in the preset wallpaper, to obtain second key points corresponding to the frame of wallpaper image; and the dynamic wallpaper video is generated based on the preset wallpaper and the second key points corresponding to the m frames of wallpaper image.

[0240] As a possible implementation manner, the face information part of n frames of wallpaper image in the dynamic wallpaper video is superimposed with interactive expression information; n is an integer greater than 0.

[0241] As a possible implementation manner, the processing unit 3003 is specifically configured to determine a start time and an end time of the interactive expression; determine the number n of frames of the interactive expression image according to the start time, the end time and a preset frame rate of the interactive expression, and acquire n frames of interactive expression images in the preset interactive expression image; determine angle information of a face orientation of the face information part in n frames of wallpaper image in a target time period in the angle information of the face orientation of the face information part in the m frames of wallpaper image of the dynamic wallpaper video; wherein the target time period is a time period formed by the start time and the end time of the interactive expression; extract third key points of the face information part in the n frames of interactive expression image, and adjust the face orientation of the third key points of the face information part in the n frames of interactive expression image according to the angle information of the face orientation of the face information part in the n frames of wallpaper image corresponding to the target time period, to obtain fourth key points corresponding to the n frames of interactive expression image; determine the second key points corresponding to the n frames of wallpaper image in the target time period in the second key points corresponding to the m frames of wallpaper image; and determine fifth key points corresponding to the n frames of wallpaper image superimposed with the interactive expression in the target time period based on the fourth key points corresponding to the n frames of interactive expression image and the second key points corresponding to the n frames of wallpaper image.

[0242] The processing unit 3003 is specifically configured to generate the dynamic wallpaper video based on the preset wallpaper, the fifth key points corresponding to the n frames of wallpaper image superimposed with the interactive expression in the target time period and the second key points corresponding to other frames of wallpaper image; wherein the second key points corresponding to other frames of wallpaper image refer to the second key points corresponding to other frames of wallpaper image except the second key points corresponding to the n frames of wallpaper image in the target time period in the second key points corresponding to the m frames of wallpaper image.

[0243] As a possible implementation manner, the processing unit 3003 is specifically configured to acquire a first time of the interactive expression; acquire a length of a target angle adjustment path corresponding to the interactive expression; and determine a second time of the interactive expression according to the length of the target angle adjustment path corresponding to the interactive expression and the first time of the interactive expression; the first time of the interactive expression includes a start time or an end time of the interactive expression; and the second time of the interactive expression includes the end time or the start time of the interactive expression.

[0244] As a possible implementation manner, the processing unit 3003 is specifically configured to determine a position offset of corresponding key points between each two adjacent frames among n frames of interactive expression images based on the fourth key points corresponding to the n frames of interactive expression images; and determine fifth key points corresponding to the n frames of wallpaper images with superimposed interactive expressions in a target time period based on the position offset of the corresponding key points between each two adjacent frames among the n frames of interactive expression images and the second key points corresponding to the n frames of wallpaper images.

[0245] As a possible implementation manner, the acquisition unit 3001 is specifically configured to acquire angle information of a face orientation of a user in response to the user gazing at the electronic device.

[0246] Corresponding to the above-mentioned embodiments, the present application also provides an electronic device. Figure 31 A structural schematic diagram of an electronic device provided by an embodiment of the present application, the electronic device 3100 can include a processor 3101, a memory 3102 and a communication unit 3103. These components communicate through one or more buses, and those skilled in the art can understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiments of the present application, which can be in a bus structure or a star structure, and can include more or fewer components than shown in the figure, or combine some components, or different component arrangements.

[0247] The communication unit 3103 is configured to establish a communication channel, so that the electronic device can communicate with other devices. Receive user data sent by other devices or send user data to other devices.

[0248] The processor 3101 is the control center of the electronic device, connects various parts of the electronic device through various interfaces and lines, and performs various functions of the electronic device and / or processes data by running or executing software programs and / or modules stored in the memory 3102 and calling data stored in the memory. The processor can be composed of an integrated circuit (IC), for example, can be composed of a single packaged IC, or can be composed of a plurality of packaged ICs connected together. For example, the processor 3101 can only include a central processing unit (CPU). In the embodiments of the present application, the CPU can be a single operation core or can include multiple operation cores.

[0249] The memory 3102 is used to store the execution instructions of the processor 3101, and the memory 3102 can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0250] When the execution instructions in the memory 3102 are executed by the processor 3101, the electronic device 3100 can perform Figure 3 some or all of the steps in the illustrated embodiments.

[0251] In specific implementations, the present application also provides a computer storage medium, wherein the computer storage medium can store a program, and the program can include some or all of the steps in the embodiments of the wallpaper interaction method provided by the present application when executed. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0252] Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be realized by means of software and necessary general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of the present application.

[0253] The same or similar parts between various embodiments in the specification can be referred to each other. Especially, for the device embodiments and the terminal embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

Claims

1. A wallpaper interaction method, characterized in that, The method is applied to an electronic device, and the method comprises: In response to a personalized wallpaper triggering operation, obtaining angle information of a face orientation of a user; Based on the angle information of the face orientation of the user, determining target angle information of a face orientation of a preset wallpaper; Obtaining the preset wallpaper, and performing angle adjustment on the face orientation of a face information part in the preset wallpaper based on the target angle information of the face orientation of the preset wallpaper, to obtain a target wallpaper; the face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face of the face information part in the target wallpaper faces the face of the user; Displaying the target wallpaper.

2. The method of claim 1, wherein, The angle information of the face orientation of the user comprises angle information between the face orientation of the user and a first direction of a target plane, and angle information between the face orientation of the user and a second direction of the target plane; the first direction and the second direction are perpendicular to each other, and the target plane is parallel to a plane on which a display screen is located.

3. The method of claim 2, wherein, The method comprises: Based on the angle information between the face orientation of the user and the first direction of the target plane, and the angle information between the face orientation of the user and the second direction of the target plane, determining target angle information between the face orientation of the face information part in the preset wallpaper and the first direction of the target plane, and target angle information between the face orientation of the face information part in the preset wallpaper and the second direction of the target plane.

4. The method of claim 3, wherein, The method comprises: Obtaining the preset wallpaper, and determining initial angle information of the face orientation of the preset wallpaper; Based on the target angle information of the face orientation of the preset wallpaper and the initial angle information of the face orientation of the preset wallpaper, determining a target angle adjustment path in at least one corresponding angle adjustment path; a starting position of the at least one corresponding angle adjustment path is the initial angle of the face orientation of the preset wallpaper, and a terminal position of the angle adjustment path is the target angle of the face orientation of the preset wallpaper; Obtaining a preset adjustment time length and a preset frame rate, and determining a frame number m of a dynamic wallpaper video; m is an integer greater than 1; Based on the target angle adjustment path and the preset adjustment time length, determining angle information of the face orientation of the face information part in each wallpaper image in the dynamic wallpaper video; wherein the angle information of the face orientation of the face information part in at least two wallpaper images in the dynamic wallpaper video is different; Based on the angle information of the face orientation of the face information part in each wallpaper image in the dynamic wallpaper video, generating the dynamic wallpaper video, wherein a first wallpaper image of the dynamic wallpaper video is the preset wallpaper with the initial angle of the face orientation of the face information part, and a last wallpaper image of the dynamic wallpaper video is the target wallpaper.

5. The method of claim 4, wherein, The method comprises: determine an angle switching speed based on the target angle adjustment path and a preset adjustment duration; determine angle information of a face orientation of a face information part in each frame of the wallpaper image in the dynamic wallpaper video based on the target angle adjustment path and the angle switching speed.

6. The method of claim 5, wherein, The angle switching speed includes a first angle switching speed and a second angle switching speed, and the first angle switching speed is different from the second angle switching speed.

7. The method according to any one of claims 4-6, characterized in that, The generating of the dynamic wallpaper video based on the angle information of the face orientation of the face information part in each frame of the wallpaper image in the dynamic wallpaper video includes: extracting a first key point of the face information part in the preset wallpaper; adjusting the face orientation of the first key point of the face information part in the preset wallpaper according to the angle information of the face orientation of the face information part in the frame of the wallpaper image in the dynamic wallpaper video to obtain a second key point corresponding to the frame of the wallpaper image; generating the dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m frames of the wallpaper image.

8. The method of claim 7, wherein, The face information part of n frames of the wallpaper image in the dynamic wallpaper video is superimposed with interactive expression information; n is an integer greater than 0.

9. The method of claim 8, wherein, Before the generating of the dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m frames of the wallpaper image, the method further includes: determining a start time and an end time of the interactive expression; determining the number n of frames of the interactive expression image according to the start time, the end time and a preset frame rate of the interactive expression, and obtaining n frames of the interactive expression image in the preset interactive expression image; determining the angle information of the face orientation of the face information part in the n frames of the wallpaper image in the target time period in the angle information of the face orientation of the face information part in the m frames of the wallpaper image in the dynamic wallpaper video; wherein the target time period is a time period formed by the start time and the end time of the interactive expression; extracting a third key point of the face information part in the n frames of the interactive expression image, and adjusting the face orientation of the third key point of the face information part in the n frames of the interactive expression image according to the angle information of the face orientation of the face information part in the n frames of the wallpaper image corresponding to the target time period to obtain fourth key points corresponding to the n frames of the interactive expression image; determining the second key points corresponding to the n frames of the wallpaper image in the target time period in the second key points corresponding to the m frames of the wallpaper image; determining fifth key points corresponding to the n frames of the wallpaper image superimposed with the interactive expression in the target time period based on the fourth key points corresponding to the n frames of the interactive expression image and the second key points corresponding to the n frames of the wallpaper image; The generating of the dynamic wallpaper video based on the preset wallpaper and the second key points corresponding to the m frames of the wallpaper image includes: generate a dynamic wallpaper video based on the preset wallpaper, the fifth key point corresponding to the n frames of wallpaper images superimposed with the interactive expression in the target time period, and the second key point corresponding to the other wallpaper images; wherein the second key point corresponding to the other wallpaper images refers to the second key point corresponding to the other frames of wallpaper images except the second key point corresponding to the n frames of wallpaper images in the target time period.

10. The method of claim 9, wherein, The determining the start time and the end time of the interactive expression includes: obtaining a first time of the interactive expression; obtaining a length of a target angle adjustment path corresponding to the interactive expression; determining a second time of the interactive expression according to the length of the target angle adjustment path corresponding to the interactive expression and the first time of the interactive expression; the first time of the interactive expression includes the start time or the end time of the interactive expression; and the second time of the interactive expression includes the end time or the start time of the interactive expression.

11. The method according to claim 9 or 10, characterized in that, The determining the fifth key point corresponding to the n frames of wallpaper images superimposed with the interactive expression in the target time period based on the fourth key point corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images includes: determining a position offset of the corresponding key point between each adjacent two frames of the fourth key point corresponding to the n frames of interactive expression images based on the fourth key point corresponding to the n frames of interactive expression images; determining the fifth key point corresponding to the n frames of wallpaper images superimposed with the interactive expression in the target time period based on the position offset of the corresponding key point between each adjacent two frames of the fourth key point corresponding to the n frames of interactive expression images and the second key point corresponding to the n frames of wallpaper images.

12. The method of claim 1, wherein, In response to the personalized wallpaper triggering operation, obtaining the angle information of the face orientation of the user includes: In response to the user gazing at the electronic device, obtaining the angle information of the face orientation of the user.

13. An interactive device, characterized by It includes: an obtaining unit configured to, in response to a personalized wallpaper triggering operation, obtain angle information of a face orientation of a user; a determining unit configured to determine target angle information of a face orientation of a preset wallpaper based on the angle information of the face orientation of the user; a processing unit configured to obtain the preset wallpaper, perform angle adjustment on a face orientation of a face information part in the preset wallpaper based on the target angle information of the face orientation of the preset wallpaper, and obtain a target wallpaper; the face orientation of the face information part in the target wallpaper is parallel to the face orientation of the user, and the face in the face information part in the target wallpaper faces the face of the user; a display unit configured to display the target wallpaper.

14. An electronic device, comprising: The computer readable storage medium includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method of any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer readable storage medium includes a stored program, wherein when the program runs, the device where the computer readable storage medium is located is controlled to execute the method of any one of claims 1-12.

Images