Animation generation method and device and electronic equipment

Abstract

The invention provides an animation generation method and device and electronic equipment, and relates to the technical field of image processing.The generation method comprises the steps that the electronic equipment obtains a first picture under the operation of a user and understands the content of the first picture, and then a first mirror moving direction is determined according to the display content of the first picture; the animation is generated according to the first mirror moving direction and the first picture, the first mirror moving direction is the moving direction of the mirror moving corresponding to the animation, and in other words, the animation displays the first picture to the user according to the first mirror moving direction. The moving direction of the moving mirror can be determined according to the display content of the picture, the moving mirror can be flexibly used according to the picture content, the picture feeling and immersion feeling of the animation are enhanced, the high-quality animation with the harmonious moving mirror can be generated, and the display effect of the animation is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an animation generation method, generation device and electronic equipment. Background Art

[0002] With the continuous development of terminal technology, more and more users are starting to use pictures to record their lives. For some satisfactory pictures, users also want to make them into animations for playback and display. In order to meet this demand, the technology of generating animations from pictures has emerged. Users can combine multiple pictures in the local gallery, splice them into an animation, and then share them on social platforms. The animation can also be used as a dynamic wallpaper for the desktop or screen saver, thereby broadening the source of dynamic wallpapers for terminals.

[0003] When the terminal device generates an animation through a picture, the picture can be displayed by moving the lens to improve the display effect of the picture in the animation. For example, the picture in the animation can be displayed from left to right, from top to bottom, or zoomed.

[0004] At present, when a terminal device generates an animation with a camera movement display effect, the terminal device can process the pictures in the animation according to the preset camera movement direction. For example, if the preset camera movement direction is from left to right, the pictures are displayed in the generated animation according to the camera movement direction from left to right. However, multiple pictures are usually added to the animation, and the current technology can only display the multiple pictures according to a fixed camera movement direction. The use of the camera movement is not flexible enough, resulting in poor animation display effect. Summary of the invention

[0005] The embodiments of the present application provide an animation generation method, generation device and electronic device, which can determine the movement direction of the camera according to the display content of the picture, can generate high-quality animation with harmonious camera movement, and improve the display effect of the animation.

[0006] In a first aspect, a method for generating an animation is provided, comprising: acquiring a first picture; determining a first camera movement direction according to the display content of the first picture; and generating the animation according to the first camera movement direction and the first picture, wherein the first camera movement direction is the movement direction of the camera corresponding to the animation.

[0007] According to the animation generation method provided by the embodiment of the present application, the moving direction of the camera (recorded as the first camera movement direction) can be first determined according to the display content of the picture (recorded as the first picture), and then the animation can be generated according to the first camera movement direction and the first picture, that is, the camera is displayed according to the first camera movement direction to generate the animation. Since the moving direction of the camera is determined according to the display content of the picture, the moving direction of the camera and the display content of the picture can be associated and matched, and the camera movement can be flexibly used according to the picture content, which enhances the picture sense and immersion of the animation, and can generate a high-quality animation with harmonious camera movement, thereby improving the display effect of the animation.

[0008] Since the movement direction of the camera is correlated with the displayed content of the picture, the use of the camera is more harmonious and smooth, the quality of the generated animation is higher and the sense of picture is stronger, thereby improving the display effect of the animation, providing users with a coherent immersive experience and improving the user's visual experience.

[0009] Optionally, the first picture may be obtained by the electronic device itself through shooting, or may come from the Internet (such as a social platform).

[0010] Optionally, the first image can also be obtained from other animations, which can be, for example, images or videos in GIF format. For example, the first image can be obtained by taking a screenshot or extracting frames from a video. This application does not limit the specific source of the first image.

[0011] Optionally, the specific method of "determining the first camera direction according to the content of the picture" is not limited in the embodiment of the present application. For example, the subject in the first picture can be identified, and the first camera direction can be determined according to the type of the subject. The subject here can be, for example, a human, an animal, a plant, a mountain, a river, a toy, jewelry, a fruit, a building, furniture, the sun, the moon, a vehicle, etc., but is not limited thereto. The first camera direction can also be determined according to a finer type of subject identified therein, for example, the category of animals can be divided into cats, dogs, lions, elephants or birds, etc., and the first camera direction corresponding to different types of animals may be different. And / or, the first camera direction can also be determined according to factors such as the number of subjects, the position in the picture, and the proportion of the entire picture. The first camera direction can also be determined according to factors such as the layout and color tone of the picture. In addition, when the subject is a person, the first camera direction can also be determined according to factors such as the expression, action, and gesture of the person.

[0012] Optionally, when an animation is generated by multiple pictures, the "animation" in "the first camera movement direction is the camera movement direction corresponding to the animation" can also be understood or replaced by an animation clip. For example, the multiple pictures include a first picture and a second picture, the animation clip corresponding to the first picture is a first animation clip, the animation clip corresponding to the second picture is a second animation clip, the first camera movement direction is the camera movement direction corresponding to the first animation clip, and the second camera movement direction is the camera movement direction corresponding to the second animation clip. The first animation clip and the second animation clip are spliced ​​together to form a complete animation.

[0013] In a possible implementation, determining the first camera movement direction according to display content of the first image includes: identifying the display content of the first image to obtain a first orientation of the target object; and determining the first camera movement direction according to the first orientation.

[0014] Determining the camera direction through the directional information of the target object in the picture can make the determined camera movement direction more closely matched with the picture content, making the camera movement more harmonious and smooth, thereby enhancing the visual and immersive feeling of the animation, and facilitating the generation of high-quality animation with harmonious camera movement, improving the display effect of the animation, and increasing playability and fun.

[0015] Optionally, the target object can be any object in the first picture, such as a person, an animal, a plant (such as a tree), a scenery (such as mountains, rivers, etc.), a building (such as a house, a bridge, etc.), a road, a toy, jewelry, fruit, furniture, the sun, the moon, a vehicle, etc., but is not limited thereto.

[0016] Optionally, the target object may be the subject of the photograph or may not be the subject of the photograph. For example, the target object may be a river or a path in the shooting background, which is not limited in the present application.

[0017] Optionally, the display content of the first picture can be understood according to an image understanding neural network model to obtain or determine the first orientation.

[0018] In a possible implementation, before identifying the display content of the first image to obtain a first pointing of the target object, the method further includes: identifying the display content of the first image to obtain a plurality of objects; and determining the target object according to the plurality of objects.

[0019] That is to say, in some scenarios, when recognizing the first image, multiple objects (such as multiple people) may be recognized. In this case, the target object can be first determined based on the multiple objects, and then the first direction corresponding to the target object can be identified. The above settings can enable the generation method provided by the present application to meet the usage requirements in more scenarios, or enable the generation method provided by the present application to have a wider scope of application. The present application does not specifically limit the specific implementation method of determining the target object based on multiple objects.

[0020] In a possible implementation, determining the target object based on the multiple objects includes: determining the subject among the multiple objects as the target object; or, determining the target object from the multiple objects based on preset priority information; or, determining the target object from the multiple objects based on the proportion of the screen occupied by the multiple objects; or, randomly selecting the multiple objects to obtain the target object.

[0021] That is to say, the electronic device understands the content of the first picture and may identify multiple objects. At this time, the electronic device can determine the aforementioned target object based on the multiple objects according to preset logical rules. Optionally, the electronic device can determine the subject (such as a person or an animal) among the multiple objects as the target object. Optionally, the electronic device can determine an object with the highest priority as the target object based on preset priority information. For example, different objects such as people, rivers, trees, buildings, roads, etc. have different priorities, and the person with the highest priority can be determined as the target object. Optionally, the electronic device can also determine the object with the largest proportion of the screen among the multiple objects as the target object. For example, if there are multiple people in the picture, the person with the largest proportion of the screen can be determined as the target object. Optionally, the multiple objects can also be randomly extracted, and the extracted objects can be determined as the target objects.

[0022] In a possible implementation manner, determining the first camera movement direction according to the first pointing includes: determining a direction along the first pointing as the first camera movement direction, or determining a direction opposite to the first pointing as the first camera movement direction.

[0023] By determining the direction along or against the first pointing direction as the first camera movement direction, the method is simple and easy to implement, and can better associate and match the first camera movement direction with the image content, thereby enhancing the visual sense and immersion of the animation and improving the user's visual experience.

[0024] Optionally, the first direction may be a sight line direction from the lower left to the upper right, and the first camera movement direction is a direction opposite to (relative to) the sight line direction, for example, the first camera movement direction may be from right to left, from top to bottom, or from the upper right to the lower left (i.e., a combination of the two directions from right to left and from top to bottom), which is not limited in this application. The situations in other directions are similar and will not be described in detail here.

[0025] Optionally, the direction along the first pointing direction may also be determined as the first camera movement direction. For the sight line direction from the lower left to the upper right, the first camera movement direction may be from left to right, from bottom to top, or from the lower left to the upper right (i.e., a combination of the two directions from left to right and from bottom to top), which is not limited in this application. The situations in other directions are similar and will not be described in detail here.

[0026] In a possible implementation manner, the first orientation includes a face orientation, a sight line direction, a moving direction, or an extending direction.

[0027] For example, the first direction may be the face orientation or sight direction of the subject (eg, a person or an animal), the moving direction of the person, animal, or object, or the extending direction of a path or a river.

[0028] For example, the target object may be a person, and the facial orientation or line of sight direction may be determined by a facial feature recognition (detection) algorithm (such as a deep neural network model).

[0029] Optionally, the target object here belongs to any object in the first picture, and may be the subject of the shooting or not. For example, the target object may be an object in the background where the subject of the shooting is located. For example, the target object may be a person, an animal, or a non-living thing that can move (such as a balloon, a white cloud, or a car) in the first picture. In this case, the first direction may be the moving direction of the person, animal, or object. For another example, the target object may also be a river or a path. In this case, the first direction may be the extension direction of the river or the path, etc., but is not limited thereto.

[0030] In a possible implementation manner, the first direction includes at least one of from front to back, from back to front, from left to right, from right to left, from top to bottom, and from bottom to top.

[0031] Optionally, the above directions can be defined with reference to the lens of the camera movement. Among them, from left to right, from right to left, from top to bottom, and from bottom to top are all plane movements within the picture (paper surface), the four directions of up, down, left, and right are the directions of up, down, left, and right of the picture seen by the user's naked eyes, and the front and back directions are the directions perpendicular to the picture (paper surface), the direction close to the paper surface is forward, and the direction away from the paper surface is backward. Therefore, the above-mentioned direction from front to back is the direction away from the picture (paper surface), and from back to front is the direction gradually approaching the picture (paper surface).

[0032] In a possible implementation, determining the first camera movement direction according to the display content of the first image includes: identifying the display content of the first image to obtain multiple directions; and determining the first camera movement direction according to the multiple directions.

[0033] That is to say, in some scenarios, when recognizing the first image, multiple first directions (such as multiple sight lines or face orientations) may be recognized, and the first camera movement direction can be determined based on the multiple directions. The above settings enable the generation method provided by the present application to meet the use requirements in more scenarios, or enable the generation method provided by the present application to have a wider range of adaptability. The present application does not specifically limit the specific implementation method of determining the first camera movement direction based on multiple directions.

[0034] In a possible implementation, determining the first camera movement direction according to the multiple pointings includes: determining a first direction from the multiple pointings according to preset priority information, and determining the first camera movement direction according to the first direction; or, determining a first direction from the multiple pointings according to other display information in the display content of the first picture, and determining the first camera movement direction according to the first direction, wherein the other display information is information other than the multiple pointings; or, determining the first camera movement direction based on the multiple pointings and a preset artificial intelligence model.

[0035] That is to say, the electronic device understands the content of the first picture and may identify multiple directions. At this time, the electronic device can determine the aforementioned first camera movement direction according to the multiple directions based on the preset logical rules. For example, the electronic device can use the highest priority direction (such as face orientation or line of sight direction) to determine the first camera movement direction according to the preset priority information. For another example, the electronic device can also determine the first direction from multiple directions in combination with other information about the content of the picture (such as the type, layout, and proportion of the screen occupied by the subject, etc.), and then determine the first camera movement direction based on the first direction. For another example, the multiple directions can also be used as input parameters to jointly determine the first camera movement direction according to preset rules or algorithms (such as AI models). This application does not impose any restrictions on the specific implementation method.

[0036] Optionally, the multiple directions include at least one of a face direction, a sight line direction, a moving direction, or an extension direction. For example, the multiple directions include multiple face directions, or multiple sight line directions, or at least one face direction (sight line direction) and a moving direction of at least one person or object. Or the multiple directions may also include an extension direction of a water flow or a path.

[0037] In a possible implementation, the generating method further includes: acquiring a depth map of the first image; and generating the animation according to the first camera movement direction and the first image includes: generating the animation according to the first camera movement direction, the depth map and the first image.

[0038] Through the above settings, the generated animation has a 3D effect. Under different camera angles, the relative size and relative position (distance) of different objects in the picture can change, thereby enhancing the picture sense and immersion of the animation, improving the display effect of the animation, and thus improving the user's viewing experience.

[0039] Optionally, the depth map can also be obtained by a stereo camera or a time of flight (TOF) camera. That is, when the camera acquires the first image, it also acquires the depth map corresponding to the first image. In other words, the first image itself contains the depth map information.

[0040] In a possible implementation manner, obtaining the depth map of the first image includes: performing monocular depth estimation on the first image to obtain the depth map.

[0041] In a possible implementation, generating the animation according to the first camera direction, the depth map and the first picture includes: performing depth layering on the depth map to identify a depth edge area; performing color filling on the depth edge area to obtain an edge background image; generating a 3D point cloud according to the first picture, the depth map and the edge background image; and generating the animation according to the first camera direction and the 3D point cloud.

[0042] Optionally, the 3D point cloud may include color information at different positions, and in this case, the animation may be directly generated according to the first camera movement direction and the 3D point cloud.

[0043] Optionally, when the 3D point cloud does not include the color information, the animation may be generated according to the first camera direction, the 3D point cloud and (the color information of) the first picture.

[0044] In a possible implementation manner, the generating method further includes: displaying the animation.

[0045] Optionally, the animation here can be an animation effect (dynamic effect) rendered and displayed (temporarily output) in real time by the electronic device when it is displayed. The temporarily output animation effect can also be generated or saved as a video file or GIF dynamic image for playback the next time it is displayed, or for users to share on social media.

[0046] In a possible implementation, the animation is displayed as a dynamic wallpaper, thereby broadening the sources of dynamic wallpapers for the terminal.

[0047] In a possible implementation, the animation is a video or a graphics interchange format (GIF) picture, but is not limited thereto. For example, the animation may also be a dynamic picture in other formats.

[0048] In a second aspect, an animation generation device is provided, comprising: an acquisition unit for acquiring a first picture; a determination unit for determining a first camera movement direction according to the display content of the first picture; and a generation unit for generating the animation according to the first camera movement direction and the first picture, wherein the first camera movement direction is the movement direction of the camera corresponding to the animation.

[0049] In a possible implementation manner, the determination unit is specifically configured to: identify display content of the first image to obtain a first orientation of a target object; and determine the first camera movement direction according to the first orientation.

[0050] In a possible implementation, before identifying the display content of the first image to obtain a first pointing of the target object, the determination unit is specifically used to: identify the display content of the first image to obtain multiple objects; and determine the target object based on the multiple objects.

[0051] In a possible implementation, the determination unit is specifically used to: determine the subject among the multiple objects as the target object; or, determine the target object from the multiple objects based on preset priority information; or, determine the target object from the multiple objects based on the proportion of the screen occupied by the multiple objects; or, randomly select the multiple objects to obtain the target object.

[0052] In a possible implementation manner, the determining unit is specifically configured to: determine a direction along the first pointing direction as the first camera movement direction, or determine a direction opposite to the first pointing direction as the first camera movement direction.

[0053] In a possible implementation manner, the first orientation includes a face orientation, a sight line direction, a moving direction, or an extending direction.

[0054] In a possible implementation manner, the first direction includes at least one of from front to back, from back to front, from left to right, from right to left, from top to bottom, and from bottom to top.

[0055] In a possible implementation manner, the determination unit is specifically configured to: identify display content of the first image to obtain a plurality of directions; and determine the first camera movement direction according to the plurality of directions.

[0056] In a possible implementation, the acquisition unit is further used to: acquire a depth map of the first image; and the generation unit is specifically used to: generate the animation according to the first camera movement direction, the depth map and the first image.

[0057] In a possible implementation manner, the acquisition unit is specifically configured to: perform monocular depth estimation on the first image to acquire the depth map.

[0058] In a possible implementation, the generating unit is specifically used to: perform depth layering on the depth map to identify a depth edge area; perform color filling on the depth edge area to obtain an edge background image; generate a 3D point cloud based on the first image, the depth map and the edge background image; and generate the animation based on the first camera direction and the 3D point cloud.

[0059] In a possible implementation manner, the generating device further includes: a display unit, configured to display the animation.

[0060] In a possible implementation, the animation is displayed as a dynamic wallpaper.

[0061] In a possible implementation, the animation is a video or a picture in GIF format.

[0062] According to a third aspect, an electronic device is provided, comprising: a memory storing instructions; and a processor, wherein when the instructions are executed by the processor, the electronic device executes the method provided by any possible implementation of the first aspect.

[0063] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is run on a computer, the computer is enabled to execute the method provided by any possible implementation of the first aspect.

[0064] In a fifth aspect, a computer program product is provided, characterized in that it comprises: a computer program code, which, when executed on a computer, enables the computer to execute a method provided by any possible implementation of the first aspect.

[0065] In a sixth aspect, a chip is provided, comprising: a processor, configured to call and run a computer program from a memory, so that an electronic device equipped with the chip executes a method provided by any possible implementation of the first aspect.

[0066] It can be understood that the animation generation device provided in the second aspect, the electronic device provided in the third aspect, the computer-readable storage medium provided in the fourth aspect, the computer program product provided in the fifth aspect, and the chip provided in the sixth aspect are all used to execute the method provided in the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a schematic diagram of a push-in lens provided in an embodiment of the present application.

[0068] Figure 2 It is a schematic diagram of a zoom lens provided in an embodiment of the present application.

[0069] Figure 3 It is a schematic diagram of a pan lens provided in an embodiment of the present application.

[0070] Figure 4 It is a schematic diagram of a panning lens provided in an embodiment of the present application.

[0071] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0072] Figure 6 It is a software structure block diagram of the electronic device provided in the embodiment of the present application.

[0073] Figure 7 It is a schematic diagram of human-computer interaction in an application scenario provided by an embodiment of the present application.

[0074] Figure 8 yes Figure 7 Schematic diagram of the playback process of the animation generated in the scene.

[0075] Fig. 9 This is a schematic diagram of human-computer interaction in another application scenario provided by an embodiment of the present application.

[0076] Fig.10 yes Fig. 9 A schematic diagram of the playback process of an example of the first picture of the animation generated in the scene.

[0077] Fig.11 This is another example of the playback process of the first picture.

[0078] Fig.12 This is another schematic diagram of the playback process of the first picture.

[0079] Fig.13 yes Fig. 9 Schematic diagram of the playback process of the second picture of the animation generated in the scene.

[0080] Fig.14 is a schematic diagram of multiple examples of the first orientation of the first picture.

[0081] Fig.15 This is another schematic diagram of the playback process of the first picture.

[0082] Fig.16 It is a schematic diagram of a process of generating a 3D animation according to a first image.

[0083] Fig.17 It is a flowchart of the animation generation method provided in the embodiment of the present application.

[0084] Fig.18 It is a schematic block diagram of an animation generation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] The technical solution in the present application will be described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0086] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0087] The term "comprising" herein indicates the presence of the described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections. The terms "comprising", "including", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized. Below, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise stated, the meaning of "multiple" is two or more.

[0088] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0089] With the continuous development of terminal technology, more and more users are beginning to use pictures to record their lives. For some satisfactory pictures, such as landscape photos, users also want to make them into animations for playback and display. In order to meet this demand, the technology of generating animations from pictures (such as slide show technology) came into being. Users can combine multiple pictures in the local gallery, splice them into an animation, and then share them on social platforms. In some cases, you can also use a picture to generate an animation, which can also be used as a dynamic wallpaper for the desktop or screen saver, thereby broadening the source of dynamic wallpapers for terminals.

[0090] Camera movement usually refers to motion shots. In the process of video shooting, camera movement is an important form of narrative expression that can reflect creative agility and artistic value. The reasonable use of camera movement in the video helps to portray the image of the characters, the characteristics of the characters, lay the scene atmosphere, and also promote the video narrative. Different camera movements can control different rhythms in the narrative and bring different visual experiences and psychological hints. When the terminal device generates animations through pictures, it can display the pictures by camera movement to improve the display effect of the pictures in the video. For example, the pictures in the video can be displayed from left to right, from top to bottom, or zoomed.

[0091] In order to improve the quality of video shooting, the industry has various camera movement shooting methods. Common camera movement methods can include push-in, pull-out, pan, rotate, follow, etc. The effects of different camera movement methods in the video are manifested as zooming in and out of the picture, approaching or moving away from the subject, translation or rotation, etc., which can increase the atmosphere and emotion of the video.

[0092] In order to facilitate understanding of the technical solution of the present application, the following first introduces the various camera movement types (methods) involved in the embodiments of the present application in conjunction with the accompanying drawings. Figure 1 This is a schematic diagram of the push lens. Figure 2 This is a schematic diagram of the pull lens. Figure 3 This is a schematic diagram of the moving lens. Figure 4 This is a schematic diagram of a panning shot.

[0093] A push shot is a shot that moves forward slowly or rapidly while the subject remains in the same position. The scene changes from a long shot to a full shot, a medium shot, a close shot, or even a close-up. This can be seen as the field of view (FOV) of this shot gradually decreasing, or the focal length gradually increasing. The main function of this shot is to highlight the subject, gradually focus the audience's visual attention, strengthen the visual experience, and create a state of scrutiny. For example, when shooting a person, as the camera moves forward, the proportion of the person in the picture gradually increases, making the person more prominent.

[0094] Figure 1 A martial artist is performing Tai Chi, and the martial artist is the subject (object). Figure 1 From the relative sizes of the warriors in different mobile phone screens, we can see that as the camera moves from back to front, the area occupied by the warriors in the entire screen gradually increases, that is, it changes from a long shot to a medium shot and a close shot in turn. This group of shots can be considered as a push shot.

[0095] The movement direction of the pull-in lens is opposite to that of the push-in lens. The lens moves backward from near to far, away from the subject, the framing range changes from small to large, the subject changes from large to small, and the distance from the audience gradually increases. The image on the screen changes from few to many, and from a part to the whole. In terms of the scene, it changes from a close-up or a close or medium shot to a panoramic or long shot. It can be seen that the FOV of this lens gradually increases, or the focal length gradually decreases. The main function of the pull-in lens is to explain the environment in which the characters are located and increase the atmosphere of the picture.

[0096] Figure 2 A martial artist is performing Tai Chi, and the martial artist is the subject of the photo. Figure 2 From the relative sizes of the warriors in different mobile phone screens, it can be seen that as the camera gradually zooms out from the front to the back, the area occupied by the warrior in the entire screen gradually becomes smaller, that is, it changes from a close shot to a medium shot and a long shot in turn. This group of shots can be considered as a pull shot.

[0097] The panning lens is similar to the push-pull lens, but the movement trajectory is different. The push-pull lens is the forward and backward movement of the lens, while the panning lens is the unidirectional movement of the lens along a fixed trajectory. "Move" can be understood as parallel movement. The direction of movement can be horizontal, vertical, or tilted at a certain angle (that is, both horizontal and vertical). The movement trajectory is usually mainly straight.

[0098] Horizontal lens movement can expand the horizontal field of view. The lens moves horizontally left and right to shoot, similar to the state of people walking and looking in life. For example, when shooting vast natural scenery, horizontal movement can be used. Vertical lens movement can expand the vertical field of view. For example, when shooting tall subjects such as buildings and mountains, vertical movement can be used.

[0099] like Figure 3 As shown, for panning, the movement direction of the lens may include from left to right, from right to left, from bottom to top, and from top to bottom, or a combination of two of the above four directions. Taking from left to right as an example, in the process of the lens gradually moving from left to right, the warrior moves to the left relative to the entire mobile phone screen, and the size of the warrior in the entire screen will not change. If there are other objects, the relative position of the warrior and other objects will not change.

[0100] Panning is similar to moving shots, but when shooting, panning means rotating the phone or camera without moving, and the lens moves in an arc. For example, if you stand still and hold the phone, and shoot from left to right, the path of the phone's movement is an arc, and the key point is to stay still. "Panning" will show the scenes in front of the camera one by one, making the picture more immersive. Panning is suitable for shooting continuous actions or large scenes, just like a person's eyes patrolling the subject in a certain direction. Panning can represent the eyes of the character, looking at everything around. It has a unique role in describing space and introducing the environment. Panning left and right is often used to introduce large scenes, and panning up and down is often used to show the majesty and steepness of tall objects.

[0101] like Figure 4 As shown, for panning, the panning direction of the lens may include from left to right, from right to left, from bottom to top, and from top to bottom, as well as a combination of two of the above four directions. Taking from left to right as an example, when the lens gradually pans from left to right, the warrior is equivalent to the entire mobile phone screen moving to the left, and the relative position and relative size of the warrior and other objects in the entire screen will change.

[0102] Push shots, pull shots, dolly shots and pan shots are several basic types of camera movements. In actual applications, camera movements may be composed of multiple movements described above, that is, camera movements may include multiple of the above four types at the same time. For example, push shots and dolly shots may be included at the same time, or pull shots and dolly shots may be included at the same time, or dolly shots and pan shots may be included at the same time, or pull (push) shots, dolly shots and pan shots may be included at the same time, so that the captured video can have a better display effect.

[0103] Table 1 shows the lens movement direction corresponding to different types of lens movement, where the movement includes movement and rotation. As shown in Table 1, in the embodiment of the present application, the movement (movement) direction of the push lens can be understood as the lens gradually approaching the paper surface in the direction perpendicular to the paper surface, that is, from back to front. The movement (movement) direction of the pull lens can be understood as the lens gradually moving away from the paper surface in the direction perpendicular to the paper surface, that is, from front to back. The movement (movement) direction of the shift lens can be understood as the lens making horizontal movement, vertical movement or tilted movement in a plane parallel to the paper surface, that is, the movement direction of the shift lens includes from left to right, from right to left, from bottom to top and from top to bottom, and a combination of two of the above four directions. The movement (shaking) direction of the pan lens can be understood as the lens rotating horizontally, vertically or tilted with a fixed point away from the paper surface as the center of the circle, that is, the shaking direction of the pan lens includes from left to right, from right to left, from bottom to top and from top to bottom, and a combination of two of the above four directions.

[0104] Table 1: Lens movement directions corresponding to different types of camera movements

[0105]

[0106] At present, when a terminal device generates an animation with a camera movement display effect based on a picture, the terminal device can process the picture in the animation according to a preset camera movement direction. For example, if the preset camera movement direction is from left to right, the picture is displayed in the generated animation according to the camera movement direction from left to right. For another example, if the preset camera movement direction is from the upper right to the lower left, the picture is displayed in the generated animation according to the camera movement direction from the upper right to the lower left. However, multiple pictures are usually added to the animation, and different pictures display different contents, which may even differ greatly. The current technology can only display the multiple pictures according to a fixed and unique camera movement direction, and the use of the camera movement is not flexible enough, resulting in poor display effect of the animation.

[0107] In view of this, an embodiment of the present application provides a method for generating an animation, which can be, for example, a video or a picture in GIF format. According to the generation method, the moving direction of the camera (recorded as the first camera direction) can be determined first according to the display content of the picture (recorded as the first picture), and then the animation is generated according to the first camera direction and the first picture, that is, the camera displays the first picture according to the first camera direction, and then the animation is generated. Since the moving direction of the camera is determined according to the display content of the picture, the moving direction of the camera and the display content of the picture can be associated and matched, and the camera can be flexibly used according to the picture content, which enhances the picture sense and immersion of the animation, and can generate a high-quality animation with harmonious camera movement, thereby improving the display effect of the animation.

[0108] Further, in the embodiment of the present application, the displayed content of the first picture can be understood by a preset algorithm (e.g., an artificial intelligence (AI) model), and at least one directional information in the picture (referred to as a first directional information) can be identified. The first directional information can be, for example, the facial orientation, sight direction, movement direction, etc. of the subject (e.g., a person or an animal), or the extension direction of a path or a river, etc. The first directional information can include, for example, at least one of front to back, back to front, left to right, right to left, top to bottom, and bottom to top.

[0109] Then, the moving direction of the camera (i.e., the first camera movement direction) is determined according to the first direction. For example, the direction along the first direction can be determined as the first camera movement direction, or the direction opposite to the first direction can be determined as the first camera movement direction, but it is not limited to this. Then, an animation can be generated according to the first camera movement direction and the first picture. Since the matching degree between the camera movement direction and the picture content is higher, the use of the camera movement is more harmonious and smooth, so that the quality of the generated animation is also higher and the picture sense is stronger, thereby improving the display effect of the animation, providing users with a coherent immersive experience, and improving the user's visual experience.

[0110] The animation generation method provided in the embodiment of the present application can be applied to electronic devices, or it can be a separate application program, which can implement the animation generation method in the present application. For example, the generation method can be applied to mobile phones, tablet computers, cameras, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA) and other electronic devices. The embodiment of the present application does not impose any restrictions on the specific type of electronic devices.

[0111] For example, Figure 5 1 is a schematic diagram of the structure of the electronic device 100 provided in the embodiment of the present application. Figure 5 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0112] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0113] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0114] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0115] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0116] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0117] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). The I2S interface can be used for audio communication. The PCM interface can also be used for audio communication to sample, quantize and encode analog signals. The UART interface is a universal serial data bus for asynchronous communication; the bus can be a bidirectional communication bus, which converts the data to be transmitted between serial communication and parallel communication. The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193; the MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. The GPIO interface can be configured by software, and the GPIO interface can be configured as a control signal or a data signal. The USB interface 130 is an interface that complies with the USB standard specification, and can specifically be a Mini USB interface, a MicroUSB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.

[0118] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0119] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0120] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0121] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0122] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0123] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0124] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0125] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering processing, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0126] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division-synchronous code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GNSS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0127] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0128] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini LED, Micro LED, quantum dot light emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0129] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0130] The ISP is used to process data fed back by the camera 193. The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos.

[0131] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0132] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, and at least one application required for a function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.). In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0133] The external memory interface 120 can be used to connect an external memory, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0134] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0135] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 are arranged in the processor 110. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" and "microphone", is used to convert sound signals into electrical signals. The headphone interface 170D is used to connect wired headphones. The headphone interface 170D can be a USB interface 130, or it can be a 3.5mm open mobile electronic equipment platform (open mobile terminal platform, OMTP) standard interface, and the cellular telecommunications industry association of the USA (cellular telecommunications industry association of the USA, CTIA) standard interface.

[0136] The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The electronic device 100 can receive the button input and generate a key signal input related to the user settings and function control of the electronic device 100. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 195, or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device 100. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0137] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0138] Figure 6 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0139] like Figure 6 As shown, the software system using a layered architecture is divided into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the software system can be divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android Runtime) and the system library, and the kernel layer.

[0140] The application layer may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0141] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer may include some predefined functions.

[0142] For example, the application framework layer includes the window manager, content provider, view system, telephony manager, resource manager, and notification manager.

[0143] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, and capture the screen.

[0144] Content providers are used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, and phone books.

[0145] The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0146] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (connected or hung up).

[0147] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, and video files.

[0148] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used for download completion notifications and message reminders. The notification manager can also manage notifications that appear in the system's top status bar in the form of icons or scrolling text, such as notifications from applications running in the background. The notification manager can also manage notifications that appear on the screen in the form of dialog windows, such as prompting text messages in the status bar, emitting reminder sounds, vibrating electronic devices, and flashing indicator lights.

[0149] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0150] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0151] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0152] The system library may include multiple functional modules, such as: surface manager, media library, 3D graphics processing library (such as open graphics library for embedded systems (OpenGL ES)) and 2D graphics engine (such as skia graphics library (SGL)).

[0153] The surface manager is used to manage the display subsystem and provide the fusion of 2D layers and 3D layers for multiple applications.

[0154] The media library supports playback and recording of multiple audio formats, playback and recording of multiple video formats, and still image files. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, moving picture experts group audio layer III (MP3), advanced audio coding (AAC), adaptive multi-rate (AMR), joint photographic experts group (JPG) and portable network graphics (PNG).

[0155] The 3D graphics processing library can be used to implement 3D graphics drawing, image rendering, compositing and layer processing.

[0156] A 2D graphics engine is a drawing engine for 2D drawings.

[0157] The kernel layer is the layer between hardware and software. The kernel layer can include driver modules such as display driver, camera driver, audio driver and sensor driver.

[0158] In the embodiment of the present application, the user opens the camera application. Figure 6 The camera application in the application layer starts the shooting function and sends instructions to the kernel layer to mobilize the camera driver, sensor driver and display driver of the kernel layer, so that the electronic device 100 can start the camera to collect images. During the camera image collection process, light is transmitted to the image sensor through the camera, and the image sensor performs photoelectric conversion on the light signal and converts it into an image visible to the user's naked eye. The output image data is transmitted to the Figure 6The system library in the 3D graphics processing library and the image processing library implement drawing, image rendering, synthesis and layer processing, etc., to generate a display layer; the surface manager performs fusion processing on the display layer, etc., and passes it to the content provider, window manager and view system of the application framework layer to control the display of the display interface. Finally, the image is displayed in the image preview area of ​​the camera application or the display screen of the electronic device 100.

[0159] Furthermore, when the user is satisfied with one or more photos taken, the photo can be used to generate an animation, such as a video or a dynamic picture in GIF format. The image processing library obtains the photo and understands the content, and then determines the direction of movement of the camera, and then displays the photo according to the direction of movement to generate an animation.

[0160] The surface manager obtains the animation data and processes it. Specifically, the surface manager can also provide layer synthesis services. The three-dimensional graphics processing library, the two-dimensional graphics engine, etc. can draw and render the synthesized layers, and send them to the display screen for display after rendering. For the playback window displayed on the display screen (such as the video playback window), data refresh, layer synthesis, drawing rendering and display sending processes can be performed according to the animation data to keep the display content of the playback window updated in real time.

[0161] With Figure 5 and Figure 6 Taking the electronic device (specifically a mobile phone) of the structure shown as an example, the animation generation method provided in the embodiment of the present application will be further introduced in combination with specific application scenarios. Figure 7 Schematic diagram of human-computer interaction in an application scenario provided by an embodiment of the present application. Figure 7 As shown, as a possible application scenario, the user can generate an animation through a static image (ie, the first image), and use the animation as a desktop wallpaper and / or a lock screen wallpaper.

[0162] Specifically, Figure 7 As shown in part (a) of the figure, the user can click or touch the "Settings" button on the mobile phone desktop to enter the settings page. Figure 7 In the setting page shown in part (b), there are options such as WLAN, Bluetooth, Mobile Network, Satellite Network, HyperTerminal, More Connections, Desktop and Personalization, and Display and Brightness. The user clicks on the "Desktop and Personalization" option. Figure 7 The desktop and personalized pages shown in part (c) include options such as themes, wallpapers, off-screen display, magazine lock screen, and icons. The user clicks the "wallpaper" option to enter the wallpaper settings page.

[0163] In such Figure 7The wallpaper page shown in part (d) includes options such as My Wallpapers, Recommended Wallpapers, and Nature Wallpapers. The user clicks the "Select from Gallery" option in My Wallpapers to enter the Gallery page. Figure 7 In the All Photos page shown in part (e), the user selects a favorite photo to use as wallpaper. The user clicks on a selected photo to automatically enter the wallpaper setting page. Figure 7 In the wallpaper setting page shown in part (f), the mobile phone can convert the 2D static photo selected in the previous step into a 3D animation under the user's operation, and further set the animation as the desktop wallpaper and / or lock screen wallpaper for use.

[0164] Specifically, Figure 7 As shown in part (f), the photo selected in the previous step is displayed on the wallpaper setting page. The user can choose to set the photo as a normal static wallpaper or a dynamic wallpaper. When the user determines that the photo needs to be used as a dynamic wallpaper, the "dynamic wallpaper" option in the page can be checked, and then the "apply" button can be clicked. In response to the user's click operation, the mobile phone converts the photo into an animation according to the animation generation method provided in the embodiment of the present application, and then displays and applies it. For example, after the user clicks the "apply" button, the photo is automatically converted into an animation, and a new page (not shown) pops up. In the new page, the user can choose to set the generated animation as a dynamic wallpaper for the desktop, or as a dynamic wallpaper for the lock screen, or as a dynamic wallpaper for both the desktop and the lock screen.

[0165] Figure 7 The mobile phone in can set the selected static image to animation according to the following logic:

[0166] The mobile phone obtains a picture selected by the user, which can be, for example, a photo of a lady. The mobile phone understands the content of the picture through a preset deep neural network model, and determines that the direction of the lady's face or line of sight is from front to back (i.e., from inside the paper to outside the paper, or from inside the screen to outside the screen), and then determines the direction opposite to the face direction - from back to front (i.e., from outside the paper to inside the paper, the direction the user looks at the screen) as the camera movement direction according to the preset logic. The camera movement direction is the movement direction of the push lens, so a push lens can be applied to the picture, that is, the camera moves from back to front to display the picture, and then an animation is generated.

[0167] Figure 8 yes Figure 7 Schematic diagram of the playback process of the animation generated in the scene. Figure 8 Three frames of this animation are shown. Figure 8From the three frames shown in parts (a), (b) and (c), it can be seen that as time goes on, the camera gradually approaches the lady in the picture, and the proportion of the lady in the picture gradually increases. In terms of scene, it changes from a distant shot to a medium shot, a close shot and even a close-up.

[0168] according to Figure 7 and Figure 8 In the embodiment shown, since the movement direction of the camera is determined according to the facial orientation or line of sight of the character in the picture, the use of the camera is more harmonious and smooth, and the camera can be flexibly used according to the content of the picture, which enhances the visual sense and immersiveness of the animation, and can generate high-quality animation with harmonious camera movement, thereby improving the display effect of the animation and thereby improving the user experience.

[0169] It is worth mentioning that in Figure 7 and Figure 8 In the illustrated embodiment, a dynamic wallpaper is generated by only one picture, and in other implementations, a dynamic wallpaper may also be generated by multiple pictures. At this time, for each of the pictures, the camera direction is determined according to the above logic, and a corresponding animation clip is generated according to the picture and its corresponding camera direction. Multiple pictures correspond to multiple animation clips, and a complete animation can be obtained by splicing the multiple animation clips, that is, a dynamic wallpaper generated by multiple pictures. It is easy to understand that the camera directions used (corresponding) by different pictures may be different, thereby making the application more flexible in the use of camera movement, making the camera direction in the animation more compatible with the picture, thereby improving the display effect of the animation.

[0170] Fig. 9 is a schematic diagram of human-computer interaction in another application scenario provided by an embodiment of the present application. Fig. 9 As shown, as another possible application scenario, the user can generate an animation through a static image, and share the animation on a social platform or send it to relatives and friends for viewing. The animation can be, for example, a video or a dynamic image in GIF format, but is not limited thereto.

[0171] like Fig. 9 As shown in part (a) of FIG, when the user is satisfied with the multiple pictures taken by the camera and / or obtained from the Internet, an animation can be generated based on the multiple pictures. At this time, the user can click the one-click to create a movie or short video clip button on the desktop. This article takes clicking one-click to create a movie as an example. After the user clicks the one-click to create a movie button, the local gallery page is entered.

[0172] like Fig. 9As shown in part (b) of FIG. 1 , the user first selects multiple pictures on the local map library page, and then clicks the one-click movie button. In response to the click operation, the mobile phone generates an animation based on the multiple pictures selected by the user.

[0173] like Fig. 9 As shown in part (c) of the figure, when the animation is generated, the animation playback page is automatically entered and the animation is automatically played. If the user is satisfied with the animation effect, he can click the export button to export and save the animation locally. The user can also share the animation on social platforms such as Weibo or Moments, and can also send the animation to relatives and friends through instant messaging software.

[0174] Fig. 9 The mobile phone in can set the selected static image to animation according to the following logic:

[0175] The mobile phone obtains two pictures selected by the user. For example, the two pictures are Fig.10 The first picture shown and Fig.13 The second picture is shown, wherein the first picture includes an animal doll and the second picture includes a dancer located in the woods.

[0176] The mobile phone first understands the content of the first picture through a preset deep neural network model, determines that the sight direction of the animal doll is towards the upper right (looking at the apple on the tree), and then determines the direction opposite to the sight direction - from right to left as the camera movement direction according to the preset logic. The camera movement direction can be the movement direction of a panning or shaking camera. Therefore, a panning and / or shaking camera can be applied to the picture, that is, the camera moves and / or shakes from right to left to display the first picture, thereby generating an animation clip corresponding to the first picture.

[0177] Fig.10 This is a schematic diagram of the playback process of an example of the first picture. Fig.10 The first picture is shown in Figure 3. The first picture is subjected to a panning and / or panning operation. Fig.10 It can be seen from the three frames shown in parts (a), (b) and (c) that, as time goes on, the camera gradually moves and / or shakes from right to left, and the animal doll gradually moves from left to right in the picture.

[0178] Optionally, the line of sight of the animal doll is towards the upper right direction, and the direction opposite to the line of sight - from top to bottom - can be determined as the camera movement direction according to preset logic. The first picture can be subjected to a panning and / or shaking shot, that is, the camera is moved and / or shaken from top to bottom to display the first picture, thereby generating an animation clip corresponding to the first picture.

[0179] Fig.11 is a schematic diagram of another example of the playback process of the first picture. Fig.11 The first picture is shown in Figure 3. The first picture is subjected to a panning and / or panning operation. Fig.11 It can be seen from the three frames shown in parts (a), (b) and (c) that, as time goes on, the camera gradually moves and / or shakes from top to bottom, and the animal doll gradually moves from bottom to top in the picture.

[0180] Optionally, the line of sight of the animal doll is from the lower left to the upper right, and the direction opposite to the line of sight - from the upper right to the lower left - can be determined as the camera movement direction according to a preset logic. A shifting and / or shaking camera can be applied to the first picture, that is, the camera moves and / or shakes from the upper right to the lower left to display the first picture, thereby generating an animation clip corresponding to the first picture.

[0181] Fig.12 is another example of the playback process of the first picture. Fig.12 The first picture is shown in Figure 3. The first picture is subjected to a panning and / or panning operation. Fig.12 It can be seen from the three frames shown in parts (a), (b) and (c) that, as time goes on, the camera gradually moves and / or shakes from the upper right to the lower left, and the animal doll gradually moves from the lower left to the upper right in the picture.

[0182] That is to say, for the sight line direction from the lower left to the upper right, the direction opposite to (relative to) the sight line direction may include the direction from right to left, from top to bottom, or from the upper right to the lower left (i.e., the combination of the two directions from right to left and from top to bottom), which is not limited in this application. The situations in other directions are similar and will not be described in detail here.

[0183] Optionally, the direction along the sight line direction may also be determined as the camera movement direction. For the sight line direction from the lower left to the upper right, the direction along the sight line direction may include the direction from left to right, from bottom to top, or from the lower left to the upper right (i.e., a combination of the two directions from left to right and from bottom to top), which is not limited in this application. The situations in other directions are similar and will not be described in detail here.

[0184] The specific process of generating an animation clip from the second picture is similar to that of the first picture. The mobile phone continues to understand the content of the second picture through a preset deep neural network model, determines that the direction of the dancer's sight in the picture is from the right front to the left back (with the direction of the camera as a reference), and then determines the direction opposite to the sight direction - from left to right as the camera direction according to the preset logic. The camera direction can be the direction of movement of the panning or shaking camera, so the panning and / or shaking camera can be applied to the second picture, that is, the camera moves and / or shakes from left to right to display the second picture, and then generates an animation clip corresponding to the second picture.

[0185] Fig.13 This is a schematic diagram of the playback process of the second picture. Fig.13 The third picture shows three frames of the animation clip corresponding to the second picture. Fig.13 It can be seen from the three frames shown in parts (a), (b) and (c) that, as time goes on, the camera gradually moves and / or shakes from left to right, and the dancer gradually moves from right to left in the picture.

[0186] Optionally, the camera direction corresponding to the second picture may also be from back to front, in which case a push-in shot may be applied to the second picture, that is, the camera gradually approaches the dancer in the picture, and the proportion of the picture occupied by the dancer gradually increases.

[0187] Optionally, the camera movement direction corresponding to the second picture may include both from back to front and from left to right. In this case, not only a push-in shot is applied to the second picture, but also a panning shot and / or a shaking shot may be applied to the second picture at the same time.

[0188] The animation clip corresponding to the first picture and the animation clip corresponding to the second picture are sequentially spliced ​​together to obtain a complete animation. The more pictures the user selects, the more animation clips need to be spliced ​​together, that is, the longer the duration of the complete animation. The user can change the order of multiple animation clips by adjusting the order of the pictures.

[0189] In an embodiment of the present application, the display content of the first image can be identified to obtain directional information (recorded as the first direction), and then the movement of the camera is determined according to the first direction. The first direction here is the directional information in the identified first image. The first direction can be, for example, the facial orientation and sight direction of the subject (such as a person or an animal) in the aforementioned embodiment, or the moving direction of a person, animal or object, or the extension direction of a path or a river. The first direction can, for example, include at least one of from front to back, from back to front, from left to right, from right to left, from top to bottom, and from bottom to top. The first direction here is further illustrated below.

[0190] Fig.14 is a schematic diagram of multiple examples of the first orientation of the first picture.

[0191] Fig.14 The first picture shown in part (a) of the figure includes a road, and the extension direction of the road in the picture is from the upper left to the lower right, so the extension direction can be determined as the first orientation, and the movement direction of the camera can be further determined based on the first orientation. Optionally, the movement direction of the camera can be against (toward) the extension direction, for example, from right to left, from bottom to top, or from bottom right to upper left. Optionally, the movement direction of the camera can be along (along) the extension direction, for example, from left to right, from top to bottom, or from upper left to lower right.

[0192] Fig.14 The first picture shown in part (b) includes a river, and the extension direction of the river in the picture is roughly from top to bottom, so this direction can be determined as the first direction, and the moving direction of the camera movement can be further determined based on the first direction. Optionally, the moving direction of the camera movement can be against the extension direction, for example, it can be from bottom to top. Optionally, the moving direction of the camera movement can be along the extension direction, for example, it can be from top to bottom. Optionally, the river has a left-right direction in the picture, and the moving direction of the camera movement can also include swinging or reciprocating in the left-right direction. That is to say, in an embodiment of the present application, the moving direction of the camera movement (i.e., the first camera movement direction) can include a specific direction, and can also include regular or irregular reciprocating swinging in a certain direction, and the present application does not limit this.

[0193] Fig.14 The first picture shown in part (c) of the figure includes a standing boy, with one hand on his waist and the fingers of the other hand pointing to the left, and the boy's line of sight is from front to back (i.e., from inside the paper to outside the paper). In other words, Fig.14 The first picture shown in part (c) includes multiple directions. At this time, the movement direction of the camera can be determined according to any one of the directions. For example, according to the direction of the finger in the picture, the movement direction of the camera can be from left to right or from right to left. Alternatively, according to the line of sight direction or the face orientation, the movement direction of the camera can be from back to front (push the camera) or from front to back (pull the camera). Which direction to use to determine the movement direction of the camera can be determined according to a preset priority, for example, the face orientation or the line of sight direction has the highest priority, or the movement direction of the camera can be determined in combination with multiple directions.

[0194] Fig.14The first picture shown in part (d) of FIG. 1 includes a man running. By understanding the content of the first picture, it can be known that the man's face direction, line of sight direction and moving direction (i.e., running direction) are all from left to right, so the first direction can be determined to be from left to right, and the moving direction of the camera can be from left to right or from right to left.

[0195] Fig.14 The first picture shown in part (e) of FIG. 1 includes a boy riding a bicycle. The boy rides the bicycle from right to left, and the boy's sight direction is from front to back (i.e., from inside the paper to outside the paper). That is, Fig.14 The first picture shown in part (e) of FIG. 1 also includes multiple directions. In this embodiment, according to the priority of the directions, the face direction or the sight direction can be used as the first direction (to determine the direction of the camera movement), and the direction of the camera movement can be from back to front (push camera movement) or from front to back (pull camera movement).

[0196] Fig.14 The first picture shown in part (f) of FIG. 1 includes an animal doll and a balloon. By understanding the content of the first picture, it can be known that the balloon is floating from bottom to top, and the animal doll's line of sight is looking toward the upper right direction at the floating balloon. In other words, Fig.14 The first picture shown in part (f) of FIG. 1 also includes multiple directions. In this embodiment, the moving direction of the camera can be determined based on information such as multiple directions (i.e., the floating direction of the balloon and the sight direction of the animal doll). For example, the moving direction of the camera can be from top to bottom or from bottom to top. In addition, the sight direction can also be used preferentially to determine the moving direction of the camera. The selection of the specific method depends on the internal implementation of the mobile phone or the pre-set rules, and this application does not make any special restrictions on this.

[0197] Fig.15 FIG. 1 is another schematic diagram of the playback process of the first picture. Fig.15 The first picture shown can be understood by understanding the content, that the picture includes two children, the two children's faces are facing different directions, the sight lines are also different, the two children are running at the same time, and the running direction is the same, both from left to right. At this time, the moving direction of the camera can be determined according to the running direction, for example, the moving direction of the camera can be from right to left or from left to right.

[0198] Fig.15 The first picture is shown in Figure 3. The first picture can be panned and / or panned. Fig.15It can be seen from the three frames shown in parts (a), (b) and (c) that, as time goes on, the camera gradually moves and / or shakes from left to right, and the child gradually moves from right to left in the picture.

[0199] In recent years, with the continuous development of image processing technology, it has become possible to generate 3D animations based on ordinary 2D static pictures. 3D pictures have better display effects and are favored by more and more users. Fig.15 In the illustrated embodiment, a depth map of the first image may also be obtained, and an animation may be generated based on the determined camera movement direction, the first image, and the depth map of the first image.

[0200] Through the above settings, the generated animation has a 3D effect. Under different camera angles, the relative size and relative position (distance) of different objects in the picture can change, thereby enhancing the picture sense and immersion of the animation, improving the display effect of the animation, and thus improving the user's viewing experience.

[0201] Optionally, a deep neural network model may be used to perform monocular depth estimation on the first image to obtain the depth map.

[0202] Optionally, the depth map can also be obtained by a stereo camera or a time of flight (TOF) camera. That is, when the camera acquires the first image, it also acquires the depth map corresponding to the first image. In other words, the first image itself contains the depth map information.

[0203] Fig.16 is a schematic diagram of a process of generating a 3D animation based on a first image. Fig.16 As shown, as a specific example, the following steps can be followed to generate a 3D animation:

[0204] Step ①: Obtain the first image, use the deep neural network model to perform monocular depth estimation on the first image to obtain a depth map, then segment the depth discontinuous areas, that is, perform depth stratification on the depth map, and then identify the depth edge area.

[0205] For example, performing depth layering on the depth map may be dividing the depth map into a foreground and a background, and identifying a depth edge region may be identifying a hole region in the background.

[0206] Step ②: Fill the depth edge area with color to obtain an edge background image. For example, the depth edge area (such as a hole area in the background) can be filled by a depth filling neural network model and a color filling neural network model to obtain an edge background image.

[0207] Step ③: Generate a 3D point cloud based on the first image, the depth map and the edge background map. The 3D point cloud here is also usually called a 3D grid or a 3D feature point cloud.

[0208] Step ④: Understand the display content of the first image through the image understanding neural network model to determine the visual direction (i.e., the line of sight direction), and then determine the moving direction of the camera according to the visual direction.

[0209] Step ⑤: Generate an animation based on the camera movement direction, 3D point cloud and first image determined in the previous step. For example, the camera movement control module controls the camera to move in the direction determined in the previous step. During the movement, as the camera angle changes, the image rendering of the new angle is performed to generate a 3D animation from the 2D static image.

[0210] Step ⑥: The generated 3D animation is presented on the electronic device, and the user watches the 3D animation. If the animation effect is satisfactory, the 3D animation can be set as a dynamic wallpaper, shared on a social platform, or sent to relatives and friends for viewing.

[0211] On the basis of the above-mentioned multiple embodiments, the embodiment of the present application continues to provide a method for generating an animation, which can be applied to the electronic device 100 provided in the above-mentioned embodiments, or to a chip or chip system in the electronic device 100. The following is a method embodiment provided by the present application, which corresponds to the above-mentioned embodiment. For example, in the following description, the first camera movement direction corresponds to or is equivalent to the camera movement direction in the above text.

[0212] Fig.17 FIG. 2 is a flow chart of a method 200 for generating an animation provided in an embodiment of the present application. Fig.17 As shown, the generation method 200 includes the following steps:

[0213] Step 210: The electronic device obtains a first picture.

[0214] Step 220: The electronic device determines a first camera movement direction according to display content of the first image.

[0215] Step 230: The electronic device generates an animation according to the first camera movement direction and the first picture, wherein the first camera movement direction is the moving direction of the camera corresponding to the animation.

[0216] Specifically, the electronic device obtains the first picture selected by the user, and understands the display content of the first picture through a preset AI model (such as a neural network model), and determines the first camera direction according to the result of the understanding. Then, an animation is generated according to the first camera direction and the first picture. The first camera direction here refers to the direction in which the camera moves, and the camera can display the first picture according to the first camera direction, thereby generating the animation. Since the camera movement direction is determined based on the display content of the picture, the camera movement direction and the display content of the picture can be associated and matched, and the camera movement can be flexibly used according to the picture content, which enhances the picture sense and immersion of the animation, and can generate high-quality animations with harmonious camera movements, thereby improving the display effect of the animation.

[0217] The embodiment of the present application does not limit the specific method of "determining the first camera direction according to the content of the picture". For example, the subject in the first picture can be identified, and the first camera direction can be determined according to the type of the subject. The subject here can be, for example, a human, an animal, a plant, a mountain, a river, a toy, jewelry, a fruit, a building, furniture, the sun, the moon, a vehicle, etc., but is not limited thereto. The first camera direction can also be determined according to a finer type of subject identified therein, for example, the category of animals can be divided into cats, dogs, lions, elephants or birds, etc., and the first camera direction corresponding to different types of animals may be different. And / or, the first camera direction can also be determined according to factors such as the number of subjects, the position in the picture, and the proportion of the entire picture. In addition, the first camera direction can also be determined according to factors such as the layout and color tone of the picture. In addition, when the subject is a person, the first camera direction can also be determined according to factors such as the expression, action, and gesture of the person.

[0218] Optionally, when an animation is generated by multiple pictures, the "animation" in "the first camera movement direction is the camera movement direction corresponding to the animation" can also be understood or replaced by an animation clip. For example, the multiple pictures include a first picture and a second picture, the animation clip corresponding to the first picture is a first animation clip, the animation clip corresponding to the second picture is a second animation clip, the first camera movement direction is the camera movement direction corresponding to the first animation clip, and the second camera movement direction is the camera movement direction corresponding to the second animation clip. The first animation clip and the second animation clip are spliced ​​together to form a complete animation.

[0219] Further, in the embodiment of the present application, in step 220, the electronic device determines the first camera movement direction according to the display content of the first picture, which specifically includes:

[0220] Step 221: The electronic device identifies the display content of the first picture to obtain a first direction of the target object.

[0221] Step 222: The electronic device determines the aforementioned first mirror movement direction according to the first orientation.

[0222] Specifically, the display content of the first image may be identified to obtain a first orientation, and then the movement of the camera may be determined according to the first orientation. The first orientation here is the orientation information in the identified first image, and the first orientation may be, for example, the facial orientation and sight direction of the photographing subject (e.g., a person or an animal) in the aforementioned embodiment, or the moving direction of a person, an animal or an object, or the extending direction of a path or a river. The first orientation may include, for example, at least one of from front to back, from back to front, from left to right, from right to left, from top to bottom, and from bottom to top.

[0223] Further, in an embodiment of the present application, the first orientation may be orientation information of a target object in the image, that is, the display content of the first image may be identified to obtain the first orientation of the target object, and then the movement of the camera may be determined according to the first orientation. The target object here may be any object in the first image, such as a person (such as the aforementioned lady, dancer, or little boy), an animal (such as the aforementioned animal doll), a plant (such as a tree), a scene (such as the aforementioned river, etc.), a building (such as a house, a bridge, etc.), a road, a toy, jewelry, fruit, furniture, the sun, the moon, a vehicle, etc., but is not limited thereto. The target object may be the subject of the photograph, or it may not be, for example, the target object may be a river or a path in the shooting background, and the present application does not limit this.

[0224] Optionally, the first image may have multiple objects, and the target object may be one of the multiple objects. For example, the display content of the first image may be identified to obtain multiple objects, and then the target object may be determined according to the multiple objects.

[0225] That is to say, in some scenarios, when recognizing the first image, multiple objects (such as multiple people) may be recognized. In this case, the target object can be first determined based on the multiple objects, and then the first direction corresponding to the target object can be identified. The above settings can enable the generation method provided by the present application to meet the usage requirements in more scenarios, or enable the generation method provided by the present application to have a wider scope of application. The present application does not specifically limit the specific implementation method of determining the target object based on multiple objects.

[0226] Optionally, determining the target object based on the multiple objects includes: determining the subject among the multiple objects as the target object; or, determining the target object from the multiple objects based on preset priority information; or, determining the target object from the multiple objects based on the proportion of the screen occupied by the multiple objects; or, randomly selecting the multiple objects to obtain the target object.

[0227] That is to say, the electronic device understands the content of the first picture and may identify multiple objects. At this time, the electronic device can determine the aforementioned target object based on the multiple objects according to preset logical rules. Optionally, the electronic device can determine the subject (such as a person or an animal) among the multiple objects as the target object. Optionally, the electronic device can determine an object with the highest priority as the target object based on preset priority information. For example, different objects such as people, rivers, trees, buildings, roads, etc. have different priorities, and the person with the highest priority can be determined as the target object. Optionally, the electronic device can also determine the object with the largest proportion of the screen among the multiple objects as the target object. For example, if there are multiple people in the picture, the person with the largest proportion of the screen can be determined as the target object. Optionally, the multiple objects can also be randomly extracted, and the extracted objects can be determined as the target objects.

[0228] Optionally, the electronic device determines the first camera movement direction according to the first direction, and may determine the direction along the first direction as the first camera movement direction, or may determine the direction opposite to the first direction as the first camera movement direction, but is not limited thereto.

[0229] By determining the direction along or against the first pointing direction as the first camera movement direction, the method is simple and easy to implement, and can better associate and match the first camera movement direction with the image content, thereby enhancing the visual sense and immersion of the animation and improving the user's visual experience.

[0230] Optionally, in other implementations, in step 220, the electronic device determines the first camera movement direction according to the display content of the first picture, specifically including:

[0231] The electronic device identifies the display content of the first picture to obtain multiple directions;

[0232] The electronic device determines the aforementioned first mirror movement direction according to the multiple directions.

[0233] That is to say, the electronic device understands the content of the first picture and may identify multiple directions. At this time, the electronic device can determine the aforementioned first camera movement direction according to the multiple directions based on the preset logical rules. For example, the electronic device can use the highest priority direction (such as face orientation or line of sight direction) to determine the first camera movement direction according to the preset priority information. For another example, the electronic device can also determine the first direction from multiple directions in combination with other information about the content of the picture (such as the type, layout, and proportion of the screen occupied by the subject, etc.), and then determine the first camera movement direction based on the first direction. For another example, the multiple directions can also be used as input parameters to jointly determine the first camera movement direction according to preset rules or algorithms (such as AI models). This application does not impose any restrictions on the specific implementation method.

[0234] Furthermore, in the embodiment of the present application, the generation method 200 further includes: the electronic device obtains a depth map of the first image. Step 230, the electronic device generates an animation according to the first camera movement direction and the first image, specifically including:

[0235] The electronic device generates an animation according to the first camera movement direction, the depth map and the first picture.

[0236] Through the above settings, the generated animation has a 3D effect. Under different camera angles, the relative size and relative position (distance) of different objects in the picture can change, thereby enhancing the picture sense and immersion of the animation, improving the display effect of the animation, and thus improving the user's viewing experience.

[0237] Optionally, a deep neural network model may be used to perform monocular depth estimation on the first image to obtain the depth map.

[0238] Optionally, the depth map can also be obtained by a stereo camera or a time of flight (TOF) camera. That is, when the camera acquires the first image, it also acquires the depth map corresponding to the first image. In other words, the first image itself contains the depth map information.

[0239] Furthermore, the electronic device generates an animation according to the first camera movement direction, the depth map, and the first picture, specifically including:

[0240] The electronic device performs depth layering on the depth map to identify depth edge regions.

[0241] The electronic device fills the depth edge area with color to obtain an edge background image.

[0242] The electronic device generates a 3D point cloud according to the first image, the depth map and the edge background map.

[0243] The electronic device generates the animation according to the first camera movement direction and the 3D point cloud.

[0244] Optionally, the 3D point cloud may include color information at different positions, and in this case, the animation may be directly generated according to the first camera movement direction and the 3D point cloud.

[0245] Optionally, when the 3D point cloud does not include the color information, the animation may be generated according to the first camera direction, the 3D point cloud and (the color information of) the first picture.

[0246] Optionally, the generating method 200 further includes: displaying the animation.

[0247] Optionally, the animation here can be an animation effect (dynamic effect) rendered and displayed (temporarily output) in real time by the electronic device when it is displayed. The temporarily output animation effect can also be generated or saved as a video file or GIF dynamic image for playback the next time it is displayed, or for users to share on social media.

[0248] Optionally, the animation is displayed as a dynamic wallpaper, thereby broadening the sources of dynamic wallpapers for the terminal.

[0249] Optionally, the animation is a video or a GIF image.

[0250] Fig.18 3 is a schematic block diagram of an animation generating device 300 provided in an embodiment of the present application. The generating device 300 may be an electronic device in an embodiment of the present application (such as the electronic device 100 or mobile phone in the aforementioned embodiment), or a processor or chip inside an electronic device. Fig.18 As shown, the generating device 300 includes an acquiring unit 310 , a determining unit 320 , a generating unit 330 and a display unit 340 .

[0251] The acquisition unit 310 is configured to acquire a first picture.

[0252] The determination unit 320 is used to determine a first camera movement direction according to the display content of the first picture.

[0253] The generating unit 330 is used to generate an animation according to the first camera movement direction and the first picture, wherein the first camera movement direction is a moving direction of the camera corresponding to the animation.

[0254] Optionally, the determination unit 320 is specifically configured to: identify display content of the first image to obtain a first orientation of a target object; and determine the first camera movement direction according to the first orientation.

[0255] Optionally, before identifying the display content of the first image to obtain a first pointing of a target object, the determination unit 320 is specifically used to: identify the display content of the first image to obtain a plurality of objects; and determine the target object according to the plurality of objects.

[0256] Optionally, the determination unit 320 is specifically used to: determine the subject among the multiple objects as the target object; or, determine the target object from the multiple objects according to preset priority information; or, determine the target object from the multiple objects according to the proportion of the screen occupied by the multiple objects; or, randomly select the multiple objects to obtain the target object.

[0257] Optionally, the determining unit 320 is specifically configured to: determine a direction along the first pointing direction as the first camera movement direction, or determine a direction opposite to the first pointing direction as the first camera movement direction.

[0258] Optionally, the first orientation includes face orientation, sight direction, movement direction or extension direction.

[0259] Optionally, the first direction includes at least one of from front to back, from back to front, from left to right, from right to left, from top to bottom, and from bottom to top.

[0260] Optionally, the determination unit 320 is specifically used to: identify the display content of the first picture to obtain multiple directions; and determine the first camera movement direction according to the multiple directions.

[0261] Optionally, the determination unit 320 is specifically used to: determine a first direction from the multiple directions according to preset priority information, and determine the first camera movement direction according to the first direction; or, determine a first direction from the multiple directions according to other display information in the display content of the first picture, and determine the first camera movement direction according to the first direction, wherein the other display information is information other than the multiple directions; or, determine the first camera movement direction based on the multiple directions and a preset artificial intelligence model.

[0262] Optionally, the acquisition unit 310 is further configured to: acquire a depth map of the first picture. The generation unit 330 is specifically configured to: generate the animation according to the first camera movement direction, the depth map and the first picture.

[0263] Optionally, the acquisition unit 310 is specifically configured to: perform monocular depth estimation on the first image to acquire the depth map.

[0264] Optionally, the generating unit 330 is specifically used to: perform depth layering on the depth map to identify depth edge areas; perform color filling on the depth edge areas to obtain an edge background image; generate a 3D point cloud based on the first picture, the depth map and the edge background image; and generate the animation based on the first camera direction and the 3D point cloud.

[0265] Optionally, the 3D point cloud may include color information at different positions, and in this case, the animation may be directly generated according to the first camera movement direction and the 3D point cloud.

[0266] Optionally, when the 3D point cloud does not include the color information, the animation may be generated according to the first camera direction, the 3D point cloud and (the color information of) the first picture.

[0267] Optionally, the generating device 300 further includes: a display unit 340, configured to display the animation.

[0268] Optionally, the animation is displayed as a dynamic wallpaper.

[0269] Optionally, the animation is a video or a picture in GIF format.

[0270] The generating device 300 corresponds to the electronic device in the aforementioned method embodiment, the acquiring unit 310 is used to execute the aforementioned step 210, the determining unit 320 is used to execute the aforementioned step 220 (for example, including step 221 and step 222), and the generating unit 330 is used to execute the aforementioned step 230.

[0271] The present application also provides an electronic device, including: a memory storing instructions; a processor, when the instructions are executed by the processor, the electronic device executes the method or steps provided in any of the above embodiments. The electronic device may be, for example, the electronic device 100 or the mobile phone in the above embodiments.

[0272] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed on a computer, the computer is enabled to execute the method or steps provided in any of the aforementioned embodiments.

[0273] An embodiment of the present application further provides a computer program product, including: a computer program code, when the computer program code is executed on a computer, the computer is caused to execute the method or step provided in any of the aforementioned embodiments.

[0274] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the methods in the above-mentioned method embodiments.

[0275] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0276] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0277] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0278] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0279] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for generating an animation, characterized in that: include: Get the first picture; Determining a first camera movement direction according to display content of the first picture; The animation is generated according to the first camera movement direction and the first picture, wherein the first camera movement direction is a moving direction of the camera corresponding to the animation.

2. The generation method according to claim 1, characterized in that: The determining the first camera movement direction according to the display content of the first picture includes: Identify the display content of the first picture to obtain a first direction of the target object; The first camera movement direction is determined according to the first orientation.

3. The generation method according to claim 2, characterized in that: Before identifying the display content of the first picture to obtain a first direction of the target object, the method further includes: Recognize the display content of the first picture to obtain multiple objects; The target object is determined according to the multiple objects.

4. The generation method according to claim 3, characterized in that: The determining the target object according to the multiple objects includes: determining a subject among the multiple objects as the target object; or, determining a target object from the plurality of objects according to preset priority information; or, determining a target object from the multiple objects according to the screen ratios occupied by the multiple objects; or, The multiple objects are randomly selected to obtain the target object.

5. The generation method according to any one of claims 2 to 4, characterized in that: The determining the first mirror movement direction according to the first orientation includes: The direction along the first pointing direction is determined as the first camera movement direction, or, A direction opposite to the first pointing direction is determined as the first camera movement direction.

6. The generation method according to any one of claims 2 to 5, characterized in that: The first orientation includes a face orientation, a sight line direction, a moving direction or an extending direction.

7. The generation method according to any one of claims 2 to 6, characterized in that: The first direction includes at least one of from front to back, from back to front, from left to right, from right to left, from top to bottom, and from bottom to top.

8. The generation method according to any one of claims 1 to 7, characterized in that: The generating method further comprises: Obtaining a depth map of the first image; Generating the animation according to the first camera movement direction and the first picture includes: The animation is generated according to the first camera movement direction, the depth map and the first picture.

9. The generation method according to any one of claims 1 to 8, characterized in that: The generating method further comprises: The animation is displayed.

10. The generation method according to any one of claims 1 to 9, characterized in that: The animation is displayed as a dynamic wallpaper.

11. The generation method according to any one of claims 1 to 10, characterized in that: The animation is a video or a picture in GIF format.

12. An animation generating device, characterized in that: include: An acquiring unit, configured to acquire a first picture; a determining unit, configured to determine a first camera movement direction according to display content of the first picture; A generating unit is used to generate the animation according to the first camera movement direction and the first picture, wherein the first camera movement direction is a moving direction of the camera corresponding to the animation.

13. The generating device according to claim 12, characterized in that: The determining unit is specifically used for: Identify the display content of the first picture to obtain a first direction of the target object; The first camera movement direction is determined according to the first orientation.

14. The generating device according to claim 13, characterized in that: Before identifying the display content of the first picture to obtain the first direction of the target object, the determining unit is specifically used to: Recognize the display content of the first picture to obtain multiple objects; The target object is determined according to the multiple objects.

15. The generating device according to claim 14, characterized in that: The determining unit is specifically used for: determining a subject among the multiple objects as the target object; or, determining a target object from the plurality of objects according to preset priority information; or, Determining a target object from the multiple objects according to the proportions of the screen occupied by the multiple objects; or, The multiple objects are randomly selected to obtain the target object.

16. The generating device according to any one of claims 13 to 15, characterized in that: The determining unit is specifically used for: The direction along the first pointing direction is determined as the first camera movement direction, or, A direction opposite to the first pointing direction is determined as the first camera movement direction.

17. The generating device according to any one of claims 13 to 16, characterized in that: The first orientation includes a face orientation, a sight line direction, a moving direction or an extending direction.

18. The generating device according to any one of claims 13 to 17, characterized in that: The first direction includes at least one of from front to back, from back to front, from left to right, from right to left, from top to bottom, and from bottom to top.

19. The generating device according to any one of claims 12 to 18, characterized in that: The acquisition unit is also used for: Obtaining a depth map of the first image; The generating unit is specifically used for: The animation is generated according to the first camera movement direction, the depth map and the first picture.

20. The generating device according to any one of claims 12 to 19, characterized in that: The generating device further comprises: A display unit is used to display the animation.

21. The generating device according to any one of claims 12 to 20, characterized in that: The animation is displayed as a dynamic wallpaper.

22. The generating device according to any one of claims 12 to 21, characterized in that: The animation is a video or a picture in GIF format.

23. An electronic device, characterized in that: include: A memory storing instructions; The processor, when the instructions are executed by the processor, causes the electronic device to execute the method as described in any one of claims 1-11.

24. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 11.

25. A computer program product, characterized in that include: A computer program code, when the computer program code is run on a computer, causes the computer to execute the method according to any one of claims 1 to 11.

Images