Animation display method and device in AR scene, terminal and storage medium
By loading resource images in AR scenes and using continuous sequence frame playback technology, the problem of slow animation rendering speed in the prior art is solved, and faster animation loading speed and more efficient rendering process are achieved.
Patent Information
- Application Number
- CN202310613749.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-26
AI Technical Summary
When the prior art displays animations in AR scenes, the computing volume of the rendering process is large and slow, causing the user to wait for the animation to be loaded for a long time.
By loading the resource image corresponding to the animation, instead of the three-dimensional model, the animation is displayed on the top of the real world picture by playing continuously sequence frames, reducing the amount of computing in the rendering process.
It improves the loading speed of AR animation effects, shortens user waiting time, and improves the efficiency of animation display.
Smart Images

Figure CN119027551A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of augmented reality (AR) technology, and in particular to an animation display method, device, terminal and storage medium in an AR scene. Background Art
[0002] AR is a technology that simulates computer-generated virtual information such as text, images, and three-dimensional models and applies them to real-world images, thereby enhancing the real world.
[0003] In the related art, a 3D model is usually loaded to display AR animation effects in the real world. For example, when a fireworks animation needs to be displayed in the real world, a fireworks model needs to be loaded and the fireworks model needs to be controlled to perform a fireworks release action.
[0004] However, since the data volume of model resources is usually large, when the AR animation effects are displayed using the methods provided by relevant technologies, the rendering process has a large amount of computation and a slow rendering speed. Summary of the invention
[0005] The embodiments of the present application provide an animation display method, device, terminal and storage medium in an AR scene, which can reduce the amount of computation in the rendering process and improve the loading speed of AR animation effects. The technical solution is as follows:
[0006] On the one hand, an embodiment of the present application provides an animation display method in an AR scene, the method comprising:
[0007] Displaying an AR interface, wherein the AR interface includes a real-world image and an animation playback control, wherein the animation playback control is used to trigger the display of an animation superimposed on the real-world image;
[0008] In response to a triggering operation on the animation playing control, loading an animation resource image corresponding to the animation;
[0009] Based on the loaded animation resource image, animation display is performed on the upper layer of the real world picture by playing a continuous sequence of frames.
[0010] On the other hand, an embodiment of the present application provides an animation display device in an AR scene, the device comprising:
[0011] An interface display module, used to display an AR interface, wherein the AR interface includes a real-world image and an animation playback control, wherein the animation playback control is used to trigger the display of an animation superimposed on the real-world image;
[0012] A loading module, used for loading an animation resource image corresponding to the animation in response to a triggering operation on the animation playing control;
[0013] The animation display module is used to display the animation on the upper layer of the real world picture by playing a continuous sequence of frames based on the loaded animation resource image.
[0014] On the other hand, an embodiment of the present application provides a terminal, which includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the animation display method in the AR scene as described in the above aspect.
[0015] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement the animation display method in the AR scene as described in the above aspects.
[0016] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the animation display method in the AR scene provided by the above aspect.
[0017] In the embodiment of the present application, the AR interface is first displayed, and the resource image corresponding to the animation is loaded by triggering the animation control set in the AR interface, wherein the data volume of the resource image is less than that of the three-dimensional model, and occupies less memory, so that the resource image is loaded faster. Based on the resource image corresponding to the animation, the real world picture and the superimposed display animation are displayed in the AR interface by playing a continuous sequence frame, replacing the way of displaying the animation with a three-dimensional model. The animation is displayed by playing a continuous sequence frame, which can reduce the amount of calculation in the animation rendering process, thereby speeding up the rendering speed, thereby reducing the time the user waits for the AR animation effect to load, and improving the loading speed of the AR animation effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0019] Figure 1 A schematic diagram showing an implementation environment provided by an exemplary embodiment of the present application is shown;
[0020] Figure 2 A flowchart of animation display in an AR scene provided by an exemplary embodiment of the present application is shown;
[0021] Figure 3 A schematic diagram showing an animation display method in an AR scene provided by an exemplary embodiment of the present application is shown;
[0022] Figure 4 A flowchart of animation display in an AR scene provided by another exemplary embodiment of the present application is shown;
[0023] Figure 5 A schematic diagram showing an animation display process provided by an exemplary embodiment of the present application is shown;
[0024] Figure 6 A schematic diagram showing an animation display process provided by another exemplary embodiment of the present application is shown;
[0025] Figure 7 A flowchart showing the texture corresponding to the sequence image to be played provided by another exemplary embodiment of the present application is shown;
[0026] Figure 8 is a schematic diagram of a target animation resource image shown in an exemplary embodiment of the present application;
[0027] Fig. 9 is a schematic diagram of a target animation resource texture provided by another exemplary embodiment of the present application;
[0028] Fig.10 is a flow chart of a custom animation display process shown in an exemplary embodiment of the present application;
[0029] Fig.11 is a schematic diagram of a shooting angle shown in an exemplary embodiment of the present application;
[0030] Fig.12 A structural block diagram of an animation display device in an AR scene provided by an exemplary embodiment of the present application is shown;
[0031] Fig.13 A schematic diagram of the structure of a terminal provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION
[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0033] It should be understood that the "several" mentioned in this article refers to one or more, and "multiple" refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0034] For ease of understanding, the nouns involved in the embodiments of the present application are explained below.
[0035] AR: Augmented reality technology is a type of computer vision technology, also known as augmented reality technology. AR can combine real-world information and virtual-world information to simulate physical information that is difficult to experience in the real world. AR uses cameras, sensors, and other devices to capture images and sounds from the real world, and combines this information with computer-generated virtual content, thereby overlaying virtual information content on the real world. Because the real environment and virtual objects overlap, they can exist in the same picture and space at the same time, and this process can be perceived by human senses, thus creating an enhanced reality experience.
[0036] Sequence frame: It is an animation production technology that uses image files one frame at a time to represent the active video. Correspondingly, sequence frame animation is to put continuous static pictures (sequence frames) together in a certain order and simulate the dynamic picture effect by playing these pictures quickly. Since each picture contains the content of a frame of animation, the faster the playback speed, the smoother the animation.
[0037] THREE.js (3D graphics processing library): THREE.js is a 3D graphics library based on JavaScript (Java scripting language). It provides a series of tools and APIs (Application Programming Interface) for creating and displaying 3D graphics scenes. Based on the WebGL (Web Graphics Library) renderer, this library can provide developers with a simpler and more efficient way to develop 3D graphics.
[0038] The animation display method in the AR scene provided by the embodiment of the present application can be applied to various scenes that need to display AR animation effects. Figure 1 , Figure 1 A schematic diagram of an application scenario provided by an exemplary embodiment of the present application is shown. The implementation environment includes a terminal 110, and an AR interface 120 is displayed on the terminal 110. The AR interface 120 includes: a real world picture 130 and an animation 140.
[0039] Among them, the terminal 110 is an electronic device with the need to display AR animation effects. The electronic device can be a smart phone, a tablet computer, a wearable device or a personal computer, etc., which is not limited in this embodiment. Figure 1 In the description, the terminal 110 is taken as an example of a smart phone used by a user.
[0040] The terminal 110 can implement the animation display method in the AR scene through applications, mini-programs, and functional modules in the application. The embodiments of the present application are not limited here. For the convenience of description, the following is an example of the terminal 110 implementing the animation display method in the AR scene through a mini-program.
[0041] In some embodiments, the terminal 110 runs the applet, and the AR interface 120 is displayed on the terminal 110. Based on the environment information of the terminal 110, the AR interface 120 displays the real world picture 130, and the animation 140 is superimposed on the real world picture 130. Figure 1 As shown, an AR interface 120 is displayed through the user's smart phone (terminal 110), and the user uses the smart phone to take a picture of the building in front of him, and uses the picture corresponding to the building as the real-world picture 130. At the same time, the terminal 110 runs a small program to load a virtual fireworks effect animation as animation 140, and superimposes the virtual fireworks effect animation (animation 140) with the picture corresponding to the building (real-world picture 130), and displays them in the AR interface 120.
[0042] It should be noted that in the process of collecting relevant data of the user, the present application may display a prompt interface, a pop-up window or output a voice prompt information, and the prompt interface, pop-up window or voice prompt information is used to prompt the user that the relevant data is currently being collected, so that the present application only starts to execute the relevant steps of obtaining the relevant data of the user after obtaining the confirmation operation issued by the user to the prompt interface or pop-up window, otherwise (that is, when the confirmation operation issued by the user to the prompt interface or pop-up window is not obtained), the relevant steps of obtaining the relevant data of the user are terminated, that is, the relevant data of the user is not obtained. In other words, the information involved in this application (including but not limited to user device information, user personal information, user's real-time location), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the real-time location of the user involved in this application is obtained with full authorization.
[0043] Please refer to Figure 2 , which shows a flowchart of animation display in an AR scene provided by an exemplary embodiment of the present application, the method includes:
[0044] Step 201 , displaying an AR interface, wherein the AR interface includes a real-world image and an animation playback control, wherein the animation playback control is used to trigger the display of an animation superimposed on the real-world image.
[0045] In some embodiments, the terminal implements the animation display method in the AR scene through software and applets, and displays the AR interface on the terminal. Among them, the AR interface can display the real world picture and display the animation on the real world picture. The real world picture can be a video picture shot in advance by the terminal, or it can be a picture shot in real time by the terminal. The displayed animation includes but is not limited to traditional video animation, vector animation (two-dimensional animation) and MG (Motion Graphic, motion image animation) and so on.
[0046] Optionally, the AR interface may include one or more animation playback controls, wherein the animation playback controls may be radio buttons, check boxes, and switch buttons (switch buttons), etc. The user operates the animation playback controls using the terminal to trigger the display animation, and the display animation is superimposed on the real world screen in the AR interface.
[0047] In a possible implementation, when the AR interface includes multiple animation playback controls, different animation playback controls are used to trigger the display of different animations superimposed on the real-world screen. The terminal can trigger multiple controls synchronously or asynchronously.
[0048] For example, an AR interface includes three animation playback controls: the first animation playback control, the second animation playback control, and the third animation playback control. The three animation playback controls correspond to: fireworks effect animation, meteor effect animation, and ribbon effect animation, respectively. The three controls are all set as radio buttons. When the user selects the first animation playback control, the first animation playback control is used to trigger the display of fireworks effect animation superimposed on the real-world screen. When the user selects the three animation playback controls at the same time, the three animation playback controls can simultaneously trigger the display of fireworks effect animation, meteor effect animation, and ribbon effect animation superimposed on the real-world screen.
[0049] like Figure 3 As shown, Figure 3 A schematic diagram of an animation display method in an AR scene is shown, including: an AR interface 310, an animation playback control 320, a real-world picture 330, and an animation 340. The terminal uses a smart phone as an example, and the terminal uses a small program to implement the animation display method in the AR scene, and displays the AR interface 310 on the terminal screen. The picture captured in real time by the camera of the terminal (the building and the surrounding environment of the building in the figure) is used as the real-world picture 330, so that the corresponding real-world picture 330 is displayed in the AR interface 310. Then, the user clicks the animation playback control 320 in the AR interface 310 (the release fireworks button in the figure), and the animation 340 (the fireworks animation in the figure) is superimposed on the real-world picture 330.
[0050] Step 202: In response to a trigger operation on an animation play control, an animation resource image corresponding to the animation is loaded.
[0051] In some embodiments, the animation may correspond to one or more animation resource images, wherein each animation resource image may contain only a single sequence frame or multiple sequence frames. In addition, when each animation resource image contains multiple sequence frames, the number of sequence frames contained in each animation resource image may be the same or different.
[0052] Optionally, the animation resource image can also be a custom animation uploaded by the user, and after the animation play control is triggered, the terminal loads the custom animation. Custom animations include but are not limited to traditional video animations, vector animations (two-dimensional animations), and MG, etc.
[0053] In some embodiments, the triggering operation of the animation playback control can be a variety of user-terminal interactive operations, including but not limited to click triggering operations, sound triggering operations, and gesture triggering operations, etc., which are not limited in the embodiments of the present application. Among them, the animation resource image can be a continuous multi-frame image corresponding to the animation. After the animation playback control is triggered, the terminal loads and displays the continuous multi-frame image corresponding to the animation.
[0054] In other possible implementations, in response to the triggering operation of the animation playback control, one or more animation resource images corresponding to the animation are loaded. Wherein, when loading an animation corresponding to an animation resource image, the animation resource image may be a collection of all continuous multi-frame images (sequence frames) corresponding to the animation; when loading an animation corresponding to multiple animation resource images, each animation resource image may be a collection of some continuous sequence frames corresponding to the animation, and each animation resource image may also be a single-frame sequence frame corresponding to the animation, and all animation resource images correspond to the complete sequence frames of the animation.
[0055] Step 203 , based on the loaded animation resource image, the animation is displayed on the upper layer of the real world picture by playing a continuous sequence of frames.
[0056] In some embodiments, when one or more loaded animation resource images are continuous frame images corresponding to an animation, a sequence frame player can be constructed to continuously play the one or more loaded animation resource images using a sequence frame player, render the one or more animation resource images into an animation, and display the animation on top of the real-world picture.
[0057] In one possible implementation, THREE.js (a three-dimensional graphics processing library) can be used to build an AR scene on top of a real-world picture, and a sequence frame player can be built based on one or more loaded animation resource images. The loaded one or more animation resource images can be played continuously using the sequence frame player, and the animation resource images can be rendered into animations, so that the animations can be displayed in the AR scene built by THREE.js, thereby allowing the animations rendered based on one or more animation resource images to be animatedly displayed on top of the real-world picture.
[0058] In summary, in the embodiment of the present application, the AR interface is first displayed, and the resource image corresponding to the animation is loaded by triggering the animation control set in the AR interface, wherein the animation resource image can be a continuous multi-frame image or a custom animation, and the memory usage is less than that of the three-dimensional model, and the animation resource image loading speed is faster. Based on the animation resource image corresponding to the animation, the real world picture and the superimposed display animation are displayed in the AR interface by playing a continuous sequence of frames, which is faster than rendering the three-dimensional model animation, thereby improving the loading speed of the AR animation effect.
[0059] In one possible application scenario, the solution provided in the embodiment of the present application can be used to play fireworks effects in an AR scene without the help of a fireworks model.
[0060] 1. Display an AR interface, which includes a real-world image and an animation playback control. The animation playback control is used to trigger the overlay of a fireworks animation on the real-world image.
[0061] In some embodiments, the fireworks animation may include: a first stage animation, a second stage animation, and a third stage animation. The first stage animation is a fireworks rising stage animation, which is used to show the process of fireworks being released and ascending to a target position, wherein the target position is the position where the fireworks bloom; the second stage animation is a fireworks blooming stage animation, which is used to show the process of fireworks blooming at the target position; the third stage animation is a fireworks descending stage animation, which is used to show the process of fireworks descending and extinguishing after blooming at the target position.
[0062] Optionally, the fireworks animation may be an animation of a single firework blooming process, or may be an animation of multiple fireworks blooming process, wherein, when the fireworks animation is the blooming process of multiple fireworks, the multiple fireworks may bloom synchronously or sequentially.
[0063] Furthermore, the target position can be determined based on a real-world image. In a possible implementation, the terminal recognizes the real-world image to obtain a recognition result, and the terminal determines the target position based on the recognition result. For example, the real-world image is an image of a square, and the terminal recognizes it and recognizes a monument in the square in the image. Based on the recognition result, the terminal determines the target position corresponding to the fireworks effect animation based on the top position of the monument in the square.
[0064] 2. In response to the triggering operation on the animation playback control, the fireworks animation resource image corresponding to the fireworks animation is loaded.
[0065] In some embodiments, the fireworks animation resource image may include: continuous sequence frames corresponding to the first stage animation, the second stage animation, and the third stage animation respectively.
[0066] In a possible implementation, the fireworks animation corresponds to a plurality of fireworks animation resource images, each of which includes a continuous sequence of frames corresponding to at least one of the first stage animation, the second stage animation, and the third stage animation. In response to a trigger operation on the animation playback control, the fireworks animation resource image is loaded as a fireworks animation resource texture.
[0067] In other possible implementations, the fireworks animation corresponds to a plurality of fireworks animation resource images, and each fireworks animation resource image may also be a frame in a continuous sequence of frames corresponding to the first stage animation, the second stage animation, or the third stage animation.
[0068] 3. Based on the loaded fireworks animation resource image, the fireworks animation is displayed on top of the real world screen by playing a continuous sequence of frames.
[0069] In some embodiments, based on the fireworks animation resource texture, displaying the fireworks animation on the real world screen by playing a continuous sequence of frames may include:
[0070] A plane model is created in a virtual space corresponding to a real-world image. The plane model is used for animation display and is a transparent model. A target texture corresponding to a sequence frame to be played is determined, and a mapping property of the plane model is set so that the plane model displays the target texture corresponding to the sequence frame to be played.
[0071] Since the fireworks animation may include three stages of animation, and the display positions of the fireworks animations at different stages may be different, in order to ensure the display effect of the fireworks animation, in a possible implementation, multiple plane models are set on the upper layer of the real-world image, and different plane models are used to display the fireworks animation resources corresponding to different animation stages. Correspondingly, based on the loaded fireworks animation resource image, the terminal displays the fireworks animation at the plane model corresponding to the fireworks animation resource image in the upper layer of the real-world image by playing a continuous sequence of frames.
[0072] In an illustrative example, the upper layer of the real-world image is provided with two plane models, namely, a first plane model located in the lower half of the upper layer of the real-world image, and a second plane model located in the upper half of the upper layer of the real-world image. The terminal displays the fireworks animation on the first plane model on the upper layer of the real-world screen by playing a continuous sequence frame based on the first fireworks animation resource image corresponding to the first stage animation (fireworks rising animation), displays the fireworks animation on the second plane model on the upper layer of the real-world screen by playing a continuous sequence frame based on the second fireworks animation resource image corresponding to the second stage animation (fireworks blooming animation), and displays the fireworks animation on the first plane model on the upper layer of the real-world screen by playing a continuous sequence frame based on the third fireworks animation resource image corresponding to the third stage animation (fireworks falling animation).
[0073] In the embodiment of the present application, the fireworks effect animation is divided into three stages of animation according to the process of fireworks blooming. After the play control is triggered, the animation resource textures corresponding to the three stages of animation are loaded, so that the corresponding textures are displayed using the plane model set on the upper layer of the real picture, and the effect of displaying the fireworks effect animation on the upper layer of the real world picture is achieved. In this process, the position of the plane model can also be adjusted according to the different stages of animation, and then the complete process of fireworks rising-fireworks blooming-fireworks falling (extinguishing) is displayed in the AR interface, which improves the viewing experience of the fireworks effect animation display in the AR scene.
[0074] In a possible application scenario, optionally, in response to a triggering operation on an animation playback control, an animation resource image corresponding to a target animation may be loaded based on a current geographic location, wherein different animations are set corresponding to different geographic locations.
[0075] In some embodiments, the AR interface includes an animation playback control, which can be triggered at different geographic locations to load animation resource images corresponding to different target animations, thereby displaying different animations on the AR interface according to the geographic location. Optionally, the terminal can set multiple location ranges in advance, each location range corresponding to a different animation, and in response to the triggering operation of the animation playback control, determine the current geographic location of the terminal, determine the location range of the terminal based on the current geographic location, and load the animation resource image of the corresponding target animation based on the location range.
[0076] For example, the terminal sets three location ranges, including parks, amusement parks, and squares, and the target animations corresponding to the three are respectively shooting star effect animation, ribbon effect animation, and firework effect animation. When the user uses the terminal to shoot a monument in the square, the playback control is triggered. In response to the triggering operation of the animation playback control, the current geographical location is determined to be the location of the monument, thereby determining that the terminal is currently within the location range of the square, and the terminal then loads the animation resource image corresponding to the fireworks effect animation of the square.
[0077] In an embodiment of the present application, different geographic locations correspond to different target animations. The current geographic location of the terminal is first determined, and then a trigger operation on the animation playback control is responded to, thereby loading the animation resource image corresponding to the target animation of the geographic location, thereby improving the richness of the animation display in the AR scene.
[0078] Please refer to Figure 4 , which shows a flowchart of animation display in an AR scene provided by another exemplary embodiment of the present application, the method comprising:
[0079] Step 401 , displaying an AR interface, wherein the AR interface includes a real-world image and an animation playback control, wherein the animation playback control is used to trigger the display of an animation superimposed on the real-world image.
[0080] The implementation of this step may refer to step 201, and this embodiment will not be described in detail here.
[0081] Step 402: In response to a trigger operation on an animation play control, the animation resource image is loaded as an animation resource texture.
[0082] In some embodiments, the animation resource image may be a continuous sequence frame image. To display the animation on top of the real-world image in an AR scenario, the continuous sequence frame image may be loaded as a corresponding animation resource texture.
[0083] In another possible implementation, the terminal processes multiple animation resource images into multiple texture maps, each animation resource image corresponds to its own texture map, the animation resource image may include multiple continuous sequence frames, and one texture map corresponds to multiple continuous sequence frames, so that the texture map corresponding to the animation resource image can be subsequently applied to the AR scene, so that the texture map is presented on the upper layer of the real-world picture, combined with the continuous sequence frame playback, to achieve the effect of displaying the animation on the upper layer of the real-world picture.
[0084] In a possible implementation, the terminal may also process the continuous sequence frame images into multiple texture maps, each sequence frame image corresponding to its own texture map, and apply the texture maps of the continuous sequence frame images to the AR scene, so that the texture maps are presented on top of the real-world picture, and combined with the continuous sequence frame playback, the animation is displayed on top of the real-world picture.
[0085] For example: THREE.js (3D graphics processing library) is used to build an AR scene on the upper layer of the real world screen, and a continuous sequence of frame images is obtained. The continuous sequence of frame images can be images of any image format supported by THREE.js, such as PNG (Portable Network Graphics) images, JPG (Joint Photographic Experts Group) images, GIF (Graphics Interchange Format) images, etc. Then, the multiple texture maps corresponding to the continuous sequence of frame images can be created by the TextureLoader (texture map loader) method in THREE.js, and its properties in TextureLoader are determined based on the continuous sequence of frame images, such as: image (source image), mapping (how the image is applied to the object / object), wrapS (how the texture map is wrapped horizontally), wrapT (how the texture map is wrapped vertically), etc. Based on the above properties, the continuous sequence of frame images is processed into corresponding multiple texture maps, so that the texture maps of the continuous sequence of frame images can be subsequently applied to the AR scene created by THREE.js, so that the texture maps are presented on the upper layer of the real world screen.
[0086] In some embodiments, in response to a trigger operation on an animation playback control, at least two animation resource images corresponding to the loaded animation are loaded. When the loaded animation corresponds to at least two animation resource images, each animation resource image may be a set of partially continuous sequence frames corresponding to the animation. When the animation corresponds to at least two animation resource images, the animation resource images corresponding to the animation are loaded in sequence based on the image order.
[0087] In a possible implementation, when the loading animation corresponds to at least two animation resource images, each animation resource image may be a collection of partially continuous sequence frames corresponding to the animation, that is, each animation resource image corresponds to multiple sequence frames. In order to play the animation normally, the sequence frames corresponding to the animation should be loaded and played in sequence, so each animation resource image as a collection of partially continuous sequence frames should also be loaded and played in sequence.
[0088] For example: Figure 5 , which shows a schematic diagram of an animation display process provided by an exemplary embodiment of the present application. The animation resource images include: a first animation resource image 501, a second animation resource image 502, a third animation resource image 503, a fourth animation resource image 504, and a fifth animation resource image 505.
[0089] Since the loading animation corresponds to five animation resource images, in response to the triggering operation of the animation playback control, the first animation resource image to the fifth animation resource image are loaded in sequence based on the order of the first animation resource image, and the first animation resource image 501 to the fifth animation resource image 505 are processed into the first animation resource texture 506, the second animation resource texture 507, the third animation resource texture 508, the fourth animation resource texture 509, and the fifth animation resource texture 510 in turn.
[0090] Step 403 , based on the animation resource texture, the animation is displayed on the upper layer of the real world picture by playing a continuous sequence of frames.
[0091] In some embodiments, based on at least two animation resource textures corresponding to the loading animation, continuous sequence frame images are rendered into animation by playing continuous sequence frames, and the animation is placed on top of the real world picture in the AR scene for display.
[0092] In one possible implementation, an AR scene is created using THREE.js, and animation resource images (continuous sequence frame images) are processed into corresponding multiple texture maps, a sequence frame player is constructed, and the continuous sequence frame images are played. The continuous sequence frame images are rendered into animations based on multiple texture maps, thereby applying the animation to the AR scene created by THREE.js, so that the animation is presented on top of the real-world picture.
[0093] For example: Figure 5 , which shows a schematic diagram of an animation display process provided by an exemplary embodiment of the present application. The animation resource images include: a first animation resource image 501, a second animation resource image 502, a third animation resource image 503, a fourth animation resource image 504, and a fifth animation resource image 505.
[0094] In order to play the animation normally, the sequence frames corresponding to the animation should be loaded and played in sequence, and the first animation resource texture 506 to the fifth animation resource texture 510 are processed in sequence, so based on the animation resource texture 506 to the fifth animation resource texture 510, the first animation resource image 501 to the fifth animation resource image 505 are rendered in sequence to obtain the first sequence frame animation 511, the second sequence frame animation 512, the third sequence frame animation 513, the fourth sequence frame animation 514 and the fifth sequence frame animation 515. Then, in the AR scene, the first sequence frame animation 511 to the fifth sequence frame animation 515 are displayed in sequence on the upper layer of the real world picture.
[0095] In some embodiments, when the number of loaded animation resource images reaches a threshold, animation display is performed on a real-world screen layer by playing a continuous sequence of frames based on the animation resource texture, wherein the remaining animation resource images are loaded during the animation display process.
[0096] Since there may be multiple animation resource images to be loaded, if all animation resource images are loaded at the same time, and they are all processed into animation resource textures and then further rendered to obtain the animations corresponding to all animation resource images, the loading time may be too long. And as the number of animation resource images increases, the loading time is also longer, resulting in slow animation loading speed, affecting the user's viewing experience. Therefore, a method of loading a part of the animation resource images first can be adopted, and the animation resource textures are generated from the first part of the animation resource images, and the corresponding sequence frame animation is generated for playback, while the remaining animation resource images are loaded in sequence during the animation playback process. By loading the animation resource images while playing the sequence frame animation corresponding to the animation resource images, the time waiting for the animation to load can be reduced.
[0097] In a possible implementation, when multiple animation resource images need to be loaded, a quantity threshold is set, for example, when five animation resource images need to be loaded, the quantity threshold is set to two. When the number of loaded animation resource images reaches the quantity threshold, based on the animation resource textures corresponding to the loaded animation resource images, the sequence frame animation corresponding to the animation resource texture is displayed in sequence on the upper layer of the real world screen by playing a continuous sequence frame.
[0098] For example: Figure 6As shown, it shows a schematic diagram of the animation display process provided by another exemplary embodiment of the present application. Five animation resource images need to be loaded, including: the first animation resource image 601 to the fifth animation resource 605, and the quantity threshold is set to two. When the first animation resource image 601 and the second animation resource image 602 have been loaded, that is, when the number of loaded images has reached the quantity threshold, based on the first animation resource texture 606 and the second animation resource texture 607 corresponding to the two, the sequence frame animations corresponding to the two animation resource textures are displayed in sequence on the upper layer of the real world screen through continuous sequence frame playback, that is, the first sequence frame animation 611 and the second sequence frame animation 612 are displayed in sequence. In addition, while displaying the first sequence frame animation 611 and the second sequence frame animation 612, the remaining animation resource images (the third animation resource image 603, the fourth animation resource image 604, and the fifth animation resource image 605) are loaded in sequence in the process of playing the animation.
[0099] In the embodiment of the present application, after responding to the trigger operation of the animation playback control, the animation resource image is loaded as an animation resource texture so that the generated animation can be displayed on the upper layer of the real world screen. Then, based on the animation resource texture, the animation is rendered and generated by playing a continuous sequence frame. Compared with the method of rendering and generating a three-dimensional model animation, the amount of calculation is less and the animation loading time is shorter. In addition, in order to further shorten the animation loading time, this embodiment loads the animation resource image in sequence, and adopts the method of playing the corresponding sequence frame animation while loading the animation resource image, which reduces the time the user waits for the animation to load and improves the loading speed of the animation in the AR scene.
[0100] During the animation display process in the AR scene, the animation resource image corresponding to the loaded animation can be one or more static images, wherein each animation resource image can contain part or all of the continuous frame images of the animation, that is, each animation resource image can contain multiple sequence frames. It can be seen that in the case where the animation resource image is a continuous frame image of part of the animation, the animation resource texture corresponding to an animation resource image can correspond to multiple sequence frames. At this time, the texture offset corresponding to the sequence frame to be played should be calculated based on the animation resource texture, so that when the animation is played, the texture corresponding to the sequence frame to be played is displayed, and then the corresponding sequence frame animation is displayed through continuous sequence frame playback.
[0101] In the case where the animation resource image is a continuous frame image of part of the animation, the texture offset corresponding to the sequence frame to be played is calculated based on the animation resource texture, so that the texture corresponding to the sequence frame to be played is displayed when the animation is played. This can be achieved by the following steps, such as Figure 7 As shown, it shows a flowchart of displaying the texture corresponding to the sequence image to be played provided by an exemplary embodiment of the present application, and the process includes:
[0102] Step 701: Create a plane model in a virtual space corresponding to a real-world image. The plane model is used for animation display and is a transparent model.
[0103] The principle of displaying animation effects in an AR scene is to superimpose the animation on the upper layer of the real-world picture. When playing the animation, the real-world picture and the animation on the upper layer can be observed at the same time. Therefore, in some embodiments, a plane model can be created so that the plane model and the real-world picture are in the same virtual space, wherein the plane model is located on the upper layer of the real-world picture. When the animation is displayed in an AR scene, the animation to be played is projected onto the plane model. Since the plane model and the real-world picture are in the same space and the plane model is located on the upper layer, the effect of displaying the animation on the upper layer of the real-world picture can be achieved.
[0104] For example, in THREE.js, a plane model (Plane Geometry) is constructed, a material is added to the plane model and its attribute is set to be transparent. During each rendering process, as the continuous sequence frames are played, the map attribute of the plane model is continuously updated according to the currently played sequence frame and its corresponding animation resource texture, so as to achieve the effect of playing the sequence frame animation on the plane model. Since the plane model is located above the real world screen, the effect of displaying the sequence frame animation on the upper layer of the real world screen is achieved.
[0105] Optionally, when the terminal moves, the model orientation of the plane model is adjusted based on the orientation of the terminal, wherein the model orientation of the plane model is consistent with the orientation of the terminal.
[0106] The terminal may move, and as the terminal moves, the real-world image provided by the terminal through real-time shooting and other methods will move accordingly. To ensure the effect of the animation display in the AR scene, the animation superimposed on the real-world image should also be adjusted accordingly. In some embodiments, when the terminal moves, the model orientation of the plane model is adjusted based on the terminal orientation so that the model orientation of the plane model is consistent with the terminal orientation, thereby achieving the effect of the animation display being adjusted synchronously with the terminal orientation.
[0107] For example, the terminal provides real-world images through real-time camera shooting. After building a plane model in THREE.js, the LookAt function is used to adjust the model orientation of the plane model to the camera orientation of the terminal. The LookAt function works by converting the world space coordinates of the vertex into the observation space coordinates. Therefore, the observer can be used as the origin to redefine the three-dimensional space. By setting the observation position, target position, and camera vector, the function can control the orientation of the target model. After the model orientation of the plane model is adjusted to be consistent with the camera orientation of the terminal, the plane model can change synchronously with the change of the terminal's camera orientation. Furthermore, the sequence frame animation displayed on the plane model will also change synchronously with the change of the real-world image.
[0108] Step 702: Determine the target animation resource texture and texture offset corresponding to the sequence frame to be played. The texture offset is the offset of the texture corresponding to the sequence frame to be played in the target animation resource texture.
[0109] In some embodiments, an animation resource image may include part or all of the continuous frame images of the animation, that is, an animation resource image may include multiple sequence frames, and therefore, the animation resource texture corresponding to an animation resource image also corresponds to multiple sequence frames. Therefore, when the animation is played, the texture offset corresponding to the sequence frame to be played should be calculated based on the target animation resource texture, so that when the animation is played, the plane model displays the texture corresponding to the sequence frame to be played. Among them, the texture offset is the offset of the texture corresponding to the sequence frame to be played in the target animation resource texture.
[0110] In a possible implementation, the target animation resource texture and texture offset corresponding to the sequence frame to be played may be obtained in the following manner:
[0111] 1. Based on the animation playback duration and frame rate, determine the frame number of the sequence frame to be played.
[0112] First, determine which frame in the current sequence frame animation is the sequence frame to be played. In some embodiments, the sequence frame to be played is determined by the animation playback time and frame rate. The frame rate is also called frame rate or frame frequency, which refers to the frequency at which a bitmap image in units of frames appears continuously on the display. By combining the frame rate with the animation playback time, the number of frames currently played can be calculated, thereby determining the frame number of the sequence frame to be played.
[0113] For example, if the frame rate of an animation is 16fps (frames per second), each animation resource image corresponds to 16 frames of sequence frame images, and the current animation has been played for 3 seconds. It can be seen that the number of frames currently played is 3 seconds × 16fps = 48 frames. Therefore, the next sequence frame to be played is frame 49.
[0114] 2. Based on the number of rows and columns of the sequence frames in the animation resource image, determine the number of sequence frames corresponding to the animation resource image.
[0115] In a possible implementation, after the frame numbers of the sequence frames to be played are determined, the number of rows and columns of the sequence frames in the animation resource image is calculated to determine the number of sequence frames corresponding to the animation resource image.
[0116] For example: Figure 8 As shown, it shows a schematic diagram of an animation resource image provided by an exemplary embodiment of the present application, including: a total of sixteen sequence frames from the first sequence frame 801 to the sixteenth sequence frame 816. The animation resource image includes four rows and four columns, corresponding to the sixteen sequence frames.
[0117] 3. Based on the frame number and the number of sequence frames corresponding to the animation resource image, determine the target animation resource texture to which the sequence frame to be played belongs.
[0118] In a possible implementation, after the frame number of the sequence frame to be played is determined, the animation resource image number corresponding to the sequence frame can be determined according to the frame number, thereby determining the target animation resource texture of the sequence frame to be played.
[0119] For example: Figure 8 As shown, it shows a schematic diagram of an animation resource image provided by an exemplary embodiment of the present application, including: a first sequence frame 801 to a sixteenth sequence frame 816, a total of sixteen sequence frames. The frame rate of a certain animation is 16fps, and each animation resource image is as follows Figure 7 The corresponding 16-frame sequence frame image is shown, and the current animation has been played for 3 seconds. It can be seen that the number of frames currently played is 3 seconds × 16fps = 48 frames. Therefore, the next sequence frame to be played is the 49th frame. Since one animation resource image corresponds to a 16-frame sequence frame image, it can be seen that 48 frames ÷ 16 frames = 3 animation resource images have been loaded and played, and the sequence frame to be played is the 49th frame, that is, the sequence frame to be played is the first sequence frame in the fourth animation resource image, and its corresponding target animation resource texture is the animation resource texture corresponding to the fourth animation resource image.
[0120] For the convenience of description, in this embodiment, each animation resource image includes 16 sequential frames. In other possible application scenarios, the animation may correspond to multiple animation resource images, wherein each animation resource image may include sequential frames with the same number of frames. The number of frames can be set in advance and is not necessarily the same as the embodiment. For example, a certain animation corresponds to 3 animation resource images, and the 3 animation resource images all include 24 frames. Optionally, each animation resource image may also include sequential frames with different numbers of frames. For example, a certain animation corresponds to 4 animation resource images, and the 4 animation resource images include 8, 24, 32 and 16 sequential frames, respectively.
[0121] 4. Based on the frame number and the number of rows and columns of the sequence frame in the target animation resource image corresponding to the target animation resource texture, determine the horizontal texture offset and the vertical texture offset of the sequence frame to be played in the target animation resource texture.
[0122] In one possible implementation, after determining the frame number of the sequence frame to be played and the target animation resource texture, the animation resource image number corresponding to the sequence frame and the position of the sequence frame in the animation resource image are determined based on the frame number, and the number of rows and columns of the sequence frame in the animation resource image is obtained, thereby determining the horizontal texture offset and the vertical texture offset of the sequence frame to be played in the target animation resource texture.
[0123] For example: each animation resource image corresponding to a certain animation contains 16 frames of sequence frame images, the number of frames currently played is 53, and the next sequence frame to be played is frame 54. Since one animation resource image corresponds to 16 frames of sequence frame images, it can be known that the sequence frame to be played is the sixth sequence frame in the fourth animation resource image, and its corresponding target animation resource texture is the animation resource texture corresponding to the fourth animation resource image. Fig. 9 As shown, it shows a schematic diagram of a target animation resource texture provided by an exemplary embodiment of the present application, including: textures corresponding to a total of sixteen sequence frames from the first sequence frame texture 901 to the sixteenth sequence frame texture 916. The sequence frame to be played with frame number 54 corresponds to two rows and two columns of sequence frames in the target animation resource image, so it corresponds to the sixth sequence frame texture 906 in the target animation resource texture. With the upper left corner of the target animation resource texture as the origin, a two-dimensional coordinate system is established, and the width and height of the target animation resource texture are both set to 1. With the two-dimensional coordinate system as a reference, the texture offsets of the first sequence frame texture 901 to the sixteenth sequence frame texture 916 can be determined. When the sequence frame to be played corresponds to the sixth sequence frame texture 906 in the target animation resource texture, it can be determined that the x-axis offset (horizontal texture offset) and the y-axis offset (vertical texture offset) of the sixth sequence frame texture 906 in the two-dimensional coordinate system are both 0.25.
[0124] Step 703: Based on the target animation resource texture and the texture offset, the texture attribute of the plane model is set so that the plane model displays the texture corresponding to the sequence image to be played.
[0125] In one possible implementation, during the rendering process of each sequence frame animation, the (texture) mapping properties of the plane model are updated according to the target animation resource texture and texture offset corresponding to the sequence frame to be played, so that the plane model displays the texture corresponding to the sequence image to be played, thereby achieving the effect of playing the sequence frame animation in the AR scene.
[0126] In the embodiment of the present application, when the animation resource image is an animation continuous sequence frame image, since the animation resource image is an aggregation of multiple continuous sequence frames, the animation resource image can be stored in a continuous memory area to avoid storing multiple sequence frame images separately and generating more memory fragments. Then, based on the animation resource texture corresponding to the animation resource image, the texture offset corresponding to the sequence frame to be played is calculated, so that the texture corresponding to the sequence frame to be played is displayed when the animation is played, and then the corresponding sequence frame animation is displayed by playing the continuous sequence frame, so as to achieve the animation display effect in the AR scene.
[0127] During the animation display process in the AR scene, the animation resource image corresponding to the loaded animation can also be a custom animation. Custom animations are different from static images and can be video animation files of types such as two-dimensional animations, three-dimensional animations, and vector animations. Based on custom animations, AR animation display is performed on the real world screen by playing continuous sequence frames. This can be achieved through the following steps, such as Fig.10 As shown, it shows a flowchart of a custom animation display process provided by an exemplary embodiment of the present application, and the process includes:
[0128] Step 1001, displaying an AR interface, wherein the AR interface includes a real-world image, an animation playback control, and an animation upload control.
[0129] In a possible implementation, the animation playback control is used to trigger the display of a custom animation superimposed on a real-world screen, and the animation upload control is used to upload a custom animation that needs to be displayed in the AR interface.
[0130] Step 1002: In response to triggering operations on the animation upload control and the animation play control, load the custom animation.
[0131] In one possible implementation, in response to a trigger operation on an animation upload control, the terminal can upload a custom animation that needs to be displayed in the AR interface, and then, in response to a trigger operation on an animation playback control, load the custom animation as an animation resource texture, so that the custom animation can be subsequently displayed on top of the real-world screen through continuous sequence frame playback based on the animation resource texture.
[0132] In a possible implementation, after triggering the animation upload control, the terminal may automatically trigger the animation playback control to load the custom animation as an animation resource texture. Optionally, after triggering the animation upload control, the user may also actively trigger the animation playback control in the AR interface.
[0133] Step 1003 , based on the loaded custom animation, the animation is displayed on the upper layer of the real world screen by playing a continuous sequence of frames.
[0134] In a possible implementation, after the custom animation is loaded as an animation resource texture, the animation resource texture is cut out, wherein the cut out process is used to remove the background in the animation resource texture. Different from the case where the animation resource image is a static image, the custom animation may carry an interfering background that is not needed in the AR animation display, so the animation resource texture of the custom animation is cut out. Then, based on the animation resource texture after the cut out process, the animation is displayed on the upper layer of the real world screen by playing a continuous sequence of frames.
[0135] For example: the real-world picture in the AR interface is a scenic spot in a tourist area, and the custom animation to be loaded is the explanation video recorded by the tour guide in the room. The effect that needs to be achieved in the AR scenario is to display the animation on top of the real-world picture, that is, to display the tour guide's explanation video on top of the scenic spot picture. The terminal identifies the custom animation to be loaded, and recognizes that the foreground is a person (tour guide) and the background is a room, where the room is an unnecessary interference background in the AR animation display. Therefore, the animation resource texture of the explanation video is cut out to remove the room background and only retain the person. Based on the explanation video after the cutout processing, the tour guide's explanation video (animation) is displayed on top of the scenic spot (real-world picture) in the AR interface through continuous sequence frame playback.
[0136] In an embodiment of the present application, when the animation resource image corresponding to the loaded animation is a custom animation, the corresponding animation resource texture is loaded based on the custom animation, the interfering background in the custom animation is removed by cutout, and based on the animation resource texture after cutout, AR animation is displayed on top of the real-world picture by playing a continuous sequence of frames, thereby increasing the diversity of animation resource image types and displaying richer animation effects in AR scenarios.
[0137] During the animation display process in the AR scene, the animation in the AR scene can be changed accordingly according to the different geographical locations of the user and the terminal held. At the same time, according to the different shooting angles of the user's terminal to the real world picture, the best viewing angle can be selected to load and display the animation to achieve a better animation display effect. This process may include:
[0138] 1. In response to a trigger operation on an animation playback control, a current shooting viewing angle is determined based on a real-world image.
[0139] In some embodiments, the terminal has a filming function, which films the current environment and generates a real-world image for display in the AR interface. By identifying the target object in the real-world image, the angle of the current terminal shooting is determined.
[0140] For example: a user holds a smartphone to take a picture of the building in front of him / her, and uses the building and the scene within the shooting range of the smartphone as the real-world picture. By identifying the position of the horizon in the real-world picture and the inclination angle of the edge of the building, the angle at which the user takes the picture with the smartphone is determined.
[0141] 2. When the current shooting angle is within the optimal viewing angle range, the animation is displayed on top of the real-world image through continuous sequence frame playback based on the loaded animation resource image.
[0142] In some embodiments, after determining the shooting angle of the terminal, the optimal viewing angle range of the animation in the AR scene is determined. When the shooting angle of the current terminal falls within the optimal viewing angle range, the animation is displayed on top of the real-world picture through continuous sequence frame playback based on the loaded animation resource image.
[0143] For example, Fig.11 As shown, Fig.11 A schematic diagram of a shooting angle provided by an exemplary embodiment of the present application is shown. After determining the shooting angle of the terminal (smartphone), the best viewing angle range 1101 of the animation in the AR scene is determined and a pop-up window 1102 is sent to the user to remind the user to adjust the terminal shooting angle. When the user adjusts the terminal shooting angle to the best viewing angle range, the animation resource image corresponding to the loaded fireworks effect animation is displayed on the upper layer of the real world screen by playing a continuous sequence of frames.
[0144] In the embodiment of the present application, the animation in the AR scene can be changed accordingly according to the different geographical locations of the user and the terminal held, thereby increasing the richness of the AR animation display content. At the same time, based on the different shooting angles of the terminal to the real world picture, the animation is loaded and displayed by selecting the best viewing angle to achieve a better animation display effect.
[0145] Fig.12 : is a structural block diagram of an animation display device in an AR scene provided by an exemplary embodiment of the present application, the device comprising:
[0146] The interface display module 1201 is used to display an AR interface, which includes a real-world image and an animation playback control, and the animation playback control is used to trigger the display of an animation superimposed on the real-world image;
[0147] The loading module 1202 is used to load the animation resource image corresponding to the animation in response to the triggering operation of the animation playing control;
[0148] The animation display module 1203 is used to display the animation on the upper layer of the real world picture by playing a continuous sequence of frames based on the loaded animation resource image.
[0149] Optionally, the loading module 1202 is further used for:
[0150] In response to a triggering operation on the animation playing control, the animation resource image is loaded as an animation resource texture.
[0151] Optionally, the animation display module 1203 is further used to:
[0152] Based on the animation resource texture, animation display is performed on the real world picture layer by playing a continuous sequence of frames.
[0153] Optionally, the loading module 1202 is further used for:
[0154] In the case where the animation corresponds to at least two of the animation resource images, the animation resource images corresponding to the animation are loaded in sequence based on the image order.
[0155] Optionally, the animation display module 1203 is further used to:
[0156] When the number of loaded animation resource images reaches a threshold, animation display is performed on the upper layer of the real-world picture by playing a continuous sequence of frames based on the animation resource texture, wherein the remaining animation resource images are loaded during the animation display process.
[0157] Optionally, the animation resource image includes multiple sequence frames, and the animation display module 1103 is further used to:
[0158] Creating a plane model in a virtual space corresponding to the real-world picture, wherein the plane model is used for animation display and is a transparent model;
[0159] Determine the target animation resource texture and texture offset corresponding to the sequence frame to be played, wherein the texture offset is the offset of the texture corresponding to the sequence frame to be played in the target animation resource texture;
[0160] Based on the target animation resource texture and the texture offset, the mapping properties of the plane model are set so that the plane model displays the texture corresponding to the sequence image to be played.
[0161] Optionally, the animation display module 1203 is further used to:
[0162] Determine the frame number of the sequence frame to be played based on the animation playback time and frame rate;
[0163] The target animation resource texture and the texture offset are determined based on the frame number and the number of rows and columns of the sequence of frames in the animation resource image.
[0164] Optionally, the animation display module 1203 is further used to:
[0165] Determine the number of sequence frames corresponding to the animation resource image based on the number of rows and columns of the sequence frames in the animation resource image;
[0166] Determine the target animation resource texture to which the sequence frame to be played belongs based on the frame number and the number of sequence frames corresponding to the animation resource image;
[0167] Based on the frame number and the number of rows and columns of the sequence frames in the target animation resource image corresponding to the target animation resource texture, the horizontal texture offset and the vertical texture offset of the sequence frame to be played in the target animation resource texture are determined.
[0168] Optionally, the animation display module 1203 is further used to:
[0169] When the terminal moves, the model orientation of the plane model is adjusted based on the terminal orientation, wherein the model orientation of the plane model is consistent with the terminal orientation.
[0170] Optionally, the loading module 1202 is further used for:
[0171] In response to a triggering operation on the animation playback control, the animation resource image corresponding to the target animation is loaded based on the current geographical location, wherein different animations are set corresponding to different geographical locations.
[0172] Optionally, the AR interface further includes an animation upload control, and the device further includes an animation upload module, which is used to:
[0173] In response to a triggering operation on the animation upload control, loading a custom animation;
[0174] Based on the loaded custom animation, the animation is displayed on the upper layer of the real world picture by playing a continuous sequence of frames.
[0175] Optionally, the animation uploading module is further used to:
[0176] Loading the custom animation as an animation resource texture;
[0177] The loaded custom animation is displayed on the upper layer of the real world screen by playing a continuous sequence of frames, including:
[0178] Performing a cutout process on the animation resource texture, wherein the cutout process is used to remove the background in the animation resource texture;
[0179] Based on the animation resource texture after the cutout processing, the animation is displayed on the upper layer of the real world picture by playing a continuous sequence of frames.
[0180] Optionally, the device further comprises a viewing angle module, configured to:
[0181] In response to a triggering operation on the animation playback control, determining a current shooting viewing angle based on the real-world picture;
[0182] The method of displaying the animation on the upper layer of the real world screen by playing a continuous sequence of frames based on the loaded animation resource image includes:
[0183] In the case that the current shooting angle of view belongs to the best viewing angle range, based on the loaded animation resource image, animation display is performed on the upper layer of the real world picture by playing a continuous sequence of frames.
[0184] To sum up, in the embodiment of the present application, the AR interface is first displayed, and the resource image corresponding to the animation is loaded by triggering the animation control set in the AR interface. Since the animation resource image can be a continuous multi-frame image or a custom animation, its data volume is less than that of the three-dimensional model, and it occupies less memory, which makes the animation resource image loading speed faster accordingly. Then, based on the animation resource image corresponding to the animation, the real-world picture and the superimposed animation are displayed in the form of continuous sequence frame playback, which can reduce the amount of calculation in the animation rendering process, thereby speeding up the rendering speed. Compared with rendering the three-dimensional model, the animation rendering speed is faster, thereby improving the loading speed of the AR animation effect.
[0185] It should be noted that: the device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.
[0186] Please refer to Fig.13 , which shows a block diagram of a terminal 1300 provided by an exemplary embodiment of the present application. The terminal 1300 may be a portable mobile terminal, such as a smart phone, a tablet computer, or a Moving Picture Experts Group Audio Layer IV (MP4) player. The terminal 1300 may also be referred to as a user device, a portable terminal, or other names.
[0187] Typically, the terminal 1300 includes a processor 1301 and a memory 1302 .
[0188] The processor 1301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1301 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1301 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1301 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.
[0189] The memory 1302 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1302 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1302 is used to store at least one instruction, which is used to be executed by the processor 1301 to implement the method provided in the embodiment of the present application.
[0190] In some embodiments, the terminal 1300 may also optionally include a peripheral device interface 1303 and at least one peripheral device.
[0191] In an embodiment of the present application, the peripheral device may include a camera. The camera is used to capture real-world images. In one embodiment, the processor 1301 may generate a real-world picture based on the real-world image captured by the camera, and display the real-world picture in the AR interface.
[0192] Those skilled in the art will understand that Fig.13 The structure shown in the figure does not constitute a limitation on the terminal 1300, and the terminal 1300 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.
[0193] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the animation display method in the AR scene as described in the above embodiments.
[0194] The embodiment of the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the animation display method in the AR scene described in the above embodiment.
[0195] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0196] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. An animation display method in an AR scene, characterized in that: The method comprises: Displaying an AR interface, wherein the AR interface includes a real-world image and an animation playback control, wherein the animation playback control is used to trigger the display of an animation superimposed on the real-world image; In response to a triggering operation on the animation playing control, loading an animation resource image corresponding to the animation; Based on the loaded animation resource image, animation display is performed on the upper layer of the real world picture by playing a continuous sequence of frames.
2. The method according to claim 1, characterized in that The step of loading the animation resource image in response to the triggering operation on the animation playing control comprises: In response to a triggering operation on the animation playback control, loading the animation resource image as an animation resource texture; The method of displaying the animation on the upper layer of the real world screen by playing a continuous sequence of frames based on the loaded animation resource image includes: Based on the animation resource texture, animation display is performed on the real world picture layer by playing a continuous sequence of frames.
3. The method according to claim 2, characterized in that The step of loading the animation resource image as an animation resource texture comprises: In a case where the animation corresponds to at least two of the animation resource images, sequentially loading the animation resource images corresponding to the animation based on the image order; The method of displaying the animation on the upper layer of the real world screen by playing a continuous sequence of frames based on the loaded animation resource image includes: When the number of loaded animation resource images reaches a threshold, animation display is performed on the upper layer of the real-world picture by playing a continuous sequence of frames based on the animation resource texture, wherein the remaining animation resource images are loaded during the animation display process.
4. The method according to claim 2, characterized in that: The animation resource image includes multiple sequence frames; The method of performing animation display on the real world screen by playing a continuous sequence of frames based on the animation resource texture includes: Creating a plane model in a virtual space corresponding to the real-world picture, wherein the plane model is used for animation display and is a transparent model; Determine the target animation resource texture and texture offset corresponding to the sequence frame to be played, wherein the texture offset is the offset of the texture corresponding to the sequence frame to be played in the target animation resource texture; Based on the target animation resource texture and the texture offset, the mapping properties of the plane model are set so that the plane model displays the texture corresponding to the sequence image to be played.
5. The method according to claim 4, characterized in that The step of determining the target animation resource texture and texture offset corresponding to the sequence frame to be played includes: Determine the frame number of the sequence frame to be played based on the animation playback time and frame rate; The target animation resource texture and the texture offset are determined based on the frame number and the number of rows and columns of the sequence of frames in the animation resource image.
6. The method according to claim 5, characterized in that The determining the target animation resource texture and the texture offset based on the frame number and the number of rows and columns of the sequence frames in the animation resource image includes: Determine the number of sequence frames corresponding to the animation resource image based on the number of rows and columns of the sequence frames in the animation resource image; Determine the target animation resource texture to which the sequence frame to be played belongs based on the frame number and the number of sequence frames corresponding to the animation resource image; Based on the frame number and the number of rows and columns of the sequence frames in the target animation resource image corresponding to the target animation resource texture, the horizontal texture offset and the vertical texture offset of the sequence frame to be played in the target animation resource texture are determined.
7. The method according to claim 4, characterized in that The method further comprises: When the terminal moves, the model orientation of the plane model is adjusted based on the terminal orientation, wherein the model orientation of the plane model is consistent with the terminal orientation.
8. The method according to claim 1, characterized in that The step of loading an animation resource image corresponding to the animation in response to a triggering operation on the animation playing control comprises: In response to a triggering operation on the animation playback control, the animation resource image corresponding to the target animation is loaded based on the current geographical location, wherein different animations are set corresponding to different geographical locations.
9. The method according to claim 1, characterized in that: The AR interface also includes an animation upload control; the method also includes: In response to a triggering operation on the animation upload control, loading a custom animation; Based on the loaded custom animation, the animation is displayed on the upper layer of the real world picture by playing a continuous sequence of frames.
10. The method according to claim 9, characterized in that The loading of custom animation includes: Loading the custom animation as an animation resource texture; The loaded custom animation is displayed on the upper layer of the real world screen by playing a continuous sequence of frames, including: Performing a cutout process on the animation resource texture, wherein the cutout process is used to remove the background in the animation resource texture; Based on the animation resource texture after the cutout processing, the animation is displayed on the upper layer of the real world picture by playing a continuous sequence of frames.
11. The method according to claim 1, characterized in that: The method further comprises: In response to a triggering operation on the animation playback control, determining a current shooting viewing angle based on the real-world picture; The method of displaying the animation on the upper layer of the real world screen by playing a continuous sequence of frames based on the loaded animation resource image includes: In the case that the current shooting angle of view belongs to the best viewing angle range, based on the loaded animation resource image, animation display is performed on the upper layer of the real world picture by playing a continuous sequence of frames.
12. An animation display device in an AR scene, characterized in that: The device comprises: An interface display module, used to display an AR interface, wherein the AR interface includes a real-world image and an animation playback control, wherein the animation playback control is used to trigger the display of an animation superimposed on the real-world image; A loading module, used for loading an animation resource image corresponding to the animation in response to a triggering operation on the animation playing control; The animation display module is used to display the animation on the upper layer of the real world picture by playing a continuous sequence of frames based on the loaded animation resource image.
13. A terminal, characterized in that: The terminal includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the animation display method in the AR scene as described in any one of claims 1 to 11.
14. A computer-readable storage medium, characterized in that: At least one instruction is stored in the readable storage medium, and the at least one instruction is loaded and executed by the processor to implement the animation display method in the AR scene as described in any one of claims 1 to 11.
15. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a server reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the server executes the animation display method in an AR scene as described in any one of claims 1 to 11.
