Dynamic image processing method, device, storage medium, equipment and program product

Abstract

The application discloses a dynamic image processing method and device, a storage medium, equipment and a program product, which are applied to video playing application, social media, game development, instant messaging and the like. The method comprises the following steps: in response to a playing scheduling instruction triggered based on a preset period, a dynamic image list is traversed, and the dynamic image list comprises at least one dynamic image to be played; for a current dynamic image traversed, if it is determined that the current dynamic image has to-be-decoded image frames, a target decoding task corresponding to the to-be-decoded image frames is generated, wherein the to-be-decoded image frames corresponding to the current dynamic image are undecoded images in a preset number of frames of images about to be played in the current dynamic image; and the target decoding task is asynchronously decoded in a sub-thread. The application improves the smoothness of dynamic image playing and enhances user experience.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, specifically to a dynamic image processing method, apparatus, storage medium, device, and program product. Background Technology

[0002] In today's digital age, moving images (including videos, GIFs, etc.) have become an important part of internet content. Whether it's short video sharing on social media, real-time video calls in online meetings, or dynamic effects in games and entertainment, moving images play an indispensable role.

[0003] However, despite the emergence of numerous animated image playback solutions, these solutions still face many challenges in practical applications. In light animated image playback scenarios, these solutions may barely maintain the user experience, but once in medium to heavy scenarios, problems arise one after another. User interface stuttering and excessive resource consumption occur frequently, severely impacting the user's smooth experience and posing a potential threat to application stability and performance. With the widespread use of animated images in internet content and the ever-increasing demands of users for playback quality, ensuring smooth playback of animated images has become a pressing technical challenge. Summary of the Invention

[0004] This application provides a dynamic image processing method, apparatus, storage medium, device, and program product, which can ensure that the dynamic image asynchronously decodes a preset number of frames of images to be played through a playback scheduling instruction triggered at a preset period, thereby reducing playback delay and stuttering and improving playback smoothness.

[0005] On one hand, embodiments of this application provide a dynamic image processing method, the method comprising:

[0006] In response to a playback scheduling command triggered based on a preset period, the dynamic image list is traversed, the dynamic image list including at least one dynamic image to be played;

[0007] For the current dynamic image that has been traversed, if it is determined that there is an image frame to be decoded in the current dynamic image, a target decoding task corresponding to the image frame to be decoded is generated, wherein the image frame to be decoded corresponding to the current dynamic image is an undecoded image in a preset number of frames that will be played in the current dynamic image.

[0008] The target decoding task is asynchronously decoded in a child thread.

[0009] On the other hand, embodiments of this application provide a dynamic image processing apparatus, the apparatus comprising:

[0010] The traversal unit is used to traverse the dynamic image list in response to a playback scheduling instruction triggered based on a preset period, wherein the dynamic image list includes at least one dynamic image to be played.

[0011] The generation unit is used to generate a target decoding task corresponding to the image frame to be decoded if it is determined that there is an image frame to be decoded in the current dynamic image that has been traversed. The image frame to be decoded corresponding to the current dynamic image is an undecoded image in a preset number of frames that will be played in the current dynamic image.

[0012] A decoding unit is used to asynchronously decode the target decoding task in a sub-thread.

[0013] In some embodiments, the apparatus further includes a rendering unit for:

[0014] If it is determined that the image frame to be played in the current dynamic image meets the playback conditions, it is determined whether there is decoded image data corresponding to the image frame to be played in the cache;

[0015] If it is determined that the decoded image data corresponding to the image frame to be played exists in the cache, the decoded image data corresponding to the image frame to be played is retrieved from the cache;

[0016] The image frame to be played is rendered and displayed based on the decoded image data corresponding to the image frame to be played.

[0017] In some embodiments, when the decoding unit is used to asynchronously decode the target decoding task in a sub-thread, it is specifically used for:

[0018] The target decoding task is asynchronously decoded in a sub-thread according to its scheduling priority. The scheduling priority of the target decoding task is determined based on whether the image to be decoded is the first frame of the current dynamic image. If the image to be decoded is the first frame of the current dynamic image, the scheduling priority of the target decoding task is high; if the image to be decoded is not the first frame of the current dynamic image, the scheduling priority of the target decoding task is low.

[0019] In some embodiments, when the decoding unit is used to asynchronously decode the target decoding task in a sub-thread according to the scheduling priority of the target decoding task, it is specifically used for:

[0020] Add the target decoding task and its scheduling priority to the decoding queue;

[0021] In the sub-thread, the target decoding task is retrieved from the decoding queue and executed according to the high scheduling priority priority policy.

[0022] In some embodiments, the apparatus further includes a cache unit for:

[0023] In response to the completion of the target decoding task, it is determined whether to cache the decoding result corresponding to the target decoding task based on the frame number of the current dynamic image.

[0024] In some embodiments, when the caching unit is used to determine whether to cache the decoding result corresponding to the target decoding task based on the frame number of the current dynamic image, it is specifically used for:

[0025] If the number of frames of the current dynamic image is not greater than the preset number of frames, the decoding result corresponding to the target decoding task is cached for use when the image frame to be decoded is played again.

[0026] In some embodiments, when the caching unit is used to cache the decoding result corresponding to the target decoding task, it is specifically used for:

[0027] The decoding results corresponding to the target decoding task are cached according to the display size of the image frame to be decoded.

[0028] In some embodiments, when the caching unit is used to cache the decoding result corresponding to the target decoding task, it is specifically used for:

[0029] The decoding result corresponding to the target decoding task is cached in the target format, where the pixel occupancy of the target format is smaller than that of the initial format of the decoding result.

[0030] In some embodiments, when the caching unit is used to cache the decoding result corresponding to the target decoding task, it is specifically used for:

[0031] The decoding results corresponding to the target decoding task are cached based on the least recently used strategy.

[0032] In some embodiments, the apparatus further includes a cache cleanup unit for:

[0033] Clean up unused decoding results in the cache that have exceeded a preset time threshold based on the preset cache cleanup interval.

[0034] In some embodiments, the number of frames in the preset number of frames is determined based on the decoding time and rendering time of the current dynamic image.

[0035] In some embodiments, the preset number of frames is two frames.

[0036] On the other hand, an embodiment of this application provides a computer-readable storage medium storing a computer program adapted for loading by a processor to execute the dynamic image processing method as described in any of the above embodiments.

[0037] On the other hand, an embodiment of this application provides a computer device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the dynamic image processing method as described in any of the above embodiments by calling the computer program stored in the memory.

[0038] On the other hand, an embodiment of this application provides a computer program product, including computer instructions, which, when executed by a processor, implement the dynamic image processing method as described in any of the above embodiments.

[0039] This embodiment of the application, in response to a playback scheduling instruction triggered based on a preset period, traverses a list of dynamic images, which includes at least one dynamic image to be played. For the current dynamic image being traversed, if it is determined that there is an image frame to be decoded in the current dynamic image, a target decoding task corresponding to the image frame to be decoded is generated. The image frame to be decoded corresponding to the current dynamic image includes a preset number of frames in the current dynamic image that will be played. Asynchronous decoding is performed on the target decoding task in a sub-thread. To achieve unified scheduling and playback of dynamic images, this embodiment of the application maintains a list of dynamic images and triggers a pre-decoding process for the dynamic images in the list in a sub-thread every preset period. This ensures that the dynamic images are pre-decoded according to a predetermined time interval, and the decoding process is executed in a sub-thread, reducing playback delay and user interface stuttering, thereby improving the smoothness of dynamic image playback. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the architecture of the dynamic image processing system provided in the embodiments of this application.

[0041] Figure 2 This is a flowchart illustrating the dynamic image processing method provided in an embodiment of this application.

[0042] Figure 3 This is a schematic diagram of the architecture of the dynamic image processing method provided in the embodiments of this application.

[0043] Figure 4 This is a flowchart illustrating the scheduling logic for a list of dynamic images, provided in an embodiment of this application.

[0044] Figure 5 The diagram shows a comparison experiment of the pre-decoding schemes provided in the embodiments of this application.

[0045] Figure 6 This is a schematic diagram of the structure of the dynamic image processing device provided in the embodiments of this application.

[0046] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] This application provides a dynamic image processing method, apparatus, storage medium, device, and program product. Exemplarily, the dynamic image processing method of this application can be executed by a computer device, which can be a terminal or server, etc. The terminal can be a smartphone, tablet, laptop, desktop computer, smart TV, smart speaker, wearable smart device, personal computer (PC), smart vehicle terminal, etc. The terminal may also include a client, which can be a video client, shopping application client, reading application client, browser client, or instant messaging client, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0049] The embodiments of this application can be applied to scenarios such as video playback applications, social media, game development, and instant messaging.

[0050] First, some of the nouns or terms that appear in the description of the embodiments of this application are explained as follows:

[0051] Dynamic images: These are images that produce a certain dynamic effect when a set of specific static images is switched at a specified frequency.

[0052] Skia is an open-source 2D graphics library that provides a cross-platform API, enabling high-performance 2D graphics rendering and animation effects across different operating systems. Its design goal is to provide high-performance, highly customizable, and easy-to-use 2D graphics rendering capabilities, which can be used to build various types of applications, including graphics editors, games, and more.

[0053] SkCodec: SkCodec is a class in the Skia graphics library used for decoding compressed image data. The SkCodec class provides a series of static methods for creating SkCodec objects that can decode images from a given data source. It supports many common image formats, including PNG, JPEG, GIF, WebP, etc.

[0054] TaskRunner: TaskRunner is an abstract class in Chromium that provides an interface for scheduling and executing tasks on a specific thread or thread pool. The main method of TaskRunner is PostTask, which accepts a Closure (which can be understood as a parameterless function object) and adds it to the task queue of the thread associated with the TaskRunner. This Closure will be executed at some future time on the thread or thread pool associated with the TaskRunner.

[0055] PostTask: The PostTask method is part of the TaskRunner class. This method accepts a Closure as a parameter, which encapsulates the task to be executed. The PostTask method adds this task to the task queue and then executes the task on the associated thread or thread pool at the appropriate time (usually when the thread that called PostTask is idle).

[0056] Pre-decoding: This refers to decoding the moving images before they are played, rather than waiting until playback time, in order to ensure smoother playback of the moving images.

[0057] The solutions provided in this application relate to technologies such as dynamic image processing, and are specifically illustrated through the following embodiments. Detailed descriptions are provided below. It should be noted that the order of description of the following embodiments is not intended to limit the priority of the embodiments.

[0058] Please see Figure 1 , Figure 1This is a schematic diagram of the architecture of a dynamic image processing system provided in an embodiment of this application. The system can implement a dynamic image processing method. The dynamic image processing system may include a terminal 1000, a server 2000, and a network 3000. The terminal 1000 and the server 2000 can interact with each other via the network 3000, which may be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0059] In some embodiments, terminal 1000 sends a dynamic image acquisition request to server 2000. In response to the dynamic image acquisition request, server 2000 sends an encoded dynamic image to terminal 1000. Terminal 1000 receives the encoded dynamic image sent by server 2000 and adds the newly received dynamic image to the dynamic image list. In response to a playback scheduling instruction triggered based on a preset period, terminal 1000 traverses the dynamic image list, which includes at least one dynamic image to be played. For the current dynamic image that has been traversed, if terminal 1000 determines that there is an image frame to be decoded in the current dynamic image, it generates a target decoding task corresponding to the image frame to be decoded. The image frame to be decoded corresponding to the current dynamic image includes a preset number of frames in the current dynamic image that will be played. Terminal 1000 performs asynchronous decoding on the target decoding task in a sub-thread. Terminal 1000 renders and displays the decoded dynamic image.

[0060] Please see Figures 2 to 5 , Figure 2 This is a flowchart illustrating the dynamic image processing method provided in an embodiment of this application. Figure 3 This is a schematic diagram of the architecture of the dynamic image processing method provided in the embodiments of this application. Figure 4 This is a flowchart illustrating the scheduling logic for a list of dynamic images provided in an embodiment of this application. Figure 5 The diagram shows a comparative experiment of the pre-decoding schemes provided in the embodiments of this application. This method can be applied to, for example... Figure 1 The method for the terminal 1000 shown may include the following steps:

[0061] Step 110: In response to a playback scheduling instruction triggered based on a preset period, traverse the dynamic image list, which includes at least one dynamic image to be played.

[0062] Specifically, the terminal maintains a dynamic image list. When the terminal device receives a new dynamic image to be played, it adds the newly received dynamic image to the dynamic image list. These dynamic images are all sent to the terminal after being encoded by the server. The terminal needs to decode and render the dynamic images in the dynamic image list. The terminal responds to the playback scheduling command triggered based on a preset period by traversing the dynamic image list.

[0063] Dynamic images take on different forms in different applications or application scenarios. For example, in social media applications, dynamic images can be animated emoticons, live images, etc.; on online video platforms, dynamic images can be short videos, animations, or live stream covers, etc.; in game applications, dynamic images may be character animations, skill effects, background animations, etc.; in design software, dynamic images may be used to create advertising animations, product demonstrations, or website animation effects; and in mobile wallpaper applications and news applications, they can also be dynamic wallpapers, news scrolling bars, or real-time weather animations, etc.

[0064] In some embodiments, such as Figure 3 As shown, the playback scheduling instruction can be triggered by a timer. The timer triggers the playback scheduling instruction once every preset period. The playback scheduling instruction is used to schedule the playback and / or pre-decode dynamic images in the dynamic image list. The preset period can be reasonably set according to the actual application scenario, and this application does not impose any restrictions on it.

[0065] In one specific embodiment provided in this application, the preset period can be 16ms.

[0066] In some embodiments, such as Figure 3 As shown, the Timer is set in the playback scheduling module 310. The playback scheduling module 310 runs on the playback logic TaskRunner of the main thread. When the Timer is called back every preset period, the playback scheduling instruction is triggered, traversing each dynamic image in the dynamic image list, and scheduling the playback and / or pre-decoding of each traversed dynamic image.

[0067] In some embodiments, when a terminal device receives a new dynamic image, it calls the interface of the playback logic TaskRunner to request an image decoder from the decoder allocator CodecAllocator320 for the newly received dynamic image, and adds the image decoder to the dynamic image list.

[0068] Step 120: For the current dynamic image that has been traversed, if it is determined that there is an image frame to be decoded in the current dynamic image, generate the target decoding task corresponding to the image frame to be decoded, wherein the image frame to be decoded corresponding to the current dynamic image is an undecoded image in the preset number of frames that will be played in the current dynamic image.

[0069] like Figure 4As shown, when scheduling playback for a list of dynamic images, step S41 is executed sequentially. In response to a playback scheduling instruction triggered based on a preset period, the list of dynamic images is traversed. The list of dynamic images includes at least one dynamic image to be played. Step S42 is to take the next dynamic image. Step S43 is to determine whether there is a frame to be decoded for the current dynamic image. If it is determined that there is a frame to be decoded for the current dynamic image, step 44 is executed to generate the target decoding task corresponding to the frame to be decoded.

[0070] Specifically, since dynamic images are composed of static image frames, the smallest unit for decoding and rendering dynamic images is also the image frame in the dynamic image. For each dynamic image, the image frames that have been decoded and the image frames currently being played are recorded. When determining whether there are image frames to be decoded in a dynamic image, it is determined whether a preset number of frames following the currently playing image frame have been decoded. If it is determined that a preset number of frames following the currently playing image frame have been decoded, then it is determined that there are no image frames to be decoded in the dynamic image. If it is determined that there are image frames that have not yet been decoded among the preset number of frames following the currently playing image frame, then it is determined that there are image frames to be decoded in the current dynamic image, and the target decoding task corresponding to the image frame to be decoded is generated.

[0071] In related technologies, the decoding process of image frames is performed only when the image frame needs to be played. This decoding scheme may cause the decoding to be slower than the playback. Compared with related technologies, the solution provided in this application sets a timer for scheduling the playback of a global list of dynamic images. Each time the dynamic images are scheduled for playback, it is determined whether there are any undecoded images in the preset number of frames that are about to be played. If so, the undecoded image frames of the dynamic images are decoded, thereby achieving pre-decoding of the dynamic images and ensuring smooth playback of the dynamic images.

[0072] The number of image frames to be decoded in advance, i.e. the specific value of the preset number, should be set reasonably. If the preset number is too large, it will lead to a waste of resources. If the preset number is too small, it may cause the playback to wait for decoding indefinitely, resulting in a slower playback speed. Therefore, setting the preset number reasonably is crucial for the playback of dynamic images.

[0073] In some embodiments, the number of frames in the preset number of frames is determined based on the decoding speed and rendering speed of the current dynamic image.

[0074] For details, please refer to Figure 5 , Figure 5This is a comparative experimental diagram of the pre-decoding scheme provided in the embodiments of this application. The decoding time is denoted as td, and the playback time is denoted as tp. For example... Figure 5 As shown:

[0075] Play-one-decode: This scenario involves no pre-decoding; the image frame is decoded only when it is needed for playback (i.e., play-one-decode). Figure 5 As shown, for example, if the playback time is 2 units and the decoding time is 1 unit, the playback of image frames always needs to wait for the decoding process, which slows down the playback speed.

[0076] Pre-decode one frame: If one frame is pre-decoded, smooth playback of dynamic images can be achieved when the decoding time td < the playback time tp. However, when the decoding time td > the playback time tp, the decoding thread is not saturated.

[0077] Pre-decode two frames: Decoding two frames can ensure that the decoding thread is saturated when the decoding time td > the playback time tp. When the decoding time td < the playback time tp, at most only one frame of decoding waste may occur. If more than two frames are pre-decoded, multiple frames of decoding waste may have occurred when playback stops.

[0078] Therefore, in a specific embodiment provided in this application, the preset number of frames can be two frames, that is, a scheme of pre-decoding two image frames is selected.

[0079] Step 130: Perform asynchronous decoding of the target decoding task in a child thread.

[0080] Specifically, after the terminal determines that there is an image frame to be decoded in the current dynamic image and generates the target decoding task corresponding to the image frame to be decoded, it will perform asynchronous decoding on the target decoding task in a sub-thread.

[0081] Image decoding is usually time-consuming. If it is executed on the main thread that is responsible for user interface updates and user interaction, it can easily cause the main thread to lag when there are many decoding tasks, resulting in unsmooth scrolling and playback of the user interface. Therefore, in the solution provided in this application embodiment, the target decoding task is set to be executed in a sub-thread to reduce user interface lag and improve the smoothness of dynamic image playback.

[0082] In some embodiments, asynchronous decoding of the target decoding task in a child thread may include:

[0083] The target decoding task is asynchronously decoded in a sub-thread according to its scheduling priority. The scheduling priority of the target decoding task is determined by whether the image to be decoded is the first frame of the current dynamic image. If the image to be decoded is the first frame of the current dynamic image, the scheduling priority of the target decoding task is high; if the image to be decoded is not the first frame of the current dynamic image, the scheduling priority of the target decoding task is low.

[0084] For a dynamic image, the first frame of the dynamic image has the greatest impact on the playback experience. Therefore, this application distinguishes the priorities of different decoding tasks, setting the decoding task corresponding to the first frame image as a high scheduling priority and the decoding task corresponding to other frames as a low scheduling priority. When scheduling and executing multiple decoding tasks, the high scheduling priority decoding task will be scheduled and executed first. Therefore, the first frame image is decoded first, improving the display speed of the first frame image and optimizing the user experience.

[0085] In some embodiments, asynchronous decoding of the target decoding task in a sub-thread according to the scheduling priority of the target decoding task may include the following steps 01 to 02:

[0086] Step 01: Add the target decoding task and its scheduling priority to the decoding queue 330;

[0087] Specifically, such as Figure 4 As shown, in step S44, after the terminal device generates the target decoding task, it will also encapsulate the target decoding task and the scheduling priority corresponding to the target decoding task, and add them to the decoding queue 330.

[0088] In some embodiments, the decoding queue 330 includes two sub-queues: one is a sub-queue corresponding to high scheduling priority, in which the decoding tasks can all be referred to as the first frame task; the other is a sub-queue corresponding to low scheduling priority, in which the decoding tasks can all be referred to as other frame tasks.

[0089] When scheduling and executing decoding tasks, the decoding tasks in the sub-queues corresponding to high scheduling priorities are scheduled and executed first to ensure that high-priority decoding tasks can be executed first.

[0090] In other embodiments, the scheduling priority of each image frame can be determined based on the display status of the image frames in each decoding task over a historical period. The more times the images are displayed, the higher the scheduling priority. The decoding tasks in the decoding queue 330 are sorted according to the scheduling priority of each decoding task.

[0091] In some embodiments, since it is necessary to prioritize the decoding tasks in the decoding queue 330, the PostTask method of the TaskRunner for the decoding thread is encapsulated and a PostTaskWithPriority interface is provided. The PostTaskWithPriority interface receives a decoding task and the scheduling priority of the decoding task as parameters.

[0092] Adding the target decoding task and its scheduling priority to the decoding queue 330 involves: calling the TaskRunner's PostTaskWithPriority interface, passing the encapsulated target decoding task and scheduling priority as parameters to the interface, and having the TaskRunner check the status of the decoding queue 330 and add the target decoding task to the appropriate position in the decoding queue 330 according to its priority.

[0093] Step 02: In the sub-thread, the target decoding task is retrieved from the decoding queue and executed according to the high scheduling priority priority policy.

[0094] In the child thread, the decoding thread continuously retrieves decoding tasks from the decoding queue and executes them in the order of highest scheduling priority.

[0095] In some embodiments, asynchronous decoding of the target decoding task in a sub-thread according to the scheduling priority of the target decoding task further includes: using the SkCodec interface to asynchronously decode the target decoding task in a sub-thread according to the scheduling priority of the target decoding task.

[0096] Specifically, using the SkCodec interface, the target decoding task is asynchronously decoded in a child thread according to the scheduling priority of the target decoding task, including:

[0097] The target decoding task, the scheduling priority of the target decoding task, and the SKCodec instance selected according to the format of the current dynamic image are encapsulated into a target decoding task object. The TaskRunner's PostTaskWithPriority interface is called, and the encapsulated target decoding task object and its scheduling priority are passed as parameters to the interface to add the target decoding task to the decoding queue 330.

[0098] In the sub-thread, the target decoding task is retrieved from the decoding queue 330 and executed according to the high scheduling priority priority policy;

[0099] In the child thread, the decoding thread calls the SKCodec instance encapsulated in the task object to execute the corresponding decoding logic, decoding the image frame to be decoded into a renderable format.

[0100] In this embodiment, the decoding process is written using the SkCodec interface. Since SkCodec itself supports common animated image formats such as WEBP, APNG, and GIF, if other formats need to be extended, it is only necessary to inherit SkCodec to implement the decoding logic, so that the decoding of extended formats can be unified into the decoding process, which greatly improves the versatility of the dynamic image processing method.

[0101] In some embodiments, the dynamic image processing method may further include steps 140 to 160:

[0102] Step 140: If it is determined that the image frame to be played in the current dynamic image meets the playback conditions, determine whether there is decoded image data corresponding to the image frame to be played in the buffer.

[0103] The playback conditions can be that the frame playback timestamp of the currently playing image frame of the current dynamic image has expired or that the first frame of the current dynamic image needs to be played. The image frame to be played in the current dynamic image can be the next frame of the currently playing image frame or the first frame of the current dynamic image.

[0104] Specifically, such as Figure 3 or Figure 4 As shown, in step S45, for the current dynamic image that has been traversed, the terminal will determine whether the image frame to be played in the current dynamic image meets the playback conditions. If it is determined that the image frame to be played in the current dynamic image meets the playback conditions, then step S46 is executed to check whether there is decoded image data corresponding to the image frame to be played in the cache 340. If yes, then step S47 is executed; if no, then step S48 is executed.

[0105] Specifically, determining whether the image frame to be played in the current dynamic image meets the playback conditions can be done by: determining whether the frame playback timestamp of the currently playing image frame of the current dynamic image has expired or whether the first frame of the current dynamic image needs to be played. If so, then the image frame to be played in the current dynamic image is determined to meet the playback conditions.

[0106] Specifically, determining whether there is decoded image data corresponding to the image frame to be played in cache 340 can be done by: checking whether there is decoded image data corresponding to the image frame to be played in cache 340. If a match is found, it is determined that there is decoded image data corresponding to the image frame to be played in cache 340. If a match is not found, it is determined that there is no decoded image data corresponding to the image frame to be played in cache 340.

[0107] Step 150: If it is determined that there is decoded image data corresponding to the image frame to be played in the cache 340, retrieve the decoded image data corresponding to the image frame to be played from the cache 340.

[0108] If the terminal finds the decoded image data corresponding to the image frame to be played in cache 340, it can directly obtain the decoded image data corresponding to the image frame to be played from the cache. If the terminal does not find the decoded image data corresponding to the image frame to be played in the cache, it needs to wait for the decoding thread to complete the decoding task corresponding to the image frame to be played in order to obtain the decoded image data corresponding to the image frame to be played.

[0109] Step 160: Render and display the image frame to be played based on the decoded image data corresponding to the image frame to be played.

[0110] like Figure 4 As shown, after the terminal obtains the decoded image data corresponding to the image frame to be played, it will execute step S47 to record the decoded image data corresponding to the image frame to be played. Further, it will execute step S48 to determine whether all images in the dynamic image list have been traversed. If not, it will continue to traverse and return to execute step S42 to retrieve the next dynamic image in the dynamic image list. If the traversal of the dynamic image list has been completed, it will execute step S49 to notify the user interface according to the decoded image data corresponding to each image frame to be played recorded during the traversal, so as to refresh the display content of the user interface. That is, the user interface renders and displays the image frame to be played according to the decoded image data corresponding to the image frame to be played.

[0111] In this embodiment, a cache is introduced to store the decoded image data. When it is necessary to play dynamic images, the dynamic images are played according to the decoded image data stored in the cache, which can improve the smoothness of dynamic image playback.

[0112] Since image decoding is a computationally intensive task, continuous decoding can lead to consistently high CPU usage. Therefore, this application uses a space-for-time strategy to cache the decoding results so that the dynamic images can be directly retrieved for subsequent playback. However, the cache space is limited and cannot be overused. Therefore, this application provides the following caching strategy for the decoding results.

[0113] In some embodiments, the dynamic image processing method may further include step 170:

[0114] Step 170: In response to the completion of the target decoding task, determine whether to cache the decoding result corresponding to the target decoding task based on the current frame number of the dynamic image.

[0115] When the target decoding task is completed, the terminal determines whether to cache the decoding result corresponding to the target decoding task based on the current frame number of the dynamic image.

[0116] In some embodiments, determining whether to cache the decoding result corresponding to the target decoding task based on the frame number of the current dynamic image includes:

[0117] If the number of frames in the current dynamic image is not greater than the preset number of frames, the decoding result corresponding to the target decoding task will be cached for use when the image frame to be decoded is played again.

[0118] The preset frame rate can be set reasonably according to the actual application scenario.

[0119] Specifically, after the terminal completes the decoding of the target decoding task, it will determine whether the number of frames of the current dynamic image is greater than the preset number of frames. If the number of frames of the current dynamic image is not greater than the preset number of frames, the terminal will cache the decoding result corresponding to the target decoding task for use when the image frame to be decoded is played next time. If the number of frames of the current dynamic image is greater than or equal to the preset number of frames, the terminal will not cache the decoding result corresponding to the target decoding task.

[0120] In this embodiment, for some animated images with a small number of frames, the decoding results of such animated images can be cached. When such animated images are played later, the cache hit rate is high, and the memory usage is small. Therefore, for such animated images, caching can be performed. The next time an animated image needs to be played, the decoded image data of the animated image can be directly read from the cache, thereby ensuring fast and smooth playback of the animated image. For some animated images with a large number of frames, it takes tens of seconds to play these animated images once. Even if the decoding results of such animated images are cached, the cache hit rate is not high, but at the same time, it will occupy a lot of memory. Therefore, for such animated images, caching is not necessary.

[0121] In some embodiments, caching the decoding result corresponding to the target decoding task may include:

[0122] The decoding results corresponding to the target decoding task are cached according to the display size of the image frame to be decoded.

[0123] Specifically, the terminal can scale the decoding result corresponding to the target decoding task according to the display size of the image frame to be decoded, and cache the scaled decoding result.

[0124] Since some dynamic images are very large, caching them at their original size can result in several megabytes per frame. However, in real-world scenarios, the display size of dynamic images is usually smaller. Caching them at their display size can significantly reduce memory usage and the amount of computation required for rendering.

[0125] In some embodiments, caching the decoding result corresponding to the target decoding task may further include:

[0126] The decoding result corresponding to the target decoding task is cached in the target format, and the pixel size of the target format is smaller than the pixel size of the initial format of the decoding result.

[0127] In some embodiments, the pixel footprint of the target format is half that of the pixel footprint of the initial format of the decoded result.

[0128] In some embodiments, the target format may be RGB565 format.

[0129] In related technologies, frame buffers commonly use the ARGB8888 format for storage. This format occupies four bytes per pixel, thus requiring relatively large buffer space. On some low-performance machines, without caching, decoding speed may consistently lag behind playback speed, impacting user experience; while full caching can easily lead to Out of Memory (OOM) errors. To address this problem, this application uses the RGB565 format, with smaller pixel size, to replace the ARGB8888 format for caching decoding results. The RGB565 and ARGB8888 formats have similar color gamuts, and although there are some differences in color granularity, these differences are not noticeable in actual use. Importantly, each pixel in the RGB565 format only occupies two bytes, reducing memory usage by half compared to the ARGB8888 format. This ensures image quality while significantly reducing storage space requirements, resulting in a more efficient and economical decoding and playback process.

[0130] In some embodiments, caching the decoding result corresponding to the target decoding task may further include:

[0131] The decoding results corresponding to the target decoding task are cached based on the least recently used strategy.

[0132] Specifically, the terminal caches the decoding results based on the least recently used strategy. When the cache space is insufficient, the system will first delete the least recently used decoding results to ensure that decoded image data that has not been used for a long time does not occupy memory.

[0133] In some embodiments, the dynamic image processing method further includes step 180:

[0134] Step 180: Clean up unused decoding results in the cache that have exceeded a preset time threshold based on a preset cache cleanup interval.

[0135] The preset cache cleanup interval is the time interval at which the terminal periodically performs the operation of cleaning up the decoding results in the cache. For example, it can be set to an hour, a day, or other time intervals according to application needs. The preset time threshold is the length of time that decoding results have not been used and need to be cleaned up. For example, if decoding results have not been accessed again within a certain period of time (such as 30 minutes, 1 hour, etc.), they may be considered no longer needed and can be cleaned up.

[0136] Specifically, at the start of each cache cleanup operation, the system's current time is first obtained, and all decoding results in the cache are traversed. For each decoding result, its last access timestamp is checked, the last access timestamp of the decoding result is compared with the current time, and the time difference is calculated. If the time difference is greater than a preset time threshold, the decoding result is considered to have been unused for a long time and should be cleaned up. All decoding results that have been unused for more than the time threshold are removed from the cache.

[0137] In this embodiment, the terminal can manage memory resources more effectively, reduce unnecessary memory usage, and improve overall performance and stability.

[0138] All of the above technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0139] This embodiment of the application, in response to a playback scheduling instruction triggered based on a preset period, traverses a list of dynamic images, which includes at least one dynamic image to be played. For the current dynamic image being traversed, if it is determined that there is an image frame to be decoded in the current dynamic image, a target decoding task corresponding to the image frame to be decoded is generated. The image frame to be decoded corresponding to the current dynamic image includes a preset number of frames in the current dynamic image that will be played. Asynchronous decoding is performed on the target decoding task in a sub-thread. To achieve unified scheduling and playback of dynamic images, this embodiment of the application maintains a list of dynamic images and triggers a pre-decoding process for the dynamic images in the list in a sub-thread every preset period. This ensures that the dynamic images are pre-decoded according to a predetermined time interval, and the decoding process is executed in a sub-thread, reducing playback delay and user interface stuttering, thereby improving the smoothness of dynamic image playback.

[0140] To facilitate better implementation of the dynamic image processing method of this application, this application also provides a dynamic image processing apparatus. Please refer to... Figure 6 , Figure 6 This is a schematic diagram of the structure of a dynamic image processing apparatus provided in an embodiment of this application. The dynamic image processing apparatus 600 may include:

[0141] The traversal unit 610 is used to traverse the dynamic image list in response to a playback scheduling instruction triggered based on a preset period. The dynamic image list includes at least one dynamic image to be played.

[0142] The generation unit 620 is used to generate a target decoding task corresponding to the image frame to be decoded if it is determined that there is an image frame to be decoded in the current dynamic image that has been traversed. The image frame to be decoded corresponding to the current dynamic image includes a preset number of image frames that will be played in the current dynamic image.

[0143] Decoding unit 630 is used to perform asynchronous decoding of the target decoding task in a sub-thread.

[0144] In some embodiments, the apparatus further includes a rendering unit for:

[0145] If it is determined that the image frame to be played in the current dynamic image meets the playback conditions, check whether there is decoded image data corresponding to the image frame to be played in the buffer;

[0146] If it is determined that the cache contains decoded image data corresponding to the image frame to be played, retrieve the decoded image data corresponding to the image frame to be played from the cache;

[0147] The image frame to be played is rendered and displayed based on the decoded image data corresponding to the image frame to be played.

[0148] In some embodiments, when the decoding unit 630 is used to asynchronously decode the target decoding task in a sub-thread, it is specifically used for:

[0149] The target decoding task is asynchronously decoded in a sub-thread according to its scheduling priority. The scheduling priority of the target decoding task is determined by whether the image to be decoded is the first frame of the current dynamic image. If the image to be decoded is the first frame of the current dynamic image, the scheduling priority of the target decoding task is high; if the image to be decoded is not the first frame of the current dynamic image, the scheduling priority of the target decoding task is low.

[0150] In some embodiments, when the decoding unit 630 performs asynchronous decoding of the target decoding task in a sub-thread according to the scheduling priority of the target decoding task, it is specifically used for:

[0151] Add the target decoding task and its scheduling priority to the decoding queue;

[0152] In the child thread, the target decoding task is retrieved from the decoding queue and executed according to the high scheduling priority priority policy.

[0153] In some embodiments, the apparatus further includes a cache unit for:

[0154] In response to the completion of the target decoding task, determine whether to cache the decoding result corresponding to the target decoding task based on the current frame number of the dynamic image.

[0155] In some embodiments, when the caching unit is used to determine whether to cache the decoding result corresponding to the target decoding task based on the frame number of the current dynamic image, it is specifically used for:

[0156] If the number of frames in the current dynamic image is not greater than the preset number of frames, the decoding result corresponding to the target decoding task will be cached for use when the image frame to be decoded is played again.

[0157] In some embodiments, when the caching unit is used to cache the decoding result corresponding to the target decoding task, it is specifically used for:

[0158] The decoding results corresponding to the target decoding task are cached according to the display size of the image frame to be decoded.

[0159] In some embodiments, when the caching unit is used to cache the decoding result corresponding to the target decoding task, it is specifically used for:

[0160] The decoding result corresponding to the target decoding task is cached in the target format, and the pixel size of the target format is smaller than the pixel size of the initial format of the decoding result.

[0161] In some embodiments, when the caching unit is used to cache the decoding result corresponding to the target decoding task, it is specifically used for:

[0162] The decoding results corresponding to the target decoding task are cached based on the Least Recently Used (LRU) strategy.

[0163] In some embodiments, the apparatus further includes a cache cleanup unit for:

[0164] Clean up unused decoding results in the cache that have exceeded a preset time threshold based on the preset cache cleanup interval.

[0165] In some embodiments, the number of frames in the preset number of frames is determined based on the decoding time and rendering time of the current dynamic image.

[0166] In some embodiments, the preset number of frames is two frames.

[0167] It should be noted that the functions of each module in the dynamic image processing device 600 in this application embodiment can be referred to the specific implementation of any embodiment in the above method embodiments, and will not be repeated here.

[0168] Each unit in the above-described device can be implemented entirely or partially through software, hardware, or a combination thereof. Each unit can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each unit.

[0169] For example, the dynamic image processing device 600 can be integrated into a terminal or server that has storage and a processor and thus computing power, or the dynamic image processing device 600 can be the terminal or server.

[0170] In some embodiments, this application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0171] Figure 7 A schematic structural diagram of the computer device provided in the embodiments of this application, such as Figure 7 As shown, the computer device 700 may include: a communication interface 701, a memory 702, a processor 703, and a communication bus 704. The communication interface 701, memory 702, and processor 703 communicate with each other via the communication bus 704. The communication interface 701 is used for data communication between the device 700 and external devices. The memory 702 can be used to store software programs and modules, and the processor 703 runs the software programs and modules stored in the memory 702, such as the software programs for the corresponding operations in the foregoing method embodiments.

[0172] In some embodiments, the processor 703 may invoke software programs and modules stored in the memory 702 to perform the following operations:

[0173] In response to a playback scheduling instruction triggered based on a preset period, the dynamic image list is traversed, which includes at least one dynamic image to be played. For the current dynamic image that is traversed, if it is determined that there is a frame to be decoded in the current dynamic image, a target decoding task corresponding to the frame to be decoded is generated. The frame to be decoded corresponding to the current dynamic image includes a preset number of frames in the current dynamic image that will be played. The target decoding task is asynchronously decoded in a sub-thread.

[0174] In some embodiments, the computer device 700 may be integrated into a terminal or server that has storage and a processor, thus possessing computing capabilities; or the computer device 700 may be the terminal or server. The terminal may be a smartphone, tablet, laptop, smart TV, smart speaker, wearable smart device, personal computer, or other similar device. The server may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0175] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in the methods described above in the embodiments of this application; for brevity, further details are omitted here.

[0176] This application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding processes in the methods described above in the embodiments of this application. For brevity, these details will not be elaborated further here.

[0177] This application also provides a computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding processes in the methods described above in the embodiments of this application. For brevity, these details will not be elaborated further here.

[0178] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0179] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0180] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0182] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0183] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0184] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] In addition, the functional units in the embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0186] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer or a server) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0187] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A dynamic image processing method, characterized in that, include: In response to a playback scheduling command triggered based on a preset period, the dynamic image list is traversed, the dynamic image list including at least one dynamic image to be played; For the current dynamic image that has been traversed, if it is determined that there is an image frame to be decoded in the current dynamic image, a target decoding task corresponding to the image frame to be decoded is generated, wherein the image frame to be decoded corresponding to the current dynamic image is an undecoded image in a preset number of frames that will be played in the current dynamic image. The target decoding task is asynchronously decoded in a child thread.

2. The method according to claim 1, characterized in that, The method further includes: If it is determined that the image frame to be played in the current dynamic image meets the playback conditions, it is determined whether there is decoded image data corresponding to the image frame to be played in the cache; If it is determined that the decoded image data corresponding to the image frame to be played exists in the cache, the decoded image data corresponding to the image frame to be played is retrieved from the cache; The image frame to be played is rendered and displayed based on the decoded image data corresponding to the image frame to be played.

3. The method according to claim 1, characterized in that, The asynchronous decoding of the target decoding task in a sub-thread includes: The target decoding task is asynchronously decoded in a sub-thread according to its scheduling priority. The scheduling priority of the target decoding task is determined based on whether the image to be decoded is the first frame of the current dynamic image. If the image to be decoded is the first frame of the current dynamic image, the scheduling priority of the target decoding task is high; if the image to be decoded is not the first frame of the current dynamic image, the scheduling priority of the target decoding task is low.

4. The method according to claim 3, characterized in that, Asynchronous decoding of the target decoding task is performed in a sub-thread according to the scheduling priority of the target decoding task, including: Add the target decoding task and its scheduling priority to the decoding queue; In the sub-thread, the target decoding task is retrieved from the decoding queue and executed according to the high scheduling priority priority policy.

5. The method according to claim 1, characterized in that, The method further includes: In response to the completion of the target decoding task, it is determined whether to cache the decoding result corresponding to the target decoding task based on the frame number of the current dynamic image.

6. The method according to claim 5, characterized in that, Determining whether to cache the decoding result corresponding to the target decoding task based on the frame number of the current dynamic image includes: If the number of frames of the current dynamic image is not greater than the preset number of frames, the decoding result corresponding to the target decoding task is cached for use when the image frame to be decoded is played again.

7. The method according to claim 6, characterized in that, Cache the decoding result corresponding to the target decoding task, including: The decoding results corresponding to the target decoding task are cached according to the display size of the image frame to be decoded.

8. The method according to claim 6, characterized in that, Cache the decoding result corresponding to the target decoding task, including: The decoding result corresponding to the target decoding task is cached in the target format, where the pixel occupancy of the target format is smaller than that of the initial format of the decoding result.

9. The method according to claim 6, characterized in that, Cache the decoding result corresponding to the target decoding task, including: The decoding results corresponding to the target decoding task are cached based on the least recently used strategy.

10. The method according to claim 5, characterized in that, The method further includes: Clean up unused decoding results in the cache that have exceeded a preset time threshold based on the preset cache cleanup interval.

11. The method according to any one of claims 1-10, characterized in that, The preset number of frames is determined based on the decoding speed and rendering speed of the current dynamic image.

12. The method according to any one of claims 1-10, characterized in that, The preset number of frames is two frames.

13. A dynamic image processing device, characterized in that, include: The traversal unit is used to traverse the dynamic image list in response to a playback scheduling instruction triggered based on a preset period, wherein the dynamic image list includes at least one dynamic image to be played. The generation unit is used to generate a target decoding task corresponding to the image frame to be decoded if it is determined that there is an image frame to be decoded in the current dynamic image that has been traversed. The image frame to be decoded corresponding to the current dynamic image is an undecoded image in a preset number of frames that will be played in the current dynamic image. A decoding unit is used to asynchronously decode the target decoding task in a sub-thread.

14. A computer device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the dynamic image processing method according to any one of claims 1-12 by executing the executable instructions.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the dynamic image processing method according to any one of claims 1-12.

16. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the dynamic image processing method according to any one of claims 1-12.

Images