Page rendering method and device, readable medium, electronic equipment and program product

Abstract

The invention discloses a page rendering method and device, a readable medium, electronic equipment and a program product. The method comprises the following steps: in response to a first frame synchronization signal triggered by a first page, determining a decoding strategy corresponding to a first image at least based on a display container and a display position of the first image under the condition that the first page comprises an undecoded first image; under the condition that the decoding strategy corresponding to the first image represents that the first image is decoded, the first image is decoded, first decoding content corresponding to the first image is obtained, and the first decoding content is used for rendering the first image in the first page. The problem that CPU threads are frequently switched due to the fact that decoding tasks are initiated for all images is avoided, performance loss and memory occupation are reduced, and then the interaction experience of a user is improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a page rendering method, apparatus, readable medium, electronic device, and program product. Background Technology

[0002] During the page rendering process, if the page contains images, it needs to go through the following steps: image download, CPU decoding, GPU texture upload, and GPU rendering onto the screen.

[0003] In related technologies, images are immediately decoded by the CPU after downloading. If there are a large number of images on the page, the large number of concentrated decoding tasks will trigger frequent switching of CPU threads, resulting in performance loss and memory consumption, and will also affect the user's interactive experience. Summary of the Invention

[0004] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] Firstly, this disclosure provides a page rendering method, the page rendering method comprising: In response to the first frame synchronization signal triggered by the first page, if the first page includes an undecoded first image, the decoding strategy corresponding to the first image is determined at least based on the display container and display position of the first image; When the decoding strategy corresponding to the first image is used to decode the first image, the first image is decoded to obtain the first decoded content corresponding to the first image. The first decoded content is used to render the first image on the first page.

[0006] Secondly, this disclosure provides a page rendering apparatus, the page rendering apparatus comprising: The determination module is configured to, in response to a first frame synchronization signal triggered by a first page, determine a decoding strategy corresponding to the first image based at least on the display container and display position of the first image when the first page includes an undecoded first image; The decoding module is used to decode the first image when the first image is decoded according to the decoding strategy representation corresponding to the first image, and to obtain the first decoded content corresponding to the first image. The first decoded content is used to render the first image in the first page.

[0007] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0008] Fourthly, this disclosure provides an electronic device, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method in the first aspect.

[0009] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0010] Through the above technical solution, in response to the first frame synchronization signal triggered by the first page, when the first page includes an undecoded first image, the decoding strategy corresponding to the first image is determined at least based on the display container and display position of the first image. If the decoding strategy indicates that the first image should be decoded, the first image is decoded to obtain the first decoded content. This first decoded content is used to render the first image on the first page. Using this method, during page rendering, the image is not immediately decoded after downloading. Instead, the decoding strategy corresponding to the image is first determined based on the image's display container and display position, and the images whose decoding strategy indicates decoding are processed. This avoids the frequent CPU thread switching caused by initiating decoding tasks for all images, reducing performance loss and memory usage, thereby improving the user's interactive experience.

[0011] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 This is a flowchart illustrating a page rendering method according to an exemplary embodiment.

[0013] Figure 2 This is a schematic diagram illustrating the process of confirming the decoding strategy according to an exemplary embodiment.

[0014] Figure 3 This is a schematic diagram illustrating the state machine scheduling process according to an exemplary embodiment.

[0015] Figure 4 This is a schematic diagram of thread interaction in a page rendering process according to another exemplary embodiment.

[0016] Figure 5 This is a schematic diagram of the structure of a page rendering apparatus according to an exemplary embodiment.

[0017] Figure 6 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0020] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0021] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0022] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0023] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0026] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0028] Meanwhile, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0029] During the page rendering process, the image content on the page typically needs to go through the following four stages in sequence: 1. Download image resources from the internet; 2. Decode the image on the CPU side to obtain bitmap data; 3. Upload the decoded bitmap data to the GPU and convert it into a texture; 4. The GPU thread composites the texture with other layers such as text, background, and border, and finally outputs it to the frame buffer.

[0030] Since steps 3 and 4 share the same GPU thread, texture uploading and subsequent compositing can only be executed serially, resulting in a longer GPU time slice. On the CPU side, decoding is usually triggered immediately after image download and the texture uploading task is submitted as soon as decoding is complete, causing the GPU thread to bear the dual load of "uploading + compositing" continuously on the page rendering path.

[0031] When a page contains a large number of images and the user scrolls rapidly, all images along the scroll path trigger decoding and texture uploading tasks simultaneously. This concentrated decoding process causes frequent CPU thread switching, resulting in performance degradation and increased memory usage. Furthermore, GPU texture uploading also blocks GPU threads, preventing the GPU from rendering other content (such as text, backgrounds, and borders that don't depend on decoding). This leads to noticeable stuttering during page scrolling, severely impacting the user experience.

[0032] In view of the above, this disclosure provides a page rendering method, apparatus, readable medium, electronic device, and program product to solve the aforementioned technical problems.

[0033] Figure 1 This is a flowchart illustrating a page rendering method according to an exemplary embodiment. For example... Figure 1 As shown, the method may include the following steps: S101: In response to the first frame synchronization signal triggered by the first page, if the first page includes an undecoded first image, determine the decoding strategy corresponding to the first image based at least on the display container and display position of the first image.

[0034] For example, the first frame synchronization signal can be a VSync (Vertical Synchronization) signal, which can be determined according to the actual business scenario. It can be a trigger signal that can be used to coordinate the rendering process, and this disclosure does not impose any restrictions on it.

[0035] S102: When the decoding strategy corresponding to the first image is used to decode the first image, the first image is decoded to obtain the first decoded content corresponding to the first image. The first decoded content is used to render the first image on the first page.

[0036] For example, the CPU thread can be invoked to decode the first image. The decoded content is the bitmap data of the first image, which is a set of binary data describing the pixel information of the image. Its essence is a "pixel-level" digital representation of the image. Subsequently, the GPU thread can be invoked to copy the bitmap data from the CPU memory to the GPU video memory, arrange it according to a specified format, and generate a texture object. During the rendering stage, the GPU performs vertex shading and rasterization on primitives containing texture coordinates to generate fragments. The fragment shader samples the texture and composites it with other layers. The resulting pixels are written to the background frame buffer. When a new vertical synchronization signal arrives, the foreground and background buffers are swapped, and the display control unit outputs the contents of the frame buffer to the screen in scan order.

[0037] In this embodiment, decoding tasks can be generated according to the decoding strategy corresponding to the image. If there are multiple images with the same strategy, it is equivalent to the decoding priority of multiple images being consistent. The execution order of the corresponding decoding tasks is determined according to the order in which the decoding strategy is determined. The specific order can be determined according to the actual business scenario, and this disclosure does not impose any restrictions on it.

[0038] In this embodiment, in response to the first frame synchronization signal triggered by the first page, a set of drawing commands can be generated based on the first content (excluding the first image) on the page, and the GPU thread can be invoked to execute the set of drawing commands to render the page content on the first page. The first content refers to the content to be rendered on the first page excluding the first image. The first content can be non-image content or a decoded image. That is, within one signal cycle, for the content to be rendered, if the image is not decoded, the CPU thread is invoked to decode it, but no drawing step is performed; for decoded images or non-image content that does not require decoding, corresponding drawing commands are generated, and the GPU thread is invoked to execute the drawing commands to render the first content onto the screen.

[0039] Using the above method, during the page rendering process, the image is not decoded immediately after downloading. Instead, the decoding strategy corresponding to the image is determined based on the image's display container and display position. Only images with the decoding strategy of decoding are decoded. This avoids the problem of frequent CPU thread switching caused by initiating decoding tasks for all images, reduces performance loss and memory usage, and thus improves the user's interactive experience.

[0040] In possible ways, the decoding strategy corresponding to the first image is determined at least based on the display container and display position of the first image, including: when the display container of the first image is a scrolling container, determining the decoding strategy corresponding to the first image based on the motion state of the first image and the positional relationship between the display position of the first image and the first visible area of ​​the scrolling container; when the display container of the first image is a non-scrolling container, determining the decoding strategy corresponding to the first image based on the positional relationship between the display position of the first image and the second visible area of ​​the non-scrolling container.

[0041] In this embodiment, as Figure 2 As shown, the page can display content through a scrollable container or a non-scrollable container. Therefore, when the UI (User Interface) thread records drawing commands, if an undecoded image is encountered, the decoding strategy for the image can be determined according to the type of different display containers to meet the display requirements of different business scenarios.

[0042] In this embodiment, we continue to refer to... Figure 2The decoding strategy can be preset to "0" to indicate that decoding is not triggered, "1" to indicate that decoding can be delayed, and "2" to indicate that decoding is triggered immediately. Decoding strategy "0" does not create a decoding task, while decoding strategies "1" and "2" create decoding tasks. However, the execution priority of the decoding task corresponding to decoding strategy "2" is higher than that of the decoding task corresponding to decoding strategy "1". The specific decoding strategy can be set according to the requirements, and this disclosure does not impose any restrictions on it.

[0043] In possible approaches, the decoding strategy includes a first decoding strategy representing immediate decoding of the image, a second decoding strategy representing delayed decoding of the image, and a third decoding strategy representing no decoding of the image. Based on the motion state of the first image and the positional relationship between the display position of the first image and the first visible area of ​​the scrolling container, the decoding strategy corresponding to the first image is determined, including: if the motion state of the first image is non-scrolling, and the display position of the first image is within the first visible area, the decoding strategy corresponding to the first image is determined to be the first decoding strategy; or, if the display position of the first image is outside the first visible area and the distance between the display position of the first image and the first visible area is less than a first preset distance, the decoding strategy corresponding to the first image is determined to be the second decoding strategy; or, if the display position of the first image is outside the first visible area and the distance between the display position of the first image and the first visible area is greater than the first preset distance, the decoding strategy corresponding to the first image is determined to be the third decoding strategy.

[0044] In this embodiment, it is assumed that the scrolling container contains a large number of images, but only a small portion of them are visible. In this case, all images will be downloaded after the Src attribute is updated, but this embodiment will not decode all images.

[0045] For example, such as Figure 2 As shown, if the image is not scrolling, the decoding strategy can be determined based on the image's visibility. Specifically, if the image's display position is within the viewport (first visible area) of the scrolling container, it is equivalent to the image being visible. In this case, the decoding strategy corresponding to the image is determined to be 2, that is, a decoding task for the image is created, and the image is decoded immediately.

[0046] If the image is displayed outside the viewport and meets the preset conditions, it is equivalent to the image being invisible but meeting the delayed decoding conditions. In this case, the decoding strategy corresponding to the image is determined to be 1, that is, a decoding task for the image is created and the decoding process for the image is delayed. For example, the image can be decoded after all other images with decoding strategy 2 have been decoded.

[0047] In addition, if the image is displayed outside the viewport and does not meet the preset conditions, it is equivalent to the image being invisible and not meeting the delayed decoding conditions. In this case, the decoding strategy corresponding to the image is determined to be 0, that is, no decoding task is created for the image and the image is not decoded.

[0048] Therefore, for images in a non-scrolling state, visible images can be decoded immediately to render the image within the user's current visible range as quickly as possible, improving the user's interactive experience. Images that are not currently visible but are close to the user's current visible range can be decoded with a delay. This allows for pre-rendering of image content the user might view next without affecting the rendering efficiency of images within the user's current visible range, further improving the user's interactive experience. Images that are not currently visible but are far from the user's current visible range can be left undecoded, reducing performance overhead and memory usage.

[0049] The preset condition is that the distance between the image display position and the viewport range is less than a preset distance. The preset distance is calculated as min(list.height, viewport.height) / 4, where list.height is the height of the scrolling container (or the width if scrolling horizontally), and viewport.height is the height of the current viewport range (or the width if scrolling horizontally). The specific values ​​can be adjusted according to requirements, and this disclosure does not impose any restrictions on them.

[0050] Taking scrolling up and down as an example, for an image above the viewport, the distance between the bottom edge of the image and the top edge of the viewport can be used as the distance between the image's display position and the viewport. For an image below the viewport, the distance between the top edge of the image and the bottom edge of the viewport can be used as the distance between the image's display position and the viewport. This allows for precise calculation of the distance between the image's display position and the viewport. Specific decoding strategies can be set according to requirements; this disclosure does not impose any limitations on this.

[0051] In one possible manner, decoding the first image to obtain the first decoded content corresponding to the first image includes: if the decoding strategy corresponding to the first image is a first decoding strategy, decoding the first image to obtain the first decoded content corresponding to the first image; if the decoding strategy corresponding to the first image is a second decoding strategy, after a first preset time period, if the motion state of the first image is a non-scrolling state and the distance between the display position of the first image and the first visible area is less than a first preset distance, decoding the first image to obtain the first decoded content corresponding to the first image.

[0052] In this embodiment, for delayed decoding images, the motion state and visibility state of the image can be determined again after a preset time. If the image is still in a non-scrolling state and the distance between the image's display position and the viewport range is less than the preset distance, the image can be decoded; otherwise, the strategy judgment is re-performed based on the current state.

[0053] For example, decoding tasks with different decoding strategies can be written into the same decoding task queue. Assuming the decoding task queue includes (task A, 2; task B, 1; task C, 2), task A is executed first. If the creation time of task B is less than a preset time, then task C is executed. If the creation time of task B is greater than or equal to the preset time, it is determined whether the image of task B is still scrolling and whether the distance between the image's display position and the viewport range is less than the preset distance. If so, task B is executed; otherwise, task B is skipped and task C is executed. Alternatively, if task B meets the above conditions after the preset time, the decoding strategy of task B is refreshed to 2, and then tasks are executed according to the queue order.

[0054] Of course, decoding tasks with different decoding strategies can be written into different decoding task queues. For example, decoding task queue 1 includes (task B, 1), and decoding task queue 2 includes (task A, 2; task C, 2). If task B meets the above conditions after a preset time, the decoding strategy of task B can be refreshed to 2 and written into decoding task queue 2. Then, the tasks are executed according to the queue order. If the conditions are not met, task B is deleted, and so on. This disclosure does not impose any restrictions on this.

[0055] In other possible implementations, if the decoding task queue does not contain an image with decoding strategy 2, then the image with decoding strategy 1 can be decoded directly. The specific implementation can be set according to the requirements, and this disclosure does not impose any restrictions on it.

[0056] Therefore, for images with delayed decoding, it is possible to further determine whether to decode them. For example, if the user quickly scrolls away from the current content, the secondary judgment will not require decoding the image, further reducing performance loss and memory usage.

[0057] It should be understood that when a page contains a large number of images and the user performs a rapid scrolling operation, all images on the scrolling path will trigger decoding and texture uploading tasks in one go; some of these images may quickly leave the visible area, and decoding and texture uploading these images is of little significance.

[0058] In one possible approach, determining the decoding strategy corresponding to the first image based on the motion state of the first image and the positional relationship between the display position of the first image and the first visible area of ​​the scrolling container includes: when the motion state of the first image is a scrolling state, determining the decoding strategy corresponding to the first image based on the scrolling state of the first image and the positional relationship between the display position of the first image and the first visible area of ​​the scrolling container; wherein the scrolling state includes a synchronous scrolling state in response to a dragging operation and an inertial scrolling state after the dragging operation ends.

[0059] For example, continue to refer to Figure 2 For images in a scrolling state, processing can be categorized based on the scrolling state type, thereby improving the flexibility and accuracy of decoding strategy judgment. Synchronous scrolling in response to drag operations can be a state where the page scrolls in response to user touch swipe operations, while inertial scrolling can be a state where the page scrolls inertially in response to user inertial swipe operations such as fling animations.

[0060] In one possible approach, the decoding strategy includes a first decoding strategy representing immediate decoding of the image and a third decoding strategy representing no decoding of the image. Based on the scrolling state of the first image and the positional relationship between the display position of the first image and the first visible area of ​​the scrolling container, the decoding strategy corresponding to the first image is determined, including: when the first image is in a synchronous scrolling state, if the display position of the first image is within the first visible area, the decoding strategy corresponding to the first image is determined to be the first decoding strategy; otherwise, the decoding strategy corresponding to the first image is determined to be the third decoding strategy; when the first image is in an inertial scrolling state, the first display position of the first image after scrolling stops is predicted; if the first display position is within the first visible area, the decoding strategy corresponding to the first image is determined to be the first decoding strategy; otherwise, the decoding strategy corresponding to the first image is determined to be the third decoding strategy.

[0061] For example, if the image is in synchronous scrolling mode, its visibility during scrolling can be determined. If the image is not visible within the viewport, the corresponding decoding strategy is set to 0; otherwise, it is set to 2. If the image is in inertial scrolling mode, its visibility after inertial scrolling ends can be determined. If the image is not visible within the viewport after inertial scrolling ends, the corresponding decoding strategy is set to 0; otherwise, it is set to 2. This is equivalent to pre-calculating the visible image after inertial scrolling ends and decoding it. If an image becomes invisible after inertial scrolling ends, no decoding is performed.

[0062] Therefore, for image scrolling scenarios, selective decoding can be performed based on the image decoding strategy to avoid decoding some images that will "quickly become invisible", thereby reducing the usage of CPU, GPU and memory and improving the smoothness of page scrolling.

[0063] In one possible approach, the decoding strategy includes a first decoding strategy that represents immediate decoding of the image and a third decoding strategy that represents no decoding of the image. The decoding strategy corresponding to the first image is determined based on the positional relationship between the display position of the first image and the second visible area of ​​the non-scrolling container. This includes: if the display position of the first image is within the second visible area, the decoding strategy corresponding to the first image is determined to be the first decoding strategy; otherwise, the decoding strategy corresponding to the first image is determined to be the third decoding strategy.

[0064] For example, continue to refer to Figure 2 For images displayed in non-scrolling containers, the decoding strategy is determined directly based on the image's visibility within the container's viewport. If the image is not visible within the viewport, the corresponding decoding strategy is set to 0; if it is visible, the corresponding decoding strategy is set to 2. This allows for selective image decoding of images in non-scrolling containers, reducing CPU performance overhead and memory usage.

[0065] In some possible ways, the page rendering method further includes: controlling the rendering thread of the graphics processor to execute the image upload task corresponding to the first image through the state machine corresponding to the graphics processor, wherein the image upload task corresponding to the first image is used to convert the first decoded content into the first texture content corresponding to the first image.

[0066] In this embodiment, the data after image decoding is not immediately uploaded to the GPU thread, but waits for state machine scheduling to avoid thread blocking caused by conflicts between the image upload task and the image drawing process.

[0067] like Figure 3As shown, the state machine provided in this embodiment needs to perform the following operations (Actions): idle, the UI thread starts recording the set of drawing commands, submits the set of drawing commands to the GPU thread after the drawing command recording is completed, the GPU thread replays the set of drawing commands and performs the actual rendering operation (drawing), and the GPU thread uploads the image texture. Specifically, it includes three types of states: BeginFrameState (frame start state) for controlling the rendering lifecycle of a frame; RasterState (rasterization state) for controlling the rasterization stage; and ImageUploadState (image upload state) for controlling the uploading of the image texture. Among them, BeginFrameState includes the IDLE state, which indicates that a new round of BeginFrame (the starting signal of a frame rendering lifecycle) can begin; the SENT state, which indicates that BeginFrame has been issued; and the COMMIT state, which indicates that the previously issued BeginFrame has been processed and the set of drawing commands can be submitted to the GPU thread. RasterState includes the IDLE state, which indicates that a new round of rasterization BeginFrame can begin; the DRAW state, which indicates that the drawing layer tree is being drawn; and the SWAP_BUFFER state, which indicates that the swap buffer is being exchanged. ImageUploadState includes the PENDING state, which indicates that the image texture has not yet been uploaded; and the READY state, which indicates that the image texture can now be uploaded.

[0068] In one possible approach, the rendering thread of the graphics processor is controlled to execute the image upload task corresponding to the first image through the state machine corresponding to the graphics processor. This includes: in response to the state machine being in a first state, the rendering thread is invoked to execute the image upload task corresponding to the first image, so as to upload the first decoded content to the video memory of the graphics processor to obtain the first texture content; wherein, the first state indicates that the rendering thread is allowed to execute the image upload task.

[0069] like Figure 3 As shown, when the UI thread starts recording drawing commands, the GPU thread will soon become busy, so ImageUploadState will change to PENDING (second state). After the GPU thread finishes drawing, ImageUploadState will change to READY (first state). Through state machine scheduling, image upload will occur while ImageUploadState is in the READY state. This avoids thread blocking caused by conflicts between image upload tasks and image drawing processes.

[0070] In one possible approach, the method further includes: in response to a second frame synchronization signal triggered by the first page, updating the state of the state machine to a second state, generating a first set of drawing commands that includes at least a first drawing command, wherein the first drawing command, when executed by the rendering thread, calls the first texture content to render a first image on the first page; wherein the second state indicates that the rendering thread is prohibited from performing image upload tasks; and in response to the rendering thread completing the first set of drawing commands, updating the state of the state machine to the first state.

[0071] For example, in response to a new second-frame synchronization signal, the UI thread begins recording drawing commands, which means the GPU thread will soon become busy. Therefore, `ImageUploadState` will change to `PENDING` (the second state). The GPU thread begins drawing, at which point it will draw the texture data obtained from the previous image upload task. After the GPU thread finishes drawing, `ImageUploadState` will change to `READY`, and the image texture upload task can resume. This allows the next frame's rendering lifecycle to continue. This avoids thread blocking caused by conflicts between the image upload task and the image drawing process.

[0072] It should be understood that the second frame synchronization signal is not strictly the next frame synchronization signal after the first frame synchronization signal, but a new frame synchronization signal after the first image is decoded. This is so that the first image can be drawn on the screen based on the image texture corresponding to the first image in the rendering process corresponding to the rendering life cycle of this frame.

[0073] It should be noted that in the relevant technologies, because the decoding is triggered immediately after the image is decoded and the image texture upload task is submitted as soon as the decoding is completed, the GPU cannot render other content (such as text, background, borders, etc. that do not depend on decoding) at this time, which causes obvious lag when scrolling the page and seriously affects the user interaction experience.

[0074] In this embodiment, when the UI thread starts recording drawing commands, imageUploadState will become PENDING. At this time, the GPU thread can first draw the content that has been decoded or does not need to be decoded onto the screen, and cannot perform image texture upload tasks. After the GPU thread finishes executing the current set of drawing commands, that is, the drawing ends, imageUploadState will become READY, and then the image texture upload task will be performed so that the newly decoded image can be drawn onto the screen in the new rendering lifecycle, avoiding obvious page lag and improving the user interaction experience.

[0075] In other words, related technologies require waiting for the undecoded image to be decoded and immediately uploaded as a texture before content can be drawn and displayed on the screen. However, this embodiment does not require decoding the entire image, and even after decoding, it is not immediately uploaded as a texture. Instead, it waits for state machine scheduling before drawing and displaying the content.

[0076] like Figure 4 As shown, the UI thread downloads the image, the GPU thread's state machine triggers the start UI frame (BeginFrame signal), requests VSync, and the UI thread responds to the VSync signal, starting to record drawing commands to determine the image to be drawn, then calculating the image decoding strategy, and submitting the image to be decoded to the decoding thread (e.g., the CPU thread). The UI thread generates a set of drawing commands based on the decoded or non-decoded content and submits it to the GPU thread. The GPU thread's state machine begins preparing for drawing; during this process, uploading image textures cannot be performed, but the decoded image can be cached by the decoding thread. During drawing, the GPU thread executes drawing commands by replaying the set of drawing commands. After drawing, the GPU thread swaps the front and back buffers (SwapBuffer), and the display controller then displays the image on the screen. After drawing, the GPU thread's state machine switches back to the READY state to prepare for uploading image textures, then uploads the decoded image to the GPU memory and generates the image texture, waiting for the next VSync cycle.

[0077] Using the above method, images are not decoded immediately after download. Instead, images are selectively decoded according to a corresponding decoding strategy, avoiding the decoding of images that will become "quickly invisible," thus reducing CPU, GPU, and memory usage and improving page scrolling smoothness. Furthermore, controlling the timing of image texture uploads using a state machine effectively avoids content rendering delays caused by GPU image texture uploads, mitigating stuttering in high-load multi-image scenarios and optimizing the user experience.

[0078] Figure 5 This is a schematic diagram illustrating the structure of a page rendering apparatus according to an exemplary embodiment. Figure 5 As shown, the page rendering device 500 includes: The determination module 501 is configured to, in response to a first frame synchronization signal triggered by a first page, determine a decoding strategy corresponding to the first image based at least on the display container and display position of the first image when the first page includes an undecoded first image; The decoding module 502 is used to decode the first image when the first image is decoded according to the decoding strategy representation corresponding to the first image, and to obtain the first decoded content corresponding to the first image. The first decoded content is used to render the first image in the first page.

[0079] Optionally, the determining module 501 is used to: When the display container of the first image is a scrolling container, the decoding strategy corresponding to the first image is determined based on the motion state of the first image and the positional relationship between the display position of the first image and the first visible area of ​​the scrolling container. When the display container of the first image is a non-scrolling container, the decoding strategy corresponding to the first image is determined based on the positional relationship between the display position of the first image and the second visible area of ​​the non-scrolling container.

[0080] Optionally, the decoding strategy includes a first decoding strategy representing immediate decoding of the image, a second decoding strategy representing delayed decoding of the image, and a third decoding strategy representing no decoding of the image. The determining module 501 is used to: When the motion state of the first image is non-scrolling, if the display position of the first image is within the first visible area, the decoding strategy corresponding to the first image is determined to be the first decoding strategy; or, If the display position of the first image is outside the first visible area and the distance between the display position of the first image and the first visible area is less than a first preset distance, then the decoding strategy corresponding to the first image is determined to be the second decoding strategy; or, If the display position of the first image is outside the first visible area and the distance between the display position of the first image and the first visible area is greater than the first preset distance, then the decoding strategy corresponding to the first image is determined to be the third decoding strategy.

[0081] Optionally, the determining module 501 is used to: When the decoding strategy corresponding to the first image is the first decoding strategy, the first image is decoded to obtain the first decoded content corresponding to the first image; When the decoding strategy corresponding to the first image is the second decoding strategy, after a first preset time period, if the motion state of the first image is the non-scrolling state and the distance between the display position of the first image and the first visible area is less than the first preset distance, the first image is decoded to obtain the first decoded content corresponding to the first image.

[0082] Optionally, the determining module 501 is used to: When the first image is in a scrolling state, the decoding strategy corresponding to the first image is determined based on the scrolling state of the first image and the positional relationship between the display position of the first image and the first visible area of ​​the scrolling container. The scrolling state includes a synchronous scrolling state in response to a dragging operation and an inertial scrolling state after the dragging operation ends.

[0083] Optionally, the decoding strategy includes a first decoding strategy that characterizes immediate decoding of the image and a third decoding strategy that characterizes no decoding of the image, and the determining module 501 is used to: When the first image is in the synchronous scrolling state, if the display position of the first image is within the first visible area, the decoding strategy corresponding to the first image is determined to be the first decoding strategy; otherwise, the decoding strategy corresponding to the first image is determined to be the third decoding strategy. When the first image is in the inertial scrolling state, predict the first display position of the first image after it stops scrolling. If the first display position is located within the first visible area, determine that the decoding strategy corresponding to the first image is the first decoding strategy; otherwise, determine that the decoding strategy corresponding to the first image is the third decoding strategy.

[0084] Optionally, the decoding strategy includes a first decoding strategy that characterizes immediate decoding of the image and a third decoding strategy that characterizes no decoding of the image, and the determining module 501 is used to: If the display position of the first image is within the second visible area, the decoding strategy corresponding to the first image is determined to be the first decoding strategy; otherwise, the decoding strategy corresponding to the first image is determined to be the third decoding strategy.

[0085] Optionally, the page rendering apparatus 500 further includes: The control module is used to control the rendering thread of the graphics processor to execute the image upload task corresponding to the first image through the state machine corresponding to the graphics processor. The image upload task corresponding to the first image is used to convert the first decoded content into the first texture content corresponding to the first image.

[0086] Optionally, the control module is used for: In response to the state machine being in the first state, the rendering thread is invoked to execute the image upload task corresponding to the first image, so as to upload the first decoded content to the video memory of the graphics processor to obtain the first texture content; wherein, the first state indicates that the rendering thread is allowed to execute the image upload task.

[0087] Optionally, the page rendering apparatus 500 further includes an update module, the update module being used for: In response to the second frame synchronization signal triggered by the first page, the state of the state machine is updated to the second state, and a first drawing command set including at least the first drawing command is generated. When the first drawing command is executed by the rendering thread, it calls the first texture content to render the first image on the first page; wherein, the second state indicates that the rendering thread is prohibited from executing the image upload task; In response to the rendering thread completing the first set of drawing commands, the state machine is updated to the first state.

[0088] The method logic executed by each functional module of the page rendering device in the above embodiments has been described in detail in the section on methods, and will not be repeated here.

[0089] Based on the same concept, embodiments of this disclosure also provide a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of any of the above-described page rendering methods.

[0090] Based on the same concept, this disclosure also provides an electronic device that may include: A storage device on which computer programs are stored; A processing device for executing a computer program stored in a storage device to implement the steps of any of the above-described page rendering methods.

[0091] Based on the same concept, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described page rendering methods.

[0092] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device 600 suitable for implementing embodiments of the present disclosure. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0093] like Figure 6As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0094] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0095] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0096] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0097] In some implementations, communication can be conducted using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can be interconnected with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0098] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0099] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: in response to a first frame synchronization signal triggered by a first page, determine a decoding strategy corresponding to the first image based at least on the display container and display position of the first image, if the first page includes an undecoded first image; and, if the decoding strategy corresponding to the first image indicates that the first image is to be decoded, perform decoding processing on the first image to obtain first decoded content corresponding to the first image, the first decoded content being used to render the first image on the first page.

[0100] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0101] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0102] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the functionality of the module itself.

[0103] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0104] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0105] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0106] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0107] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. A method of rendering a page, the method comprising: The page rendering method comprises: in response to a first frame synchronization signal triggered by a first page, in a case where the first page comprises a first image which is not decoded, determining a decoding strategy corresponding to the first image based on at least a display container and a display position of the first image; in a case where the decoding strategy corresponding to the first image represents decoding the first image, performing decoding processing on the first image to obtain first decoding content corresponding to the first image, the first decoding content being used to render the first image in the first page.

2. The page rendering method of claim 1, wherein, The determining of the decoding strategy corresponding to the first image based on at least the display container and the display position of the first image comprises: in a case where the display container of the first image is a scroll container, determining the decoding strategy corresponding to the first image based on a motion state of the first image and a positional relationship between the display position of the first image and a first visible region of the scroll container; in a case where the display container of the first image is a non-scroll container, determining the decoding strategy corresponding to the first image based on a positional relationship between the display position of the first image and a second visible region of the non-scroll container.

3. The page rendering method of claim 2, wherein, The decoding strategy comprises a first decoding strategy representing immediate decoding of an image, a second decoding strategy representing delayed decoding of an image, and a third decoding strategy representing non-decoding of an image, and the determining of the decoding strategy corresponding to the first image based on the motion state of the first image and the positional relationship between the display position of the first image and the first visible region of the scroll container comprises: in a case where the motion state of the first image is a non-scroll state, if the display position of the first image is located within the first visible region, determining the decoding strategy corresponding to the first image as the first decoding strategy; or if the display position of the first image is located outside the first visible region and a distance between the display position of the first image and the first visible region is less than a first preset distance, determining the decoding strategy corresponding to the first image as the second decoding strategy; or if the display position of the first image is located outside the first visible region and the distance between the display position of the first image and the first visible region is greater than the first preset distance, determining the decoding strategy corresponding to the first image as the third decoding strategy.

4. The page rendering method of claim 3, wherein, The performing of the decoding processing on the first image to obtain the first decoding content corresponding to the first image comprises: in a case where the decoding strategy corresponding to the first image is the first decoding strategy, performing the decoding processing on the first image to obtain the first decoding content corresponding to the first image; in a case where the decoding strategy corresponding to the first image is the second decoding strategy, after a first preset time length, if the motion state of the first image is the non-scroll state and the distance between the display position of the first image and the first visible region is less than the first preset distance, performing the decoding processing on the first image to obtain the first decoding content corresponding to the first image.

5. The page rendering method of claim 2, wherein, determining the decoding strategy corresponding to the first image based on the motion state of the first image and the positional relationship between the display position of the first image and the first visible region of the scroll container, comprising: in the case that the motion state of the first image is a scroll state, determining the decoding strategy corresponding to the first image based on the scroll state of the first image and the positional relationship between the display position of the first image and the first visible region of the scroll container; wherein the scroll state comprises a synchronous scroll state in response to a drag operation and an inertial scroll state after the end of the drag operation.

6. The page rendering method of claim 5, wherein, The decoding strategy comprises a first decoding strategy representing immediate decoding of an image and a third decoding strategy representing no decoding of an image, and the determining of the decoding strategy corresponding to the first image based on the scroll state of the first image and the positional relationship between the display position of the first image and the first visible region of the scroll container comprises: in the case that the first image is in the synchronous scroll state, if the display position of the first image is located within the first visible region, determining the decoding strategy corresponding to the first image as the first decoding strategy, otherwise determining the decoding strategy corresponding to the first image as the third decoding strategy; in the case that the first image is in the inertial scroll state, predicting a first display position of the first image after stopping scrolling, if the first display position is located within the first visible region, determining the decoding strategy corresponding to the first image as the first decoding strategy, otherwise determining the decoding strategy corresponding to the first image as the third decoding strategy.

7. The page rendering method of claim 2, wherein, The decoding strategy comprises a first decoding strategy representing immediate decoding of an image and a third decoding strategy representing no decoding of an image, and the determining of the decoding strategy corresponding to the first image based on the positional relationship between the display position of the first image and the second visible region of the non-scroll container comprises: if the display position of the first image is located within the second visible region, determining the decoding strategy corresponding to the first image as the first decoding strategy, otherwise determining the decoding strategy corresponding to the first image as the third decoding strategy.

8. The page rendering method according to any of claims 1-7, wherein, The page rendering method further comprises: controlling, by a state machine corresponding to the graphics processor, a rendering thread of the graphics processor to execute an image uploading task corresponding to the first image, the image uploading task corresponding to the first image being used to convert the first decoded content into first texture content corresponding to the first image.

9. The page rendering method of claim 8, wherein, The controlling, by a state machine corresponding to the graphics processor, a rendering thread of the graphics processor to execute an image uploading task corresponding to the first image comprises: in response to the state machine being in a first state, invoking the rendering thread to execute the image uploading task corresponding to the first image to upload the first decoded content to the video memory of the graphics processor to obtain the first texture content; wherein the first state represents that the rendering thread is allowed to execute an image uploading task.

10. The page rendering method of claim 9, wherein, The method further comprises: In response to a second frame synchronization signal triggered by the first page, a state of the state machine is updated to a second state, a first drawing command set including at least a first drawing command is generated, the first drawing command, when executed by the rendering thread, calls the first texture content to render the first image in the first page; wherein the second state represents that the rendering thread is prohibited to execute an image uploading task; In response to the rendering thread executing the first drawing command set, the state of the state machine is updated to the first state.

11. A page rendering apparatus characterized by comprising: The page rendering apparatus comprises: A determining module is configured to, in response to a first frame synchronization signal triggered by a first page, determine a decoding strategy corresponding to a first image based on at least a display container and a display position of the first image, in a case that the first image is not decoded in the first page; A decoding module is configured to, in a case that the decoding strategy corresponding to the first image represents that the first image is to be decoded, decode the first image to obtain first decoded content corresponding to the first image, the first decoded content being used to render the first image in the first page.

12. A computer readable medium having stored thereon a computer program, characterized in that, The computer program is executed by the processing apparatus to implement the steps of the method of any one of claims 1-10.

13. An electronic device, comprising: comprising: a storage device having stored thereon a computer program; a processing device configured to execute the computer program in the storage device to implement the steps of the method of any one of claims 1-10.

14. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processing device to implement the steps of the method of any one of claims 1-10. the computer program is executed by the processing device to implement the steps of the method of any one of claims 1-10.

Images