Display method and device and readable storage medium

Abstract

The invention relates to the technical field of image processing, and discloses a display method and device and a readable storage medium. The method is applied to the electronic equipment, a display screen of the electronic equipment comprises a first image and a second image, and the method comprises the steps that a first interface is displayed, and the first interface comprises the first image and the second image; in response to the sliding state of the first image and the second image, drawing a to-be-drawn layer of the first image and a to-be-drawn layer of the second image in a first drawing mode; and obtaining a to-be-displayed image frame based on the drawn to-be-drawn layer, and displaying the to-be-displayed image frame in the first interface. In this way, the display continuity of the image in the display screen in the sliding state can be improved, and image jamming in the display screen is avoided.

Description

[0001] This application is a divisional application of the invention patent application with application number 202310880429.3 submitted to the China Patent Office, application date July 18, 2023, and invention name “Display method, device and readable storage medium”. Technical Field

[0002] The present application relates to the field of image processing technology, and in particular to a display method, device and readable storage medium. Background Art

[0003] With the continuous development of terminal technology, the performance of electronic devices such as mobile phones and computers has been greatly improved, and users have higher and higher requirements for the performance of electronic devices. Among them, when the user slides up and down on the image displayed on the display screen of the electronic device, the display continuity of the image in the sliding state is one of the performance of the electronic device. Therefore, how to ensure the display continuity of the image in the sliding state, that is, to avoid the freeze of the image in the display screen, is a technical problem that needs to be solved. Summary of the invention

[0004] The embodiment of the present application provides a display method, device and readable storage medium. After the second image stops sliding, the method will also determine whether the first image is in a sliding state. If so, the layer to be drawn corresponding to the first image will continue to be drawn in advance, which can improve the coherence of the image display of the electronic device.

[0005] In a first aspect, the present application provides a display method, the method comprising: displaying a first interface, the first interface comprising a first image and a second image; detecting that the first image and the second image are both in a sliding state, and drawing the layer to be drawn of the first image and the layer to be drawn of the second image using a first drawing method; detecting that the first image is in a sliding state and the second image is in a sliding stop state, continuing to draw the layer to be drawn corresponding to the first image using the first drawing method, and stopping the drawing operation on the layer to be drawn of the second image; obtaining an image frame to be displayed based on the layer to be drawn that has been drawn, and displaying the image frame to be displayed in the first interface.

[0006] Exemplarily, the electronic device may be a folding screen device, in which case the first interface may include display interfaces of two different display screens of the folding screen device, such as the display screen 101 and the display screen 102 mentioned later. Then, the first image may be the display image 103 in the display screen 101, and the second image may be the display image 104 in the display screen 102, and the first image and the second image may be controlled separately. Therefore, in this case, the first image and the second image may be in a sliding state at the same time, or the second image may stop sliding first and the first image may stop sliding later.

[0007] For example, in addition to the folding screen device mentioned above, the electronic device may also be a single screen device, as long as the image displayed by the single screen device is monitored by two sliding modules. In this case, the schematic diagram of the first interface may be as follows: Figure 4 As shown, the first interface is Figure 4 In the article recommendation page 401 , the first image is an upper area 402 of the article recommendation page 401 , and the second image is a lower area 403 of the article recommendation page 401 .

[0008] It can be understood that the first drawing method used for the layer to be drawn is the advance drawing involved in the following description. The first drawing method includes: receiving a first synchronization message, drawing a first layer to be drawn, and when the first layer to be drawn is drawn and the second synchronization message is not received, starting to draw the second layer to be drawn.

[0009] It can be understood that the sending interval of the first synchronization message and the second synchronization message is determined based on the frame rate of the display screen of the electronic device. Exemplarily, if the frame rate of the display screen of the electronic device is 60 Hz, then the sending interval of the first synchronization message and the second synchronization message is 16.6 milliseconds. In addition, the first synchronization message and the second synchronization message can be the VSync_APP signal involved in the following text, and the first synchronization message is the VSync_APP signal corresponding to the current moment, and the second synchronization message is the VSync_APP signal corresponding to the next moment of the current moment.

[0010] In the display method provided in the present application, when the electronic device detects that the second image stops sliding but the first image is still in a sliding state, it will still perform an advance drawing operation on the to-be-drawn layer corresponding to the first image, which can improve the display continuity of the first image in the sliding state.

[0011] In a possible implementation of the first aspect above, the method further includes: detecting that both the first image and the second image are in a sliding stop state, and stopping the drawing operation on the to-be-drawn layers respectively corresponding to the first image and the second image based on the first drawing mode.

[0012] It can be understood that if the first image and the second image are both in a sliding stop state, that is, the first image and the second image are not sliding, there is no need to perform drawing operations on the to-be-drawn layers corresponding to the first image and the second image respectively based on the first drawing method.

[0013] In a possible implementation of the first aspect above, the electronic device includes a first sliding module and a second sliding module, and detecting that the first image and the second image are in a sliding state at the same time includes: if the state of the first sliding module is a sliding state, and the state of the second sliding module is a sliding state, determining that the first image and the second image are in a sliding state at the same time; if the state of the first sliding module is a sliding state, and the state of the second sliding module is a sliding stop state, determining that the first image is in a sliding state and the second image is in a sliding stop state.

[0014] It can be understood that if the electronic device has two display screens, and the two display screens can be touched and slid respectively, so that the first image and the second image respectively contained in the two display screens are in a scrolling display state, then usually in this case, one image corresponds to one sliding module. Exemplarily, the first image corresponds to the first sliding module, and the second image corresponds to the second sliding module. It can also be understood that the state of the first sliding module is used to reflect the sliding state of the first image, and the state of the second sliding module is used to reflect the sliding state of the second image.

[0015] Therefore, if it is detected that the states of the first sliding module and the second sliding module are both in the sliding state, it can be determined that the first image and the second image are both in the sliding state; similarly, if it is detected that the state of the first sliding module is in the sliding state, but the state of the second sliding module is in the sliding stop state, it can be determined that the first image is in the sliding state and the second image is in the sliding stop state.

[0016] In a possible implementation of the first aspect above, the method further includes: if the states of the first sliding module and the second sliding module are both sliding states, setting the identifiers of the first sliding module and the second sliding module to the first identifier; based on the identifiers of the first sliding module and the second sliding module being the first identifier, adding the first sliding module and the second sliding module to the module group to be analyzed, and using the first drawing method to draw the layers to be drawn of the images respectively corresponding to the sliding modules included in the module group to be analyzed; if the state of the first sliding module is the sliding state and the state of the second sliding module is the sliding stop state, setting the identifier of the second sliding module to the second identifier; based on the identifier of the second sliding module being the second identifier, deleting the second sliding module from the module group to be analyzed, continuing to use the first drawing method to draw the layer to be drawn corresponding to the first image, and stopping the drawing operation of the layer to be drawn of the second image based on the first drawing method.

[0017] It can be understood that the first identifier is used to indicate the sliding state of the sliding module, and exemplarily, the first identifier can be True; the second identifier is used to indicate the sliding end state of the sliding module, and exemplarily, the second identifier can be False. The first identifier is used to add the sliding module corresponding to the first identifier to the module group to be analyzed, so as to use the first drawing method to draw the layers to be drawn of the images corresponding to the sliding modules included in the module group to be analyzed. The second identifier is used to delete the sliding module corresponding to the second identifier from the module group to be analyzed, so that the drawing operation of the layers to be drawn of the images corresponding to the deleted sliding module is no longer performed based on the first drawing method.

[0018] In a possible implementation of the first aspect above, the method also includes: if the state of the first sliding module is a sliding stop state, setting the identifier of the first sliding module to a second identifier; based on the identifier of the first sliding module being the second identifier, deleting the first sliding module from the module group to be analyzed, and stopping the drawing operation of the layer to be drawn of the first image based on the first drawing method.

[0019] It can be understood that after the second sliding module is deleted from the module group to be analyzed, the module group to be analyzed only includes the first sliding module, and the state of the first sliding module is the sliding state. If it is detected that the state of the first sliding module is the sliding stop state, the identifier of the first sliding module will be set to the second identifier, and then the first sliding module will be deleted from the module group to be analyzed based on the second identifier, so as not to perform a drawing operation on the to-be-drawn layer of the first image based on the first drawing method.

[0020] In a possible implementation of the first aspect, the method further includes: if there is no sliding module in the module group to be analyzed, stopping the drawing operation on the layers to be drawn corresponding to the first image and the second image respectively based on the first drawing mode.

[0021] If there is no sliding module in the module group to be analyzed, it means that the first image and the second image are both in a sliding stop state, and at this time, there is no need to perform drawing operations on the layers to be drawn corresponding to the first image and the second image respectively based on the first drawing method.

[0022] In a possible implementation of the first aspect above, the electronic device also includes a choreographer and a dynamic buffer, and when it is detected that the first image and the second image are in a sliding state at the same time, a first drawing method is used to draw the layer to be drawn of the first image and the layer to be drawn of the second image, including: after the dynamic buffer detects that the first image is in a sliding state according to the state of the first sliding module, and detects that the second image is in a sliding state according to the state of the second sliding module, an instruction to perform the first drawing method is sent to the choreographer; the choreographer draws the layer to be drawn of the first image and the layer to be drawn of the second image using the first drawing method according to the instruction of the first drawing method.

[0023] It can be understood that this implementation method introduces the interaction process of each module included in the display method. Exemplarily, after the sliding module detects the state of the image corresponding to the sliding module, the state is sent to the dynamic buffer, and the dynamic buffer analyzes the state. If the state is a sliding (middle) state, the dynamic buffer will issue an instruction to the choreographer to perform the first drawing method. After receiving the instruction, the choreographer will perform a drawing operation on the to-be-drawn layer corresponding to the image based on the first drawing method.

[0024] In a possible implementation of the first aspect above, the electronic device includes a rendering thread, a display merging module and a hardware display module; an image frame to be displayed is obtained based on the layer to be drawn that has been drawn, and the image frame to be displayed is displayed in a first interface, including: sending the layer to be drawn that has been drawn using the first drawing method to the rendering thread for rendering operation to obtain the rendered layer to be merged; the rendering thread sends the rendered layer to be merged to the display merging module for merging operation to obtain the image frame to be displayed; the display merging module sends the image frame to be displayed to the hardware display module; the hardware display module displays the image frame to be displayed in the first interface through the display screen of the electronic device.

[0025] Based on the above content, it can be understood that the rendering thread is used to render the completed layers to be drawn; the display merging module is used to merge the rendered layers to obtain the image frames to be displayed; and the hardware display module is used to display the image frames to be displayed based on the first interface.

[0026] In a second aspect, the present application provides an electronic device comprising: one or more processors; one or more memories; one or more memories storing one or more programs, when the one or more programs are executed by one or more processors, the electronic device executes the first aspect and any possible display method of the first aspect.

[0027] In a third aspect, the present application provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, causes the computer to execute the first aspect and any possible display method of the first aspect.

[0028] In a fourth aspect, the present application also provides a computer program product, including: execution instructions, the execution instructions are stored in a readable storage medium, at least one processor of the electronic device can read the execution instructions from the readable storage medium, and at least one processor executes the execution instructions to enable the electronic device to implement the first aspect and any possible display method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1According to some embodiments of the present application, a schematic diagram of a scenario in which a user slides a displayed image through two display screens of a foldable screen mobile phone in a split-screen scenario is shown;

[0030] Figure 2 According to some embodiments of the present application, a first timing diagram of image display under the action of a vertical synchronization signal is shown;

[0031] Figure 3 According to some embodiments of the present application, a second timing diagram of image display under the action of a vertical synchronization signal is shown;

[0032] Figure 4 According to some embodiments of the present application, a schematic diagram of an article recommendation page in a news application A is shown;

[0033] Figure 5A According to some embodiments of the present application, a flow chart of a display method is shown;

[0034] Figure 5B According to some embodiments of the present application, a schematic structural diagram of a first electronic device is shown;

[0035] Figure 6 According to some embodiments of the present application, a schematic diagram of the interaction process of each module in a display method is shown;

[0036] Fig. 7A According to some embodiments of the present application, a schematic diagram of an effective time drawn in advance in the case of a dual sliding module is shown;

[0037] Figure 7B According to some embodiments of the present application, a schematic diagram of the timing of early drawing taking effect in the present application and related technologies is shown;

[0038] Figure 8 According to some embodiments of the present application, a schematic structural diagram of a second electronic device is shown. DETAILED DESCRIPTION

[0039] Illustrative embodiments of the present application include, but are not limited to, display methods, devices, and readable storage media.

[0040] The following first explains the special terms involved in the embodiments of the present application.

[0041] User Interface (UI) thread: It can also be called Main Thread in the field of image processing. UI thread usually uses CPU (Central Processing Unit) to process logic related to user interface. In the embodiment of the present application, UI thread can be used to perform drawing operations on the image layer to be drawn.

[0042] Render Thread: In the field of image processing, the rendering thread usually uses the GPU (Graphics Processing Unit) to render the layer drawn by the UI thread. In the processing of a frame of image, it is usually necessary to use the UI thread to perform the drawing operation first, and then use the rendering thread to perform the rendering operation, so that the rendered image can be displayed on the display screen of the electronic device.

[0043] Vertical Synchronization (VSync) signal: It can also be understood as a timing interrupt signal, which is a periodic signal, and the signal period of the signal is determined based on the frame rate of the display screen of the electronic device. Vertical synchronization signals (VSync signals) can generally be divided into three categories: vertical synchronization-application (VSync_APP) signal, vertical synchronization-display synthesis (VSync_Surface Flinger, also known as VSync_SF) signal and hardware-vertical synchronization (HW_VSync) signal.

[0044] Among them, the VSync_APP signal can be used to trigger the drawing operation of one or more layers, and perform rendering operations on the drawn layers. It can also be understood that the VSync_APP signal can be used to trigger the UI thread to draw one or more layers, and trigger the Render thread to render one or more layers drawn by the UI thread. The VSync_SF signal can be used to trigger the layer synthesis of one or more rendered layers to obtain an image frame operation. The HW_VSync signal is a hardware signal triggered by the liquid crystal display driver of the electronic device. The signal period of the signal can be determined based on the frame rate of the display screen of the electronic device. For example, the signal period can be the inverse of the frame rate of the display screen. For example, if the frame rate of the display screen of the electronic device is 60Hz, then the signal period of the HW_VSync signal is 16.6 milliseconds (ms), that is, the HW_VSync signal is sent every 16.6ms.

[0045] It can be understood that the VSync_APP signal, VSync_SF signal and HW_VSync signal are all periodic signals, that is, the VSync_APP signal, VSync_SF signal and HW_VSync signal send signals periodically according to their respective signal periods. And the VSync_APP signal and VSync_SF signal are generated based on the HW_VSync signal, that is, the HW_VSync signal is the signal source of the VSync_APP signal and the VSync_SF signal. It can also be understood that the VSync_APP signal and the VSync_SF signal are synchronized with the HW_VSync signal, that is, the signal periods of the three signals are the same. According to the previous text, if the signal period of the HW_VSync signal is 16.6ms, then the signal periods of the VSync_APP signal and the VSync_SF signal are also 16.6ms.

[0046] Frame Rate: It is the frequency (rate) at which a bitmap image in units of frames appears continuously on the display. It can also be called the frame frequency and is expressed in Hertz (Hz). Taking a mobile phone as an example, the frame rate of the mobile phone display can usually be 60Hz, 70Hz, etc. For example, if the frame rate of the mobile phone display is 60Hz, the image is refreshed 60 times per second, that is, the image on the mobile phone display is refreshed once every 16.6 milliseconds (ms).

[0047] The following first illustrates an application scenario of the display method provided in the present application.

[0048] At present, with the popularization of electronic devices, more and more users will browse news, entertainment, and popular science information through electronic devices. Taking the mobile phone as an example, there are not only single-screen mobile phones, but also folding screen mobile phones. The embodiment of this application does not limit the folding method of the folding screen mobile phone. For example, the folding screen mobile phone can fold the two parts of the display screen in the left and right direction, or in the up and down direction. Regardless of which of the above folding methods of the folding screen mobile phone is used, the user can unfold the two parts of the display screen of the folding screen mobile phone to browse the relevant information.

[0049] Taking a mobile phone as an example, Figure 1 The schematic diagram shows a scenario in which a user slides the displayed image through the two parts of the display screen of a foldable screen mobile phone in a split screen scenario. The split screen can be split screen display between the same application or between different applications. Figure 1 As shown, the foldable screen mobile phone 100 can be folded in the left and right directions. Figure 1The state in is the unfolded state, that is, the folding screen mobile phone 100 includes a display screen 101 and a display screen 102. In addition, the display screen 101 and the display screen 102 respectively include a display image 103 in the first application and a display image 104 in the second application. Exemplarily, the user can slide the corresponding area of ​​the display screen 101, and the display image 103 will be scrolled in the display screen 101; the user can slide the corresponding area of ​​the display screen 102, and the display image 104 will be scrolled in the display screen 102. In the embodiment of the present application, the scrolling display state of the display image 103 and the display image 104 needs to be realized by sliding the display screen 101 and the display screen 102 respectively.

[0050] Therefore, in some scenarios, regardless of whether the user touches or slides display screen 101 and display screen 102, display image 103 and display image 104 can be scrolled, and display image 103 and display image 104 can be in a scrolling display state at the same time, or can be in a scrolling display state one after another, which is not limited in this embodiment of the present application.

[0051] for Figure 1 For the folding screen mobile phone shown in the figure, the images in the two parts of the display screen of the mobile phone can be controlled separately, so at this time there are usually two sliding modules in the mobile phone, that is, one sliding module corresponds to the image in one part of the display screen.

[0052] by Figure 1 For example, Figure 1 The display image 103 and the display image 104 involved in the above may correspond to a sliding module respectively. For example, the display image 103 may correspond to a first sliding module, that is, the first sliding module may determine the sliding state of the display image 103; the display image 104 may correspond to a second sliding module, that is, the second sliding module may determine the sliding state of the display image 104.

[0053] Based on the above application scenarios, the principle of image display on the display screen of an electronic device is explained below.

[0054] Take the electronic device as a mobile phone, and the user performs a sliding operation on the display screen of the mobile phone as an example. When the user performs the sliding operation, the user interface (UI) framework of the mobile phone will obtain the user's sliding operation on the mobile phone. Among them, the UI framework contains a UI thread and a rendering (Render) thread. Then, the UI thread in the UI framework first draws the layer corresponding to the current time t1, and the Render thread then renders the drawn layer and sends the rendered layer to the display (Display) framework. The Display framework uses the internal Surface Flinger module to perform layer synthesis operations on the rendered layer to obtain an image frame, and sends the image frame to the liquid crystal display panel (LiquidCrystal Display, LCD) driver in the hardware display module. After the LCD driver receives the image frame, it can display the image frame through the LCD.

[0055] Exemplarily, the timing for the UI thread to draw a layer may be, in response to the UI framework receiving the VSync_APP signal at the current time t1, calling the UI thread to draw the layer corresponding to the current time t1, and then calling the Render thread to render the drawn layer; then, in response to the Display framework receiving the VSync_SF signal, starting to use the Surface Flinger module inside the Display framework to perform synthesis operations on the layer to obtain an image frame; in response to the hardware display module receiving the HW_VSync signal, displaying the image frame through the LCD inside the module.

[0056] It can be understood that the drawing and display process of the image frame at the next moment (i.e., moment t2) of the current moment t1 is the same as the above process. Therefore, it can be seen from the above content that the LCD periodically displays the acquired image frames based on the signal period of the HW_VSync signal, so that different image frames corresponding to different moments are displayed in the display screen of the electronic device in chronological order, thus presenting a sliding display state of the content on the display screen that can be visually observed by the user.

[0057] Figure 2 FIG. 2 shows a timing diagram of image display under the action of a vertical synchronization signal. Figure 2As shown, the VSync_APP signal arrives at t1, t2, t3 and t4 respectively; the VSync_SF signal arrives at t1, t2, t3 and t4 respectively; the HW_VSync signal arrives at t1, t2, t3 and t4 respectively. The time interval between t1 and t2 is the same as the time interval between t2 and t3, and the time interval between t3 and t4, and this time interval is also the signal period mentioned above. Taking the signal period of 16.6ms as an example, Figure 1 The time interval between adjacent moments in is also 16.6ms.

[0058] When the VSync_APP signal arrives at time t1, the first drawing and rendering operations are performed on the first layer corresponding to time t1, and when the VSync_SF signal arrives at time t2, the first synthesis operation can be performed on the rendered first layer to obtain the first image frame. When the HW_VSync signal arrives at time t3, the first image frame can be displayed, and the user can see the displayed first image frame on the display screen of the mobile phone.

[0059] Still like Figure 2 It can be seen that when the VSync_APP signal arrives at time t2, the second drawing and second rendering operations are started on the second layer corresponding to time t2, and when the VSync_SF signal arrives at time t3, the second layer after the second rendering can be synthesized to obtain the second image frame. When the HW_VSync signal arrives at time t4, the second image frame can be displayed, and the user can see the displayed second image frame on the display of the mobile phone. The processing methods of the layers corresponding to different times are the same as the principles described above, and will not be repeated here.

[0060] However, if Figure 3 As shown in the figure, if the second drawing operation is started on the second layer corresponding to time t2 when the VSync_APP signal arrives at time t2, but the second drawing operation on the second layer is not completed before time t3 arrives, the second rendering operation on the second layer will be completed between time t3 and time t4. Then, when the VSync_SF signal arrives at time t4, the second synthesis operation is started on the second layer after the second rendering operation to obtain the second image frame, and the second image frame can be displayed only when the HW_VSync signal arrives at time t5.

[0061] This will cause the first image frame of the first layer corresponding to time t1 to be displayed at time t3-t4, but the second image frame of the second layer corresponding to time t2 to be displayed at time t5-t6, that is, there is no image frame that can be displayed at time t4-t5. This situation is manifested as the image on the display screen being stuck or blank at time t4-t5, that is, frame loss. This situation will affect the smoothness of the image display on the display screen, and thus affect the user experience.

[0062] Therefore, in order to solve the above technical problems, the embodiment of the present application provides a display method, which includes, when the corresponding states of each sliding module in the electronic device are all sliding states, the layers to be drawn of the images corresponding to each sliding module are drawn in advance. When it is detected that the state of a sliding module in the electronic device has changed to the sliding end state, after stopping the advance drawing operation of the layers to be drawn of the image in the display screen corresponding to the sliding module, it will be determined again whether there are other sliding modules in the sliding state. If so, the layers to be drawn of the images in the display screen corresponding to the other sliding modules are still drawn in advance until the states of the other sliding modules change to the sliding stop state, and then the advance drawing operation is stopped. This method determines whether the layers to be drawn of the images corresponding to the sliding modules are drawn in advance according to the sliding states of each sliding module. That is, when one sliding module is in the sliding end state but another sliding module is still in the sliding state, the method can use the advance drawing operation for the layers to be drawn of the images corresponding to the sliding module that is still in the sliding state, thereby ensuring the smoothness of the sliding of the image and improving the user experience.

[0063] It can be understood that the early drawing operation means that there is no need to wait until the next signal cycle VSync_APP signal arrives to start drawing the image's to-be-drawn layer. After the drawing of the image's to-be-drawn layer corresponding to the current moment is completed, the drawing operation of the image's to-be-drawn layer corresponding to the next moment can be started. After the drawing is completed, the drawn layer is still rendered in the aforementioned manner, the layer is synthesized using the SF module, and sent to the LCD driver for display using the LCD.

[0064] It can be understood that the display method provided in the embodiment of the present application is applicable to any electronic device with communication function, including but not limited to mobile phones, tablet computers, wearable devices, augmented reality (AR) devices and other electronic devices. The embodiment of the present application does not limit the type and form of the electronic device.

[0065] It can be understood that the display method in the embodiment of the present application can be applied to any scenario with multiple sliding modules, such as the split-screen scenario of the folding mobile phone mentioned above, or the split-screen scenario of the single-screen mobile phone, or any electronic device with multiple sliding module settings. For example, for some applications, or some interfaces of some applications, when there is no split screen, that is, when a single screen is displayed, the sliding state is also monitored by two sliding modules. At this time, the above-mentioned display method can still be used when sliding these application interfaces.

[0066] For example, Figure 4 A schematic diagram of an article recommendation page 401 in a news application A is shown, wherein the sliding state of the page can also be monitored by two modules. For example, the sliding state of the upper area 402 of the article recommendation page 401 is monitored by the first sliding module, and the sliding state of the lower area 403 is monitored by the second sliding module. Therefore, the image frame can also be drawn and displayed based on the display method in the embodiment of the present application, and the smoothness of the image display can also be guaranteed to avoid the occurrence of stuttering problems. It can be understood that the above-mentioned monitoring method of the sliding state of the display interface by the sliding module is only an example, and the monitoring method of the sliding state of the display interface by the sliding module can be set according to actual needs, and this application does not limit it.

[0067] The following is a detailed description of the display method provided in the embodiment of the present application, which can be executed by an electronic device. Figure 5A As shown, the method may include the following steps:

[0068] 501: Display a first interface, where the first interface includes a first image and a second image.

[0069] In an embodiment of the present application, the first interface is an interface corresponding to the display screen of the electronic device. Exemplarily, the electronic device may be a folding screen device. In this case, the first interface may include display interfaces of two different display screens of the folding screen device, such as the display screen 101 and the display screen 102 mentioned above. Then, the first image may be the display image 103 in the display screen 101, and the second image may be the display image 104 in the display screen 102, and the first image and the second image may be controlled separately. Therefore, in this case, the first image and the second image may be in a sliding state at the same time, or the second image may stop sliding first, and the first image may stop sliding later, and so on.

[0070] For example, in addition to the folding screen device mentioned above, the electronic device may also be a single-screen device, as long as the image displayed by the single-screen device is monitored by two sliding modules. In this case, the schematic diagram of the first interface may be as shown above. Figure 4 As shown, the first interface is Figure 4In the article recommendation page 401 , the first image is an upper area 402 of the article recommendation page 401 , and the second image is a lower area 403 of the article recommendation page 401 .

[0071] In addition, in an embodiment of the present application, the status of the first image and the second image can be monitored by different sliding modules respectively. For example, the electronic device can include a first sliding module and a second sliding module, and the first sliding module can be used to determine the sliding status of the first image, and the second sliding module can be used to determine the sliding status of the second image.

[0072] 502: It is detected that the first image and the second image are in a sliding state at the same time, and the to-be-drawn layer of the first image and the to-be-drawn layer of the second image are drawn in a first drawing manner.

[0073] According to the previous step 501, the first image and the second image can be controlled separately, that is, the first image and the second image may be in a sliding state at the same time. In this case, the electronic device can use the first drawing method to draw the layers to be drawn corresponding to the first image and the second image respectively. Among them, the first drawing method includes: receiving a first synchronization message, drawing the first layer to be drawn, and starting to draw the second layer to be drawn after the first layer to be drawn is drawn and the second synchronization message is not received. It can also be understood that the first drawing method is advance drawing, that is, after the first layer to be drawn is drawn, there is no need to wait for the arrival of the second synchronization message, but the second layer to be drawn can be started immediately.

[0074] The first synchronization message and the second synchronization message may be vertical synchronization-application (VSync_APP) signals, and the sending interval of the first synchronization message and the second synchronization message is one signal cycle of the VSync_APP signal. The signal cycle is determined based on the frame rate of the display screen of the electronic device. For example, if the frame rate of the display screen of the electronic device is 60 Hz, then the sending interval of the first synchronization message and the second synchronization message is 16.6 milliseconds.

[0075] Exemplarily, if the states of the first sliding module and the second sliding module are both in a sliding state, it can be determined that the first image and the second image are in a sliding state at the same time, and the first drawing method (advance drawing method) is used to draw the to-be-drawn layer of the first image and the to-be-drawn layer of the second image.

[0076] 503: It is detected that the first image is in a sliding state and the second image is in a sliding stop state, and the layer to be drawn corresponding to the first image continues to be drawn in the first drawing mode, and the drawing operation on the layer to be drawn of the second image is stopped.

[0077] If the state of the first sliding module is the sliding state and the state of the second sliding module is the sliding end state, it can be determined that the first image is in the sliding state and the second image is in the sliding stop state (sliding end state), which means that it is necessary to continue to perform the drawing operation on the layer to be drawn corresponding to the first image, and there is no need to perform the drawing operation on the layer to be drawn corresponding to the second image.

[0078] 504: Obtaining an image frame to be displayed based on the completed layer to be drawn, and displaying the image frame to be displayed in the first interface.

[0079] In an embodiment of the present application, the completed layer to be drawn can be sent to the rendering thread for rendering operation to obtain the layer to be merged after rendering; then the layer to be merged is merged to obtain the image frame to be displayed, and then the image frame to be displayed is displayed in the first interface.

[0080] In the display method provided in the present application, when the electronic device detects that the state of the second sliding module is the sliding stop state, but the state of the first sliding module is still the sliding state, it will still perform an advance drawing operation on the to-be-drawn layer of the first image corresponding to the first sliding module. This method can improve the display continuity of the first image in the sliding state and avoid the jamming phenomenon of the first image.

[0081] Figure 5B FIG. 1 is a schematic diagram showing the structure of an electronic device to which the method provided in an embodiment of the present application is applied. Figure 5B As shown, the display method provided in the embodiment of the present application is mainly completed on the application (APP) side and the display synthesis (Surface Flinger, SF) side. Among them, the APP side mainly includes a choreographer (Choreographer), a dynamic buffer (DynBuf), a sliding (Over Scroller) module and a buffer queue (Buffer Queue); the SF side includes an SF module. Buffer Queue is used for image transfer between the SF modules on the App side and the SF side. The buffer queue has a buffer queue core (Buffer QueueCore), which defines some basic properties of the Buffer Queue.

[0082] Among them, the main purpose of the Choreographer is to receive the vertical synchronization (Vertical Synchronization, VSync) signal sent by the display synthesis (Surface Flinger, SF) module, and uniformly manage the execution timing of tasks such as input (Input), animation (Animation) and layout (Layout) of the application (App). It can also be understood that Choreographer controls the drawing timing of each frame layer based on the received VSync signal. In addition, it can be understood that Choreographer is configured in the UI thread mentioned above.

[0083] Dynamic Buffer (DynBuf) can calculate the buffer pressure based on some parameters of the current buffer, such as elimination rate, idleness, etc. If the threshold set by the system is reached, the buffer will be expanded immediately to control the buffer pressure within an acceptable range to meet application requirements. In this application, DynBuf can also determine whether the layer can be drawn in advance based on the current state of the sliding module, maintain the timestamp, and calculate the sliding curve.

[0084] Over Scroller module: It can determine whether the image on the display screen is in a sliding state at the current moment, and send the obtained image sliding state to DynBuf to help DynBuf determine whether it needs to be drawn in advance at the current moment. In addition, the sliding module can also calculate the position of each component on the display screen at the current moment based on the timestamp and sliding curve returned by DynBuf, so as to facilitate the subsequent drawing, rendering and other operations of each component based on this position information.

[0085] Display synthesis (Surface Flinger, SF) module: In the embodiment of the present application, this module is used to send a VSync signal to Choreographer, so that Choreographer starts to perform operations such as drawing the current frame image based on the received VSync signal. In addition, this module can also receive data buffers that have been drawn and rendered, synthesize these data buffers, and then send them to the hardware display module for display.

[0086] Figure 5BIn the process, the SF module sends an onVSync signal to Choreographer, which may be a VSync_APP signal, and may be used to instruct Choreographer to start drawing the layer to be drawn corresponding to the current moment. After receiving the onVSync signal sent by the SF module, Choreographer starts to process the drawing operation of the layer to be drawn corresponding to the current moment. The drawing operation may include processing animation events, traversal events, and layout events of the layer to be drawn. It can also be understood that after Choreographer processes the above animation events in sequence, it can complete the drawing operation of the layer to be drawn.

[0087] For example, Choreographer processes the animation event of the layer to be drawn, which may be that Choreographer controls the sliding module to update the sliding (fling) state of the image in the display screen at the current moment, that is, Choreographer controls the sliding module to update the sliding state of the sliding module. For example, the sliding state includes but is not limited to the sliding start state, the sliding end state and the sliding state.

[0088] Then, the sliding module sends the updated sliding state to DynBuf. DynBuf determines whether to perform an advance drawing operation based on the received sliding state. If so, DynBuf sends an advance drawing instruction to Choreographer so that after Choreographer completes the drawing operation of the to-be-drawn layer of the image corresponding to the current moment, it can directly start drawing the to-be-drawn layer of the image corresponding to the next moment without waiting for the VSync_APP signal of the next signal cycle to start drawing.

[0089] In addition to sending pre-drawing instructions to Choreographer, DynBuf also returns a timestamp and a sliding position curve to the sliding module. The function of the returned timestamp is that the sliding module determines the position of each component contained in the image on the display screen at the current moment according to the timestamp and the sliding position curve. The embodiment of the present application does not limit the form of the timestamp. For example, the timestamp can represent the current moment in the form of time, that is, the unit of the timestamp is milliseconds, etc. In addition, the timestamp can also represent the current moment in the number of signal cycles. For example, if the timestamp is 2 signal cycles and one signal cycle is 16.6ms, it means that the current moment is the moment when the second signal cycle arrives, that is, 33.2ms.

[0090] Exemplarily, the sliding position curve can be determined by DynBuf calculating the sliding curve by integrating the timestamp before the current moment and the sliding position information of each component corresponding to different timestamps. It can also be understood that the sliding curve can reflect the sliding position of each component at different timestamps. The sliding module searches for the curve based on the timestamp sent by DynBuf and determines the position information corresponding to the timestamp.

[0091] After the sliding module determines the position information of each component at the current moment, it sends the position information to Choreographer. Choreographer executes other callback functions based on the position of each component to handle traversal events, layout events, etc., and completes the drawing operation of the to-be-drawn layer of the image corresponding to the current moment.

[0092] After the drawing of the layer is completed, the rendering thread needs to be used to render the completed layer. After the rendering operation is completed, the rendered layer is stored in the buffer, and then the buffer is placed in the buffer queue. Subsequently, the SF module consumes the buffer, that is, the SF module merges the rendered layers to obtain image frames, and then sends the image frames to the liquid crystal display panel (LCD) for display.

[0093] It should be noted that if DynBuf sends an advance drawing instruction to Choreographer, after Choreographer completes the drawing operation of the to-be-drawn layer of the image corresponding to the current moment and sends the drawn layer to the rendering thread for rendering, it can immediately start drawing the to-be-drawn layer of the image corresponding to the next moment, and then send the drawn layer of the to-be-drawn image corresponding to the next moment to the rendering thread for rendering, synthesize the layer to obtain the image frame, and send it to the LCD driver for display using the LCD. It can be understood that the processing operation of the image corresponding to the next moment is similar to the processing operation principle of the image corresponding to the current moment, and will not be repeated here.

[0094] For ease of description, the present application takes the electronic device as a mobile phone as an example for subsequent introduction. The following is a detailed introduction to the display method involved in the embodiment of the present application based on some structures in the above electronic device.

[0095] Taking the electronic device including a first sliding module and a second sliding module as an example, Figure 6 The schematic diagram of the interaction process of each module in the display method provided in the embodiment of the present application is shown. Figure 6 As shown, the method comprises the following steps:

[0096] 601: The SF module sends a VSync_APP signal to the Choreographer.

[0097] It can be understood that the SF module determines the signal period of the VSync_APP signal according to the frame rate of the display screen of the electronic device, and then periodically sends the VSync_APP signal to the Choreographer according to the signal period. For example, according to the above content, if the frame rate of the display screen of the electronic device is 60Hz, then the signal period is 16.6ms. That is, the SF module sends the VSync_APP signal to the Choreographer every 16.6ms.

[0098] 602: Choreographer processes the animation event of the to-be-drawn layer of the target image corresponding to the current moment, obtains the position information of each component in the to-be-drawn layer of the target image, and sends the state of the target sliding module obtained in the process of processing the animation event to DynBuf.

[0099] Exemplarily, before the Choreographer receives the VSync_APP signal, it can be in a sleep state. After receiving the VSync_APP signal sent by the SF module, the Choreographer can start processing the animation events of the target image's to-be-drawn layer. In an exemplary embodiment, the Choreographer processing the animation events of the target image's to-be-drawn layer can also be understood as the Choreographer processing the animation events by executing the animation callback function. Specifically, the Choreographer can calculate the positions of the components included in the target image's to-be-drawn layer in the display screen of the electronic device at the current moment by executing the animation callback function. Among them, one component corresponds to a part of the layer to be drawn. For example, a picture in the layer to be drawn corresponds to a component, a text corresponds to a component, and so on. Based on the position of each component at the current moment, the Choreographer can perform drawing operations on the display screen according to the position of each component.

[0100] It can be understood that in the embodiment of the present application, the electronic device includes a first sliding module and a second sliding module, the first sliding module can correspond to the first image in the display screen, and the second sliding module can correspond to the second image in the display screen. Therefore, after the Choreographer receives the VSync_APP signal sent by the SF module, if the first image and the second image are both in a sliding state, the Choreographer needs to process animation events for the to-be-drawn layer of the first image and the to-be-drawn layer of the second image, respectively. Similarly, if the first image is in a sliding state, but the second image is not in a sliding state, the Choreographer only needs to process animation events for the to-be-drawn layer of the first image. It can be seen from this that the target image can be the first image, the second image, or the first image and the second image.

[0101] Specifically, Choreographer may process the animation event of the to-be-drawn layer of the target image corresponding to the current moment by the following steps (not shown in the figure):

[0102] 6021: Choreographer controls the target sliding module to update the sliding state of the target image.

[0103] It can be understood that the sliding state of the target image in the display screen at the current moment includes but is not limited to the sliding start state, the sliding end state and the sliding state. In addition, according to the above content, if the target image is the first image, the target sliding module is the first sliding module; if the target image is the second image, the target sliding module is the second sliding module; if the target image is the first image and the second image, the target sliding modules are the first sliding module and the second sliding module.

[0104] 6022: Choreographer controls the target sliding module to send the updated sliding state to DynBuf.

[0105] After receiving the updated sliding state, DynBuf can determine whether early drawing is required according to the sliding state. If the state of the sliding module is the sliding start state or the sliding state, DynBuf will send an early drawing instruction to Choreographer, indicating that early drawing can be performed at the current moment; if the state of the sliding module is the sliding end state, DynBuf will stop sending early drawing instructions to Choreographer, indicating that early drawing is no longer required.

[0106] 6023: DynBuf sends the updated timestamp and calculated sliding position curve to the target sliding module.

[0107] Among them, the updated timestamp is used to characterize the current moment. It can also be understood that when Choreographer receives the VSync_APP signal, it does not know the specific time of the current moment. Because it is necessary to determine the position information of each component in the display screen at the current moment based on the current moment in the future to complete the drawing of the target image at the current moment, the current moment (that is, the updated timestamp) is sent by DynBuf to the sliding module. In addition, DynBuf can also calculate the sliding position curve of each component in the display screen. Exemplarily, DynBuf can calculate the sliding position curve based on the integration of the timestamp before the current moment and the sliding position information of each component corresponding to different timestamps. It can also be understood that the sliding curve can reflect the sliding position information of each component at different timestamps.

[0108] 6024: The target sliding module calculates the position of each component in the display screen at the current moment according to the timestamp and the sliding position curve.

[0109] After the target sliding module obtains the current timestamp and sliding position curve, it can find the sliding position corresponding to the timestamp in the sliding position curve based on the timestamp, and then determine the position of each component in the to-be-drawn layer of the target image on the display screen at the current moment.

[0110] 603: Choreographer executes traversal events, layout events, etc. based on the positions of each component, and completes the drawing operation of the to-be-drawn layer of the target image corresponding to the current moment.

[0111] After Choreographer obtains the position of each component at the current moment through the target sliding module, it can perform native operations, such as traversal events and layout events. Among them, a traversal event refers to visiting each node once and only once in sequence along a certain search route. A layout event is an event that redraws the layer to be drawn in the display screen according to the position of each component at the current moment. In an embodiment of the present application, the traversal time and layout events work together to draw the layer to be drawn of the target image corresponding to the current moment by traversing the position information corresponding to each component.

[0112] It can be understood that after Choreographer processes the layout event at the current moment, it completes the drawing operation of the to-be-drawn layer of the target image corresponding to the current moment.

[0113] 604: Choreographer sends the to-be-drawn layer of the completed target image to the Render thread.

[0114] The process in which Choreographer sends the to-be-drawn layer of the target image that has been drawn to the Render thread can also be understood as the to-be-drawn layer of the image that Choreographer and the Render thread have drawn synchronously.

[0115] 605: The Render thread performs a rendering operation on the to-be-drawn layer of the target image that has been drawn, stores the to-be-drawn layer of the rendered target image in a buffer, and puts the Buffer into a buffer queue.

[0116] It can be understood that in the process of drawing and rendering the target image's to-be-drawn layer, it is usually necessary to cache the drawn or rendered image's to-be-drawn layer in the corresponding buffer. In the normal drawing process, the DynBuf module usually provides four buffers (Buffer) for drawing and rendering operations of the to-be-drawn layers of different images corresponding to different times.

[0117] 606: The SF module obtains and consumes the buffer in the buffer queue, and performs a merging operation on the to-be-drawn layers of the rendered target image to obtain a target image frame.

[0118] Because Buffer Queue is used for image transfer between the App side and the SF module on the SF side, the SF module can obtain the buffer containing the to-be-drawn layer of the target image in the App side buffer queue, and then merge the to-be-drawn layer of the target image in the buffer to obtain the target image frame.

[0119] 607: The SF module sends the target image frame to the LCD driver.

[0120] Among them, LCD driver is a chip device for adjusting the phase, peak value, frequency, etc. of the potential signal applied to the liquid crystal display electrode, establishing a driving electric field, and realizing the display effect of the liquid crystal display. It can be understood that LCD driver can control the liquid crystal display panel to display image frames.

[0121] 608: LCD driving displays the target image frame through a liquid crystal display panel (LCD).

[0122] 609: DynBuf analyzes whether the state of the target sliding module sent by Choreographer is a sliding state. If so, DynBuf sends an advance drawing instruction to Choreographer.

[0123] In the embodiments of the present application, the issuance of the pre-drawn instructions is determined by DynBuf based on the sliding state detected by the target sliding module. As can be seen from the foregoing content, the target sliding can be the first sliding module, the second sliding module, or both the first sliding module and the second sliding module. Therefore, if the target sliding module is the first sliding module, in this step, DynBuf determines whether to issue the pre-drawn instructions based on the state of the first sliding module. If the target sliding module is the second sliding module, in this step, DynBuf determines whether to issue the pre-drawn instructions based on the state of the second sliding module. If the target sliding module is both the first sliding module and the second sliding module, then when at least one of the two sliding modules is in the sliding state, DynBuf will issue the pre-drawn instructions.

[0124] Exemplarily, the target sliding module can determine the sliding state of the target image in the display screen at the current moment and then send this state to DynBuf. Among them, the sliding state detected by the target sliding module can also be simply referred to as the state of the target sliding module. The state of the target sliding module includes, but is not limited to, the sliding start state, the sliding state, and the sliding end state.

[0125] Exemplarily, if the sliding state of the target sliding module is the sliding state, DynBuf can send a pre-drawn instruction to Choreographer, indicating that pre-drawing can be performed at the current moment; if the sliding state of the target sliding module is the sliding end state, DynBuf will not send a pre-drawn instruction to Choreographer, that is, it is not necessary for Choreographer to perform the pre-drawing operation. Exemplarily, the way for DynBuf to obtain the state of the sliding module can be that after Choreographer controls the target sliding module to obtain the sliding state of the target image corresponding to the current moment, the sliding state is sent to the DynBuf module.

[0126] As can be seen from the above content, when the state of the target image corresponding to the current moment is the sliding state, DynBuf sends a pre-drawn instruction to Choreographer. This instruction is used to indicate that after the drawing operation of the layer to be drawn of the target image corresponding to the current moment is completed, the drawing operation of the layer to be drawn of the image corresponding to the next moment can be immediately started. Exemplarily, if the drawing operation of the layer to be drawn of the target image corresponding to the t1 moment is completed at the tm moment, and at this time the VSync_APP signal has not arrived at the t2 moment, that is, tm < t2, at this time, the drawing of the layer to be drawn of the image corresponding to the tm moment can be directly started without waiting for the VSync_APP signal at the t2 moment to arrive before performing the drawing operation.

[0127] In addition, because the electronic device in the embodiment of the present application includes a first sliding module and a second sliding module, the embodiment of the present application adds a method for separately judging the states of the two sliding modules, and the judgment logic corresponding to this method may be as follows.

[0128] First, if the state of the sliding module is in the sliding state, then the state of the sliding module mIsFling will be set to True. Then the sliding module in the sliding state is added to the module group to be analyzed mOverScrollers. If the states of the first sliding module and the second sliding module are both in the sliding state, then the first sliding module and the second sliding module exist in the module group to be analyzed mOverScrollers at the same time. Exemplarily, the corresponding logical representation can be mOverScrollers = {overscroller_1, overscroller_2, ...}, where overscroller_1 represents the first sliding module and overscroller_2 represents the second sliding module.

[0129] Taking the example that the first sliding module and the second sliding module exist in the module group to be analyzed mOverScrollers, DynBuf sends an advance drawing instruction to Choreographer, for instructing Choreographer to perform advance drawing operations on the to-be-drawn layers of the first image and the to-be-drawn layers of the second image in the display screens respectively corresponding to the first sliding module and the second sliding module.

[0130] If the state of the first sliding module changes to the sliding end state, then the state of the sliding module mIsFling will be set to False, and the first sliding module will be deleted from the module group to be analyzed, that is, the pre-drawing operation will no longer be performed on the layer to be drawn of the first image corresponding to the first sliding module. After deleting the first sliding module, it is determined whether there are other sliding modules in the module group to be analyzed. If there are no other sliding modules in the module group to be analyzed, the determination operation is terminated, and DynBuf does not send a pre-drawing instruction to Choreographer; if there are other sliding modules in the module group to be analyzed, it is determined whether the state of the other sliding module (the second sliding module) is in the sliding state. If so, DynBuf continues to send a pre-drawing instruction to Choreographer, instructing Choreographer to continue to pre-draw the layer to be drawn of the second image in the display screen corresponding to the second sliding module. If not, DynBuf does not send a pre-drawing instruction to Choreographer, indicating that it is no longer necessary to pre-draw the layer to be drawn of the second image corresponding to the second sliding module, and then delete the second sliding module from the sliding module group to be analyzed.

[0131] 610: Choreographer performs a drawing operation on the to-be-drawn layer of the third image corresponding to the next moment after the current moment.

[0132] 611: Choreographer sends the to-be-drawn layer of the third image that has been drawn to the Render thread.

[0133] 612: The Render thread performs a rendering operation on the to-be-drawn layer of the third image that has been drawn, stores the rendered to-be-drawn layer of the third image in a buffer, and puts the Buffer into a buffer queue.

[0134] However, if advance drawing is required, the time interval for the four buffers to support the completion of drawing becomes shorter, resulting in insufficient number of buffers to support the advance drawing operation of the image's layers to be drawn, which will still cause image display jamming on the display. Therefore, in this case, the number of buffers in the DynBuf module can be increased to improve the smoothness of image display.

[0135] 613: The SF module obtains and consumes the buffer in the buffer queue, and performs a merging operation on the to-be-drawn layers of the rendered third image to obtain a third image frame.

[0136] 614: The SF module sends the third image frame to the LCD driver.

[0137] 615: LCD driving displays the third image frame through a liquid crystal display panel (LCD).

[0138] Among them, the above steps 610 to 615 are similar in content and principle to the above steps 603 to 608, and will not be repeated here.

[0139] The principles of the display methods in some embodiments are described below.

[0140] Some embodiments use advance drawing operations in the process of displaying images. Figure 1 In the scenario shown, as long as the sliding end state of any one of the two sliding modules is detected, the early drawing operation of the to-be-drawn layers of the image on the display screen corresponding to all sliding modules will be ended. If there are other sliding modules in the sliding state at this time, then the to-be-drawn layers of the image corresponding to the sliding modules in the sliding state cannot be drawn in advance.

[0141] Specifically, see Fig. 7A , Fig. 7A A schematic diagram showing the effective time drawn in advance in the case of a double sliding module is shown. Fig. 7AThe first sliding module and the second sliding module are included, and the first sliding module and the second sliding module correspond to a shaded rectangle respectively. The moment corresponding to the leftmost side 001 of the rectangle corresponding to the first sliding module indicates the moment when sliding starts, and the moment corresponding to the rightmost side 002 of the rectangle indicates the moment when sliding ends. Fig. 7A It can be seen that the sliding start time of the second sliding module is the same as that of the first sliding module, but in some cases the second sliding module ends sliding earlier than the first sliding module. In this case, the early drawing operation will no longer be performed. Therefore, in this case, the effective time of the early drawing in this method is as follows: Fig. 7A However, at this time, the first sliding module is still in the sliding state. If the image on the display screen corresponding to the first sliding module is finished in advance drawing operation at this time, that is, Fig. 7A The image processing in the time period corresponding to 004 does not adopt the advance drawing method, which may still cause the sliding process of the image to be stuck.

[0142] From the above content, it can be seen that when an electronic device has dual sliding modules and the sliding end states of the dual sliding modules are in different times, when one sliding module is in the sliding end state but the other sliding module is still in the sliding state, the method of introducing early drawing cannot perform early drawing operation on the image corresponding to the sliding module that is still in the sliding state, which may cause a freeze in the image during the sliding process and reduce the user experience.

[0143] Figure 7B The schematic diagram shows the timing of early drawing effectiveness in the present application and related technologies. Specifically, Figure 7B As shown, Figure 7B Figure A1 in the figure includes a first sliding module and a second sliding module. The filled portion corresponding to the first sliding module represents that the image is in a sliding state. According to Figure A1, the start sliding time and the end sliding time of the first sliding module and the second sliding module are the same, and the effective time of the advance drawing in the related art in this case is also the same (as shown in the dotted box 701 in Figure A1). For the case where the start sliding time and the end sliding time of the first sliding module and the second sliding module in Figure A1 are the same, as shown in Figure A2, the effective time of the advance drawing in the method provided by the present application is the same as the effective time of the advance drawing in the related art (as shown in the dotted box 702 in Figure A2).

[0144] According to Figure B1, the first sliding module starts sliding first and then ends sliding, and the second sliding module starts sliding after the first sliding module ends sliding and then ends sliding. In this case, the effective time of early drawing of the image corresponding to the first sliding module in the related art is shown as 703 in the figure, and the effective time of early drawing of the image corresponding to the second sliding module is shown as 704 in Figure B1. For the case where the second sliding module starts sliding after the first sliding module in Figure B1 ends sliding, as shown in Figure B2, in the method provided by the present application, the effective time of early drawing of the first sliding module and the second sliding module is the same as the effective time of early drawing in the related art (as shown in the dotted boxes 705 and 706 in Figure B2).

[0145] According to Figure C1, the first sliding module and the second sliding module have the same start sliding time, but the first sliding module ends sliding a little later than the second sliding module. In this case, the effective time of the advance drawing in the related art can be shown as the dotted box 707 in the figure, that is, the advance drawing is not effective in the sliding state of the 708 part corresponding to the first sliding module. For the case where the first sliding module and the second sliding module have the same start sliding time in Figure C1, but the first sliding module ends sliding a little later than the second sliding module, as shown in Figure C2, the effective time of the advance drawing in the method provided by the present application can be shown as the dotted box 709 in Figure C2, that is, the advance drawing is effective when both the first sliding module and the second sliding module are in the sliding state.

[0146] According to Figure D1, the first sliding module starts sliding first, and then the second sliding module starts sliding, and the first sliding module ends sliding first, and then the second sliding module ends sliding. In this case, the effective time of the advance drawing in the related technology can be shown as the dotted box 710 in the figure, that is, the advance drawing is not effective in the sliding state of the 711 part corresponding to the second sliding module. For the sliding situation of the first sliding module and the second sliding module in Figure D1, as shown in Figure D2, the effective time of the advance drawing in the method provided by the present application can be shown as the dotted box 712 in Figure D2, that is, the advance drawing is effective when the first sliding module and the second sliding module are in the sliding state.

[0147] Therefore, it can be seen from the above content that compared with the method in the related art, after one sliding module finishes sliding, no matter whether the other sliding modules are in the sliding process, no advance drawing will be performed. In the technical solution provided in the embodiment of the present application, when two sliding modules correspond to different images, if one sliding module finishes sliding, the state of the other sliding module will be judged. If the state of the other sliding module is in the sliding state, the advance drawing operation will continue to be performed on the to-be-drawn layer of the image in the display screen corresponding to the other sliding module, thereby avoiding the jamming phenomenon when the image is in the sliding state in the case of double sliding modules.

[0148] Figure 8 1 shows a schematic diagram of the structure of an electronic device 800. The electronic device 800 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, and the like.

[0149] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 800. In other embodiments of the present application, the electronic device 800 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0150] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0151] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0152] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the above-mentioned memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved. The processor may be used to execute the display method mentioned in the present application.

[0153] The wireless communication function of the electronic device 800 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0154] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 800 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0155] The mobile communication module 150 can provide wireless communication solutions including 2G / 3G / 4G / 5G etc. applied to the electronic device 800 .

[0156] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0157] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 800. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0158] In some embodiments, the antenna 1 of the electronic device 800 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 800 can communicate with the network and other devices through wireless communication technology. The electronic device 800 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0159] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 800. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0160] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 800 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 800 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0161] The SIM card interface 195 is used to connect a SIM card.

[0162] It will be understood that, as used herein, the term "module" may refer to or include an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and / or memory that executes one or more software or firmware programs, a combinational logic circuit, and / or other appropriate hardware components that provide the described functionality, or may be part of these hardware components.

[0163] It can be understood that in each embodiment of the present application, the processor can be a microprocessor, a digital signal processor, a microcontroller, etc., and / or any combination thereof. According to another aspect, the

[0164] The processor may be a single-core processor, a multi-core processor, etc., and / or any combination thereof.

[0165] The various embodiments disclosed in the present application may be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application may be implemented as a computer program or program code executed on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0166] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0167] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0168] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Therefore, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.

[0169] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0170] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0171] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0172] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present application.

Claims

1. A display method, characterized in that: Using an electronic device, the method comprises: Displaying a first interface, wherein the first interface includes a first image and a second image; In response to the first image and the second image being in a sliding state, drawing the to-be-drawn layer of the first image and the to-be-drawn layer of the second image in a first drawing manner; Obtaining an image frame to be displayed based on the completed layer to be drawn, and displaying the image frame to be displayed in the first interface; The first drawing method includes: receiving a first synchronization message and drawing a first layer to be drawn; and in response to drawing the first layer to be drawn and not receiving a second synchronization message, drawing a second layer to be drawn.

2. The method according to claim 1, characterized in that The method further comprises: In response to the first image being in a sliding state and the second image being in a sliding stop state, the first drawing mode is used to draw the layer to be drawn corresponding to the first image, and the drawing operation on the layer to be drawn of the second image is stopped.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: In response to the first image and the second image being in a sliding stop state, the drawing operation on the to-be-drawn layers of the first image and the second image is stopped.

4. The method according to claim 1 or 2, characterized in that: The electronic device includes a first sliding module and a second sliding module, the state of the first sliding module is used to reflect the sliding state of the first image, and the state of the second sliding module is used to reflect the sliding state of the second image.

5. The method according to claim 4, characterized in that The method further comprises: If the states of the first sliding module and the second sliding module are in a sliding state, setting the identifiers of the first sliding module and the second sliding module to a first identifier; In response to the identifiers of the first sliding module and the second sliding module being the first identifier, the first sliding module and the second sliding module are added to the module group to be analyzed, and the first drawing method is used to draw the layers to be drawn of the images corresponding to the sliding modules included in the module group to be analyzed.

6. The method according to claim 5, characterized in that The method further comprises: If the state of the first sliding module is a sliding state and the state of the second sliding module is a sliding stop state, setting the identifier of the second sliding module to a second identifier; In response to the identifier of the second sliding module being the second identifier, the second sliding module is deleted from the group of modules to be analyzed, and the drawing operation on the layer to be drawn of the second image is stopped.

7. The method according to claim 6, characterized in that The method further comprises: If the state of the first sliding module is a sliding stop state, setting the identifier of the first sliding module to the second identifier; In response to the identifier of the first sliding module being the second identifier, the first sliding module is deleted from the group of modules to be analyzed, and the drawing operation on the layer to be drawn of the first image is stopped.

8. The method according to claim 7, characterized in that The method further comprises: If there is no sliding module in the module group to be analyzed, the drawing operation on the layers to be drawn of the first image and the second image is stopped.

9. The method according to claim 4, characterized in that The electronic device further includes a choreographer module and a dynamic buffer, and in response to the first image and the second image being in a sliding state, drawing the to-be-drawn layer of the first image and the to-be-drawn layer of the second image in a first drawing manner, comprising: The dynamic buffer sends an instruction for performing the first drawing mode to the choreographer module; The choreographer module draws the to-be-drawn layer of the first image and the to-be-drawn layer of the second image using the first drawing method.

10. The method according to any one of claims 1 to 9, characterized in that: The electronic device comprises a rendering thread, a display merging module and a hardware display module; The obtaining the image frame to be displayed based on the completed layer to be drawn, and displaying the image frame to be displayed in the first interface includes: Sending the to-be-drawn layer drawn by using the first drawing method to the rendering thread for rendering operation to obtain the rendered to-be-merged layer; The rendering thread sends the rendered layers to be merged to the display merging module for merging to obtain image frames to be displayed; The display merging module sends the image frame to be displayed to the hardware display module; The hardware display module displays the image frame to be displayed in the first interface through the display screen of the electronic device.

11. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the display method described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the display method according to any one of claims 1 to 10.

Images