Processing method of terminal equipment and related device

Abstract

The invention provides a processing method of terminal equipment and a related device. The terminal equipment comprises a display, and the processing method comprises the steps that when it is detected that a first thread in the terminal equipment calls a first system function and the first thread is not a system frame drawing thread, the first thread is determined to be a third-party frame drawing thread; the first system function is a function called by the frame drawing thread, the first system function is used for submitting a buffer drawn by the frame drawing thread to a buffer queue, and the terminal equipment displays the buffer drawn by the frame drawing thread on the display by consuming the buffer queue. According to the method, the third-party frame drawing thread in the terminal equipment can be identified.

Description

Technical Field

[0001] The present application relates to the field of terminal equipment, and in particular to a processing method and related apparatus for terminal equipment. Background Art

[0002] When a user uses a terminal device, each frame displayed by the terminal device needs to be drawn by at least one frame drawing thread. If a frame cannot be drawn within the rated time, the user will feel a freeze, which greatly affects the user experience.

[0003] Currently, in order to reduce lag and improve user experience, the terminal device will identify whether it is a frame drawing thread. If it is identified as a frame drawing thread, a higher priority will be assigned to the identified frame drawing thread so that more computing resources can be scheduled for the frame drawing thread during system scheduling, thereby reducing the occurrence of lag and unsmoothness.

[0004] However, the drawing threads that terminal devices can currently identify are mainly concentrated on the system drawing thread (also known as System Render). For some apps in terminal devices, when drawing each frame, in addition to the system drawing thread (also known as System Render), they may also be drawn based on the drawing threads in some third-party drawing frameworks (such as Flutter, webview, etc.). Currently, there is no solution to identify the drawing threads in third-party drawing frameworks. Summary of the invention

[0005] The present application provides a processing method and related apparatus of a terminal device, which are used to identify a frame drawing thread in a third-party frame drawing framework.

[0006] In a first aspect, the present application provides a processing method of a terminal device, wherein the terminal device includes a display, and the method includes: when it is detected that a first thread in the terminal device calls a first system function and the first thread is not a system frame drawing thread, determining that the first thread is a third-party frame drawing thread;

[0007] Among them, the first system function is a function called after the frame drawing thread fills the drawn data into the applied buffer buffer, and the first system function is used to submit the buffer filled by the frame drawing thread to the BufferQueue, wherein the terminal device displays the data in the buffer filled by the frame drawing thread on the display by consuming the BufferQueue.

[0008] When the frame drawing thread is drawing, the frame drawing thread will first apply for a buffer, and then fill the drawn data into the buffer. Therefore, in this application, the buffer data filled by the frame drawing thread is also referred to as a drawn buffer.

[0009] Or in other words, in this application, the buffer drawn by the frame drawing thread is the data drawn by the frame drawing thread.

[0010] Since the frame drawing threads have a common feature: after completing the drawing, the frame drawing thread will submit the drawn buffer to the BufferQueue through the system function (also referred to as the first system function in this application), so that the terminal device can display the data in the buffer drawn by the frame drawing thread on the display by consuming the buffer in the BufferQueue.

[0011] Therefore, in this embodiment, the third-party frame drawing thread in the terminal device is identified based on this feature.

[0012] It should be understood that, through the technical solution provided by this application, the terminal device can fully identify the third-party frame drawing thread in the terminal device. Or to put it another way, the technical solution provided by this application will not miss the frame drawing thread in the terminal device.

[0013] Furthermore, after identifying the third-party frame drawing thread, the terminal device can, for example, assign a high priority to the third-party frame drawing thread so that more computing resources are allocated to the third-party frame drawing thread, thereby reducing the time it takes for the third-party frame drawing thread to draw the buffer, and further reducing the impact of the third-party frame drawing thread on the duration of the entire frame, thereby achieving smoothness in the runtime of the APP that is drawn by the frame drawing thread in the third-party frame framework.

[0014] It should be understood that when a frame is drawn by at least one frame drawing thread, some of the at least one frame drawing thread may draw buffers of small sizes, while some of the frame drawing threads may draw buffers of large sizes.

[0015] How to define the large size and the small size may depend on the actual scenario. For example, a threshold may be set, and a buffer size larger than the threshold is considered to be a large size, and a buffer size smaller than or equal to the threshold is considered to be a small size.

[0016] It should be understood that the larger the drawing buffer size, the longer the drawing time is required. Therefore, compared with the drawing thread of a small size, the drawing thread of a large size is the key to affecting the ability to draw a frame and complete the drawing. In view of this, as an optional embodiment, after identifying the third-party drawing thread, the application first determines whether the third-party drawing thread is a key drawing thread, and then only when the third-party drawing thread is a key drawing thread, the first thread is assigned a high priority; wherein the key drawing thread is a drawing thread that can affect the delay of a frame.

[0017] Specifically, in the present application, a frame drawing thread whose drawing buffer size is larger than a preset threshold is considered as a frame drawing thread that affects the latency of one frame.

[0018] For example, determining whether the first thread is a key frame thread includes:

[0019] Get the screen size of the display;

[0020] Determine whether the size of the buffer drawn by the first thread is greater than the product of the screen size and a first preset percentage;

[0021] If the size of the buffer drawn by the first thread is larger than the product of the screen size and the first preset percentage, the first thread is determined to be a key frame thread.

[0022] For another example, determining whether the first thread is a key frame thread includes:

[0023] Get the target size, and the terminal device displays a frame within the target size. The target size is smaller than the screen size of the display;

[0024] Determine whether the size of the buffer drawn by the first thread is greater than the product of the target size and the second preset percentage;

[0025] If the size of the buffer drawn by the first thread is larger than the product of the target size and the second preset percentage, the first thread is determined to be a key frame thread.

[0026] For another example, determining whether the first thread is a key frame thread includes:

[0027] Get the screen size of the display;

[0028] Determine whether the size of the buffer drawn by the first thread is greater than the product of the screen size and a first preset percentage;

[0029] If the size of the buffer drawn by the first thread is larger than the product of the screen size and the first preset percentage, it is determined whether the size of the buffer drawn by the first thread is larger than the product of the target size and the second preset percentage, and the terminal device displays a frame within the target size, and the target size is smaller than the screen size of the display;

[0030] If the size of the buffer drawn by the first thread is larger than the product of the target size and the second preset percentage, the first thread is determined to be a key frame thread.

[0031] In combination with the first aspect, in a possible implementation manner, the first system function includes any one of the following: a queue buffer queuebuffer function and an onFrameAvailable function.

[0032] In combination with the first aspect, in a possible implementation manner, the method further includes: when it is detected that a second thread in the terminal device calls the first system function and the second thread is a system frame drawing thread, assigning a high priority to the second thread.

[0033] In a second aspect, the present application provides a processing device, which may include various functional modules for implementing the method in the first aspect. For example, the device includes a detection module and an identification module.

[0034] For example, the detection module may be used to detect whether a first thread in the terminal device calls a first system function;

[0035] an identification module, configured to identify whether the first thread is a system frame drawing function when the detection module detects that the first thread in the terminal device calls the first system function, and if not, determine that the first thread is a third-party frame drawing thread;

[0036] Among them, the first system function is a function called after the frame drawing thread fills the drawn data into the applied buffer buffer, and the first system function is used to submit the buffer buffer drawn by the frame drawing thread to the BufferQueue, wherein the terminal device displays the data in the buffer filled by the frame drawing thread on the display by consuming the BufferQueue.

[0037] In conjunction with the second aspect, in a possible implementation manner, the terminal device further includes a scheduling module, and the scheduling module is used to assign a high priority to the first thread;

[0038] Among them, the higher the priority assigned by the terminal device to the thread, the more computing resources the terminal device provides for the thread.

[0039] In combination with the second aspect, in a possible implementation manner, the identification module is further used to determine whether the first thread is a key drawing frame thread;

[0040] The scheduling module is specifically used for: when the first thread is a key frame drawing thread, assigning a high priority to the first thread; wherein the key frame drawing thread is a frame drawing thread that can affect the delay of drawing a frame of picture.

[0041] In combination with the second aspect, in a possible implementation, the identification module identifies whether the first thread is a key frame thread, including: obtaining the screen size of the display; determining whether the size of the buffer filled by the first thread is larger than the product of the screen size and a first preset percentage; if the size of the buffer filled by the first thread is larger than the product of the screen size and the first preset percentage, determining that the first thread is a key frame thread.

[0042] In combination with the second aspect, in a possible implementation, the identification module identifies whether the first thread is a key frame thread, including: obtaining a target size, the terminal device displays a frame of the picture within the target size, and the target size is smaller than the screen size of the display; determining whether the size of the buffer filled by the first thread is greater than the product of the target size and a second preset percentage; if the size of the buffer filled by the first thread is greater than the product of the target size and the second preset percentage, determining that the first thread is a key frame thread.

[0043] In combination with the second aspect, in a possible implementation method, the identification module identifies whether the first thread is a key frame thread, including: obtaining the screen size of the display; determining whether the size of the buffer filled by the first thread is larger than the product of the screen size and a first preset percentage; if the size of the buffer filled by the first thread is larger than the product of the screen size and the first preset percentage, determining whether the size of the buffer filled by the first thread is larger than the product of the target size and a second preset percentage, and the terminal device displays a frame of picture within the target size, and the target size is smaller than the screen size of the display; if the size of the buffer filled by the first thread is larger than the product of the target size and the second preset percentage, determining that the first thread is a key frame thread.

[0044] In combination with the second aspect, in a possible implementation manner, the first system function includes any one of the following: a queue buffer queuebuffer function and an onFrameAvailable function.

[0045] In combination with the second aspect, in a possible implementation, when the detection module detects that the second thread in the terminal device calls the first system function and the identification module identifies that the second thread is a system frame drawing thread, the scheduling module is also used to assign a high priority to the second thread.

[0046] In some implementations, these modules may be implemented by software and / or hardware. In this implementation, the device may include a processor and a transceiver. For example, the detection module and the identification module may be implemented by the processor executing a program code stored in a memory, and the acquisition module may be implemented by a transceiver. Optionally, the device may also include a memory.

[0047] In a third aspect, the present application provides a terminal device, which may include various functional modules for implementing the method in the first aspect. In some implementations, these modules may be implemented in software and / or hardware.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium storing a program code for execution by a communication device, wherein the program code includes instructions for implementing the method in the first aspect.

[0049] In a fifth aspect, the present application provides a computer program product comprising instructions, and when the computer program product is executed on a communication device, the communication device implements the method in the first aspect.

[0050] It can be understood that the effects that can be obtained in the second, third, fourth and fifth aspects can refer to the description in the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0052] Figure 2 A software structure block diagram of a terminal device applicable to the embodiments of the present application;

[0053] Figure 3 A schematic diagram of a frame of picture displayed by a terminal device provided in one embodiment of the present application;

[0054] Figure 4 An architecture of a method applicable to the present application provided for one embodiment of the present application;

[0055] Figure 5 A framework applicable to the method of the present application provided for another embodiment of the present application;

[0056] Figure 6 A schematic diagram of a process for identifying whether a first thread is a third-party frame drawing thread provided by an embodiment of the present application;

[0057] Figure 7 A schematic diagram of a process for identifying whether a first thread is a third-party frame drawing thread provided in another embodiment of the present application;

[0058] Figure 8 A schematic diagram of target size provided for another embodiment of the present application;

[0059] Fig. 9 A schematic diagram of a process for identifying whether a first thread is a third-party frame drawing thread provided in yet another embodiment of the present application;

[0060] Fig.10 A schematic diagram of the structure of a processing device provided in one embodiment of the present application;

[0061] Fig.11 A schematic diagram of the structure of a processing device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0062] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0063] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0064] It should be noted that, in this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0065] In addition, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c, where a, b, c can be single or multiple.

[0066] The processing method of the terminal device provided in the embodiment of the present application can also be called a method for identifying a third-party drawing frame thread, which can be applied to a terminal device with a display function. The terminal device can be a wearable terminal device such as a mobile phone, a tablet computer, a personal computer (PC), a smart screen, a car machine device, and a smart watch. It can also be various teaching auxiliary tools (such as learning machines, early education machines), smart toys, portable robots, personal digital assistants (PDA), augmented reality technology (AR) equipment, virtual reality (VR) equipment, etc. It can also be a device with mobile office functions, a device with smart home functions, a device with audio and video entertainment functions, a device that supports smart travel, etc. It should be understood that the embodiments of the present application do not limit the specific technology and specific device form adopted by the terminal device.

[0067] In order to better understand the embodiment of the present application, the hardware structure of the terminal device of the embodiment of the present application is introduced below. Figure 1 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application.

[0068] Figure 1 The terminal device may include a processor 110, an internal memory 121, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a sensor module 180, a button 190, a display screen 194, a timer 196, and the like.

[0069] Optionally, the sensor module 180 may include a pressure sensor 180A, a touch sensor 180B, and the like.

[0070] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0071] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, etc. Among them, different processing units may be independent devices or integrated into one or more processors. A memory may also be provided in the processor 110 for storing instructions and data.

[0072] The charging management module 140 is used to receive charging input from a charger, which may be a wireless charger or a wired charger. The power management module 141 is used to connect the charging management module 140 to the processor 110 .

[0073] The wireless communication function of the terminal device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0074] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The antenna in the terminal device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna.

[0075] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to terminal devices. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.

[0076] The wireless communication module 160 can provide wireless communication solutions applied to terminal devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), etc.

[0077] In some embodiments, antenna 1 of the terminal device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that the mobile phone can communicate with the network and other devices through wireless communication technology.

[0078] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the terminal device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0079] As an example, each frame displayed on the display screen 194 needs to be drawn by a frame drawing process in the terminal device.

[0080] The terminal device can realize the display function through a graphics processing unit (GPU), a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The GPU can also be called a display core, a visual processor, a display chip, etc.

[0081] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store applications required for at least one function in the operating system (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the terminal device (such as audio data, a phone book, etc.).

[0082] The terminal device can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, and the application processor.

[0083] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The terminal device can listen to music or listen to hands-free calls through the speaker 170A. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. When the terminal device answers a call or voice message, the voice can be answered by placing the receiver 170B close to the human ear. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0084] The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be arranged on the display screen 194. When a touch operation acts on the display screen 194, the terminal device can detect the intensity of the touch operation according to the pressure sensor 180A. The terminal device can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. The touch sensor 180B can be arranged on the display screen 194, and the touch sensor 180B and the display screen 194 form a touch screen, also known as a "touch screen". The touch sensor 180B is used to detect touch operations acting on or near it. The touch sensor 180B can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180B can also be arranged on the surface of the mobile phone, which is different from the position of the display screen 194.

[0085] The button 190 includes a power button, a volume button, etc. The button 190 may be a mechanical button. It may also be a touch button. The terminal device may receive the button input and generate a key signal input related to the user settings and function control of the terminal device. The indicator 192 may be an indicator light, which may be used to indicate the charging status, the change of the power, or may be used to indicate messages, missed calls, notifications, etc. The timer 196 may be used to record time information. For example, when the terminal device records a video, the start time and the end time of the video may be recorded, and the duration of the recorded video may be recorded.

[0086] Figure 2A software structure block diagram of a terminal device applicable to the embodiments of the present application. The layered architecture divides the software system of the terminal device into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into four layers, from top to bottom, namely, the application layer (applications), the application framework layer (application framework), the Android runtime (Android runtime) and the system library, and the kernel layer (kernel).

[0087] The application layer may include a series of application packages, and the application layer runs applications by calling the application programming interface (API) provided by the application framework layer.

[0088] For example, an application package may include desktop, calendar, memo, map, navigation, Bluetooth, music, JD.com, and short message applications.

[0089] The application framework layer provides API and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0090] For example, the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0091] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to applications. The data can include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, the display interface including the SMS notification icon can include a view for displaying text and a view for displaying pictures. The phone manager is used to provide communication functions for terminal devices. For example, the management of call status (including connected, hung up, etc.). The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the top status bar of the system in the form of an icon or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, the terminal device vibrates, the indicator light flashes, etc.

[0092] The Android system runtime includes the core library and the virtual machine. The Android system runtime is responsible for the scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection. The system library can contain modules with multiple functions, such as: surface manager, media library, and 3D graphics processing library.

[0093] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple common audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing.

[0094] The kernel layer is a layer between hardware and software. The kernel layer is used to drive the hardware to make the hardware work. The kernel layer at least includes display driver, screen driver, camera driver and sensor driver, etc., which are not limited in the embodiments of the present application.

[0095] for Figure 1 and Figure 2 For example, assuming that the terminal device displays 40 frames per second, the terminal device currently displays the following: Figure 3 A frame of picture is shown, wherein the frame of picture is drawn by at least one frame drawing thread.

[0096] Specifically, the process from drawing data from each frame drawing thread to displaying a drawn frame of data on the display of the terminal device is: each frame drawing thread fills the drawn data into a buffer applied for in advance, and then calls the system function (also referred to as the first system function in this application) to submit the filled buffer to the buffer queue BufferQueue. After that, the terminal device calls the frame consumption module to consume BufferQueue to display the data drawn by each frame drawing thread on the display.

[0097] For example, after each frame drawing thread calls the system function to submit the buffer to the BufferQueue, the terminal device calls the frame consumption module surfacelinger to display the drawn data on the display. The surfacelinger module is used to receive the data drawn by each frame drawing thread in the BufferQueue, synthesize these data into a frame and then send it to the display.

[0098] As described above, since each frame drawing thread first applies for a buffer when drawing, and then fills the drawn data into the buffer, in this application, drawing by each frame drawing thread is also referred to as drawing a buffer.

[0099] In addition, it should be noted that in the present application, the terminal device calling the frame consumption module to consume the BufferQueue can also be referred to as consuming the bufffer in the BufferQueue, or it can also be referred to as consuming the data in the bufffer in the BufferQueue, which does not constitute a limitation of the present application.

[0100] It is understandable that if a frame cannot be drawn within the rated time, the user will feel a lag, which will affect the user experience.

[0101] Therefore, in order to reduce lag and improve user experience, the terminal device will identify the frame drawing thread, and then assign a higher priority to the identified frame drawing thread so that more computing resources can be scheduled for the frame drawing thread during system scheduling, so that the frame drawing thread takes less time to complete the drawing buffer, thereby reducing the occurrence of lag and unsmoothness.

[0102] However, the drawing threads that terminal devices can currently identify are mainly concentrated on the system drawing thread (also known as System Render). For some apps in terminal devices, when drawing each frame, in addition to System Render, they may also be based on the drawing threads in some third-party drawing frameworks (such as Flutter, webview, etc.).

[0103] For example, for shopping apps, the graphics of each frame are becoming more and more complex. In addition to the information and pictures of the displayed products, it may also be necessary to draw video information. In this case, the video information may be drawn by a third-party frame drawing thread.

[0104] For another example, for some mini-programs in the WeChat APP, some of the screens displayed to users by these mini-programs may also be based on third-party drawing frame threads. For example, the mini-programs are game-type mini-programs. However, there is currently no solution for identifying drawing frame threads in third-party drawing frame frameworks. At this time, since the terminal device cannot identify the third-party drawing frame thread, when users use the APP that is drawn based on the third-party drawing frame framework, the APP may have problems such as freeze and unsmoothness, thus affecting the user experience.

[0105] by Figure 3 Take the frame shown as an example, for example, Figure 3 The live bar 301 shown in the frame needs to be drawn based on a third-party frame drawing thread. At this time, if the terminal device cannot identify the third-party frame drawing thread, it is possible that the terminal device has less computing resources scheduled for the third-party frame drawing thread, making the third-party frame drawing thread take a long time to draw. At this time, there may be frame loss and freezes caused by the long drawing time of the third-party frame drawing thread.

[0106] In view of this, the present application provides a processing method and a processing device of a terminal device, which are used to identify a drawing thread in a third-party drawing frame framework, further improve the smoothness of an APP that is drawn through the drawing thread in the third-party drawing frame framework during operation, and further improve the user experience.

[0107] The processing method of the terminal device provided by the present application is described below in conjunction with the accompanying drawings.

[0108] Before introducing the processing method of the terminal device provided by this application, the system architecture of this application is first introduced. Figure 4 The system architecture includes a default rendering framework 401 , a third-party rendering framework 402 , and a third-party rendering framework 403 .

[0109] The default drawing framework 401 includes a system UI thread and a system Render thread (ie, System Render thread). The detailed description of the UI thread and the System Render thread can refer to the description in the related art, which will not be repeated here.

[0110] The third-party drawing framework 402 and the third-party drawing framework 403 include a third-party drawing frame thread for drawing frames and / or at least one wake-up thread, wherein the at least one wake-up thread is used to wake up the third-party drawing thread.

[0111] For example, Figure 4 As shown, threads 1 to 5 included in the third-party drawing framework 402 can be considered as wake-up threads of third-party drawing frame thread 1 in the third-party drawing framework 402 , and threads 1 ′ to 5 ′ included in the third-party drawing framework 403 can be considered as wake-up threads of third-party drawing frame thread 2 in the third-party drawing framework 403 .

[0112] Specifically, thread 1 in the third-party drawing frame 402 points to thread 2, which means thread 1 wakes up thread 2; thread 2 points to thread 3, which means thread 2 wakes up thread 3; thread 3 points to third-party drawing frame thread 1, which means thread 3 wakes up third-party drawing frame thread 1; thread 4 points to thread 5, which means thread 4 wakes up thread 5; thread 5 points to third-party drawing frame thread 1, which means thread 5 wakes up third-party drawing frame thread 1. It can be understood that thread 1, thread 2, and thread 4 can be considered as indirect wake-up threads of third-party drawing frame thread 1, and thread 3 and thread 5 can be considered as direct wake-up threads of third-party drawing frame thread 1.

[0113] Similarly, thread 1 ′, thread 2 ′, and thread 4 ′ in the third-party drawing frame 403 can be considered as indirect wake-up threads of the third-party drawing frame thread 2 , and thread 3 ′ and thread 5 ′ can be considered as direct wake-up threads of the third-party drawing frame thread 2 .

[0114] Specifically, the system Render thread, the third-party frame drawing thread 1 and the third-party frame drawing thread 2 jointly complete the drawing of one frame.

[0115] It should be noted that this embodiment does not limit the specific form of at least one third-party drawing frame.

[0116] For example, at least one third-party drawing framework may be the same or different. Figure 4 The third-party drawing framework 402 and the third-party drawing framework 403 may both be Flutter drawing frameworks, or both may be webview frameworks, or one may be the Flutter drawing framework and the other may be the webview framework.

[0117] It is also stated here that Figure 4 The example of including two third-party frame drawing threads is taken only, but does not constitute a limitation of the present application.

[0118] For example, only one third-party frame drawing thread may be included. In this case, it can be considered that the system Render thread and the third-party frame drawing thread jointly complete the drawing of one frame.

[0119] For another example, more third-party frame drawing threads may be included, for example, there are a total of N third-party frame drawing threads. In this case, it can be considered that the system Render thread and the N third-party frame drawing threads jointly complete the drawing of one frame.

[0120] It should be understood that the size of the buffer drawn by the frame drawing thread may be different. The size of the buffer can be understood as follows: for example, if the frame drawing thread applies for a buffer with a length of M and a width of P, then the size of the buffer is M*N. In addition, it should be understood that the larger the size of the buffer drawn by the frame drawing thread, the more data the frame drawing thread draws, and accordingly, the longer the time required.

[0121] For the buffers drawn by the third-party drawing frame threads in at least one third-party drawing frame, the sizes of some of the buffers drawn by the third-party drawing frame threads may be relatively large, while the sizes of some of the buffers drawn by the third-party drawing frame threads may be relatively small. It should be noted that this embodiment does not limit how to define the large size or the small size.

[0122] During implementation, a threshold may be set, and then a buffer whose size is larger than the threshold is considered as a large-size buffer, and a buffer whose size is smaller than the threshold is considered as a small-size buffer.

[0123] For example, Figure 5 As shown, the buffer drawn by the third-party frame drawing thread 1 is a large-sized buffer, and the buffer drawn by the third-party frame drawing thread 2 is a small-sized buffer. It should be noted that this is only an example and does not constitute a limitation. For example, the third-party frame drawing thread 1 and the third-party frame drawing thread 2 may both draw large-sized buffers, or may both draw small-sized buffers.

[0124] like Figure 4 and Figure 5As shown, since all frame drawing threads have the following characteristics when drawing: after completing the drawing, the frame drawing thread will submit the drawn buffer to the BufferQueue through the system function (i.e., the first system function), so that the frame consumption module in the terminal device consumes the buffer in the BufferQueue to display the buffer drawn by the frame drawing thread on the display.

[0125] As an example, the first system function is a queue buffer queuebuffer function and an onFrameAvailable function.

[0126] It is noted here that when there are multiple third-party frame drawing threads, the system functions used by the multiple third-party frame drawing threads to submit buffers can be the same or different, which does not constitute a limitation of this application. Figure 4 or Figure 5 For example, the system function used by the third-party frame drawing thread 1 is the queuebuffer function, and the system function used by the third-party frame drawing thread 2 is the onFrameAvailable function. Alternatively, the system function used by the third-party frame drawing thread 1 is the onFrameAvailable function, and the system function used by the third-party frame drawing thread 2 is the queuebuffer function.

[0127] As an example, the frame consumption module is surfaceflinger.

[0128] Therefore, the method for identifying a third-party frame drawing thread provided by the present application includes: when it is detected that a first thread in a terminal device calls a first system function and the first thread is not a system frame drawing thread, determining that the first thread is a third-party frame drawing thread; wherein the first system function is a function called by the frame drawing thread, and the first system function is used to submit the buffer drawn by the frame drawing thread to the buffer queue BufferQueue, wherein the terminal device displays the buffer drawn by the frame drawing thread on the display by consuming the BufferQueue.

[0129] Or to put it another way, in this application, the terminal device determines the thread that submits the buffer using the first system function as the frame drawing thread, and then compares it with the system frame drawing thread to identify whether it is a third-party frame drawing thread.

[0130] It should be understood that the technical solution provided by this application can enable the terminal device to fully identify the third-party frame drawing thread in the terminal device. Or to put it another way, the technical solution provided by this application will not miss the third-party frame drawing thread in the terminal device.

[0131] Furthermore, after identifying the third-party frame drawing thread, the terminal device can, for example, assign a high priority to the third-party frame drawing thread so that more computing resources are allocated to the third-party frame drawing thread, thereby reducing the time it takes for the third-party frame drawing thread to draw the buffer, and further reducing the impact of the third-party frame drawing thread on the duration of the entire frame, thereby achieving smoothness in the runtime of the APP that is drawn by the frame drawing thread in the third-party frame framework.

[0132] As mentioned above, when drawing a frame through at least one drawing thread, some of the at least one drawing thread draw small sizes, and some of the drawing threads may draw large sizes. Because the larger the drawing size, the longer the drawing time is required. Therefore, compared with the drawing thread of small size, the drawing thread of large size is the key to affecting the ability to draw a frame. In other words, compared with the drawing thread of large size, the drawing thread of small size may have little effect on the time of drawing a frame.

[0133] In view of this, in some implementations provided by the present application, after a third-party frame drawing thread is identified, when a high priority is assigned to the third-party frame drawing thread, a high priority is assigned to the first thread only when the third-party frame drawing thread is a key frame drawing thread; wherein the key frame drawing thread is a frame drawing thread that can affect the delay of a frame of the picture.

[0134] Three embodiments of identifying the first thread as a key frame thread are given below.

[0135] Example 1

[0136] The method comprises: when a first thread calls a first system function to submit a drawn buffer, the terminal device calls a recognition module.

[0137] For example, the first system function is the queuebuffer function and the onFrameAvailable function.

[0138] In specific implementation, when the first frame drawing thread submits the buffer through the first system function, a function call stub can be inserted into the first system function, or also called inserting an identification module interface, so as to enter the identification process of the third-party frame drawing thread.

[0139] Among them, when the terminal device calls the identification module, Figure 6 As shown, execute:

[0140] S601, determine whether the first thread is a third-party frame drawing thread, if so, execute S602.

[0141] S602: Obtain the screen size of the display.

[0142] For example, obtain the physical size of the display of the terminal device hardware.

[0143] S603, determining whether the size of the buffer drawn by the first thread is larger than the product of the screen size and a first preset percentage, if so, executing S604.

[0144] For example, the first preset percentage is expressed as x%, that is, it is determined whether the size of the buffer drawn by the first thread is greater than the screen size*x%.

[0145] In this embodiment, a buffer whose size is larger than the screen size*x% is considered as a large-size buffer, and a buffer whose size is smaller than or equal to the screen physical size*x% is considered as a small-size buffer.

[0146] It should be noted that this embodiment does not limit the specific value of the first preset percentage.

[0147] S604: Determine that the first thread is a key frame thread.

[0148] It can be seen that in this embodiment, by comparing the size of the buffer drawn by the first thread with the screen size*x%, it is determined whether the buffer drawn by the first thread is of large size or small size, and then when it is determined that the buffer drawn by the first thread is of large size, the first thread is determined as a key frame drawing thread that affects the drawing of one frame.

[0149] Furthermore, when it is determined that the first thread is a third-party key frame thread that affects the drawing of a frame, more computing power is allocated to the first thread, thereby reducing the time it takes for the third-party key frame thread to draw the buffer, to ensure that the drawing of a frame can be completed within the rated time, thereby improving the smoothness of the APP that is drawn through the third-party first frame thread during operation.

[0150] In this embodiment, the identification process can be executed in the application process so as not to rely on the operating system. In other words, when executed in the application process, the identification process can be completed regardless of the type of operating system used by the terminal device, such as Android system or IOS system.

[0151] Example 2

[0152] The method comprises: when a first thread calls a first system function to submit a drawn buffer, the terminal device calls a recognition module.

[0153] For example, the first system function is the queuebuffer function and the onFrameAvailable function.

[0154] Similarly, in specific implementation, when the first frame drawing thread submits the buffer through the first system function, a function call stub can be inserted at the first system function, or also called inserting the identification module interface, so as to enter the identification process of the third-party frame drawing thread.

[0155] Among them, when the terminal device calls the identification module, Figure 7 As shown, execute:

[0156] S701, determine whether the first thread is a third-party frame drawing thread, if so, execute S702.

[0157] S702: Notify the frame consumption module of the size of the buffer drawn by the first thread.

[0158] S703, obtaining a target size.

[0159] It should be understood that after each frame drawing thread used to draw a frame completes drawing, when the terminal device displays it, the target size used to display the drawn frame may be different from the screen size of the terminal device's display, that is, when the terminal device displays a frame on the screen of the display, it may only use a partial area for display.

[0160] For example Figure 8 As shown, the terminal device needs to display a drawn frame in the area within the dotted frame 801 instead of covering the entire display.

[0161] In this embodiment, the target size is also referred to as the display size. It is emphasized here that the display size in this embodiment is different from the display size. Figure 6 The screen sizes of the displays in the embodiments are different. Figure 6 The screen size of the display in the embodiment is determined by the hardware. Exemplarily, the screen size can be understood as the size of the entire screen, such as including the status bar above, etc., while the display size is determined by the system process.

[0162] S704, determining whether the size of the buffer drawn by the first thread is larger than the product of the target size and the second preset percentage; if so, executing S704.

[0163] For example, the second preset percentage is expressed as m%, that is, it is determined whether the size of the buffer drawn by the first thread is greater than the target size*m%.

[0164] In this embodiment, a buffer whose buffer size is larger than the target size*m% is considered as a large-size buffer, and a buffer whose buffer size is smaller than or equal to the target size*m% is considered as a small-size buffer.

[0165] It should be noted that this embodiment does not limit the specific value of the second preset percentage.

[0166] S705: Determine that the first thread is a key frame thread.

[0167] It can be seen that in this embodiment, by comparing the size of the buffer drawn by the first thread with the target size * m%, it is determined whether the buffer drawn by the first thread is of large size or small size, and then when it is determined that the buffer drawn by the first thread is of large size, the first thread is determined as a key frame drawing thread that affects the drawing of one frame.

[0168] Furthermore, when it is determined that the first thread is a third-party key frame thread that affects the drawing of a frame, more computing power is allocated to the first thread, thereby reducing the time it takes for the third-party key frame thread to draw the buffer, to ensure that the drawing of a frame can be completed within the rated time, thereby improving the smoothness of the APP that is drawn through the third-party first frame thread during operation.

[0169] This embodiment and Figure 6 The difference between the shown embodiments is that Figure 6 The embodiment shown is to determine whether the first drawing frame thread is a key drawing frame thread based on the buffer size drawn by the first drawing frame thread and the screen size of the hardware display, while this embodiment is to determine whether the first drawing frame thread is a key drawing frame thread based on the buffer size drawn by the first drawing frame thread and the target size used when actually displaying a frame. Figure 7 Example compared to Figure 6 The embodiment can improve the accuracy of identifying key drawing frame threads.

[0170] Example 3

[0171] The method comprises: when a first thread calls a first system function to submit a drawn buffer, the terminal device calls a recognition module.

[0172] For example, the first system function is the queuebuffer function and the onFrameAvailable function.

[0173] Similarly, in specific implementation, when the first frame drawing thread submits the buffer through the first system function, a function call stub can be inserted at the first system function, or also called inserting the identification module interface, so as to enter the identification process of the third-party frame drawing thread.

[0174] Among them, when the terminal device calls the identification module, Fig. 9 As shown, execute:

[0175] S901, determine whether the first thread is a third-party frame drawing thread, if so, execute S902.

[0176] S902, obtaining the screen size of the display.

[0177] S903, determining whether the size of the buffer drawn by the first thread is larger than the product of the screen size and a first preset percentage; if so, executing S904.

[0178] The above process may be executed in the application process space.

[0179] S904: Notify the frame consumption module of the size of the buffer drawn by the first thread.

[0180] S905, the frame consumption module determines whether the size of the buffer drawn by the first thread is larger than the product of the target size and the second preset percentage. If so, execute S906.

[0181] S906: Determine that the first thread is a key frame thread.

[0182] It can be seen that this embodiment and Fig. 9 The embodiments shown and Figure 7 The difference between the embodiments shown is that this embodiment first compares the size of the buffer drawn by the first thread with the product of the screen size and a first preset percentage (which can be considered as the first level of screening). After the size of the buffer drawn by the first frame drawing thread is greater than the product of the screen size and the first preset percentage, it is also necessary to meet the condition that the size of the buffer drawn by the first thread is greater than the product of the target size and a second preset percentage (which can be considered as the second level of screening). Only then will the first thread be determined as a key frame drawing thread that affects the drawing of one frame. That is, the terminal device determines whether the first thread is a key frame drawing thread through double-layer screening. It should be understood that through the double-layer screening method, when there are multiple third-party frame drawing threads, compared Figure 6 Embodiments can improve the accuracy of identifying key drawing frame threads, compared to Figure 7 In the embodiment, since the frame consumption module on the system side no longer identifies all third-party frame drawing threads, but only identifies a part of the third-party frame drawing threads, the overhead on the system side can be reduced.

[0183] Furthermore, when it is determined that the first thread is a third-party key frame thread that affects the drawing of a frame, more computing power is allocated to the first thread, thereby reducing the time it takes for the third-party key frame thread to draw the buffer, to ensure that the drawing of a frame can be completed within the rated time, thereby improving the smoothness of the APP that is drawn by the third-party first frame thread during operation.

[0184] Above, three embodiments for determining key frame drawing threads are introduced. In implementation, in one example, if you only want to identify through the application, you can choose embodiment 1. If you want to identify through the system-application, you can use embodiments 2 and 3. However, compared with embodiment 2, embodiment 3 is more efficient: because when there are multiple frame drawing threads, all of them are handed over to the system at one time to identify which are the key frame drawing threads, which will reduce system consumption than first filtering through the application and then handing over some to the system for filtering. Fig.10 A schematic diagram of the structure of a processing device of a terminal device provided in one embodiment of the present application. Fig.10 As shown, the processing device 1000 includes a detection module 1001 and an identification module 1002 .

[0185] As an example, the detection module 1001 may be used to execute an operation of detecting whether the first thread calls the first system function, or the detection module 1001 may be used to execute an operation of detecting whether the system frame drawing thread calls the first system function.

[0186] As an example, the identification module 1002 can be used to perform Figure 6 The process of identifying whether the first thread is a key frame thread in the illustrated embodiment.

[0187] As another example, the identification module 1002 can be used to perform Figure 7 The process of identifying whether the first thread is a key frame thread in the illustrated embodiment.

[0188] As another example, the identification module 1002 can be used to perform Fig. 9 The process of identifying whether the first thread is a key frame thread in the illustrated embodiment.

[0189] Fig.11 A schematic diagram of the structure of a processing device of a terminal device provided in another embodiment of the present application. Fig.11 As shown, the processing device 1100 may include a processor 1101 and an interface circuit 1102. The processor 1101 and the interface circuit 1102 are coupled to each other. It is understood that the interface circuit 1102 may be a transceiver or an input-output interface. Optionally, the processing device 1100 may also include a memory 1103 for storing instructions executed by the processor 1101 or storing input data required by the processor 1101 to execute instructions or storing data generated after the processor 1101 executes instructions.

[0190] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0191] The method steps in the embodiments of the present application can be implemented by hardware, or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and the storage medium can also be present in a network device or a terminal device as discrete components.

[0192] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server, data center, etc. that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc; it may also be a semiconductor medium, for example, a solid-state hard disk.

[0193] In the various embodiments of the present application, unless otherwise specified or provided for in any logical conflict, the terms and / or descriptions between the different embodiments are consistent and may be referenced to each other, and the technical features in the different embodiments may be combined to form new embodiments according to their inherent logical relationships.

[0194] It is understood that the various numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.

Claims

1. A processing method of a terminal device, characterized in that: The terminal device includes a display, and the method includes: When it is detected that a first thread in the terminal device calls a first system function and the first thread is not a system frame drawing thread, determining that the first thread is a third-party frame drawing thread; The first system function is a function called after the frame drawing thread fills the drawn data into the applied buffer buffer, and the first system function is used to submit the buffer filled by the frame drawing thread to the buffer queue BufferQueue, wherein the terminal device displays the data in the buffer filled by the frame drawing thread on the display by consuming the BufferQueue.

2. The method according to claim 1, characterized in that After identifying the first thread as a third-party frame drawing thread, the method further includes: assigning a high priority to the first thread; Among them, the higher the priority assigned by the terminal device to the thread, the more computing resources the terminal device provides for the thread.

3. The method according to claim 2, characterized in that Before assigning a high priority to the first thread, the method further includes: Determine whether the first thread is a key frame thread; The allocating a high priority to the first thread specifically includes: When the first thread is a key frame drawing thread, assigning a high priority to the first thread; The key frame drawing thread is a frame drawing thread that can affect the delay of drawing a frame of picture.

4. The method according to claim 3, characterized in that The determining whether the first thread is a key drawing frame thread includes: Obtaining the screen size of the display; Determine whether the size of the buffer filled by the first thread is greater than the product of the screen size and a first preset percentage; If the size of the buffer filled by the first thread is larger than the product of the screen size and a first preset percentage, the first thread is determined to be a key frame thread.

5. The method according to claim 3, characterized in that: The determining whether the first thread is a key drawing frame thread includes: Acquire a target size, the terminal device displays a frame of picture within the target size, and the target size is smaller than the screen size of the display; Determine whether the size of the buffer filled by the first thread is greater than the product of the target size and a second preset percentage; If the size of the buffer filled by the first thread is larger than the product of the target size and a second preset percentage, the first thread is determined to be a key frame thread.

6. The method according to claim 3, characterized in that: The determining whether the first thread is a key drawing frame thread includes: Obtaining the screen size of the display; Determine whether the size of the buffer filled by the first thread is greater than the product of the screen size and a first preset percentage; If the size of the buffer filled by the first thread is larger than the product of the screen size and a first preset percentage, determining whether the size of the buffer filled by the first thread is larger than the product of a target size and a second preset percentage, the terminal device displays a frame within the target size, and the target size is smaller than the screen size of the display; If the size of the buffer filled by the first thread is larger than the product of the target size and a second preset percentage, the first thread is determined to be a key frame thread.

7. The method according to any one of claims 1 to 6, characterized in that The first system function includes any one of the following: a queue buffer queuebuffer function and an onFrameAvailable function.

8. The method according to any one of claims 1 to 6, characterized in that The method further comprises: When it is detected that a second thread in the terminal device calls a first system function and the second thread is a system frame drawing thread, a high priority is assigned to the second thread.

9. A processing device, characterized in that: The method comprises a functional module for implementing the method according to any one of claims 1 to 8.

10. A processing device, characterized in that: include: Memory and processor; The memory is used to store program instructions; The processor is configured to execute program instructions in the memory to implement the method according to any one of claims 1 to 8.

11. A terminal device, characterized in that: Comprising the device as claimed in claim 9 or 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code for computer execution, wherein the program code includes instructions for implementing the method according to any one of claims 1 to 8.

13. A computer program product, characterized in that The computer program product comprises instructions for implementing the method according to any one of claims 1 to 8.

Images