Sliding frame loss detection method, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a sliding frame loss detection method, electronic equipment, a storage medium and a program product, and relates to the technical field of terminals, and the method can obtain a frame loss detection result capable of reflecting frame loss continuity, and can more accurately analyze a lagging condition in a sliding process based on the frame loss detection result. The method is applied to the electronic equipment comprising a screen, and in the process that the electronic equipment responds to sliding operation to control page sliding in the screen, frame loss detection processing is carried out on a frame image generated in the page sliding process based on a set sliding window. And in the frame loss detection processing, marking the frame loss images, and analyzing the continuity between the frame loss images to obtain a frame loss detection result.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of terminal technology, and in particular to a sliding frame loss detection method, an electronic device, a storage medium, and a program product. Background Art

[0002] Since the display interface of the screen is limited, the information that can be presented to the user is also limited. At this time, the user's finger can slide on the screen of the electronic device, and the content on the screen can be changed through the user's sliding operation to browse more information. Multiple continuous frame images will be generated during the start and end of the sliding process to change the content displayed on the screen.

[0003] However, frame loss is prone to occur during the sliding process. The more frames are lost, the more obvious the freeze is. Electronic devices can detect frame loss and send the frame loss detection results to the cloud to analyze the freeze that occurs during the sliding process. However, the current frame loss detection results can only determine which frame of the image has frame loss, resulting in limited information expressed by the frame loss detection results, and cannot accurately analyze the freeze during the sliding process. Summary of the invention

[0004] The embodiments of the present application provide a sliding frame loss detection method, an electronic device, a storage medium, and a program product, which can obtain a frame loss detection result that can reflect the continuity of frame loss, and based on the frame loss detection result, can more accurately analyze and determine the frame loss situation and the jamming situation during the sliding process.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a sliding frame loss detection method is provided, which is applied to an electronic device, wherein the electronic device includes a screen. The electronic device can respond to a sliding operation on the screen to control a page in the screen to slide. During the page sliding process, the electronic device can perform frame loss detection processing on the frame images generated during the page sliding process based on a set sliding window, that is, mark the frame loss images (frame images with frame loss) in the sliding window, and analyze the continuity between the frame loss images to obtain a frame loss detection result.

[0007] In the above scheme, the sliding window can perform frame loss detection on the frame images generated during the sliding process, and mark the frame images where frame loss occurs during the sliding process. After the sliding operation is completed, it is determined whether there is continuous frame loss during the sliding process by judging whether the marked frame images are continuous, thereby obtaining a frame loss detection result that can reflect the continuity of frame loss. Based on the frame loss detection result, the frame loss situation and the freeze situation during the sliding process can be analyzed and determined more accurately.

[0008] In a possible implementation manner of the first aspect, the electronic device may determine the length of the sliding window based on data of historical single sliding operations, that is, determine the length of the sliding window according to the statistical duration of the single sliding operation. The statistical duration of the single sliding operation is obtained by statistically analyzing the duration of the historical single sliding operations.

[0009] In the above scheme, the length of the sliding window is determined by counting the duration of a single sliding operation, so that the determined sliding window length is more appropriate. Therefore, based on the more appropriate sliding window, frame loss detection processing can be performed more efficiently, and unnecessary processing due to inaccurate window length is avoided, thereby reducing resource consumption to a certain extent.

[0010] In another possible implementation of the first aspect, the electronic device will predetermine the correspondence between the duration of a single sliding operation and the probability value. According to the correspondence, the electronic device can select the duration when the probability value is greater than a preset probability threshold as the length of the sliding window.

[0011] In the above scheme, the corresponding relationship between the duration of a single sliding operation and the probability value makes it easier to find a suitable window length (i.e., the length of the sliding window) that can cover most sliding scenarios. Then, when the user slides the screen once, all frame images generated during the sliding process can be detected using a single window detection, and the frame loss detection result generated by a single window detection can be used to determine the jamming situation during the entire sliding process, which improves efficiency and avoids unnecessary resource consumption caused by multiple window detections.

[0012] In another possible implementation of the first aspect, after the page starts sliding, the electronic device can use the first frame loss image in the page sliding process as the detection starting point of the sliding window, and start this window detection, so that in this window detection, the frame image located in the sliding window can be subjected to frame loss detection processing.

[0013] In the above scheme, since there are many frames generated during the sliding process, but the number of frame images that can be detected by the sliding window at one time is limited, if the first frame image in the sliding process is used as the detection starting point of the sliding window, the number of detections of the sliding window during the sliding process will increase, wasting computing power. Therefore, using the first lost frame image during the page sliding process as the detection starting point of the sliding window can reduce the number of detections of the sliding window during the sliding process and reduce the waste of computing power.

[0014] In another possible implementation of the first aspect, if the current window detection is completed but the sliding operation is not completed, frame images will continue to be generated. Therefore, the electronic device can use the first frame image with frame loss in the frame images generated after the current window detection is completed as the detection starting point of the sliding window during the next window detection, and use the next window detection as the new current window detection, and continue to perform frame loss detection processing on the new frame images in the sliding window until the sliding operation is completed.

[0015] In the above scheme, when a window detection is completed but the sliding operation is not yet completed, the electronic device can continue to use the first lost frame image after the current window detection as the detection starting point of the sliding window, thereby avoiding incorporating the non-lost frame image after the current window detection into the sliding window, reducing the number of sliding window detections during the sliding process, and reducing the waste of computing power.

[0016] In another possible implementation of the first aspect, in the frame loss detection process, a judgment will be made as to whether each frame image is lost. Specifically, in the frame loss detection process, for each frame image to be detected, the electronic device can obtain the processing time of the frame image to be detected; the processing time of the frame image to be detected refers to the time actually consumed for processing the frame image to be detected, and the processing time of the frame image to be detected is compared with the frame period. When the difference between the two exceeds a preset time threshold, the frame image to be detected is determined to be a frame loss image. The frame period refers to the time required to generate a frame image under normal circumstances.

[0017] In the above scheme, whether the frame image is lost is determined by whether the difference between the processing duration and the frame period exceeds the preset duration threshold. This can more accurately determine whether the frame image is lost, and is also convenient for subsequent determination and analysis of the cause of frame loss through frame loss detection results.

[0018] In another possible implementation of the first aspect, the continuity between the lost frame images is analyzed to obtain detection information related to continuous frame loss. In addition to the detection information related to continuous frame loss, the frame loss detection result also includes at least one of the information of the lost frame image, global statistical information of the frame loss, a quantization value of the degree of jamming, or the length of the sliding window.

[0019] In the above scheme, the frame loss detection result includes information of multiple dimensions. Therefore, the frame loss detection result can be used to more comprehensively and holistically determine the frame loss situation during the sliding process, and more accurately determine the degree of jamming during the sliding process.

[0020] In another possible implementation of the first aspect, in the frame loss detection process, the electronic device may obtain the quantification value of the degree of jamming in the following manner: the electronic device may determine the quantification value of the degree of jamming based on the processing time of each frame loss image, the weight of each frame loss image, and the length of the sliding window. The weight of each frame loss image is positively correlated with the order of each frame loss image.

[0021] In the above scheme, each frame loss detection result obtained by the sliding window detection has a corresponding quantified value of the degree of jamming. The quantified value of the degree of jamming is determined according to the processing time and weight of the frame loss image in each sliding window detection process, so that the quantified value of the degree of jamming can measure the degree of jamming of the sliding scene from the dimension of user perception.

[0022] In another possible implementation of the first aspect, the quantified value of the jamming degree includes a normalized frame loss time. The electronic device may determine the normalized frame loss time according to a ratio between the second value and the length of the sliding window. The second value is a value obtained by weighted summing the processing time corresponding to each frame loss image according to the corresponding weight.

[0023] In the above scheme, the standardized frame loss time is determined by the ratio of the result of weighted summation of the processing time corresponding to each frame loss image according to the corresponding weight and the length of the sliding window. This is equivalent to combining the frame loss time of multiple frame loss images to more accurately quantify the degree of jamming, that is, the degree of jamming of the sliding scene can be more accurately measured from the dimension of user perception.

[0024] In another possible implementation of the first aspect, the electronic device may determine the weight of each frame image in the following manner: determine a sliding curve, where the sliding curve is used to characterize the pixel change of each frame image compared to the previous frame image during the page sliding process. In the sliding curve, the amount of pixel change of the frame image is negatively correlated with the order in which the frame images are generated during the page sliding process. The electronic device may determine the weight of each lost frame image based on the sliding curve and the order in which each lost frame image is generated during the page sliding process.

[0025] In the above scheme, since the pixel points of the frame image change greatly at the beginning of sliding, if frame loss occurs at this time, the user will be more likely to perceive the jamming, and the weight of the frame loss image is larger; at the end of sliding the screen, the pixel points of the frame image change less, if frame loss occurs at this time, the user is less likely to perceive the jamming, and the weight of the frame loss image is smaller. Therefore, by determining the weight of each frame loss image by the amount of pixel change of the frame image, the differentiated impact of different frame images on the jamming degree after frame loss can be taken into account. Therefore, the standardized frame loss time can more accurately measure the jamming degree of the sliding scene from the dimension of user perception.

[0026] In a second aspect, the present application provides an electronic device, the electronic device comprising a memory, a screen and a processor; the memory and the screen are coupled to the processor. The memory stores a computer program code, the computer program code comprises a computer instruction, and when the computer instruction is executed by the processor, the electronic device executes the method in the first aspect and any possible implementation thereof.

[0027] In a third aspect, an embodiment of the present application provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect and any possible implementation thereof.

[0028] In a fourth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method of the first aspect and any possible implementation thereof. The computer may be an electronic device in the second aspect and any possible implementation thereof.

[0029] It can be understood that the beneficial effects that can be achieved by the electronic device of the second aspect, the computer storage medium of the third aspect, and the computer program product of the fourth aspect provided above can be referred to as the beneficial effects in the first aspect and any possible implementation method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of the software architecture structure of an electronic device provided by a conventional method;

[0031] Figure 2 Schematic diagram of frame loss detection results provided by traditional methods;

[0032] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of the software architecture structure of an electronic device provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of a flow chart of a sliding frame loss detection method provided in an embodiment of the present application;

[0035] Figure 6 A schematic diagram of a cumulative distribution function curve provided in an embodiment of the present application;

[0036] Figure 7 A schematic diagram of a sliding interval provided in an embodiment of the present application;

[0037] Figure 8A schematic diagram of a frame loss type provided in an embodiment of the present application;

[0038] Fig. 9 A schematic diagram of a sliding curve provided in an embodiment of the present application;

[0039] Fig.10 A schematic diagram of a frame loss detection process provided in an embodiment of the present application;

[0040] Fig.11 Another schematic diagram of a sliding interval provided for an embodiment of the present application;

[0041] Fig.12 The embodiments of the present application provide Fig.11 Schematic diagram of the corresponding frame loss detection results. DETAILED DESCRIPTION

[0042] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0043] Before introducing specific embodiments, in order to facilitate understanding, some descriptions of concepts related to the embodiments of the present application are given by way of example for reference.

[0044] Frame refers to: a single picture that is the smallest unit in the interface display. A frame can be understood as a static picture. Displaying multiple connected frames quickly and continuously can create a dynamic effect of object movement.

[0045] A sliding scenario means that during the operation of an electronic device by a user, the user's finger can slide on the screen of the electronic device, and the content on the screen can be changed through the user's sliding operation. A specific sliding scenario may be: when the display interface of the electronic device shows the content of a certain application, since there is a lot of content to be displayed in the application and the screen interface is limited, the user can slide up and down on the screen to allow the user to view the content of the application that is not yet displayed on the screen. The sliding operation can be: swiping up, swiping down, etc. For example, swiping up can be: after the finger touches the bottom of the screen, it slides up from the bottom of the screen.

[0046] In the sliding scenario, when the electronic device receives the user's sliding operation, the content on the screen changes with the sliding operation. The electronic device first determines the force with which the user presses on the screen when sliding on the screen, and converts the force data into the initial sliding rate through a preset algorithm or function. At this time, the sliding begins, and the timing of the end of the sliding operation is determined by the preset damping coefficient, that is, when the sliding rate becomes 0, the sliding ends. In the process from the beginning to the end of the sliding operation (i.e., the sliding process), multiple continuous frame images will be generated, and multiple continuous frame images constitute a sliding interval. Among them, the frame images in the sliding interval have a sequence, and the frame images are displayed on the screen in sequence. The preset algorithm or function for converting the force data into the initial sliding rate, and the damping coefficient are set by the electronic device when it leaves the factory. The algorithms or functions and damping coefficients set by different manufacturers are different.

[0047] Therefore, when the user performs a sliding operation, the electronic device can generate multiple frame images and display the multiple frame images on the screen, so that the content on the screen changes, new content is presented on the screen, and a sliding effect of the screen is achieved.

[0048] It is understandable that it usually takes drawing, rendering, synthesis and other processes to generate a frame image. If the drawing, rendering, synthesis and other processes take a long time to generate a frame image, that is, the frame image processing time is long and exceeds the screen refresh cycle, some frames will not be displayed on the screen on time, then frame loss will occur, resulting in discontinuity in the animation or sliding process. Therefore, the more frames are lost, the more severe the interface freezes that users feel when the screen achieves the sliding effect.

[0049] In traditional methods, electronic devices can detect frame loss and send the frame loss detection results to the cloud, and then determine the cause of frame loss based on the frame loss detection results to reduce the occurrence of frame loss.

[0050] Take the mobile phone as an example, Figure 1 Schematic diagram of the software architecture structure of the electronic device provided by the traditional method.

[0051] like Figure 1 As shown, the layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the local service layer, and the kernel layer.

[0052] The application layer may include a series of application packages, which may include shopping applications, chat applications, browsers, short messages, etc.

[0053] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0054] like Figure 1 As shown, the application framework layer may include a scroller, a synchronization module, etc. The scroller may be OverScroller, which is an auxiliary class used to achieve smooth scrolling of the view, and OverScroller may be used to identify the start and end of sliding. The start of sliding refers to when the frame image on the screen begins to change, and the end of sliding refers to when the frame image on the screen stops changing. The synchronization module may be hiviewtunnel, which is used to report the frame loss detection results in the problem scenario. hiviewtunnel may obtain the frame loss detection results in the kernel layer and send the frame loss detection results to the cloud.

[0055] Among them, OverScroller is mainly used to realize the detection of sliding scenes.

[0056] The local service layer is located between the application framework layer and the kernel layer. The local service layer may include a layer integrator, a layer detector, and a fault detection module. The layer integrator is used to manage, synthesize, and render the graphical interface of the system to ensure that the interface of the application and the interface of the system can be displayed correctly and efficiently on the screen. The layer detector is used to detect frame images in different layers and record and store the frame loss detection results in the kernel layer. The fault detection module is used to count the frame loss detection results recorded in the kernel layer and transmit the frame loss detection results to the synchronization module in the application framework layer.

[0057] In some embodiments, the layer integrator may be SurfaceFlinger, the layer detector may be LayerDetector, and the fault detection module may be hiview.

[0058] Among them, the layer integrator and layer detector constitute the fluency detection model, which is mainly used to realize the fluency detection during the sliding process.

[0059] The kernel layer is the layer between hardware and software. The kernel layer includes drivers for various hardware.

[0060] In some embodiments, the kernel layer may include a storage driver, a digital signal processor driver, and an image processor driver, etc. Among them, the digital signal processor driver is used to drive the digital signal processor to process images, the image processor driver is used to drive the graphics processor to process images, and the storage driver is used to ensure effective access and control of the storage device, so that the layer detector can store the generated frame loss detection result in the / dev / hwlog_jank file, and the fault detection module can obtain the frame loss detection result in the / dev / hwlog_jank file.

[0061] The frame loss detection process in the traditional method is as follows:

[0062] After the electronic device receives the user's sliding operation, OverScroller in the application framework layer identifies the start and end of the sliding, and OverScroller then transmits the relevant information of the start and end of the sliding to SurfaceFlinger in the local service layer, and SurfaceFlinger then determines the layer where the frame image produced during the sliding process is located. LayerDetector in the local service layer performs frame loss detection on the frame image generated on the layer during the sliding process, for example: detecting the processing time of the frame image, that is, detecting the processing time of the drawing, rendering, synthesis and other processes when generating each frame image, and storing the frame loss detection results in the kernel layer. Finally, hiview in the local service layer counts the frame loss detection results in the kernel layer, and transmits the frame loss detection results of a single frame image to hiviewtunnel in the application framework layer. hiviewtunnel can upload the frame loss detection results of a single frame image to the cloud, so that developers can determine whether frame loss occurs during the sliding process and determine the cause of frame loss based on the frame loss detection results of a single frame image, and make corresponding improvements to reduce the occurrence of frame loss.

[0063] Based on the above description, it can be known that in the traditional frame loss detection process, only frame images among multiple frame images can be detected to have frame loss, but it is impossible to detect whether there is continuous frame loss between these frame loss images. For example: N consecutive frame images from frame 1 to frame N show the sliding effect, assuming that frame 2 and frame 3 have frame loss, the traditional solution can only determine that frame 2 and frame 3 have frame loss, but cannot determine that frame 2 and frame 3 are continuous frame loss.

[0064] For example, Figure 2 This is a schematic diagram of the frame loss detection results provided by the traditional method, such as Figure 2 As shown, the frame loss detection results obtained by the traditional method only obtain relevant information of each frame loss image (i.e., the frame image where frame loss occurs), such as: the layer where the frame loss image is located, the processing time of the frame loss image, the frame period and the information of the frame loss image.

[0065] Among them, the layer where the dropped frame image is located can refer to: on which application layer the dropped frame image is generated during the sliding process, that is: assuming that the page currently displayed on the screen is a browser page, when the user slides on the screen, a frame image will be generated on the browser layer during the sliding process. If the generated frame image has frame drops, the layer where the dropped frame image is located is the browser layer.

[0066] The frame period refers to the time required to generate a frame image under normal circumstances. The frame period can be obtained through the frame rate. For example, the frame period can be the reciprocal of the frame rate, that is, the frame period = 1 / frame rate. The frame rate refers to the frequency (rate) at which multiple frames appear continuously on the display. It can also be understood as the number of times the graphics processor refreshes the screen per second.

[0067] The information of the dropped frame image may include: the ID of the dropped frame image and the processing time of the dropped frame image, for example, the processing time in the rendering stage, the processing time in the synthesis stage, and the total processing time in the rendering, synthesis, and drawing stages, etc.

[0068] Now Figure 2 The following is an example to explain this. Figure 2 As shown, the layer where the frame loss image is located is the layer of shopping application A, and frame loss occurs in the frame images with ids 11366304, 11366326, and 11366331 during the sliding process, and the processing time in the rendering, synthesis, and drawing stages during the generation of these three frame images. The total processing time during the generation of the frame image with id 11366304 is 32 milliseconds, the total processing time during the generation of the frame image with id 11366326 is 33 milliseconds, and the total processing time during the generation of the frame image with id 11366331 is 25 milliseconds, and the frame period is 833333 milliseconds.

[0069] Therefore, according to the frame loss detection results obtained by the traditional scheme, it can be seen that the current detection method can only obtain the frame loss information of a single frame and cannot reflect the continuity of the frame loss.

[0070] However, if continuous frame loss occurs during the sliding process, the stuttering effect is more obvious. For example, N continuous frame images are generated during the sliding process, that is, frame 1, frame 2, frame 3, frame 4, frame 5... frame N. In the first case, frame 1 and frame 5 are lost, and in the second case, frame 1 and frame 2 are lost. In this case, the stuttering perceived by the user is more obvious than that in the first case.

[0071] However, the traditional method can only determine the frame loss information of the lost frame image, that is, the frame loss detection result can only determine which frame image has frame loss, and the amount of frame loss information obtained is limited. It is impossible to determine the overall frame loss situation during the sliding process. For example, it is impossible to determine whether there is continuous frame loss, the cumulative frame loss duration during the sliding process, the maximum number of continuous frame losses during the sliding process, etc. through the frame loss detection result. Therefore, it is impossible to accurately analyze the frame loss situation and the degree of jamming during the sliding process. For example, it is impossible to understand whether the frame loss situation affects the smoothness of the interface, whether the jamming during the sliding process is obvious, and thus developers cannot better find the cause of frame loss through the frame loss detection results.

[0072] To this end, in an embodiment of the present application, a sliding frame loss detection method is provided. In the process of page sliding in response to a sliding operation control screen, frame loss detection processing is performed on frame images generated during page sliding based on a set sliding window. In the frame loss detection processing, the frame loss images are marked, and the continuity between the frame loss images is analyzed to obtain a frame loss detection result. Among them, the frame loss image refers to a frame image with frame loss in the sliding window in the frame loss detection processing.

[0073] It can be understood that when the electronic device receives the user's sliding operation and slides the page on the screen, the sliding window can perform frame loss detection on the frame images generated during the sliding process, and mark the frame images where frame loss occurs during the sliding process. After the sliding operation is completed, it is determined whether there is continuous frame loss during the sliding process by judging whether the marked frame images are continuous, thereby obtaining a frame loss detection result that can reflect the continuity of frame loss. Based on the frame loss detection result, the frame loss situation and the freeze situation during the sliding process can be analyzed more accurately.

[0074] In some embodiments of the present application, the electronic device may perform frame loss detection on the frame images generated during the sliding process through a sliding window. It can be understood that for the entire sliding process of a sliding operation, several detections can be performed during the sliding process through the sliding window, and each detection is called a window detection. In each window detection, the frame images in the sliding window will be processed for frame loss detection. Specifically, in each window detection, the electronic device can mark the frame images in the sliding window where frame loss occurs (i.e., mark the frame loss image). After the current window detection is completed, the continuity between the marked frame images in this window detection is determined to obtain detection information related to continuous frame loss, and the detection information related to continuous frame loss obtained in this window detection is reported to the cloud as one of the information of the frame loss detection result.

[0075] Exemplarily, the electronic device may be a mobile phone, a tablet computer, a smart remote controller, a wearable device (such as a smart bracelet, a smart watch or smart glasses, etc.), a PDA, an augmented reality (AR) / virtual reality (VR) device. Alternatively, the electronic device 300 may also be other types of electronic devices such as a portable multimedia player (PMP), a media player, etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0076] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0077] Please refer to Figure 3 In this application, the electronic device 300 is taken as an example of a mobile phone to introduce the electronic device 300 provided by this application. Figure 3 As shown, the electronic device 300 may include: a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a sensor module 380, a button 390, an indicator 392, a camera 393, and a display screen 394. Among them, the sensor module 380 may include a pressure sensor 380A, a touch sensor 380B, etc.

[0078] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 300. In other embodiments of the present application, the electronic device 300 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware. For ease of understanding, the above components will be briefly introduced below.

[0079] The processor 310 may include one or more processing units, for example, the processor 310 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0080] In the embodiment of the present application, after receiving the sliding operation performed by the user on the display screen 394 through the pressure sensor 380A and the touch sensor 380B, the processor 310 generates a plurality of continuous frame images through the image processor, and displays these frame images on the display screen 394, so that new page content can be presented on the display screen, and the sliding effect on the display screen is achieved. The processor 310 can detect these frame images to determine whether there is continuous frame loss in these frame images.

[0081] The controller may be the nerve center and command center of the electronic device 300. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0082] The processor 310 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. The memory may store instructions or data that the processor 310 has just used or cyclically used. If the processor 310 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.

[0083] In some embodiments, the processor 310 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0084] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 300. In other embodiments of the present application, the electronic device 300 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0085] The charging management module 340 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 340 may receive charging input from a wired charger through the USB interface 330. In some wireless charging embodiments, the charging management module 340 may receive wireless charging input through a wireless charging coil of the electronic device 300. While the charging management module 340 is charging the battery 342, it may also power the electronic device through the power management module 341.

[0086] The power management module 341 is used to connect the battery 342, the charging management module 340 and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340, and supplies power to the processor 310, the internal memory 321, the external memory, the display screen 394, the camera 393, and the wireless communication module 360. The power management module 341 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 341 can also be set in the processor 310. In other embodiments, the power management module 341 and the charging management module 340 can also be set in the same device.

[0087] The wireless communication function of the electronic device 300 can be implemented through the antenna 1, the antenna 2, the mobile communication module 350, the wireless communication module 360, the modem processor and the baseband processor.

[0088] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 300 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0089] The mobile communication module 350 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 300. The mobile communication module 350 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 350 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 350 can be set in the processor 310. In some embodiments, at least some of the functional modules of the mobile communication module 350 can be set in the same device as at least some of the modules of the processor 310.

[0090] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 370A, a receiver 370B, etc.), or displays an image or video through a display screen 394. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 310 and be set in the same device as the mobile communication module 350 or other functional modules.

[0091] The wireless communication module 360 ​​can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 300. The wireless communication module 360 ​​can be one or more devices integrating at least one communication processing module. The wireless communication module 360 ​​receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 310. The wireless communication module 360 ​​can also receive the signal to be sent from the processor 310, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0092] In some embodiments, the antenna 1 of the electronic device 300 is coupled to the mobile communication module 350, and the antenna 2 is coupled to the wireless communication module 360, so that the electronic device 300 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS) and / or the satellite based augmentation system (SBAS).

[0093] The electronic device 300 implements the display function through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs, which execute program instructions to generate or change display information.

[0094] The display screen 394 is used to display images, videos, etc. The display screen 394 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 300 may include 1 or N display screens 394, where N is a positive integer greater than 1.

[0095] In an embodiment of the present application, a plurality of continuous frame images generated by the processor 310 through an image processor may be displayed through the display screen 394 .

[0096] The electronic device 300 can realize the shooting function through ISP, camera 393, video codec, GPU, display screen 394 and application processor.

[0097] The ISP is used to process the data fed back by the camera 393. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 393.

[0098] The camera 393 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 300 may include 1 or N cameras 393, where N is a positive integer greater than 1.

[0099] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 300 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0100] Video codecs are used to compress or decompress digital videos. The electronic device 300 may support one or more video codecs. Thus, the electronic device 300 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0101] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 300 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0102] The external memory interface 320 can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of the electronic device 300. The external memory card communicates with the processor 310 through the external memory interface 320 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0103] The internal memory 321 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 310 executes various functional applications and data processing of the electronic device 300 by running the instructions stored in the internal memory 321. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 300 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0104] The electronic device 300 can implement audio functions such as music playing and recording through the audio module 370, the speaker 370A, the receiver 370B, the microphone 370C, the headphone jack 370D, and the application processor.

[0105] The audio module 370 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 370 can also be used to encode and decode audio signals. In some embodiments, the audio module 370 can be arranged in the processor 310, or some functional modules of the audio module 370 can be arranged in the processor 310.

[0106] The speaker 370A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 300 can listen to music or listen to a hands-free call through the speaker 370A.

[0107] The receiver 370B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 300 receives a call or voice message, the voice can be received by placing the receiver 370B close to the human ear.

[0108] Microphone 370C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 370C to input the sound signal into the microphone 370C. The electronic device 300 can be provided with at least one microphone 370C. In other embodiments, the electronic device 300 can be provided with two microphones 370C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 300 can also be provided with three, four or more microphones 370C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0109] The earphone interface 370D is used to connect a wired earphone and can be a USB interface 330 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0110] The pressure sensor 380A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 380A can be set on the display screen 394. There are many types of pressure sensors 380A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 380A, the capacitance between the electrodes changes. The electronic device 300 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 394, the electronic device 300 detects the touch operation intensity according to the pressure sensor 380A. The electronic device 300 can also calculate the touch position according to the detection signal of the pressure sensor 380A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0111] The touch sensor 380B is also called a "touch panel". The touch sensor 380B can be arranged on the display screen 394, and the touch sensor 380B and the display screen 394 form a touch screen, also called a "touch screen". The touch sensor 380B is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 394. In other embodiments, the touch sensor 380B can also be arranged on the surface of the electronic device 300, which is different from the position of the display screen 394.

[0112] In the embodiment of the present application, the touch sensor 380B and the pressure sensor 380A can be combined to determine the start and end of the sliding operation. Specifically, after the touch sensor 380B receives the user's sliding operation, the pressure sensor 380A determines the force of the user pressing on the display screen 394 when sliding on the display screen 394, and the processor 310 converts the force data into the initial sliding rate, at which point the sliding starts. Subsequently, the timing of the end of the sliding operation is determined by the preset damping coefficient, that is, the end time of the user's sliding operation on the display screen 394 is determined.

[0113] The key 390 includes a power key, a volume key, etc. The key 390 may be a mechanical key or a touch key. The electronic device 300 may receive key input and generate key signal input related to user settings and function control of the electronic device 300.

[0114] In summary, in an embodiment of the present application, after the electronic device 300 determines the sliding operation performed by the user on the display screen 394 through the pressure sensor 380A and the touch sensor 380B, the processor 310 generates a plurality of continuous frame images and displays the plurality of frame images on the display screen 394, so as to change the content on the display screen 394 through the user's sliding operation, and detects continuous frame loss during the sliding process through the processor 310.

[0115] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0116] Figure 4 A schematic diagram of the software architecture structure of an electronic device provided in an embodiment of the present application.

[0117] In an embodiment of the present application, a continuous frame loss detection module may be added to the local service layer to implement a sliding frame loss detection method. The method in the embodiment of the present application is described in detail below in combination with the above hardware structure and system structure:

[0118] 1. The electronic device 300 receives a sliding operation from the user, the sliding starts, and the electronic device 300 starts to generate multiple continuous frame images and display these frame images on the display screen 394.

[0119] This step 1 is performed continuously. After the electronic device 300 receives the sliding operation performed by the user on the display screen 394 through the pressure sensor 380A and the touch sensor 380B, the processor 310 generates a plurality of continuous frame images and displays these frame images on the display screen 394, so that a sliding effect is generated on the display screen 394.

[0120] 2. The electronic device 300 performs frame loss detection on the frame image to determine the continuous frame loss during the sliding process.

[0121] This step 2 is carried out continuously. OverScroller in the application framework layer identifies the start and end of sliding, and OverScroller then transmits the relevant information of the start and end of sliding to SurfaceFlinger in the local service layer, and then SurfaceFlinger determines the layer where the frame image produced during the sliding process is located, and then LayerDetector in the local service layer detects the continuous frame loss of the frame image generated on the layer during the sliding process. The detection process can be: a continuous frame loss detection module is set in LayerDetector in the local service layer, and a sliding window is set through the continuous frame loss detection module. The sliding window detects the frame image generated during the sliding process, and collects the information of the frame loss image during each detection process. Subsequently, the sliding window reports the information of the frame loss image. Specifically, the electronic device detects the frame image generated during the sliding process. When the first frame loss image is detected, the first frame loss image is used as the detection starting point of the sliding window, and the current window detection is started. Subsequently, the frame image in the sliding window is subjected to frame loss detection processing. When the current window detection is completed, the frame loss detection result corresponding to the current window detection is obtained. If the current window detection is finished but the sliding operation is not finished, the first frame loss image after the current window detection is used as the detection starting point for the next window detection, and the next window detection is started. This cycle is repeated until the sliding operation is finished and the detection is stopped. The frame loss detection results obtained from each window detection are then stored in the kernel layer. For example, they can be written to the / dev / hwlog_jank file, and then transmitted to the hiviewtunnel in the application framework layer through the hiview in the local service layer, and then uploaded to the cloud by the hiviewtunnel, so that developers can determine whether continuous frame loss occurs during the sliding process and judge the degree of jamming during the sliding process based on the generated frame loss detection results.

[0122] The sliding frame loss detection method of the present application is described in detail below with reference to the accompanying drawings of specific embodiments.

[0123] Figure 5 A schematic flow chart of a sliding frame loss detection method provided in an embodiment of the present application, the method is applied to an electronic device, and specifically includes the following steps:

[0124] S501: Determine the length of the sliding window.

[0125] In some embodiments, the length of the sliding window can be determined based on the duration of a single sliding operation that is statistically analyzed. Among them, the duration of a single sliding operation that is statistically analyzed is obtained by statistically analyzing the duration of a single sliding operation in history. It can be understood that the duration of a single sliding operation that is statistically analyzed by big data may not be limited to only statistically analyzing the duration of a single sliding operation in the history of users using the electronic device, but is determined by statistically analyzing the duration of single sliding operations in the history of multiple users in the entire big data platform. In addition, the duration of a single sliding operation that is statistically analyzed by big data may also be determined by statistically analyzing the duration of a single sliding operation in the history of users using the electronic device. In this way, the length of the sliding window that is more suitable for different users' sliding habits can be determined, thereby making frame loss detection more accurate.

[0126] In some embodiments, determining the length of the sliding window according to the statistical duration of a single sliding operation includes: obtaining a preset corresponding relationship, the preset corresponding relationship refers to the corresponding relationship between the duration of the single sliding operation and the probability value; the probability value is used to characterize the probability of occurrence of the duration of the single sliding operation. According to the preset corresponding relationship, the duration whose probability value is greater than the preset probability threshold is selected as the length of the sliding window.

[0127] In this embodiment, the preset corresponding relationship can be represented by a sliding time curve. For example, the sliding time curve can be a cumulative distribution function curve, that is, the preset corresponding relationship between the duration of a single sliding operation and the probability value can be represented by a cumulative distribution function curve.

[0128] In some examples, Figure 6 A schematic diagram of a cumulative distribution function curve provided in an embodiment of the present application. Figure 6 The cumulative distribution function (CDF) curve of the duration of each screen sliding of the user obtained from the big data is shown in FIG. Figure 6 You can see the specific duration of a single screen slide by the user in Figure 6 The horizontal axis represents the duration of a single swipe by the user, and the vertical axis represents the probability value. Assume that the preset probability threshold is 90%, and Figure 6 It can be seen that when the user continuously slides the screen for less than or equal to 2 seconds, the probability value is 94%, which is greater than the preset probability value 90%. Therefore, the probability that the user slides the screen for less than 2 seconds is 94%, that is, the duration of the user sliding the screen is mostly less than 2 seconds. In the embodiment of the present application, the length of the sliding window can be set to 2 seconds.

[0129] It can be understood that in this embodiment, based on the statistical duration of a single sliding operation, the duration of a single sliding operation with a higher probability of occurrence can be selected as the length of the sliding window. Then, when the user slides the screen once, all frame images generated during the sliding process can be detected using a single window detection, and the frame loss detection result generated by the single window detection can be used to determine the sliding situation during the entire sliding process.

[0130] In other embodiments, the length of the sliding window may also be set by the user, or may be determined by the average duration of the user sliding the screen. In the embodiments of the present application, the length of the sliding window is not limited.

[0131] In other examples, the electronic device counts the duration of each screen sliding operation of the user within a preset time (i.e., the duration of a single sliding operation), and takes the average of the above-stated durations to obtain the average duration of the user sliding the screen, and then uses the average duration as the length of the sliding window. It should be noted that the electronic device performs statistical analysis on the duration of each screen sliding operation of the user to determine the length of the sliding window with the authorization of the user.

[0132] It can be understood that the length of the sliding window is used to determine the number of frame images for each window detection. That is, the frame images located in the same sliding window are the frame images that need to be detected in one window detection. It should be noted that since the length of the sliding window is determined from the time dimension, there may be a frame image at the end of the sliding window where only a part of it falls into the sliding window. In this case, the frame image that partially falls into the end of the sliding window and the other frame images that all fall into the sliding window can be used together as the frame images that need to be detected in this window detection.

[0133] S502: performing frame loss detection processing on frame images generated during the sliding process through a sliding window. In the frame loss detection processing, the frame loss images are marked, and the continuity between the frame loss images is analyzed to obtain a frame loss detection result.

[0134] In some embodiments of the present application, frame loss detection processing is performed on frame images generated during the page sliding process through a sliding window, including: after the page starts sliding, the first frame loss image in the page sliding process is used as the detection starting point of the sliding window, and this window detection is started, so that in this window detection, frame loss detection processing is performed on the frame image located in the sliding window.

[0135] It can be understood that the first frame loss image in the page sliding process can be used as the starting point of the sliding window, that is, the first frame loss image in the page sliding process is the first frame image of the first window detection, and the first frame loss image and a preset number of frame images after the first frame loss image can be used as frame images that need to be detected for frame loss in the first window detection.

[0136] For ease of understanding, we now combine Figure 7 How to determine the detection starting point of the sliding window is schematically described. Figure 7 This is a schematic diagram of a sliding interval provided in an embodiment of the present application. Figure 7 A total of 17 frames are shown in FIG. Figure 7 When the third frame image (i.e. Figure 7 When the third frame image is lost, the third frame image can be used as the detection starting point of this sliding window, that is, the third frame image is used as the first frame image to be detected in this sliding window. Figure 7 It can be seen that the thirteenth frame image (i.e. Figure 7 If a part of the frame image (the frame image marked with number 13 in the figure) falls within the sliding window, the thirteenth frame image can be used as the frame image that needs to be detected for frame loss in this window detection, that is, after completing the detection of the thirteenth frame image, the frame loss detection results of the frame images between the third frame image and the thirteenth frame image are reported as the frame loss detection results of this window detection.

[0137] In some embodiments, a large number of frames are generated during the sliding process, but the number of frame images that can be detected by the sliding window at one time is limited. If the first frame image generated during the sliding process is used as the starting point for the sliding window detection, or the sliding window is used to detect all the frame images during the sliding process, the number of detections of the sliding window during the sliding process will increase. Then, the number of statistics on the frame loss detection data will also increase. However, the non-frame loss images during the sliding process will not affect the jamming during the sliding process. Therefore, without affecting the jamming degree analysis, the first frame loss image during the page sliding process can be used as the starting point of the sliding window, reducing the number of detections of the sliding window and reducing the number of statistics on the frame loss detection data, thereby reducing computing power.

[0138] In each window detection, the detection information related to the continuous frame loss can be obtained by analyzing the continuity between the frame loss images. In some embodiments of the present application, in addition to the detection information related to the continuous frame loss, the frame loss detection result obtained by the sliding window detection may also include at least one of the information of the frame loss image, the global statistical information of the frame loss, the first value or the length of the sliding window (which can be represented by detection ion Duration). That is, in each window detection, in addition to analyzing the continuity between the frame loss images, the electronic device can also calculate the first value, calculate the information of the frame loss image, and perform statistics on the global information of the frame loss. The first value is a quantization value of the degree of jamming, which can also be referred to as the quantization value of the degree of jamming.

[0139] In some embodiments of the present application, the detection information related to continuous frame loss may include: frame loss type (which can be represented by ARG3), the longest number of continuous frame losses (which can be represented by longestContinuousDropFrames), the longest continuous frame loss duration, etc.

[0140] in, Figure 8 A schematic diagram of a frame loss type provided in an embodiment of the present application, in Figure 8 There are two types of frame loss: Figure 8 There is no frame image with continuous frame loss in the sliding window shown in (a). Figure 8 (b) in the figure shows a frame image with continuous frame loss in the sliding window. In the embodiment of the present application, the frame loss type can be represented by a numerical value. For example, 0 and 1 can be used to represent the frame loss type during the sliding process. When the numerical value is 0, it means that there is no frame image with continuous frame loss in the sliding window (i.e., there is no continuous frame loss). When the numerical value is 1, it means that there is a frame image with continuous frame loss in the sliding window (i.e., there is continuous frame loss).

[0141] The longest continuous frame loss number may refer to the maximum number of continuous frame loss when there is continuous frame loss in the sliding window. Figure 8 Taking (b) in the figure as an example, there are two consecutive frame losses in the sliding window, so the longest number of consecutive frame losses is 2.

[0142] The longest continuous frame loss duration may refer to the sum of the longest continuous frame loss frame image processing duration (i.e., the duration of generating frame images) when continuous frame loss exists in the sliding window. Figure 8 Taking (b) in FIG. 1 as an example, assuming that the processing time of two consecutive frame loss images is 32 milliseconds and 35 milliseconds respectively, the longest continuous frame loss time is 57 milliseconds.

[0143] In some embodiments of the present application, the information of the lost frame image may include: the layer where the lost frame image is located (which may be represented by ARG1), related information of the lost frame image (which may be represented by FrameInfo), for example: attribute information such as the ID of the lost frame image and the processing time of each stage of the lost frame image. The processing time of each stage of the lost frame image may include: the processing time of drawing, rendering, synthesis, etc. in the process of generating the lost frame image.

[0144] In some embodiments of the present application, the global statistical information of frame loss may include: the cumulative frame loss duration (which can be represented by totalLostFrameTime), the number of lost frames (which can be represented by lostFrameCount), etc.

[0145] The cumulative frame loss duration may refer to the processing duration of all frame loss images within the statistical sliding window. Figure 8 Taking (a) in the figure as an example, there are 5 frame-dropping images in the sliding window. Then, determine the duration of generating these 5 frame-dropping images, that is, the processing duration of generating these 5 frame-dropping images, and then add up the processing durations corresponding to the 5 frame-dropping images to obtain the cumulative frame-dropping duration.

[0146] The number of dropped frames can refer to the number of dropped frame images in the statistical sliding window. Figure 8 Taking (a) in the figure as an example, the number of lost frames in the sliding window is 5.

[0147] In some embodiments of the present application, the frame loss detection result may further include a first value.

[0148] In traditional methods, some parameters for measuring the degree of jamming during sliding are different from the degree actually perceived by the user. In an embodiment of the present application, the degree of jamming during sliding can be accurately analyzed and quantified based on the frame loss detected in the sliding window. That is, the first numerical value can be obtained based on the frame loss detection result reported by the sliding window, and the quantified result is close to the degree of jamming actually perceived by the user.

[0149] Therefore, the first value can measure the degree of freeze during the user's screen sliding process from the dimension of user perception. Among them, the more frame loss, the more obvious the freeze of the screen display process can be perceived by the user. Therefore, the freeze perceived by the user during the screen display process can be reflected by the frame loss situation.

[0150] When the first value is greater than the preset threshold, it means that the freeze caused by the user's current screen sliding process can be perceived by the user. When the first value is less than or equal to the preset threshold but greater than 0, it means that frame loss occurs during the user's current screen sliding process, but the freeze caused by the frame loss is not obvious and is not easily perceived by the user.

[0151] For ease of understanding, the first value is now described in conjunction with a more specific example. For example, the preset threshold can be set to 0.25. If the first value corresponding to the current user sliding the screen is 0.56, it means that the freeze during the current user sliding the screen can be perceived by the user. If the first value corresponding to the current user sliding the screen is 0.06, it means that the freeze during the current user sliding the screen is not easily perceived by the user.

[0152] In this embodiment, the first value may be determined in a manner that: in the frame loss detection process, the first value is determined based on the processing time of each frame loss image, the weight of each frame loss image, and the length of the sliding window, wherein the weight of each frame loss image is positively correlated with the order of each frame loss image.

[0153] It can be understood that each frame loss detection result obtained by the sliding window detection has a corresponding first value. The first value is determined according to the processing time and weight of the frame loss image in each sliding window detection process, so that the first value can reflect the freeze situation perceived by the user in each sliding window detection process.

[0154] In some embodiments, the first value may be a standardized frame loss time, and the first value is determined based on the processing time of each frame loss image, the weight of each frame loss image, and the length of the sliding window, including: determining the standardized frame loss time according to the ratio between the second value and the length of the sliding window. The second value is a value obtained by weighted summing the processing time corresponding to each frame loss image according to the corresponding weight.

[0155] The specific calculation formula can be: Where n is the number of dropped frame images in the sliding window, t i is the processing time of each lost frame image, τ i is the weight of each lost frame image, which is calculated according to the sliding curve, T is the length of the sliding window, T d is the normalized frame drop time.

[0156] Furthermore, the calculation method of the weight of each lost frame image may include: determining a sliding curve, which is used to characterize the pixel change of each frame image compared with the previous frame image during the page sliding process. In the sliding curve, the amount of pixel change of the frame image is negatively correlated with the order in which the frame images are generated during the page sliding process. Based on the sliding curve and the order in which each lost frame image is generated during the page sliding process, the weight of each lost frame image is determined.

[0157] It is understandable that at the beginning of sliding, due to the faster initial sliding rate, the changes in the frame images are more obvious, and the pixel changes of each frame image compared to the previous frame image are also more obvious. Therefore, at the beginning of sliding, if frame loss occurs at this time, users will be more likely to perceive the jamming, and the weight of the frame loss image is larger. As the sliding gradually stops, the frame image on the screen no longer changes, that is, it may stay on a certain frame image. At this time, the pixel points of the frame image no longer change, and the pixel changes of each frame image compared to the previous frame image are less and less obvious. Therefore, at the end of the user sliding the screen, the pixel points of the frame image change less, and the sliding rate gradually slows down. At this time, if frame loss occurs, users are less likely to perceive the jamming, and the weight of the frame loss image is smaller.

[0158] Therefore, the sequence of frame images generated during the sliding process can be numbered. The earlier the frame number, the greater the weight of the frame image, and the later the frame number, the smaller the weight of the frame image.

[0159] In some examples, the sliding curve of the electronic device may be determined first, and then the frame number of the lost frame image during the sliding process may be determined, and the weight corresponding to the frame number of the frame image may be determined based on the sliding curve. Fig. 9 A schematic diagram of a sliding curve provided in an embodiment of the present application, such as Fig. 9 As shown in Figure 2, the sliding curve is a decreasing exponential function. Fig. 9 The horizontal axis represents the sequence number of the frame image, and the vertical axis represents the number of pixels changed in the frame image. The number of pixels changed in the frame image represents the weight of the frame image.

[0160] In some embodiments, the sliding curve formula may be y = a·e bx , and the sliding curve is determined by the manufacturer of the electronic device, that is, different electronic devices may have different values ​​of a and b. For example, a=16.317, b=-0.046.

[0161] In some embodiments of the present application, one window detection may not be able to complete the detection of all frame images generated during the sliding process, so multiple (at least two) window detections are required. Specifically, if the current window detection is completed but the sliding operation is not completed, the electronic device can use the first frame loss image after the current window detection as the detection starting point of the sliding window during the next window detection, and use the next window detection as the new current window detection, and return to execute the frame loss detection process for the frame images located in the sliding window in the current window detection until the sliding operation is completed. Among them, the first frame loss image after the current window detection refers to the first frame image with frame loss in the frame images generated after the current window detection is completed.

[0162] Exemplarily, in the process of using a sliding window to sequentially detect the frame images generated during the sliding process, the first frame loss image (i.e., the frame image where the frame loss occurs for the first time) can be determined as the detection starting point of this window detection, that is, the frame loss image is used as the first frame image to be detected in this sliding window, and after the frame loss image is marked, the frame loss of subsequent frame images in the sliding window is continuously detected until the frame images in the sliding window are detected, and then it is determined that the current window detection is over. Furthermore, the electronic device can report the frame loss detection result of this window detection to the cloud, and use the sliding window to start the next window detection, that is, determine the detection starting point of the sliding window during the next window detection, and continue to perform frame loss detection processing on the frame images in the sliding window, which will not be described in detail.

[0163] For ease of understanding, multiple window detection is schematically illustrated below by taking an example.

[0164] Assume that the duration of the user's sliding operation is 3 seconds, that is, the process from the start to the end of the sliding is 3 seconds, and 20 continuous frame images are generated within 3 seconds, forming a sliding interval, and the sliding window starts to detect from the first frame image. Assuming that the first frame image is lost, the first frame image is used as the detection starting point of this window detection, that is, as the first frame image detected in this sliding window. Assuming that after the detection of the eleventh frame image is completed, the position of the frame image currently detected by the sliding window reaches the length of the preset sliding window from the detection starting point, it can be determined that this window detection is over. The electronic device can generate the frame loss detection results of the frame images between the first frame image and the eleventh frame image. At this time, if the sliding operation has not ended, the twelfth frame image to the twentieth frame image will continue to be generated, then the electronic device can continue to determine whether the frame images after the twelfth frame image are lost. Assuming that there is no frame loss from the twelfth to the sixteenth frame image, but there is a frame loss in the seventeenth frame image, the seventeenth frame image is used as the detection starting point of the sliding window during the second window detection, and the frame images after the seventeenth frame image are detected. After the sliding window completes the detection of the twentieth frame image, the sliding operation ends, and the sliding window also ends the detection. It can be understood that during the sliding process, the sliding window generates two frame loss detection results, and the detection starting point of the second detection of the sliding window is the first frame loss image after the first detection of the sliding window, that is, the seventeenth frame image, and the end time of the second detection of the sliding window is determined by the end time of the sliding operation.

[0165] It should be noted that in the embodiment of the present application, the electronic device performs frame loss detection in real time during the sliding operation of the user. That is, while the electronic device receives the sliding operation of the user, the electronic device also generates multiple continuous frame images based on the sliding operation of the user, and while these frame images are displayed on the screen, the electronic device also synchronously performs frame loss detection on the displayed frame images.

[0166] Fig.10 A schematic diagram of a frame loss detection process provided in an embodiment of the present application, in Fig.10 In the above example, the detection process of the sliding window can be:

[0167] (1) Slide to start.

[0168] After the sliding operation begins, the electronic device generates continuous frame images in real time and displays them on the screen in response to the user's sliding operation.

[0169] (2) Collect statistics on the current frame.

[0170] The electronic device uses a sliding window to determine whether the current frame image is lost in sequence according to the order of the frame images displayed on the screen. That is, the electronic device can Fig.10 The information of the "current frame" in (3) is counted, for example, the processing time of the current frame image is counted. It can be understood that the information of the current frame image can be used as a basis for determining whether a frame is lost in (3).

[0171] (3) Determine whether the current frame is lost.

[0172] In some embodiments, it is possible to determine whether a frame is lost based on the information of the current frame image counted in (2). Specifically, the method for determining whether a frame image is lost may include: in the frame loss detection process, for each frame image to be detected, obtaining the processing time of the frame image to be detected. The processing time of the frame image to be detected refers to the time actually consumed for processing the frame image to be detected. When the difference between the processing time and the frame period exceeds a preset time threshold, the frame image to be detected is determined to be a frame loss image. The preset time threshold may be 25 milliseconds. It can be understood that the time actually consumed for processing the frame image to be detected may refer to the time generated by the process of generating the frame image. The process of generating the frame image may include drawing, rendering, and synthesis processes. Therefore, the processing time of the frame image to be detected may include the time generated by the drawing, rendering, and synthesis processes when generating the frame image.

[0173] If the current frame image has no frame loss, it is determined whether the detection of the sliding window has started. If not (i.e., the detection of the sliding window has not started), the electronic device can perform frame loss detection on the next frame image of the current frame image, that is, take the next frame image as the new current frame image, and continue to return to perform the statistics of the current frame information and subsequent steps; if yes (i.e., the detection of the sliding window has been detected), the electronic device can execute (4) to determine whether the detection of this sliding window has ended.

[0174] If the current frame image is frame-dropped, the electronic device can determine whether the frame-dropped image is the first frame-dropped image detected by the current sliding window, that is, the first frame-dropped image detected by the sliding window to be started. If it is the first frame-dropped image, the electronic device can use the frame-dropped image as the starting point of the sliding window (also called the detection starting point of the current sliding window detection), that is, the frame image is the first frame image located in the sliding window during the current window detection, and the frame image and a preset number of frame images after the frame image are used as the frame images detected in the current sliding window to detect whether the frame images in the sliding window are frame-dropped, and the frame images with frame-dropped conditions are marked as frame-dropped (that is, the frame-dropped images are marked). Then, the electronic device can execute (4) to determine whether the detection of the current sliding window is completed.

[0175] (4) Determine whether the detection of this sliding window is completed.

[0176] If the detection of this sliding window has ended, the frame loss detection result of the sliding window is reported, and the frame loss detection result is cleared to prepare for the next window detection.

[0177] It can be understood that after preparing to start the next window detection and before officially starting the next window detection, the electronic device can determine whether the sliding is completed, that is, execute (5).

[0178] (5) Determine whether the sliding is completed.

[0179] If the sliding has not ended, a frame loss detection is performed on the next frame image of the current frame image, that is, the next frame image is used as the new current frame image, and the process returns to execute (2) to count the information of the current frame and subsequent steps.

[0180] If the sliding is completed, execute (6).

[0181] (6) Determine whether there is any reported frame loss detection result.

[0182] If there is a reported frame loss detection result, the frame loss detection result is reported, and then the continuous frame loss detection is terminated.

[0183] If there is no reported frame loss detection result, the continuous frame loss detection is terminated directly.

[0184] The following is a specific example to introduce the execution process of the sliding frame loss detection method of the present application.

[0185] Take swiping on a page in shopping app A as an example. Fig.11 Another sliding interval schematic diagram provided for an embodiment of the present application is as follows: Fig.11 As shown, when the electronic device receives a sliding operation of the user on the page of the shopping application A, the page of the shopping application A will be controlled to change according to the sliding operation. Therefore, 15 consecutive frame images will be generated from the beginning to the end of the sliding, and the 15 frame images will constitute a sliding interval. The electronic device can first determine the processing time for generating the first frame image, and then determine that the difference between the processing time for the first frame image and the frame period is less than 25 milliseconds, so the first frame image has no frame loss.

[0186] Since the sliding operation has not ended, the processing time for generating the second frame image is further determined. At this time, the difference between the processing time of the second frame image and the frame period exceeds 25 milliseconds, that is, the frame image with id 11366304 is detected to be lost, and the second frame image is determined to be lost. The electronic device can use the second frame image as the detection starting point of this sliding window, that is, use the second frame image as the first frame image to be detected in this sliding window, and start the detection of this sliding window. The electronic device can mark the second frame image again to indicate that the second frame image is lost.

[0187] Subsequently, since the window detection has not been completed and the sliding operation has not been completed, the electronic device can continue to determine the processing time for generating the third frame image, and so on, until determining the processing time for generating the sixth frame image. Among them, the difference between the processing time and the frame period of the third frame image and the sixth frame image is less than 25 milliseconds, so the third frame image and the sixth frame image are not lost.

[0188] Subsequently, since the detection of the sliding window has not yet ended, the processing time for generating the seventh frame image is further determined. At this time, the difference between the processing time of the seventh frame image and the frame period exceeds 25 milliseconds, and it is determined that the seventh frame image is lost, that is, the frame image with ID 11366326 is detected to be lost. The electronic device can mark the seventh frame image to indicate that the seventh frame image is lost.

[0189] Subsequently, since the detection of the sliding window has not yet ended and the sliding operation has not yet ended, the electronic device can continue to determine the processing time for generating the eighth frame image. At this time, the difference between the processing time of the eighth frame image and the frame period exceeds 25 milliseconds, and it is determined that the eighth frame image is lost, that is, the frame image with id 11366331 is detected to be lost. The electronic device can mark the eighth frame image to characterize the frame loss of the eighth frame image. Since both the seventh frame image and the eighth frame image are lost, and the frame images are in a continuous relationship, the seventh frame image and the eighth frame image are continuous frame losses, and the seventh frame image and the eighth frame image are marked as continuous frame losses in this window detection.

[0190] Until the position of the detected frame image exceeds the length of the sliding window from the detection starting point, it indicates that the current window detection is over. The electronic device can then analyze the frame image targeted by this window detection to obtain the frame loss detection result. For example, the electronic device can analyze the continuity between the frame loss images marked in the sliding window during this window detection, obtain the frame loss detection result and report it to the cloud.

[0191] After the current window detection is completed, since the sliding operation has not yet ended, after the frame loss detection result of the current window detection is reported, the reporting record is cleared, and the processing time of the next frame image generated during the sliding process is continued to be determined. If frame loss occurs again, the frame loss image is used as the detection starting point for the next window detection, and the next window detection is performed. This cycle is repeated until the sliding operation ends.

[0192] Fig.12 The embodiments of the present application provide Fig.11 Schematic diagram of the corresponding frame loss detection result. Since the frame images generated during the sliding process are all generated on the layer of the shopping application A, the layer where the frame loss images detected by the sliding window are located is the layer of the shopping application A.

[0193] The electronic device can calculate the standardized frame loss time corresponding to the sliding window. Specifically, the sliding curve formula can be set as y = a·e bx , a=16.317, b=-0.046, and the value is 0.56.

[0194] Depend on Fig.11It can be seen that since the seventh and eighth frame images in the frame loss detection results reported by the sliding window have continuous frame loss, the value of the frame loss type is 1, indicating that there are frame images with continuous frame loss in the sliding window. Since the second frame image also loses frames, the number of frame losses in the sliding window is 3 (i.e., the second frame image, the seventh frame image, and the eighth frame image have frame losses). The electronic device can add the frame loss time of the second frame image, the seventh frame image, and the eighth frame image as the cumulative frame loss time, i.e., 90 milliseconds. The length of the sliding window is 2000 milliseconds (i.e., 2 seconds). The information of all frame images with frame loss includes the frame IDs corresponding to the second frame image, the seventh frame image, and the eighth frame image, as well as the processing time of the drawing, rendering, synthesis, etc. processes during the generation of the second frame image, the seventh frame image, and the eighth frame image.

[0195] In other embodiments of the present application, assuming that the sliding window performs at least two window detections and obtains at least two frame loss detection results, the frame loss detection results corresponding to the at least two window detections can be analyzed simultaneously to determine whether there is continuous frame loss between the frame images detected in the critical position in any two window detections, and then the analysis results are uploaded to the cloud. For example: in the first window detection, frame loss detection processing is performed on the second to fifth frame images generated during the sliding process, wherein the second frame image and the fifth frame image are both frame loss images, and in the first window detection, frame loss detection processing is performed on the sixth to ninth frame images generated during the sliding process, wherein the sixth frame image is a frame loss image. It can be seen that the fifth frame image and the sixth frame image are frame images at the critical position, and from the two frame loss detection results generated by the sliding window, it can be seen that there is a continuous frame loss relationship between the fifth frame image and the sixth frame image.

[0196] S503: Report the frame loss detection result.

[0197] In an embodiment of the present application, hiview can be used to count the frame loss detection results generated during the sliding window detection process, and the frame loss detection results can be transmitted to hiviewtunnel, and then uploaded to the cloud by hiviewtunnel.

[0198] Some other embodiments of the present application provide an electronic device, which may include: the above-mentioned screen (such as a touch screen, a display screen), a memory, and one or more processors. The screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device performs each function or step in the above-mentioned method embodiment. The structure of the electronic device can refer to Figure 3 The structure of the electronic device 300 is shown.

[0199] An embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step executed by the electronic device in the above-mentioned method embodiment.

[0200] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes each function or step executed by the electronic device in the above method embodiment.

[0201] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0202] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0203] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0204] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0205] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0206] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A sliding frame loss detection method, characterized in that: The method is applied to an electronic device, wherein the electronic device includes a screen, and the method includes: In the process of controlling the page sliding in the screen in response to the sliding operation, performing frame loss detection processing on the frame images generated in the process of page sliding based on the set sliding window; In the frame loss detection process, the frame loss images are marked, and the continuity between the frame loss images is analyzed to obtain the frame loss detection result; The frame loss image refers to a frame image in which frame loss occurs within the sliding window during the frame loss detection process.

2. The method according to claim 1, characterized in that Before performing frame loss detection processing on the frame images generated during page sliding based on the set sliding window, the method further includes: The length of the sliding window is determined according to the statistical duration of a single sliding operation; the statistical duration of a single sliding operation is obtained by statistically analyzing the duration of historical single sliding operations.

3. The method according to claim 2, characterized in that Determining the length of the sliding window according to the statistical duration of a single sliding operation includes: Obtaining a preset corresponding relationship; the preset corresponding relationship refers to the corresponding relationship between the duration of a single sliding operation and a probability value; the probability value is used to characterize the probability of occurrence of the duration of the single sliding operation; According to the preset corresponding relationship, the duration in which the probability value is greater than the preset probability threshold is selected as the length of the sliding window.

4. The method according to any one of claims 1 to 3, characterized in that The frame loss detection process of the frame image generated during the page sliding process based on the set sliding window includes: After the page starts to slide, the first frame loss image in the page sliding process is used as the detection starting point of the sliding window, and the current window detection is started, so that in the current window detection, the frame image located in the sliding window is subjected to frame loss detection processing.

5. The method according to claim 4, characterized in that The method further comprises: If the current window detection is completed but the sliding operation is not completed, the first frame loss image after the current window detection is used as the detection starting point of the sliding window in the next window detection, and the next window detection is used as the new current window detection, and the frame loss detection process is performed on the frame image located in the sliding window in the current window detection until the sliding operation is completed; The first frame-loss image after the current window detection refers to the first frame image with frame loss among the frame images generated after the current window detection is completed.

6. The method according to claim 1, characterized in that In the frame loss detection process, marking the frame loss image includes: In the frame loss detection process, for each frame image to be detected, the processing time of the frame image to be detected is obtained; the processing time of the frame image to be detected refers to the time actually consumed for processing the frame image to be detected; When the difference between the processing time and the frame period exceeds a preset time threshold, the frame image to be detected is determined to be a lost frame image; the frame period refers to the time required to generate a frame image under normal circumstances.

7. The method according to any one of claims 1 to 6, characterized in that The analysis of the continuity between the frame loss images is used to obtain detection information related to continuous frame loss. In addition to the detection information related to continuous frame loss, the frame loss detection result also includes at least one of the information of the frame loss images, global statistical information of frame loss, a first value or the length of the sliding window.

8. The method according to any one of claims 1 to 6, characterized in that The frame loss detection result includes a first value; and the method further includes: In the frame loss detection process, the first value is determined based on the processing time of each of the frame loss images, the weight of each of the frame loss images and the length of the sliding window; wherein the first value is a quantified value of the degree of freeze; wherein the weight of each of the frame loss images is positively correlated with the order of each of the frame loss images.

9. The method according to claim 8, characterized in that The first value includes a normalized frame drop time; The determining the first value based on the processing time of each of the lost frame images, the weight of each of the lost frame images and the length of the sliding window includes: Determining the standardized frame loss time according to a ratio between the second value and the length of the sliding window; The second value is a value obtained by weighted summing the processing times corresponding to the respective lost frame images according to corresponding weights.

10. The method according to claim 8, characterized in that Before determining the first value based on the processing time of each of the lost frame images, the weight of each of the lost frame images and the length of the sliding window, the method further includes: Determine a sliding curve, wherein the sliding curve is used to characterize the pixel change of each frame image compared with the previous frame image during the page sliding process; wherein, in the sliding curve, the pixel change amount of the frame image is negatively correlated with the order in which the frame images are generated during the page sliding process; The weight of each of the lost frame images is determined based on the sliding curve and the order in which each of the lost frame images is generated during the page sliding process.

11. An electronic device, characterized in that: The electronic device includes a memory, a screen and a processor; the memory and the screen are coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the method as described in any one of claims 1 to 10.

12. A computer storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method according to any one of claims 1 to 10.

13. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 10.

Images