Video lag detection method and related device

Abstract

The embodiment of the invention provides a video lag detection method and a related device, and relates to the technical field of terminals. The method comprises the steps that the electronic equipment displays a video playing interface, the video playing interface comprises a first control and a second control, the first control is used for displaying a first type of view, the second control is used for displaying a second type of view, and the first type of view does not comprise a view locally loaded from the electronic equipment; the second class of views comprise views locally loaded from the electronic equipment; when the frame interval of the view displayed in the first control is greater than or equal to a preset value, the electronic equipment determines that the video is blocked; wherein the frame interval of the view displayed in the second control is smaller than, equal to or larger than a preset value. In this way, the accuracy of judging whether the video application interface is lagged or not can be improved, and lagging of the video application interface can be reduced.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a video freeze detection method and related devices. Background Art

[0002] With the development of electronic technology, some electronic devices support video playback. Users can watch videos, shoot videos, and start live broadcasts through video applications. However, when the electronic device plays the content of the video application, the interface of the electronic device may become stuck, affecting the user experience. Summary of the invention

[0003] The video freeze detection method and related devices provided in the embodiments of the present application enable the electronic device to perform frame rate refresh detection on drawing controls that do not load local data when detecting freezes in the video application interface, thereby not being affected by controls that load local data, improving the accuracy of determining whether the video application interface is freezed, and further helping to reduce freezes in the video application interface.

[0004] In a first aspect, the video freeze detection method provided by the embodiment of the present application includes:

[0005] The electronic device displays a video playback interface, the video playback interface includes a first control and a second control, the first control is used to display a first type of view, the second control is used to display a second type of view, the first type of view does not include a view loaded locally from the electronic device; the second type of view includes a view loaded locally from the electronic device; when the frame interval of the view displayed in the first control is greater than or equal to a preset value, the electronic device determines that the video is stuck; wherein the frame interval of the view displayed in the second control is less than, equal to, or greater than the preset value. In this way, the accuracy of determining whether the video application interface is stuck can be improved, thereby helping to reduce the stuckness of the video application interface.

[0006] In a possible implementation, before the electronic device determines that the video is stuck, the electronic device also includes: the electronic device determines that no preset event occurs in the video playback interface, and the preset event includes one or more of the following: an event in which a progress bar control in the video playback interface is triggered, an event for indicating that the video playback interface is paused, and an event for indicating that the video playback interface is switched to the background. In this way, it can be determined whether the current interface is paused due to an event triggered by the user, and thus it can be more accurately determined whether the interface is stuck.

[0007] In a possible implementation, the preset value includes a first threshold value, and before the electronic device determines that the video is stuck, it also includes: the electronic device determines that the layer texture identifier of the view displayed in the first control has not changed; when the frame interval of the view displayed in the first control is greater than or equal to the preset value, the electronic device determines that the video is stuck, including: when the frame interval of the view displayed in the first control is greater than or equal to the first threshold value, and no event of the progress bar control being triggered occurs in the video playback interface, and / or an event for indicating the pause of the video playback interface occurs, the electronic device determines that the video is stuck. In this way, it can be determined that the interval time between two frames of images exceeds the threshold value due to the user's pause operation, thereby accurately judging whether the video is stuck.

[0008] In a possible implementation, the preset value includes a second threshold value, and before the electronic device determines that the video is stuck, it also includes: the electronic device determines that the layer texture identifier of the view displayed in the first control has changed; when the frame interval of the view displayed in the first control is greater than or equal to the preset value, the electronic device determines that the video is stuck, including: when the frame interval of the view displayed in the first control is greater than or equal to the second threshold value, and no event occurs in the video playback interface to indicate switching the video playback interface to the background, the electronic device determines that the video is stuck. In this way, it can be further determined that the interval time between two frames of images exceeds the threshold value due to the user's background switching operation, thereby accurately judging whether the video is stuck.

[0009] In a possible implementation, the electronic device determines that a video freeze occurs, including: the electronic device records a log related to the freeze, so that further analysis and statistics of the freeze can be performed later.

[0010] In a possible implementation, the logs related to the jamming include one or more of the following: scene resource logs, scene event logs. In this way, the electronic device records different types of logs to facilitate subsequent different types of analysis of the jamming, thereby specifically analyzing and solving a certain type of problem.

[0011] In a possible implementation, the electronic device determines that the video freezes, including: the electronic device performs processing related to preset performance indicators; the preset performance indicators include one or more of the following: process CPU utilization of the electronic device, memory application time of the electronic device, and network signal of the electronic device. In this way, after the electronic device performs relevant processing on the above preset performance indicators, the performance of the electronic device can be improved, thereby helping to reduce the freeze of the video interface.

[0012] In a possible implementation, the electronic device performs processing related to the preset performance index, including: the electronic device determines that the CPU utilization of the process is abnormal; the electronic device performs detection related to the CPU utilization of the process, and the detection related to the CPU utilization of the process includes one or more of the following: frequency limit detection, scheduling priority detection, and inter-core running time detection; the electronic device records a log related to the CPU utilization of the process; the electronic device increases the CPU running frequency and / or increases the priority of the process; the electronic device determines that the memory application duration is abnormal; the electronic device performs detection related to the memory application duration, and the detection related to the memory application duration includes one or more of the following: available memory detection, slow path application times detection; the electronic device records a log related to the memory application duration; the electronic device cleans up the memory of the electronic device; the electronic device determines that the network signal is abnormal; the electronic device performs detection related to the network signal, and the detection related to the network signal includes one or more of the following: network delay detection, network packet loss detection; the electronic device records a log related to the network signal; the electronic device switches the network signal. In this way, the performance of the electronic device can be improved, thereby helping to reduce the freeze of the video interface.

[0013] In a possible implementation, the method further includes: when the electronic device determines that a preset condition is met, the electronic device records the frame time; when the electronic device determines that a preset event occurs in the video playback interface, the electronic device sets the frame time to a target value; the electronic device determines that there is buffer data related to the playback video, including: when the frame time is not the target value, the electronic device determines that there is buffer data related to the playback video; wherein, satisfying the preset condition includes any of the following: there is no buffer data related to the playback video; there is buffer data related to the playback video, and the frame interval of the view displayed in the first control is less than the preset value. In this way, the electronic device determines whether there is buffer data related to the playback video, which can reduce unnecessary execution steps and improve the execution efficiency of the code.

[0014] In a possible implementation, the first control includes a texture view control or an independent layer view control. In this way, the accuracy of determining whether the video application interface is stuck can be improved, which helps to reduce the stuck of the video application interface.

[0015] In a second aspect, an embodiment of the present application provides a device for video jamming detection, which may be an electronic device, or a chip or chip system in an electronic device. The device may include a processing unit and a display unit. The processing unit is used to implement any method related to processing performed by the electronic device in the first aspect or any possible implementation of the first aspect. The display unit is used to implement any method related to display performed by the electronic device in the first aspect or any possible implementation of the first aspect. When the device is an electronic device, the processing unit may be a processor. The device may also include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the electronic device implements the method described in the first aspect or any possible implementation of the first aspect. When the device is a chip or chip system in an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit so that the electronic device implements the method described in the first aspect or any possible implementation of the first aspect. The storage unit may be a storage unit in the chip (for example, a register, a cache, etc.), or a storage unit in the electronic device located outside the chip (for example, a read-only memory, a random access memory, etc.).

[0016] Exemplarily, the processing unit is used to determine that the video is stuck. The display unit is used to display the video playback interface.

[0017] In a possible implementation, the processing unit is used to determine that no preset event occurs in the video playback interface.

[0018] In a possible implementation, the processing unit is used to determine that the layer texture identifier of the view displayed in the first control has not changed, and is also used to determine that a video freeze occurs.

[0019] In a possible implementation, the processing unit is used to determine whether a layer texture identifier of a view displayed in a first control has changed, and is also used to determine whether a video freeze has occurred.

[0020] In a possible implementation, the processing unit is used to record logs related to the jam.

[0021] In one possible implementation, the logs related to lag include one or more of the following: scene resource logs and scene event logs.

[0022] In a possible implementation, the processing unit is used to execute processing related to a preset performance indicator.

[0023] In one possible implementation, the processing unit is used to determine whether an abnormality has occurred in the CPU utilization of a process, and is also used to perform detections related to the CPU utilization of the process, and is also used to record logs related to the CPU utilization of the process, and is specifically used to increase the operating frequency of the CPU and / or increase the priority of the process; is used to determine whether an abnormality has occurred in the memory application duration, and is also used to perform detections related to the memory application duration, and is also used to record logs related to the memory application duration, and is specifically used to clear the memory of the electronic device; is used to determine whether an abnormality has occurred in the network signal, and is also used to perform detections related to the network signal, and is also used to record logs related to the network signal, and is specifically used to switch the network signal.

[0024] In a possible implementation, the processing unit is used to record the frame time, and is also used to set the frame time as a target value, and is specifically used to determine whether there is buffer data related to the playing video.

[0025] In a possible implementation, the first control includes a texture view control or an independent layer view control.

[0026] In a third aspect, an embodiment of the present application provides a terminal device, including a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.

[0027] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0028] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0029] In a sixth aspect, the present application provides a chip or a chip system, the chip or chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0030] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).

[0031] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of an interface of a video application provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of module interaction of a video freeze detection method provided in an embodiment of the present application;

[0036] Figure 5 A schematic diagram of a flow chart of a video freeze detection method provided in an embodiment of the present application;

[0037] Figure 6 A schematic diagram of a video freeze detection method provided in an embodiment of the present application;

[0038] Figure 7 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0040] 1. Terminology

[0041] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0042] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0043] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0044] With the development of electronic technology, some electronic devices support video playback. Users can watch videos, shoot videos, and start live broadcasts through video applications. However, when electronic devices play the content of video applications, the interface of the electronic devices may freeze, affecting the user experience. Therefore, electronic devices are paying more and more attention to the freeze detection of video applications, so that when the application freezes, the electronic devices can perform corresponding processing, thereby reducing the freeze of video applications.

[0045] In one possible implementation of detecting application jams, the electronic device can detect application jams based on a detection method for software drawing frames. The detection method may include counting the time intervals of frame images drawn on the application interface. For example, the electronic device can call the doframe() method for drawing frame images to determine whether the application is jammed. The electronic device can determine in the doframe() method for drawing frame images whether the time interval between the current frame image and the drawing of the previous frame image exceeds a preset threshold. If it exceeds the preset threshold, it can be considered that the application interface is jammed. Otherwise, it can be considered that the application interface is not jammed.

[0046] In another possible implementation of detecting application jams, the electronic device can detect application jams based on the layer blending service SurfaceFlinger. For example, when the electronic device cannot send frames stably, the update time interval between the previous layer and the next layer may be unstable for a period of time, causing the application interface to jam. The SurfaceFlinger service can count the update time intervals of the layers. If the update time interval of the layers exceeds the preset value, it can be considered that the application interface is jammed. Otherwise, it can be considered that the application interface is not jammed.

[0047] However, the above two implementations of detecting application freezes can be understood as detecting based on the update frequency of the application interface. However, it is impossible to detect the scenario where the application interface is still being refreshed but part of the content in the application interface is not displayed.

[0048] For example, Figure 1 a is an interface of a video application, which includes a video playback control 100 and an avatar display control 101, wherein the avatar display control 101 can display animation effects. When the interface of the video application freezes, in a possible scenario, the video playback control 100 has no displayed content, but the avatar display control 101 can still display animation effects, which can be animation effects locally loaded by the video application.

[0049] It is understandable that if Figure 1 As shown in b, in the above-mentioned stuck scene, due to poor network signal, the CPU of the electronic device has no resources to parse network data packets, and the codec of the electronic device has problems, the video playback control 100 may not be able to refresh the video at a certain frame rate, causing the video playback control 100 to play stuck. However, since the animation effect of the avatar display control 101 is an animation effect loaded locally by the video application, it does not involve the network environment, and does not require the electronic device to parse the network data packet. Therefore, the interface of the video application can still continue to refresh the avatar display control 101 at a certain frame rate, that is, the refresh rate of the video application interface remains unchanged, and the electronic device is still drawing frame pictures at a certain time interval.

[0050] When the above-mentioned detection method based on the software drawing framework is used to detect freezes in the video application, since the detection method determines that the time interval between drawing the frame image of the current video application interface and drawing the previous frame image does not exceed the preset threshold, it is considered that there is no freeze in the interface of the video application.

[0051] When the above-mentioned detection method based on the layer blending service SurfaceFlinger is used to detect the jamming of the video application, since the detection method determines that the update time interval between the previous layer and the next layer of the video application interface does not exceed the preset value, it is considered that there is no jamming in the video application interface.

[0052] It can be seen that the above two detection methods cannot accurately detect whether the interface of the video application is stuck, and then the stuck is not processed, resulting in more frequent stuck of the video application interface.

[0053] In view of this, the video freeze detection method provided in the embodiment of the present application can, when the electronic device detects the freeze of the video application interface, detect the frame rate refresh of the drawing controls that do not load local data, so as not to be affected by the controls that load local data, thereby improving the accuracy of judging whether the video application interface is freezed, and further helping to reduce the freeze of the video application interface.

[0054] It can be understood that the electronic device of the embodiment of the present application can also be a terminal device in any form. For example, the electronic device can include a handheld device, a vehicle-mounted device, etc. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, electronic devices in 5G networks or future evolved public land mobile communication networks (PLANs). The electronic devices in the network (PLMN) and the like are not limited to this in the embodiments of the present application.

[0055] As an example but not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0056] In addition, in the embodiments of the present application, the electronic device may also be an electronic device in an Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0057] The electronic devices in the embodiments of the present application may also be referred to as: user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0058] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0059] For example, Figure 2 A schematic structural diagram of an electronic device is shown.

[0060] The electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0061] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0062] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0063] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0064] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0065] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a USB interface, etc.

[0066] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0067] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor. For example, the method of an embodiment of the present application may be executed.

[0068] The electronic device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information. The electronic device can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0069] Figure 3 It is a software structure block diagram of the electronic device of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

[0070] The application layer can include a series of application packages. Figure 3 As shown, the application package may include video, live broadcast, video call, etc. Applications may include system applications and third-party applications.

[0071] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0072] like Figure 3 As shown, the application framework layer may include a view system, a window processing device, a performance logging interface, a content provider, and a resource manager, etc. In addition, the application framework layer may also include a drawing management class of a view control, a jamming log system, and the like.

[0073] The view system can be used for interface drawing and event processing of the application. The view system can include texture view controls and independent layer view controls.

[0074] The window processing device can be used to obtain relevant data of the electronic device window interface, such as the window name, etc. The window processing device can also detect freeze events.

[0075] The performance logging interface can be used to call the jamming related interface to write the jamming information into the performance jamming log virtual device of the kernel layer.

[0076] Content providers are used to implement data sharing between different applications, allowing one program to access data in another program while ensuring the security of the accessed data.

[0077] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0078] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0079] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0080] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection. For example, in the embodiment of the present application, the virtual machine can be used to perform functions such as drawing interface frame images, detecting jam events, and processing jam events.

[0081] The system library can also be called the Native layer, which can include multiple functional modules, such as function libraries, graphics processing libraries (such as OpenGL ES), etc. The Native layer can also include log maintenance and diagnostic services.

[0082] Log maintenance and diagnosis services may include performance log diagnosis services and resource log collection services, etc. Among them, the performance log diagnosis service can be used to detect freeze events in the window interface, and the resource log collection service can be used to cache system resource logs.

[0083] The function library provides developers with API interfaces for multiple services, making it easy for developers to quickly integrate and implement various functions.

[0084] The graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing.

[0085] The kernel layer is the layer between hardware and software. The kernel layer may include various driver modules such as display driver and audio driver, and may also include a performance jam log virtual device. The performance jam log virtual device may be used to save the jam information transmitted by the performance logging interface of the application framework layer into the jam log.

[0086] It should be noted that the embodiments of the present application are only illustrated using the Android system as an example. In other operating systems (such as Windows system, IOS system, etc.), as long as the functions implemented by each functional module are similar to those in the embodiments of the present application, the solutions of the present application can also be implemented.

[0087] Below Figure 4 Take the interaction between related modules in the video freeze detection method of the embodiment of the present application as an example.

[0088] The application layer of the electronic device may include video applications, live broadcast applications, and video call applications. When the application wants to instruct the electronic device to display a certain interface, the application can use the view control of the application framework layer to draw the interface. The view control may include a texture view control and an independent layer view control. For example, the texture view control may include a TextureView control, and the independent layer view control may include a SurfaceView control. Among them, the texture view control may also be called a texture control, and the independent layer view control may also be called a canvas control.

[0089] It is understandable that for different video scenes, the application can use different view controls. For example, in the video playback scene, the application can use the texture control; in the live broadcast scene, the application can use the canvas control. The application can decide which view control to use, and the embodiment of the present application is not limited. For the convenience of description, the video scene is used as an example for explanation.

[0090] When an application uses a texture control, the application can call related functions in the texture control to implement interface drawing. For example, the application can call a window-canvas association function, which can bind the canvas to the window interface of the electronic device. The application can also call a canvas frame completion function, which can display the content on the canvas on the window interface of the bound electronic device, thereby achieving the effect of video playback. In addition, the application can also call a window-canvas de-association function to destroy the canvas when no interface drawing is needed.

[0091] Similarly, when an application uses a canvas control, the application can call related functions in the canvas control to implement interface drawing. For example, the application can call a window-canvas association function, which can bind the canvas to the window interface of the electronic device. The application can also call a layer update function, which can update the content on the layer to the window interface of the electronic device, thereby achieving the effect of video playback. In addition, the application can also call a window-canvas de-association function to destroy the canvas when no interface drawing is needed.

[0092] In addition, the above canvas frame completion function or layer update function can also call the video frame processing interface in the window processing device to pass the relevant data of each frame of the video playback to the window processing device. In this way, the window processing device can detect whether the current window interface is stuck.

[0093] The window processing device may include a texture control switching device, a texture control frame length calculation device, a canvas control frame length calculation device, etc. The texture control switching device and the texture control frame length calculation device may be used to detect a jam event in the texture control, and the canvas control frame length calculation device may be used to detect a jam event in the canvas control.

[0094] Among them, the texture control switching device can be used to detect whether a freeze occurs when the window interface switches, wherein the window interface switching may include the electronic device responding to the user's operation of sliding the video and playing the previous video or the next video; the window interface switching may also include the electronic device responding to the user's operation of switching the application to the background and the video interface being cut to the background.

[0095] For example, when the refresh time of the interface reaches the preset time threshold, if the texture control switching device detects an event of the user sliding the video, it can be considered that the video switching is currently occurring, the refresh time is acceptable, and the interface is not stuck; if the texture control switching device does not detect an event of the user sliding the video, it can be considered that the video switching is currently not occurring, the refresh time is long, and the interface is stuck. The specific value of the preset time threshold is not limited in the embodiment of the present application.

[0096] The texture control frame length calculation device can be used to detect whether a certain video is stuck during playback. For example, in the same video, when the refresh time interval between the current and subsequent frames exceeds a preset value, if the texture control frame length calculation device detects that the user clicks to pause playback, it can be considered that the video is not stuck; if the texture control frame length calculation device does not detect that the user clicks to pause playback, it can be considered that the video is stuck. The specific value of the preset value is not limited in the embodiment of the present application.

[0097] The canvas control frame length calculation device can be used to detect whether a freeze occurs during the playback of a certain video. Similarly, the detection process of the canvas control frame length calculation device can refer to the relevant description of the texture control frame length calculation device above, which will not be repeated.

[0098] It can be understood that the window processing device can set corresponding jamming identification IDs for different types of jamming scenes. For example, the jamming scene detected by the texture control switching device can correspond to the jamming identification ID1, the jamming scene detected by the texture control frame length calculation device can correspond to the jamming identification ID2, and the jamming scene detected by the canvas control frame length calculation device can correspond to the jamming identification ID3.

[0099] When the window processing device detects that the video interface is stuck, the window processing device can pass the stuck information to the performance logging interface, which can write the stuck information to the stuck log in the performance stuck log virtual device of the kernel layer through the log-related interface of the Native layer and the log-related interface of the kernel layer.

[0100] In the Native layer, the performance log diagnosis service in the log maintenance and diagnosis service can analyze and process the jamming events of the video interface according to the jamming logs mentioned above. The performance log diagnosis service can include a performance log reading device, a texture video playback jamming processor, and an independent layer video playback jamming processor.

[0101] In a possible implementation, the performance log reading device can poll and read the jam log in the performance jam log virtual device of the kernel layer. The performance log diagnostic service can determine whether the current jam log is analyzed and processed by the texture class video playback jam processor or the independent layer class video playback jam processor based on the jam identifier in the jam log. For example, when the jam identifier is ID1 or ID2, the current jam log can be analyzed and processed by the texture class video playback jam processor; when the jam identifier is ID3, the current jam log can be analyzed and processed by the independent layer class video playback jam processor.

[0102] When the texture video playback jamming processor or the independent layer video playback jamming processor analyzes and processes the jamming event, the resource log collection service can be called to obtain the system resource log. It is understandable that the system resource log can record in detail various log information when the electronic device is running, including CPU resource information, input and output information, memory information, network status information and other resource log information. Compared with the system resource log, the above jamming log records relatively less information. The jamming log can record jamming time, jamming scene, jamming degree and other information related to jamming.

[0103] For the analysis and processing flow of the texture video playback jamming processor or the independent layer video playback jamming processor for jamming events, please refer to the following Figure 5 The relevant description of the video playback freeze detection process in the corresponding embodiment (2) is not repeated here.

[0104] Figure 5 A schematic diagram of a video jam detection method is shown. The video jam detection method may include the following processes: (1) a fault collection process, (2) a video playback jam detection process, and (3) a fault diagnosis and recovery process.

[0105] (1) Fault collection process.

[0106] The fault collection process can be implemented by a fault collection device, which can collect events reported by some system controls in the electronic device. Figure 4 In the corresponding embodiment, the window processing device of the application framework layer, the system control can be understood as the above Figure 4 The corresponding texture control or canvas control in the embodiment. The events reported by the system control may include one or more of the following events: event 1, event of judging the video scene, event 2, event of change of visible attribute of the control, event 3, event of user click, event 4, event of change of foreground and background status of the interface.

[0107] Event 1: Determine the event of the video scene.

[0108] During the interface rendering process of the electronic device, when the system control receives the rendering thread signal of the application, the rendering thread signal can be transmitted to the fault collection device. The fault collection device can determine whether the current scene is a video scene according to the rendering thread signal.

[0109] In a possible implementation, the fault collection device can determine the name of the interface currently displayed by the electronic device and the application package name corresponding to the interface based on the rendering thread signal. It can be understood that a white list of video scenes is set in the electronic device, and the white list may include application package names and interface names. When the interface name determined by the fault collection device matches the interface name in the white list, and the application package name determined by the fault collection device matches the application package name in the white list, it means that the current scene is a video scene; otherwise, the current scene is not a video scene.

[0110] If the fault acquisition device determines that the current scene is a video scene, it can record the cache time FrameBufferReady and the layer texture identifier SurfaceLayerID of the current frame image, where the layer texture identifier can also be called the layer texture ID.

[0111] Event 2: The event when the visible property of the control changes.

[0112] The fault collection device can obtain visible property change events of controls such as progress bars. The visible property change events of the controls can include change events of the visible property Visible of the controls and events of transparency change AlphaChanged. For ease of description, the following uses the progress bar control as an example to illustrate visible property change events and transparency change events.

[0113] In a possible implementation, the user can click on the video interface to pause the video. In response to the user's click event, the display state of the progress bar control in the electronic device interface can change. At this time, the electronic device can report the change value of the visible attribute Visible of the progress bar control. For example, when the value of Visible is true, it means that the progress bar control is visible, and when the value of Visible is false, it means that the progress bar control is invisible. In addition, the electronic device can also report the value of the transparency change of the progress bar control. It can be understood that the transparency value can be in the range of 0 to 255. The smaller the value, the more transparent the display of the progress bar control, and the larger the value, the more opaque the display of the progress bar control.

[0114] Event 3: User click event.

[0115] The fault collection device can detect various click events InputEvent of the user on the screen of the electronic device. For example, the click event InputEvent can include a press event ACTION_DOWN, a slide event ACTION_MOVE, a lift event ACTION_UP, etc. When the user presses the screen and then lifts his finger, the fault collection device can obtain the lift event ACTION_UP and record the time of the lift.

[0116] It is understandable that when the electronic device responds to the user clicking to pause the video playback operation, the progress bar control of the electronic device interface can become visible. At this time, the electronic device has both an event of a change in the visible attribute of the progress bar and an event of a user clicking. In other words, the fault collection device can detect the occurrence of event 2 and event 3.

[0117] Event 4: the event of the foreground and background status change of the interface.

[0118] The fault collection device can register the event of the interface switching to the background with the electronic device. When the application switches to the background, the electronic device can report the event of the application switching to the background to the fault collection device.

[0119] (2) Video playback freeze detection process.

[0120] The video playback jam detection process can be implemented by a video playback jam detection device, which can be understood as the above-mentioned Figure 4 The corresponding embodiment includes a texture video playback jamming processor of the Native layer and an independent layer video playback jamming processor. S501: Determine whether there is buffered data.

[0121] In an embodiment of the present application, the video playback jam detection device can determine whether there is buffered data. In a possible implementation, the video playback jam detection device can obtain the cache time of the previous frame image, and determine whether there is buffered data based on the cache time. Among them, the video playback jam detection device can obtain the cache time of the previous frame image from the cache time FrameBufferReady of the frame image recorded by the fault acquisition device.

[0122] For example, if the current frame is the first frame, the cache time before the video playback freeze detection device obtains the frame can be 0, that is, there is no buffered data before the first frame. 0 can also be set to other preset values, as long as the preset value can be used to indicate that there is no buffered data before the first frame.

[0123] When the cache time of the previous frame image obtained by the video playback jamming detection device is 0, it means that there is no buffered data, and step S506 can be executed; when the cache time of the previous frame image obtained by the video playback jamming detection device is not 0, it means that there is buffered data, and step S502 can be executed.

[0124] S502: Determine whether the video texture ID has changed.

[0125] When the video playback jam detection device determines that there is buffered data, the video playback jam detection device can further determine whether the video texture ID has changed. If the video texture ID has changed, it means that the current scene is a video switching scene. For example, the video switching scene may include the electronic device responding to the user's operation of sliding the video to play the previous video or the next video, and may also include the electronic device responding to the user's operation of switching the application to the background, and the current video interface switches to the background. If the video texture ID has not changed, it means that the current scene is a video playback scene.

[0126] If the video texture ID changes, step S505 may be performed. If the video texture ID does not change, step S503 may be performed.

[0127] S503: Determine whether the frame interval time exceeds threshold A.

[0128] When the video texture ID does not change, it indicates that the current scene is a video playback scene, and the video playback freeze detection device can determine whether the interval between the first and second frames exceeds a threshold value A. The threshold value A can be preset by the electronic device, for example, the threshold value A can be set to about 80 milliseconds. The specific value of the threshold value A is not limited in the embodiment of the present application.

[0129] When the interval between the current and the next two frames exceeds the threshold value A, it means that the refresh time interval between the two frames is relatively long, and the video playback freeze detection device needs to further determine whether the current interface freezes, and then step S505 can be executed. When the interval between the current and the next two frames does not exceed the threshold value A, it means that the previous interface refreshes normally and there is no freeze, and then step S506 can be executed.

[0130] S504: Determine whether the frame interval time exceeds threshold B.

[0131] When the video texture ID changes, it indicates that the current scene is a video switching scene, and the video playback freeze detection device can determine whether the interval between the previous and next two frames exceeds the threshold B. The threshold B can be preset by the electronic device, for example, the threshold B can be set to about 155 milliseconds. The specific value of the threshold B is not limited in the embodiment of the present application.

[0132] When the interval between the current and the next two frames exceeds the threshold value B, it means that the refresh time interval between the two frames is relatively long, and the video playback jam detection device needs to further determine whether the current interface is jammed, and then step S505 can be executed. When the interval between the current and the next two frames does not exceed the threshold value B, it means that the current interface is refreshed normally and there is no jamming, and then step S506 can be executed.

[0133] S505: Determine whether the following events occur: a progress bar state change event, a user click event, or an application switching to the background event.

[0134] In one possible scenario, in the above step S502, if the video texture ID has not changed, it indicates a video playback scenario. The video playback freeze detection device can further determine whether the interval time between two frames of images exceeds threshold A due to an event in which the visible properties of the control change and / or a user click event.

[0135] It is understandable that the video playback freeze detection device can determine whether a control visible property change event and / or a user click event occurs within a period of time, wherein the period of time may cover the time when the video texture ID changes. The specific value of the period of time is not limited in the embodiment of the present application.

[0136] If the video playback jam detection device detects an event that a control visible property change has occurred, and / or a user click event, it means that the interval between two frames of images exceeds the threshold value A due to the user's pause operation, and it can be considered that the video playback is not jammed, and step S506 can be executed. If the video playback jam detection device does not detect an event that a control visible property change has occurred, and / or a user click event, it means that the video playback is jammed, and step S508 can be executed.

[0137] In another possible scenario, in the above step S502, if the video texture ID changes, indicating a video switching scene, the video playback freeze detection device can further determine whether the interval time between two frames of images exceeds threshold B due to an application background switching event.

[0138] If the video playback jam detection device detects that an application background switching event has occurred, it means that the interval between two frames of images exceeds the threshold value B due to the user's background switching operation, and it can be considered that the video playback is not jammed, and step S506 can be executed. If the video playback jam detection device does not detect that an application background switching event has occurred, it means that the video playback is jammed, and step S508 can be executed.

[0139] S506: Record frame time.

[0140] The video playback jamming detection device can record the frame time, wherein the frame time can be understood as the cache time FrameBufferReady of the frame image reported to the video playback jamming detection device by the above-mentioned fault collection process.

[0141] S507: Record the video texture ID.

[0142] The video playback jamming detection device can record the video texture ID, wherein the video texture ID can be understood as the video texture ID reported to the video playback jamming detection device by the above-mentioned fault collection process.

[0143] S508: Record scene resource logs.

[0144] The scene resource log can be used by the video playback jam detection device from the above Figure 4 In the corresponding embodiment, some system resource logs are obtained from the cache of the resource log collection service of the Native layer. The video playback jam detection device can record the scene resource log to the disk file of the electronic device, and can also package the disk file corresponding to the scene resource log and upload it to the server to facilitate subsequent analysis of the scene resource log.

[0145] S509: Record scene event log.

[0146] The scene event log can be Figure 4 The corresponding embodiment is a partial jam log obtained in the virtual device of the kernel layer performance jam log. The video playback jam detection device can record the scene event log to the disk file of the electronic device, and can also package the disk file corresponding to the scene event log and upload it to the server, so as to facilitate the subsequent analysis of the scene event log.

[0147] S510: Run the fault diagnosis and recovery device.

[0148] The video playback freeze detection device may also include a running fault diagnosis and recovery device. The specific running fault diagnosis and recovery device can refer to the relevant description in the following (3) fault diagnosis and recovery process, which will not be repeated here.

[0149] Optionally, in the above step S501, when the video playback jam detection device detects a progress bar state change event and / or a user click event, the video playback jam detection device may set the cache time corresponding to the previous frame image to 0. In this way, when the video playback jam detection device obtains the cache time of the previous frame image as 0, it can also indicate that there is no buffered data before the current frame image. Among them, 0 can also be set to other preset values, as long as the preset value can be used to indicate that there is no buffered data before the current frame image.

[0150] Exemplarily, if the N+1th frame image is not the first frame image, assuming that the cache time of the Nth frame image is 100 milliseconds, the cache time of the N+1th frame image is 200 milliseconds. When a progress bar state change event and / or a user click event occurs at 110 milliseconds, the video playback freeze detection device can set the cache time corresponding to the Nth frame image to 0, indicating that there is no buffered data before the N+1th frame image. This is because when the electronic device responds to the user clicking to pause playback, the electronic device interface content no longer changes, that is, the electronic device does not refresh the interface, and therefore, it can be considered that there is no buffered data before the N+1th frame image.

[0151] In a possible scenario, when a progress bar state change event and / or a user click event occurs at 110 milliseconds, the video playback jam detection device sets the cache time corresponding to the previous frame image to 0. However, since the electronic device sends frames at a faster speed, the electronic device sends one more frame image at 120 milliseconds, causing the cache time corresponding to the previous frame image to become 120 milliseconds, causing the video playback jam detection device to determine that there is currently buffered data and execute step S502. When the video playback jam detection device detects that the frame interval time exceeds the threshold, step S505 can be executed to determine the progress bar state change event and / or the user click event again.

[0152] It is understandable that the video playback jam detection device can consider the occurrence time of the user click event to be the time when the user clicks the screen plus the preset finger lift time. For example, if the user clicks the screen for 110 milliseconds, the preset finger lift time can be 50 milliseconds. In this way, the video playback jam detection device can consider the occurrence time of the user click event to be about 160 milliseconds. Therefore, when the video playback jam detection device determines that the 120 milliseconds of the cache time corresponding to the previous frame of the image is before 160 milliseconds, it can be considered that a click event has occurred, and the cache time corresponding to the previous frame of the image can also be set to 0. This can help improve the accuracy of the video playback jam detection device in determining jam events.

[0153] (3) Fault diagnosis and recovery process.

[0154] The fault diagnosis and recovery process can be implemented by a fault diagnosis and recovery device, and the processes that can be executed by the fault diagnosis and recovery device include (3.1) the processing flow when the process CPU utilization is abnormal, (3.2) the processing flow when the memory application duration is abnormal, and (3.3) the processing flow when the network signal is abnormal.

[0155] (3.1) Processing flow when the process CPU utilization is abnormal.

[0156] When the process CPU utilization is greater than or equal to the preset CPU utilization threshold, the process CPU utilization can be considered abnormal, for example, the preset CPU utilization threshold can be 40%. It is understandable that the specific value of the preset CPU utilization threshold is not limited in the embodiment of the present application.

[0157] When the CPU utilization of the process is abnormal, the fault diagnosis and recovery device can perform one or more of the following tests: (a) frequency limit detection, (b) scheduling priority detection, and (c) inter-core running time detection.

[0158] (a) Frequency limit detection.

[0159] Frequency limiting can be understood as a thermal frequency limiting strategy in electronic devices, and the thermal frequency limiting strategy can be understood as a strategy for limiting the CPU frequency of electronic devices according to a preset temperature threshold. The CPU frequency can also be understood as the operating frequency of the CPU.

[0160] Exemplarily, when the temperature of the electronic device reaches a preset temperature threshold, the electronic device can reduce the CPU frequency. For example, the preset temperature threshold can be 40 degrees, and the CPU frequency can be reduced to 2.0 GHz. Among them, the specific value of the preset temperature threshold and the reduction range of the CPU frequency are not limited in the embodiments of the present application. It can be understood that frequency limiting can reduce the power consumption of the electronic device while roughly maintaining the basic performance of the electronic device, so that the temperature of the electronic device will not be too high, and the battery life of the electronic device will be increased.

[0161] When an electronic device executes a thermal frequency limiting strategy, relevant logs can be printed into a system resource log. A fault diagnosis and recovery device can obtain thermal frequency limiting strategy related logs from the system resource logs, and record the thermal frequency limiting strategy related logs into a scheduling log of a disk file of the electronic device, to facilitate subsequent analysis and statistics of thermal frequency limiting problems.

[0162] (b) Scheduling priority detection.

[0163] The fault diagnosis and recovery device can detect the priority level of the current video application. If the priority level of the current video application is low, the fault diagnosis and recovery device can send a message to the CPU scheduling module to instruct the CPU scheduling module to increase the priority level of the video application.

[0164] In a possible implementation, the fault diagnosis and recovery device can determine whether the priority of the video application is greater than or equal to a preset priority threshold. If the priority of the video application is greater than or equal to the preset priority threshold, it means that the priority level of the video application is low, and the fault diagnosis and recovery device can send a message to the CPU scheduling module to instruct the CPU scheduling module to increase the priority of the video application. Among them, the preset priority threshold can be 120, and the specific value of the preset priority threshold is not limited in the embodiment of the present application.

[0165] It is understandable that the fault diagnosis and recovery device can record the relevant logs of the CPU scheduling priority into the above-mentioned scheduling log to facilitate the subsequent analysis and statistics of the priority scheduling problem.

[0166] (c) Inter-core runtime detection.

[0167] The electronic device may include multiple CPU large cores and multiple CPU small cores, and the fault diagnosis and recovery device may detect the running time and running frequency of each CPU large core or CPU small core. For the convenience of subsequent description, taking the CPU large core as an example, when the application interface freezes due to the low running frequency of a certain CPU large core, the fault diagnosis and recovery device may send a message to the CPU scheduling module, instructing the CPU scheduling module to increase the running frequency of the CPU large core, thereby reducing the freeze of the application interface. Among them, adjusting the running frequency of the CPU large core can also be called frequency modulation.

[0168] For example, the application process displayed on the current interface is running on a CPU core, and the operating frequency of the CPU core is 1.3GHz. Since the operating frequency of the CPU core is relatively low, the application interface is stuck, and the fault diagnosis and recovery device can send a message to the CPU scheduling module to instruct the CPU scheduling module to increase the operating frequency of the CPU core. When the CPU scheduling module increases the operating frequency of the CPU core to 2.0GHz, the application interface can be less stuck.

[0169] It is understandable that the fault diagnosis and recovery device can record the relevant logs of the inter-core running time into the above-mentioned scheduling log, so as to facilitate the subsequent analysis and statistics of the inter-core running time problem.

[0170] In the embodiment of the present application, the execution order of (a) frequency limit detection, (b) scheduling priority detection and (c) inter-core running time detection may not be limited.

[0171] (3.2) Processing flow when memory request duration is abnormal.

[0172] When the memory application duration of the electronic device exceeds the preset application duration threshold within the preset time period, the memory application duration can be considered abnormal. For example, the preset time period can be 100 milliseconds, and the preset application duration threshold can be 8 milliseconds. The specific values ​​of the preset time period and the preset application duration threshold are not limited in the present embodiment.

[0173] When the memory application duration is abnormal, the fault diagnosis and recovery device may perform one or more of the following tests: (d) available running memory test, (e) slow path application times test.

[0174] (d) Available running memory detection.

[0175] When the memory application time is abnormal, the fault diagnosis and recovery device can determine the remaining space of the running memory of the electronic device. When the remaining space of the running memory is less than the preset running memory threshold, the fault diagnosis and recovery device can perform a background cleanup operation to free up more running memory space.

[0176] Exemplarily, for an electronic device with an operating memory of 8G, the corresponding preset operating memory threshold may be 800MB. When the fault diagnosis and recovery device determines that the remaining space of the operating memory is less than 800MB, a background cleaning operation may be performed. It is understandable that different electronic devices have different corresponding operating memory sizes, so different electronic devices may have different preset operating memory thresholds. The specific value of the preset operating memory threshold is not limited in the embodiments of the present application.

[0177] It is understandable that the fault diagnosis and recovery device can record the relevant logs of the available memory detection into the memory log of the disk file of the electronic device, so as to facilitate the subsequent analysis and statistics of the memory problem.

[0178] (e) Apply for slow path times detection.

[0179] When an application requests memory from an electronic device, if the electronic device does not have sufficient memory space, the electronic device needs to clean up the memory first and reclaim some memory before allocating memory to the application. This way of allocating memory can be understood as applying for a slow path.

[0180] In a possible implementation, when the fault diagnosis and recovery device determines that the proportion of the number of times the application applies for the slow path is greater than the preset number of times proportion threshold, and / or the proportion of the duration of the application applying for the slow path is greater than the preset duration proportion threshold, the fault diagnosis and recovery device can perform a background cleaning operation. Exemplarily, the preset number of times proportion threshold can be 80%, and the preset duration proportion threshold can be 20%. The specific values ​​of the preset number of times proportion threshold and the preset duration proportion threshold are not limited in the embodiments of the present application.

[0181] It is understandable that the fault diagnosis and recovery device can record the relevant logs of the number of slow path application detections into the above-mentioned memory logs, so as to facilitate the subsequent analysis and statistics of the slow path application problems.

[0182] (3.3) Processing flow when network signal is abnormal.

[0183] In a possible implementation, when the network signal is lower than the preset signal threshold of the full signal, the network signal can be considered abnormal, for example, the preset signal threshold can be 60%. It is understandable that the specific value of the preset signal threshold is not limited in the embodiment of the present application.

[0184] In another possible implementation, when the network signal strength is not within the preset range of signal strength, the network signal may be considered abnormal. For example, the preset range of signal strength may be -90dBm to -60dBm. It is understood that the specific value of the preset range of signal strength is not limited in the present embodiment.

[0185] When the network signal is abnormal, the fault diagnosis and recovery device can perform one or more of the following tests: (f) network delay detection, (g) network packet loss detection.

[0186] (f) Network delay detection.

[0187] When the fault diagnosis and recovery device detects that the network delay is greater than the preset network delay threshold, the fault diagnosis and recovery device can perform an operation of switching the network signal. The operation of switching the network signal can include switching the wifi signal to the data signal and switching the data signal to the wifi signal. The preset network delay threshold can be pre-set by the electronic device, for example, the preset network delay threshold can be 100 milliseconds, and the specific value of the preset network delay threshold is not limited in the embodiment of the present application.

[0188] It is understandable that the fault diagnosis and recovery device can record the relevant logs of network delay detection into the network log of the disk file of the electronic device, so as to facilitate the subsequent analysis and statistics of the network delay problem.

[0189] (g) Network packet loss detection.

[0190] When the fault diagnosis and recovery device detects that the network packet loss rate is greater than a preset packet loss rate threshold, the fault diagnosis and recovery device can perform an operation of switching the network signal. The preset packet loss rate threshold can be pre-set by the electronic device, for example, the preset packet loss rate threshold can be 20%, and the specific value of the preset packet loss rate threshold is not limited in the embodiment of the present application.

[0191] It is understandable that the fault diagnosis and recovery device can record relevant logs of network packet loss detection into the above-mentioned network log to facilitate subsequent analysis and statistics of the network packet loss problem.

[0192] In the embodiment of the present application, the execution order of (3.1) the processing flow when the process CPU utilization is abnormal, (3.2) the processing flow when the memory application duration is abnormal, and (3.3) the processing flow when the network signal is abnormal may not be limited.

[0193] The following specific embodiments are used to describe the method of the present application in detail. The following embodiments may be combined with each other or implemented independently, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0194] Figure 6 The video freeze detection method of the embodiment of the present application is shown. The method includes:

[0195] S601. An electronic device displays a video playback interface, where the video playback interface includes a first control and a second control, where the first control is used to display a first type of view, and the second control is used to display a second type of view, where the first type of view does not include views loaded locally from the electronic device; and the second type of view includes views loaded locally from the electronic device.

[0196] In the embodiment of the present application, the first control can be understood as the above Figure 1 In the corresponding embodiment of the video playback control 100, the second control can be understood as the above Figure 1 In the corresponding embodiment of the avatar display control 101, the first type of view may be a view of a video playback scene, a view of a video switching scene, etc. displayed in the first control.

[0197] S602: When a frame interval of the view displayed in the first control is greater than or equal to a preset value, the electronic device determines that a video freeze occurs; wherein a frame interval of the view displayed in the second control is less than, equal to or greater than the preset value.

[0198] In the embodiment of the present application, the preset value can be pre-set by the electronic device, and the specific value of the preset value is not limited in the embodiment of the present application. Figure 5 The corresponding threshold value A or threshold value B in the embodiment.

[0199] When the electronic device determines that the video is stuck, the electronic device can print a log related to the stuck and can also perform relevant processing on the stuck, which can help reduce the stuck of the video interface.

[0200] When detecting the freeze of the video application interface, the electronic device can detect the drawing controls that do not load local data without being affected by the controls that load local data, which can improve the accuracy of judging whether the video application interface is freezed, thereby helping to reduce the freeze of the video application interface.

[0201] Optional, in Figure 6 On the basis of the corresponding embodiment, before the electronic device determines that the video is stuck, it may also include: the electronic device determines that no preset event occurs in the video playback interface, the preset event includes one or more of the following: an event in which the progress bar control in the video playback interface is triggered, an event for indicating the pause of the video playback interface, and an event for indicating the switching of the video playback interface to the background.

[0202] In the embodiment of the present application, the event that the progress bar control in the video playback interface is triggered can be understood as the above Figure 5 The corresponding event 2 in the embodiment, the event of the visible attribute change of the control, the event for indicating the pause of the video playback interface can be understood as the above Figure 5The event 3 in the corresponding embodiment, the user click event, which is used to indicate the event of switching the video playback interface to the background, can be understood as the above Figure 5 The corresponding event 4 in the embodiment is the event of the interface front and back state change. For specific preset events, please refer to the above Figure 5 The relevant description in the fault collection process (1) of the corresponding embodiment will not be repeated here.

[0203] The electronic device determines whether a preset event occurs in the video playback interface, so as to judge whether the current interface is paused due to an event triggered by the user, and thus can more accurately judge whether the interface is stuck.

[0204] Optional, in Figure 6 On the basis of the corresponding embodiment, the preset value includes a first threshold value. Before the electronic device determines that the video is stuck, it may also include: the electronic device determines that the layer texture identifier of the view displayed in the first control has not changed; when the frame interval of the view displayed in the first control is greater than or equal to the preset value, the electronic device determines that the video is stuck, including: when the frame interval of the view displayed in the first control is greater than or equal to the first threshold value, and no event of the progress bar control being triggered occurs in the video playback interface, and / or an event for indicating the pause of the video playback interface occurs, the electronic device determines that the video is stuck.

[0205] In the embodiment of the present application, the first threshold value can be understood as the above Figure 5 The threshold value A in the corresponding embodiment, the specific first threshold value can refer to the relevant description of the threshold value A, which will not be repeated here.

[0206] The layer texture identifier can be understood as the above Figure 5 The layer texture ID in the corresponding embodiment can be specifically judged whether the layer texture identifier has changed by referring to the above Figure 5 The relevant description in S502 of the corresponding embodiment is not repeated here. Figure 5 The relevant description in S505 of the corresponding embodiment is not repeated here.

[0207] The electronic device determines the event of the progress bar control being triggered and / or the event used to indicate the pause of the video playback interface, and can further determine that the interval time between two frames of images exceeds the threshold A due to the user's pause operation, thereby accurately determining whether the video is stuck.

[0208] Optional, in Figure 6On the basis of the corresponding embodiment, the preset value includes a second threshold value. Before the electronic device determines that the video is stuck, it may also include: the electronic device determines that the layer texture identifier of the view displayed in the first control has changed; when the frame interval of the view displayed in the first control is greater than or equal to the preset value, the electronic device determines that the video is stuck, including: when the frame interval of the view displayed in the first control is greater than or equal to the second threshold value, and no event occurs in the video playback interface to indicate switching the video playback interface to the background, the electronic device determines that the video is stuck.

[0209] In the embodiment of the present application, the second threshold value can be understood as the above Figure 5 The threshold B in the corresponding embodiment and the specific second threshold can refer to the relevant description of the threshold B, which will not be repeated here. It can be understood that the values ​​of the first threshold and the second threshold can be the same or different.

[0210] In a possible scenario, since the frame image refresh time when the electronic device switches to the previous video or the next video, or switches the video to the background is longer than the frame image refresh time when the same video is played, the second threshold can be appropriately greater than the first threshold, which can be more in line with the judgment logic of the frame time of the specific scenario.

[0211] In addition, the electronic device can refer to the above Figure 5 The relevant description in S505 of the corresponding embodiment is not repeated here.

[0212] The electronic device determines the event for indicating switching the video playback interface to the background, and can further determine that the interval time between two frames of images exceeds the threshold value B due to the user's switching to the background operation, thereby accurately determining whether the video is stuck.

[0213] Optional, in Figure 6 On the basis of the corresponding embodiment, the electronic device determining that a video freeze occurs may include: the electronic device recording a log related to the freeze.

[0214] In the embodiment of the present application, the logs related to the jam can be recorded by referring to the above Figure 5 The relevant descriptions in S508 and S509 of the corresponding embodiment are not repeated here.

[0215] Electronic devices can record logs related to lags to facilitate further analysis and statistics of the lags. For example, electronic devices can upload logs to a server, and developers can optimize performance parameters of the electronic devices based on the log information. Electronic devices can also send logs to video application manufacturers, who can further optimize program codes, thereby helping to reduce lags in video playback.

[0216] Optional, in Figure 6Based on the corresponding embodiment, the logs related to the jamming include one or more of the following: scene resource logs and scene event logs.

[0217] In the embodiment of the present application, the scene resource log can refer to the above Figure 5 The relevant description in S508 of the corresponding embodiment will not be repeated here. The scene event log can refer to the above Figure 5 The relevant description in S509 of the corresponding embodiment is not repeated here.

[0218] Electronic devices record different types of logs to facilitate subsequent different types of analysis of the jam, thereby analyzing and solving certain types of problems in a targeted manner.

[0219] Optional, in Figure 6 On the basis of the corresponding embodiments, the electronic device determines that the video is stuck, which may include: the electronic device performs processing related to preset performance indicators; the preset performance indicators include one or more of the following: the process CPU utilization of the electronic device, the memory application time of the electronic device, and the network signal of the electronic device.

[0220] In the embodiment of the present application, the electronic device can perform processing related to the preset performance indicator by referring to the above Figure 5 The relevant description in the fault diagnosis and recovery process (3) of the corresponding embodiment is not repeated here. After the electronic device performs relevant processing on the above preset performance indicators, the performance of the electronic device can be improved, thereby helping to reduce the freeze of the video interface.

[0221] Optional, in Figure 6 On the basis of the corresponding embodiments, the electronic device performs processing related to the preset performance indicators, which may include: the electronic device determines that the CPU utilization of the process is abnormal; the electronic device performs detection related to the CPU utilization of the process, and the detection related to the CPU utilization of the process includes one or more of the following: frequency limit detection, scheduling priority detection, and inter-core running time detection; the electronic device records logs related to the CPU utilization of the process; the electronic device increases the operating frequency of the CPU, and / or increases the priority of the process; the electronic device determines that the memory application duration is abnormal; the electronic device performs detection related to the memory application duration, and the detection related to the memory application duration includes one or more of the following: available memory detection, slow path application times detection; the electronic device records logs related to the memory application duration; the electronic device cleans up the memory of the electronic device; the electronic device determines that the network signal is abnormal; the electronic device performs detection related to the network signal, and the detection related to the network signal includes one or more of the following: network delay detection, network packet loss detection; the electronic device records logs related to the network signal; the electronic device switches the network signal.

[0222] In the embodiment of the present application, the electronic device can perform processing related to the preset performance indicator by referring to the above Figure 5 The relevant description in the fault diagnosis and recovery process (3) of the corresponding embodiment is not repeated here. After the electronic device performs relevant processing on the above preset performance indicators, the performance of the electronic device can be improved, thereby helping to reduce the freeze of the video interface.

[0223] Optional, in Figure 6 On the basis of the corresponding embodiments, the method may also include: when the electronic device determines that a preset condition is met, the electronic device records the frame time; when the electronic device determines that a preset event occurs in the video playback interface, the electronic device sets the frame time to a target value; the electronic device determines that there is buffered data related to playing the video, including: when the frame time is not the target value, the electronic device determines that there is buffered data related to playing the video; wherein, meeting the preset condition includes any one of the following: there is no buffered data related to playing the video; there is buffered data related to playing the video, and the frame interval of the view displayed in the first control is less than the preset value.

[0224] In the embodiment of the present application, the target value can be used to indicate that there is no buffered data before the current frame image. For example, the target value can be the above Figure 5 The 0 in step S501 of the corresponding embodiment may also be other values, which is not limited in the embodiment of the present application.

[0225] The electronic device determines whether there is buffer data related to playing the video, which can reduce unnecessary execution steps, such as the above Figure 5 In a corresponding embodiment, when the electronic device determines that there is no buffer data related to playing the video, the electronic device may not execute steps S502 to S505, etc., which can improve the execution efficiency of the code.

[0226] Optional, in Figure 6 Based on the corresponding embodiment, the first control includes a texture view (TextureView) control or an independent layer view (SurfaceView) control.

[0227] In the embodiment of the present application, the texture view (TextureView) control or the independent layer view (SurfaceView) control can be a drawing control that does not load local data. By detecting the frame rate refresh of the drawing control that does not load local data, the electronic device can improve the accuracy of judging whether the video application interface is stuck, which helps to reduce the stuck of the video application interface.

[0228] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0229] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the method steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0230] The embodiment of the present application can divide the functional modules of the device implementing the method according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0231] like Figure 7 FIG. 7 is a schematic diagram of a chip structure provided by an embodiment of the present application. The chip 700 includes one or more (including two) processors 701 , a communication line 702 , a communication interface 703 and a memory 704 .

[0232] In some implementations, the memory 704 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.

[0233] The method described in the above embodiment of the present application can be applied to the processor 701, or implemented by the processor 701. The processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware or software instructions in the processor 701. The above processor 701 can be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components. The processor 701 can implement or execute the methods, steps and logic block diagrams related to each processing disclosed in the embodiment of the present application.

[0234] The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. Among them, the software module can be located in a mature storage medium in the field such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read only memory (EEPROM). The storage medium is located in the memory 704, and the processor 701 reads the information in the memory 704 and completes the steps of the above method in combination with its hardware.

[0235] The processor 701 , the memory 704 , and the communication interface 703 may communicate with each other via the communication line 702 .

[0236] In the above embodiments, the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory, or may be downloaded and installed in the memory in the form of software.

[0237] The embodiment of the present application also provides a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. For example, the available medium can include a magnetic medium (e.g., a floppy disk, a hard disk or a tape), an optical medium (e.g., a digital versatile disc (digital versatile disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0238] The present application also provides a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer-readable media may include computer storage media and communication media, and may also include any medium that can transfer a computer program from one place to another. The storage medium may be any target medium that can be accessed by a computer.

[0239] As a possible design, the computer readable medium may include a compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disc storage; the computer readable medium may include a magnetic disk storage or other magnetic disk storage device. Moreover, any connection line may also be appropriately referred to as a computer readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave), the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while optical discs reproduce data optically using lasers.

[0240] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

Claims

1. A video freeze detection method, characterized in that: The method comprises: The electronic device displays a video playback interface, the video playback interface including a first control and a second control, the first control is used to display a first type of view, the second control is used to display a second type of view, the first type of view does not include a view locally loaded from the electronic device; the second type of view includes a view locally loaded from the electronic device; When a frame interval of the view displayed in the first control is greater than or equal to a preset value, the electronic device determines that a video freeze occurs; wherein a frame interval of the view displayed in the second control is less than, equal to or greater than the preset value.

2. The method according to claim 1, characterized in that Before the electronic device determines that the video freezes, the method further includes: The electronic device determines that no preset event occurs in the video playback interface, and the preset event includes one or more of the following: an event in which a progress bar control in the video playback interface is triggered, an event for indicating that playback of the video playback interface is paused, and an event for indicating that the video playback interface is switched to the background.

3. The method according to claim 1 or 2, characterized in that: The preset value includes a first threshold value, and before the electronic device determines that a video freeze occurs, the electronic device further includes: The electronic device determines that the layer texture identifier of the view displayed in the first control has not changed; When the frame interval of the view displayed in the first control is greater than or equal to a preset value, the electronic device determines that a video freeze occurs, including: When the frame interval of the view displayed in the first control is greater than or equal to the first threshold, and the event of triggering the progress bar control and / or the event for indicating pausing the video playback interface does not occur in the video playback interface, the electronic device determines that the video is stuck.

4. The method according to claim 1 or 2, characterized in that: The preset value includes a second threshold value, and before the electronic device determines that the video freezes, the electronic device further includes: The electronic device determines that a layer texture identifier of a view displayed in the first control changes; When the frame interval of the view displayed in the first control is greater than or equal to a preset value, the electronic device determines that a video freeze occurs, including: When the frame interval of the view displayed in the first control is greater than or equal to the second threshold, and the event for indicating switching the video playback interface to the background does not occur in the video playback interface, the electronic device determines that the video is stuck.

5. The method according to any one of claims 1 to 4, characterized in that: The electronic device determines that a video freeze occurs, including: The electronic device records a log related to the freeze.

6. The method according to claim 5, characterized in that The logs related to the jamming may include one or more of the following: scene resource logs and scene event logs.

7. The method according to any one of claims 1 to 6, characterized in that: The electronic device determines that a video freeze occurs, including: The electronic device performs processing related to the preset performance indicator; the preset performance indicator includes one or more of the following: process CPU utilization of the electronic device, memory application duration of the electronic device, and network signal of the electronic device.

8. The method according to claim 7, characterized in that The electronic device performs processing related to the preset performance indicator, including: The electronic device determines that the CPU utilization of the process is abnormal; The electronic device performs detection related to the CPU utilization of the process, wherein the detection related to the CPU utilization of the process includes one or more of the following: frequency limit detection, scheduling priority detection, and inter-core running time detection; The electronic device records a log related to the CPU utilization of the process; The electronic device increases the operating frequency of the CPU and / or increases the priority of the process; The electronic device determines that an abnormality occurs in the memory application duration; The electronic device performs a detection related to the memory application duration, wherein the detection related to the memory application duration includes one or more of the following: an available memory detection and a slow path application times detection; The electronic device records a log related to the duration of the memory application; The electronic device clears the memory of the electronic device; The electronic device determines that an abnormality occurs in the network signal; The electronic device performs detection related to the network signal, wherein the detection related to the network signal includes one or more of the following: network delay detection and network packet loss detection; The electronic device records a log related to the network signal; The electronic device switches the network signal.

9. The method according to any one of claims 2 to 8, characterized in that: The method further comprises: When the electronic device determines that the preset condition is met, the electronic device records the frame time; When the electronic device determines that the preset event occurs in the video playback interface, the electronic device sets the frame time to a target value; The electronic device determines that there is buffer data related to playing a video, including: When the frame time is not the target value, the electronic device determines that there is buffer data related to the played video; Wherein, the satisfying of the preset condition includes any one of the following: There is no buffering data associated with playing the video; There is buffer data related to playing the video, and the frame interval of the view displayed in the first control is less than the preset value.

10. The method according to any one of claims 1 to 9, characterized in that: The first control includes a texture view control or an independent layer view control.

11. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the method according to any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the computer executes the method according to any one of claims 1 to 10.

13. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables an electronic device to execute the method according to any one of claims 1 to 10.

Images