Dynamic effect detection method and equipment

Abstract

The invention discloses a dynamic effect detection method and equipment, and the method comprises the steps: generating a first change curve according to the display characteristics of each frame in a target dynamic effect display process; and determining the fluency of the target dynamic effect display process according to the change trend of the first change curve. According to the technical scheme, the consistency of the detection result and the actual perception of the user can be improved while the fluency of the target dynamic effect display process is effectively detected.

Description

Technical Field

[0001] The present invention relates to the field of terminal technology, and in particular to a motion effect detection method and device. Background Art

[0002] There are many "motion effects" or animation effects in terminal devices, such as the motion effects when starting and exiting an application, and the lock screen and unlock effects. Taking the motion effect when the application starts as an example, when the user clicks the application icon, the motion effect when the application starts is triggered, and the application icon gradually transforms into an application interface that fills the entire screen. During the motion effect display process, frame loss may occur, causing the user to perceive the motion effect as stuck. To this end, it is necessary to provide a detection mechanism for the motion effect display process to determine the smoothness of the motion effect display process. Summary of the Invention

[0003] The purpose of the present invention is to provide a motion effect detection method and device for detecting the motion effect display process and determining the smoothness of the motion effect display process.

[0004] In a first aspect, the present invention provides a motion effect detection method, comprising: generating a first change curve based on the display characteristics of each frame during the target motion effect display process; and determining the smoothness of the target motion effect display process based on the change trend of the first change curve.

[0005] In the above implementation, the display characteristics of each frame during the target motion effect display process may include the display size of each frame of the motion effect screen, the position of the center point of each frame of the motion effect screen, the brightness of each frame of the motion effect screen, etc. Based on the changing trend of the change curve corresponding to the actual display process of the target motion effect, the smoothness of the target motion effect display process can be effectively characterized. Specifically, the closer the changing trend of the change curve is to the changing trend under the ideal state, the higher the smoothness of the target motion effect display process, and vice versa, the worse the smoothness of the target motion effect display process.

[0006] Furthermore, in the stage where the target motion effect changes at a faster rate, if frame loss occurs at this time, the user will perceive a more obvious freeze. Correspondingly, the rate of change of the corresponding change curve is also large, which will have a greater impact on the change trend of the change curve. At this time, based on the above technical solution, a detection result of poor smoothness of the target motion effect display process will be obtained, which is consistent with the user's perception. In the stage where the target motion effect changes at a slower rate, if frame loss occurs at this time, the user will perceive a less obvious freeze. Correspondingly, the rate of change of the corresponding change curve is also small, which will not have a greater impact on the change trend of the change curve. At this time, based on the above technical solution, a detection result of good smoothness of the target motion effect display process will be obtained, which is consistent with the user's perception.

[0007] It can be seen that the above implementation scheme can effectively detect the smoothness of the target motion effect display process while improving the consistency between the detection results and the user's actual perception.

[0008] In certain implementations of the first aspect of the motion effect detection method described above, the method further includes: obtaining display features of each frame during the display of the target motion effect.

[0009] In certain implementations of the first aspect of the motion effect detection method described above, obtaining display features of each frame during the display of a target motion effect includes: obtaining display features of a target layer in each frame during the display of the target motion effect, where the target layer is the layer where the target motion effect is located.

[0010] In the above implementation method, since the terminal device controls the merging of different layers and then controls the display of the merged layers, during the display of the target motion effect, the display characteristics of the layer where the target motion effect of each frame is located can be used to represent the display characteristics of each frame during the display of the target motion effect.

[0011] In certain implementations of the first aspect of a motion effect detection method as described above, obtaining the display characteristics of the target layer of each frame during the display of the target motion effect includes: obtaining the display characteristics of the target layer of each frame in which no frames are lost during the display of the target motion effect; performing frame interpolation processing on the display characteristics of the target layer of each frame in which no frames are lost, to obtain the display characteristics of the target layer of each frame during the display of the target motion effect.

[0012] In the above implementation, the change curve is generated based on the acquired display features of each frame. Therefore, if a frame is lost, the display features of the lost frame cannot be acquired. As a result, the display features of that frame will be missing from the generated change curve, resulting in the change curve being unable to accurately represent the smoothness of the target animation display process. To prevent this from happening, the above implementation can first perform frame infill processing on the display features of the lost frame, so that the generated change curve fully includes the display features of each frame.

[0013] In some implementations of the first aspect of a motion effect detection method as described above, display characteristics of target layers of each frame in which no frame is lost during the display of the target motion effect are obtained, including: detecting a trigger signal for a display operation on the target layer of the Nth frame, and obtaining the display characteristics of the target layer of the Nth frame when the rendering of the target layer of the Nth frame is completed; and determining not to obtain the display characteristics of the target layer when the rendering of the target layer of the Nth frame is not completed; wherein the Nth frame is any frame in the display process of the target motion effect.

[0014] In certain implementations of the first aspect of the motion effect detection method described above, the display characteristics of the target layer include: the size of the target layer, or the position of the center point of the target layer.

[0015] In some implementations of the first aspect of a motion effect detection method as described above, the smoothness of the target motion effect display process is determined based on the changing trend of the first change curve, including: calculating the similarity between the first change curve and the second change curve in the changing trend, the second change curve is used to describe the target display progress of the target motion effect in each frame, and the similarity is positively correlated with the smoothness of the target motion effect display process.

[0016] In the above implementation, the target display progress refers to the proportion of the current animation process to the total animation process when there are no display anomalies such as frame loss. The second change curve is the change curve of the display progress of each frame in the target motion effect display process when there are no display anomalies such as frame loss. The closer the change trend of the first change curve corresponding to the actual display process is to the change trend of the second change curve under ideal conditions, the better the smoothness of the actual display process of the target motion effect. Otherwise, the smoothness of the actual display process of the target motion effect is worse.

[0017] In a second aspect, the present technical solution provides an electronic device comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, enable the device to execute the method in the first aspect or any possible implementation of the first aspect.

[0018] In a third aspect, the present invention further provides a chip comprising a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface and executes the method in the first aspect or any possible implementation of the first aspect.

[0019] Optionally, as an implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method in the first aspect or any possible implementation of the first aspect.

[0020] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable medium stores program code for execution by a device, wherein the program code includes instructions for executing the method in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a schematic diagram of a scenario of the motion effect detection method provided in an embodiment of the present application;

[0022] Figure 2This is another scenario diagram of the motion effect detection method provided in an embodiment of the present application;

[0023] Figure 3 This is a timing control diagram of the dynamic effect display process provided by an embodiment of the present application;

[0024] Figure 4 This is another timing control diagram of the dynamic effect display process provided by an embodiment of the present application;

[0025] Figure 5 is a progress curve diagram of the dynamic effect display process provided by the embodiment of the present application;

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0027] Figure 7 This is a schematic flow chart of the motion effect detection method provided in an embodiment of the present application;

[0028] Figure 8 This is another scenario diagram of the motion effect detection method provided in an embodiment of the present application;

[0029] Figure 9 This is another scenario diagram of the motion effect detection method provided in an embodiment of the present application;

[0030] Figure 10 This is another scenario diagram of the motion effect detection method provided in an embodiment of the present application;

[0031] Figure 11 This is another scenario diagram of the motion effect detection method provided in an embodiment of the present application;

[0032] Figure 12 This is another scenario diagram of the motion effect detection method provided in an embodiment of the present application;

[0033] Figure 13 This is a schematic diagram of a software structure of an electronic device provided in an embodiment of the present application;

[0034] Figure 14 This is another schematic flowchart of the motion effect detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] There are various "animation effects", namely motion effects, in terminal devices. For example, in the scenario where the user launches an application by clicking the application icon in the main interface, the application launch animation effect will be displayed, which is used to complete the switch from the application icon to the application interface in the form of animation; for example, in the scenario where the user exits the application by swiping up, the application exit animation effect will be displayed, which is used to complete the switch from the application interface to the application icon in the form of animation.

[0036] For the application startup scenario, for example, take setting up the application as an example, Figure 1 As shown in the figure, in the startup scene of the application, the display process of the application startup animation is that the application icon gradually enlarges and switches from the icon form to the application interface form, and then the application interface gradually enlarges until it is displayed in full screen. For the scene of exiting the application by swiping up, such as Figure 2 As shown, the display process of the application exit animation is that the application interface gradually shrinks and switches from the application interface form to the icon form, and then the application icon gradually shrinks until it becomes the size of a regular application icon.

[0037] During the display of the above-mentioned dynamic effects, frame loss may occur. For ease of understanding, the control process of dynamic effect display is briefly described first. Specifically, the terminal device can control the drawing and display of each frame of the display screen according to the vertical synchronization (Vsync) signal. During the dynamic effect display process, if Figure 3 As shown in the figure, when the N-1th Vsync signal is detected, the relevant functional units in the terminal device start drawing and rendering the N-1th frame display screen, and at the same time, display the rendered N-2th frame display screen. When the Nth Vsync signal is detected, the drawing and rendering of the Nth frame display screen is triggered, and the rendered N-1th frame display screen is displayed. In this cycle, normal dynamic effect display can be achieved. However, in some scenarios, such as Figure 4 As shown, after receiving the Nth Vsync signal, the drawing and rendering time of the Nth frame may be too long due to reasons such as excessive load on the terminal device, even exceeding the Vsync signal interval. At this time, if the N+1th Vsync signal is detected, the Nth frame display image has not yet been rendered, and the terminal device will not be able to display the Nth frame display image, but will continue to display the N-1th frame display image. In this way, the Nth frame is dropped, and the user's perspective will see the display image stay at the N-1th frame for a long time, which will cause the user to experience screen freezes.

[0038] In view of the above situation, it is necessary to detect the motion effect display process and determine the smoothness of the motion effect display process, which will help to make targeted improvements to the terminal equipment.

[0039] One technical solution involves detecting the cumulative duration of frame drops during the animation display process and using this duration to describe the smoothness of the animation. Longer cumulative frame drops indicate more noticeable stuttering and poorer smoothness during the animation display process. Conversely, shorter frame drops indicate less noticeable stuttering and better smoothness during the animation display process.

[0040] However, the smoothness of the motion effect display process described based on the cumulative frame loss duration may not be consistent with the smoothness perceived by the user. The reason is that in the motion effect design stage, in order to make the change process of the motion effect more in line with the needs of human perception, the display process of many motion effects is not a linear change process, but a nonlinear process, such as first changing quickly and then slowly, or first changing slowly and then changing quickly. Then, for the same cumulative frame loss duration, when frame loss occurs in different display stages of the motion effect, the degree of freeze perceived by the user is different. Specifically, compared with the slow-changing stage of the motion effect, the freeze perceived by the user is more obvious when frame loss occurs in the fast-changing stage of the motion effect.

[0041] For ease of understanding, Figure 5 A curve chart of the application startup dynamic effect display process is given, which can be used to describe the display progress of the dynamic effect at different moments in the dynamic effect display process. Figure 5 It can be seen that during the T1 period, the rate of change of the animation display progress is faster, so the difference between the two adjacent frames of the display will be larger. Therefore, if frame loss occurs at this time, when the display screen does not change for a long time, the user may perceive a noticeable freeze in the display screen. During the T2 period, the rate of change of the animation display progress tends to be slower, so the difference between the two adjacent frames of the display will decrease. Therefore, if frame loss occurs at this time, even if the display screen does not change for a long time, the user will not feel a noticeable freeze.

[0042] In response to the above problems, an embodiment of the present application can provide a motion effect detection method, which can be used to detect the motion effect display process, determine the smoothness of the motion effect display process, and at the same time, improve the consistency between the detection results and user perception.

[0043] Figure 6 A schematic structural diagram of an electronic device 100 provided in an embodiment of the present application is shown.

[0044] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a fingerprint sensor 180H, a pressure sensor 180A, a touch sensor 180K, an ambient light sensor 180L, a proximity light sensor 180G, a temperature sensor 180J, a distance sensor 180F, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a bone conduction sensor 180M, etc.

[0045] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0046] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0047] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0048] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0049] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0050] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0051] The MIPI interface can be used to connect the processor 110 and the display screen 194 . The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100 .

[0052] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0053] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0054] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0055] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0056] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as an unlocking function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as fingerprint data, lock screen interface data, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0057] The following examples of this application will be Figure 6 Taking the electronic device with the structure shown as an example, the motion effect detection method provided in the embodiment of the present application is specifically explained.

[0058] Figure 7 This is a schematic flow chart of the motion effect detection method provided in the embodiment of the present application. Figure 7 As shown, the motion effect detection method provided in the embodiment of the present application includes:

[0059] 101. The terminal device determines that the target animation effect starts.

[0060] 102. The terminal device determines a theoretical change curve corresponding to the target motion effect.

[0061] In the embodiment of the present application, the target animation can be any animation provided by the terminal device system, such as application startup animation, application exit animation, lock screen animation, unlock animation, switching animation between different pages of the main interface, notification bar display animation, etc. Alternatively, the target animation can also be any animation provided by the target application in the terminal device, such as switching animation between different application interfaces of the target application. Among them, the target application can be, for example, a self-developed application of the terminal device manufacturer, or, in another implementation method, the target application can also be a third-party application.

[0062] The terminal device can determine the theoretical change curve corresponding to the target motion effect after determining that the target motion effect has started. In an embodiment of the present application, the theoretical change curve can be used to describe the change curve of the display progress of each frame during the display of the target motion effect in the absence of display anomalies such as frame loss. Among them, the display progress can be characterized by the proportion of the current animation process to the total animation process. According to the above description, it can be known that the display process of the target motion effect is a nonlinear change process, such as a rapid change first and then a slow change, etc., that is, the display progress of the target motion effect changes nonlinearly with time, wherein, for example, assuming that the display duration of the target motion effect is T, then, in At this moment, the displayed progress of the target animation may not be 50%.

[0063] The following describes how to determine the theoretical change curve of the target motion effect.

[0064] First, after determining that the target animation has started, the terminal device can obtain the animation display parameters corresponding to the target animation. The animation display parameters may include, but are not limited to: the preset display duration of the animation, and the animation interpolator. Among them, the preset display duration of the animation specifies the display duration of the target animation, and for the same animation, its display duration remains fixed each time it is displayed. The animation interpolator can be used to control the rate of change of the animation so that the animation changes at a preset rate, such as accelerating first and then decelerating, or decelerating first and then accelerating.

[0065] The terminal device can then determine the timestamp of the first frame of the target animation after it begins drawing. Furthermore, the terminal device can calculate the theoretical change curve of the target animation based on parameters such as the display duration of the target animation, the animation interpolator, the first frame drawing timestamp, and the vsync period.

[0066] 103. The terminal device obtains display features of each frame during the display of the target motion effect.

[0067] 104. The terminal device generates an actual change curve corresponding to the target motion effect according to the display characteristics of each frame during the display of the target motion effect.

[0068] The display characteristics of each frame during the target motion effect display process may be the size of each frame of the motion effect screen, the position of the center point of the motion effect screen, or the brightness of the motion effect screen, etc.

[0069] Specifically, the display of a terminal device often involves multiple layers. During a display process, the terminal device needs to first draw and render each layer to be displayed, and then synthesize each layer according to the set layer order, and finally display it. In the scene of displaying a target motion effect, the display characteristics of each frame during the display of the target motion effect can be characterized by the display characteristics of the layer where the target motion effect is located (hereinafter referred to as the "target layer").

[0070] Taking the application startup scenario as an example, Figure 8 A schematic diagram of the target layer's changes during the target animation display process is given, as shown in the following figure: Figure 8 As shown in , during the application startup process, the size of the target layer gradually increases, and at the same time, the position of the target layer center point gradually changes, wherein the position of the target layer center point can be the position within the display screen. Figure 9 As shown, during the process of exiting the application, the size of the target layer gradually becomes smaller and the position of the center point of the target layer gradually changes.

[0071] It can be understood that, in accordance with the change in the display progress of the target motion effect, the change in the target layer size and the change in the position of the target layer center point within the display screen are also nonlinear change processes, and in the absence of display anomalies such as frame loss, the change trend of the change curve corresponding to the target layer size and the center point position should be consistent with the change trend of the aforementioned theoretical change curve. Therefore, in an embodiment of the present application, during the display of the target motion effect, the terminal device can obtain the display characteristics of the target layer of each frame, and after the target motion effect ends, based on the display characteristics of the target layer of each frame, generate an actual change curve corresponding to the display process of the target motion effect. Among them, the display characteristics of the target layer of each frame can be, for example, the layer size, or the position of the center point of the layer, the brightness of the layer, etc.

[0072] 105. The terminal device determines the smoothness of the target motion effect display process according to the actual change curve and the theoretical change curve.

[0073] According to the above description, it can be understood that in the absence of display anomalies such as frame loss, during the display of the target motion effect, the change trend of the actual change curve of the target layer should be close to the change trend of the theoretical change curve. Accordingly, in the embodiment of the present application, the terminal device can detect the degree of similarity between the actual change curve and the theoretical change curve in the change trend. This similarity can be used to characterize the smoothness of the target motion effect display process. The higher the similarity, the higher the smoothness of the target motion effect display process. Conversely, the worse the smoothness of the target motion effect display process.

[0074] Figure 10 A schematic diagram of the fitting effect of the actual change curve of the target layer and the theoretical change curve corresponding to the target animation effect in the application startup scenario without frame loss is given, as shown in Figure 10 As shown in the figure, when there is no frame loss, the two have a good fit and their change trends are similar.

[0075] Figure 11The following is a diagram showing the fitting effect of the actual change curve of the target layer and the theoretical change curve corresponding to the target animation in the application startup scenario, when the target animation change rate is fast and two frames are lost. If two frames are lost, the terminal device will control the previous frame target layer to be displayed again. Therefore, the display characteristics of the target layer corresponding to the two lost frames remain unchanged compared to the previous frame. Figure 11 As shown in FIG, since the target motion effect changes at a fast rate, frame loss will cause obvious discontinuity between adjacent frames, resulting in a significant decrease in the similarity of the change trends between the two change curves.

[0076] Figure 12 The following is a diagram showing the fitting effect of the actual change curve of the target layer and the theoretical change curve corresponding to the target animation in the application startup scenario, when the target animation change rate is slow and the number of frames lost is two. Consistent with the above, for two frames lost, the terminal device will control the previous frame target layer to be displayed again. Therefore, the display characteristics of the target layer corresponding to the two lost frames do not change compared to the previous frame. However, if Figure 12 As shown, since the target motion effect changes slowly at this time, frame loss does not cause obvious discontinuity between adjacent frames, and thus will not significantly reduce the similarity of the change trends between the two change curves.

[0077] From the above, it can be understood that in the scenario where frame loss occurs when the target animation changes at a fast rate, the sense of stuttering from the user's perspective caused by frame loss will be more obvious. Consistent with user perception, based on the technical solution provided in the embodiment of this application, the degree of similarity in the change trend between the actual change curve of the target layer and the theoretical change curve corresponding to the target animation is significantly reduced. In the scenario where frame loss occurs when the target animation changes at a slow rate, the sense of stuttering from the user's perspective caused by frame loss will be relatively mild. Consistent with user perception, based on the technical solution provided in the embodiment of this application, the degree of similarity in the change trend between the actual change curve of the target layer and the theoretical change curve corresponding to the target animation is not significantly reduced.

[0078] It can be seen that the technical solution provided by the embodiment of the present application can effectively detect the smoothness of the target motion effect display process, and can improve the consistency between the detection results and the user's actual perception.

[0079] In another embodiment of the present application, the motion effect detection method provided in the embodiment of the present application is described in combination with the software structure of the electronic device 100.

[0080] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0081] Figure 13 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0082] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer (APP), the application framework layer (Framework), the native layer (Native), and the kernel layer (not shown).

[0083] The application layer may include a target application, which can be any application installed on the terminal device. In the target application's usage scenario, the target application can display the target motion effect. At the same time, the application layer may also include a choreographer. The choreographer may be located in the target application's user interface (UI) thread. The choreographer may be used to send the timestamp of the first frame drawing to the fluency detector (FluencyDetector) after detecting the first frame drawing of the target motion effect.

[0084] The application framework layer may include a FluencyDetector, which can be used to obtain the target animation's display duration and interpolator information from the application layer. Furthermore, when the first frame of the target animation begins drawing, the FluencyDetector can also obtain the timestamp of the target animation's first frame drawing through Choreographer. Furthermore, the FluencyDetector can be used to calculate the theoretical change curve of the target animation's display process based on information such as the target animation's display duration, interpolator, and first frame drawing timestamp.

[0085] The local layer may include the display system service SurfaceFlinger, which is a service in the Android system that can be used to obtain layer data from different application layer programs, synthesize it, and display it. In an embodiment of the present application, during the display of the target motion effect, SurfaceFlinger can be used to obtain the display characteristics of the target layer, such as the layer size attribute matrix, the layer center point position information, etc., before each frame of the target layer is displayed. In addition, SurfaceFlinger can also be used to generate the actual change curve corresponding to the target motion effect display process based on the display characteristics of the target layer of each frame obtained, and determine the degree of similarity between the actual change curve and the theoretical change curve in the change trend.

[0086] Furthermore, the local layer may also include a maintenance and testing unit, such as a hiview service set. This maintenance and testing unit can be used to diagnose faults in the display process of the target animation based on the degree of similarity between the actual change curve and the theoretical change curve. If an anomaly is detected in the display process of the target animation, the maintenance and testing unit can also collect relevant logs and upload the fault diagnosis results and related logs to the cloud server.

[0087] The method provided in the embodiment of the present application can be applied to the internal testing stage of the terminal device before it leaves the factory. In this implementation, the fault diagnosis results and log data uploaded by the maintenance and testing unit to the cloud server can be used by developers to locate the specific cause of the target animation freeze, and optimize the terminal device accordingly to improve the user experience. Alternatively, the method provided in the embodiment of the present application can be applied to the process of the user actually using the terminal device. In this implementation, in order to protect the user's privacy data, the data uploaded to the cloud server by the maintenance and testing unit may not include log data, but include fault diagnosis results. The fault diagnosis results may specifically include the degree of similarity between the actual change curve and the theoretical change curve in the change trend, information about the target application, etc. In this implementation, the fault diagnosis results can be used by developers to perform online upgrades to the version of the terminal device's system or application.

[0088] In another embodiment of the present application, Figure 13 Taking the electronic device with the shown architecture as an example, the motion effect detection method provided in the embodiment of the present application is further explained.

[0089] Figure 14 is another schematic flow chart of the motion effect detection method provided in the embodiment of the present application, such as Figure 14 As shown, the motion effect detection method provided in the embodiment of the present application includes:

[0090] 201, the target application determines that the target animation effect starts.

[0091] 202. The target application sends the display duration and interpolator of the target animation to FluencyDetector.

[0092] In an embodiment of the present application, the target application may maintain information on the display durations corresponding to different animations in the target application and animation interpolators. The information on the display duration of the animation refers to the theoretical display duration of the animation, which is a preset fixed value. The target application may control the start of the target animation. When the target animation starts, the target application may send the display duration and interpolator corresponding to the target animation to the FluencyDetector. Specifically, after determining that the target animation has started, the target application may call the interface function beginDetect() provided in an embodiment of the present application, and through this interface function, pass the display duration and interpolator corresponding to the target animation to the FluencyDetector.

[0093] 203. FluencyDetector sends a notification message to SurfaceFlinger to start the detection process.

[0094] 204. FluencyDetector registers a notification message of the timestamp of the first frame of the target animation with Choreographer.

[0095] After receiving the display duration and interpolator corresponding to the target motion effect, on the one hand, FluencyDetector can call the beginDetect() function to send a notification message of the detection process to SurfaceFlinger. On the other hand, FluencyDetector can obtain the timestamp information of the first frame drawing after the first frame drawing of the target motion effect begins. In an exemplary implementation, FluencyDetector can set a first frame drawing callback function (Callback) on the Choreographer of the UI thread of the target application. Then, when the first frame drawing starts, the callback function can be triggered. At this time, Choreographer can send the timestamp of the first frame drawing to FluencyDetector. In an embodiment of the present application, the callback function can be automatically destroyed after Choreographer sends the timestamp of the first frame drawing to FluencyDetector.

[0096] The embodiment of the present application does not limit the execution order of the above-mentioned step 203 and step 204. In another implementation, step 204 can be executed first and then step 203, or step 203 and step 204 can be executed simultaneously.

[0097] 205, Choreographer determines that the first frame drawing starts.

[0098] 206, Choreographer sends the timestamp of the first frame drawn to FluencyDetector.

[0099] 207, FluencyDetector determines a theoretical change curve of the target motion effect display process according to the display duration, interpolator, vsync period, and first frame drawing timestamp of the target motion effect.

[0100] In the embodiments of the present application, for example, the theoretical change curve of the target motion effect display process can be described in the form of an array, where each element in the array represents the display progress of each frame during the motion effect display process. Then, at each moment when the target motion effect changes rapidly, the numerical fluctuations between the corresponding elements in the array are large, while at each moment when the target motion effect changes slowly, the numerical fluctuations between the corresponding elements in the array are small.

[0101] 208, SurfaceFlinger obtains the display characteristics of the target layer of each frame.

[0102] Based on the above step 203, after receiving the notification message of starting the detection process, SurfaceFlinger can obtain the display characteristics of the target layer of each frame during the target animation display process. It should be noted that in this embodiment of the application, step 208 can be started at any time after step 203, without waiting for steps 204 to 207 to be completed.

[0103] During the display process of the target animation, for any frame of the target layer, after the vsync period is reached, SurfaceFlinger can be used to call the doTrasactions() function to perform the display operation. In the embodiment of the present application, the onLayerPresent() function can be added to the doTrasactions() function. Then, after SurfaceFlinger calls the doTrasactions() function, SurfaceFlinger can obtain the size attribute matrix of the target layer by calling the onLayerPresent() function.

[0104] In one possible implementation, the calling condition of the onLayerPresent() function is that it is called after the doTrasactions() function is called. Based on this implementation, after each vsync signal cycle is reached, if the target layer of the frame to be displayed has been rendered, then SurfaceFlinger can obtain the size information of the target layer of the frame to be displayed. If the target layer of the frame to be displayed has not been rendered, then SurfaceFlinger can obtain the size information of the target layer of the previous frame of the frame to be displayed. Specifically, when the vsync signal used to trigger the display of the Nth frame arrives, if the rendering of the Nth frame is completed, then SurfaceFlinger can control the display of the Nth frame by calling the doTrasactions() function, and SurfaceFlinger can obtain the size attribute matrix of the target layer of the Nth frame by calling the onLayerPresent() function. If the Nth frame is not rendered, SurfaceFlinger can control the display of the N-1th frame by calling the doTrasactions() function, and SurfaceFlinger can obtain the size attribute matrix of the target layer of the N-1th frame by calling the onLayerPresent() function. That is to say, based on this implementation method, when the Nth frame is dropped, SurfaceFlinger will control the display of the target layer of the N-1th frame again, and SurfaceFlinger will obtain the size information of the target layer of the N-1th frame again.

[0105] It can be understood that in this implementation, after the target animation ends, if there is a frame loss during the target animation display process, for the lost frame N, the size information of the target layer of each frame obtained by SurfaceFlinger still corresponds to the data of that frame, and the data of that frame is consistent with the data of the N-1th frame. Based on this implementation, in the subsequent step 212, SurfaceFlinger can directly generate the actual change curve of the target animation display process based on the size information of the target layer of each frame that has been obtained.

[0106] In another possible implementation, the calling condition of the onLayerPresent() function is that it is called after the doTrasactions() function is called and the rendering of the frame to be displayed is completed. Based on this implementation, after each vsync signal cycle is reached, if the target layer of the frame to be displayed has been rendered, then SurfaceFlinger can obtain the size information of the target layer of the current frame. If the target layer of the frame to be displayed has not been rendered, it means that the frame to be displayed has been lost. Then, SurfaceFlinger can determine not to call the onLayerPresent() function and there is no need to obtain the size information of the target layer. For example, when the vsync signal used to trigger the display of the Nth frame arrives, if the rendering of the Nth frame is completed, then SurfaceFlinger can control the display of the target layer of the Nth frame by calling the doTrasactions() function, and SurfaceFlinger can obtain the size attribute matrix of the target layer of the Nth frame by calling the onLayerPresent() function. When the vsync signal used to trigger the display of the Nth frame arrives, if the Nth frame is not rendered, it means that the Nth frame is lost. At this time, SurfaceFlinger only needs to control the display of the target layer of the N-1th frame by calling the doTrasactions() function, without calling the onLayerPresent() function to obtain the size information of the target layer.

[0107] It can be understood that in this implementation, when the target animation effect ends, if there is frame loss in the target animation effect display process, then the size information of the target layer of each frame obtained by SurfaceFlinger will not contain the data corresponding to the lost frame. Based on this implementation, in the subsequent step 212, SurfaceFlinger can first perform frame filling processing, add the size information of the target layer of the lost frame to the size information of the target layer of each frame obtained, and then generate the actual change curve of the target animation effect display process according to the size information of the target layer of each frame obtained after the frame filling processing.

[0108] 209. The target application determines that the target animation effect ends.

[0109] 210. The target application sends a notification message to FluencyDetector indicating that the target animation has ended.

[0110] After determining that the target animation has ended, the target application can send a notification message of the end of the target animation to FluencyDetector by calling the endDetect() function, which can then trigger FluencyDetector to send the calculated theoretical change curve of the target animation display process to SurfaceFlinger.

[0111] 211. FluencyDetector sends a theoretical change curve of the target motion effect display process to SurfaceFlinger.

[0112] 212, SurfaceFlinger determines the actual change curve of the target motion effect display process according to the display characteristics of the target layer of each frame.

[0113] After receiving the theoretical change curve sent by FluencyDetector, SurfaceFlinger can determine that the target animation has ended. At this time, SurfaceFlinger can process the size information of the target layer of each frame during the target animation display process by calling the ProcessLayerPresent() function to generate the actual change curve corresponding to the target animation display process. For example, the actual change curve of the target animation display process can also be described in the form of an array, and each element in the array can represent the size of the target layer of each frame during the animation display process.

[0114] 213, SurfaceFlinger determines the similarity between the theoretical change curve and the actual change curve of the target motion effect display process in terms of their change trends.

[0115] In the embodiment of the present application, although the physical meanings represented by the values ​​corresponding to each moment on the theoretical change curve and the actual change curve are different, in the absence of frame loss, the change trends of the two are close. Therefore, the degree of similarity between the theoretical change curve and the actual change curve in the target motion effect display process in terms of change trends can be determined, and the degree of smoothness of the actual display process of the target motion effect can be represented by the degree of similarity between the two in terms of change trends. The higher the degree of similarity between the two in terms of change trends, the higher the degree of flow of the actual display process of the target motion effect, and vice versa, the lower the degree of flow of the actual display process of the target motion effect.

[0116] Moreover, when frame loss occurs during a period when the target motion effect changes slowly, the impact of frame loss on the similarity between the theoretical change curve and the actual change curve is relatively small, and the degree of lag perceived by the user is also relatively mild. Conversely, when frame loss occurs during a period when the target motion effect changes rapidly, the impact of frame loss on the similarity between the theoretical change curve and the actual change curve is relatively large, and the degree of lag perceived by the user is also less obvious. It can be seen that the method provided in the embodiment of the present application can make the obtained detection results more consistent with user perception.

[0117] It should be understood that the electronic devices here are embodied in the form of functional units. The term "unit" here can be implemented in the form of software and / or hardware, without specific limitation. For example, a "unit" can be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a proprietary processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a merged logic circuit, and / or other suitable components that support the described functions. Whether a function is executed in hardware or in a computer software-driven hardware manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments.

[0118] The module division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other divisions may be employed. For example, functional modules may be divided according to their respective functions, or two or more functions may be integrated into a single processing module. These integrated modules may be implemented in hardware.

[0119] An embodiment of the present application also provides an electronic device, which includes a storage medium and a central processing unit. The storage medium can be a non-volatile storage medium, in which a computer executable program is stored. The central processing unit is connected to the non-volatile storage medium and executes the computer executable program to implement the above-mentioned motion effect detection method.

[0120] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a computer, the computer executes each step of the motion effect detection method of the embodiment of the present application.

[0121] The embodiment of the present application also provides a computer program product containing instructions. When the computer program product is run on a computer or any at least one processor, it enables the computer to execute each step of the motion effect detection method of the embodiment of the present application.

[0122] An embodiment of the present application also provides a chip, including a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to execute corresponding operations and / or processes performed by the motion effect detection method provided in the present application.

[0123] Optionally, the chip further includes a memory, which is connected to the processor via a circuit or wire, and the processor is configured to read and execute a computer program in the memory. Further optionally, the chip further includes a communication interface, to which the processor is connected. The communication interface is configured to receive data and / or information to be processed, and the processor obtains the data and / or information from the communication interface and processes the data and / or information. The communication interface may be an input / output interface.

[0124] The memory may be a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer.

[0125] In the embodiments of the present application, "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. A and B may be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0126] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented using a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0127] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0128] In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0129] The above description is merely a specific embodiment of the present application. Any person skilled in the art may easily conceive of variations or substitutions within the technical scope disclosed in this application, and such variations or substitutions shall be within the scope of protection of this application. The scope of protection of this application shall be subject to the scope of protection of the claims.

Claims

1. A method for detecting dynamic effects, characterized in that: include: generating a first change curve according to display characteristics of each frame during the display of the target motion effect; The smoothness of the target motion effect display process is determined according to the change trend of the first change curve.

2. The method according to claim 1, characterized in that The method further comprises: Obtain the display features of each frame during the target motion effect display process.

3. The method according to claim 2, characterized in that Obtain the display characteristics of each frame during the target animation display process, including: Obtain display characteristics of a target layer in each frame during the display of the target motion effect, where the target layer is the layer where the target motion effect is located.

4. The method according to claim 3, characterized in that Obtain the display characteristics of the target layer in each frame during the target animation display process, including: Obtain the display characteristics of the target layer of each frame without frame loss during the display of the target motion effect; Performing frame supplementation processing on the display characteristics of the target layer of each frame that has not been lost, to obtain the display characteristics of the target layer of each frame during the target motion effect display process.

5. The method according to claim 4, characterized in that Obtain the display characteristics of the target layer for each frame without frame loss during the target animation display process, including: detecting a trigger signal for a display operation of a target layer of the Nth frame, and obtaining display characteristics of the target layer of the Nth frame if rendering of the target layer of the Nth frame is complete; and determining not to obtain display characteristics of the target layer if rendering of the target layer of the Nth frame is not complete; The Nth frame is any frame in the process of displaying the target motion effect.

6. The method according to claim 3, characterized in that The display characteristics of the target layer include: the size of the target layer, or the position of the center point of the target layer.

7. The method according to any one of claims 1 to 6, characterized in that Determining the smoothness of the target motion effect display process according to the change trend of the first change curve includes: The similarity between the first change curve and the second change curve in the change trend is calculated, where the second change curve is used to describe the target display progress of the target motion effect in each frame, and the similarity is positively correlated with the smoothness of the target motion effect display process.

8. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, cause the device to perform the method according to any one of claims 1 to 7.

9. A chip, characterized in that: The chip includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface and executes the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores program instructions, which, when executed on an electronic device, enable the electronic device to execute the method according to any one of claims 1 to 7.

Images