A motion effect detection method and apparatus

Abstract

The application discloses a dynamic effect detection method and device, the dynamic effect detection method comprises the following steps: generating a first change curve according to the display characteristics of each frame in a target dynamic effect display process; and determining the fluency of the target dynamic effect display process according to the change trend of the first change curve. The above technical solution can effectively detect the fluency of the target dynamic effect display process, and improve the consistency between the detection result and the actual perception of a user.

Description

Technical Field

[0001] This invention relates to the field of terminal technology, and in particular to a motion effect detection method and device. Background Technology

[0002] Terminal devices contain various "motion effects," or animations, such as app launch and exit animations, and screen lock and unlock animations. Taking app launch animations as an example, when a user clicks the app icon, the launch animation is triggered, and the app icon gradually transforms into an app interface that fills the entire screen. During the animation display, frame drops may occur, causing a perceived stuttering effect for the user. Therefore, it is necessary to provide a detection mechanism for the animation display process to determine its smoothness. Summary of the Invention

[0003] The purpose of this invention is to provide a motion effect detection method and device for detecting motion effect display processes and determining the smoothness of the motion effect display process.

[0004] In a first aspect, the present invention provides a motion effect detection method, comprising: generating a first change curve based on the display characteristics of each frame during the display of a target motion effect; and determining the smoothness of the display process of the target motion effect based on the changing trend of the first change curve.

[0005] In the above implementation, the display characteristics of each frame during the target motion effect display process can include the display size of each frame's motion effect image, the position of the center point of each frame's motion effect image, and the brightness of each frame's motion effect image. Based on the changing trend of the change curve corresponding to the actual display process of the target motion effect, the smoothness of the target motion effect display process can be effectively characterized. Specifically, the closer the changing trend of the changing curve is to the changing trend under the ideal state, the higher the smoothness of the target motion effect display process; conversely, the lower the smoothness of the target motion effect display process.

[0006] Furthermore, during phases where the target animation changes rapidly, if frame drops occur, the user will perceive noticeable stuttering. Correspondingly, the rate of change of the corresponding change curve will also be large, significantly impacting the trend of the change curve. In this case, based on the above technical solution, the detection result for the smoothness of the target animation display process will be poor, consistent with the user's perception. Conversely, during phases where the target animation changes slowly, if frame drops occur, the user will not perceive noticeable stuttering. Correspondingly, the rate of change of the corresponding change curve will be small, not significantly impacting the trend of the change curve. In this case, based on the above technical solution, the detection result for the smoothness of the target animation display process will be good, consistent with the user's perception.

[0007] As can be seen, the above implementation scheme can effectively detect the smoothness of the target animation display process while improving the consistency between the detection results and the user's actual perception.

[0008] In some implementations of the motion effect detection method described above, the method further includes: obtaining the display features of each frame during the display of the target motion effect.

[0009] In one of the motion effect detection methods described above, in some implementations of the first aspect, obtaining the display features of each frame during the display of the target motion effect includes: obtaining the display features of the target layer in each frame during the display of the target motion effect, wherein the target layer is the layer where the target motion effect is located.

[0010] In the above implementation, since the terminal device controls the merging of different layers and then controls the display of the merged layer, the display characteristics of each frame in the target animation display process can be characterized by the display characteristics of the layer where the target animation is located in each frame.

[0011] In one of the motion effect detection methods described above, in some implementations of the first aspect, obtaining the display features of the target layer in each frame during the display of the target motion effect includes: obtaining the display features of the target layer in each frame without dropped frames during the display of the target motion effect; performing frame interpolation processing on the display features of the target layer in each frame without dropped frames to obtain the display features of the target layer in each frame during the display of the target motion effect.

[0012] In the above implementation, since the change curve is generated based on the display features of each frame, when a frame is lost, the display features of that lost frame cannot be obtained. Therefore, the resulting change curve will lack the display features of that frame, causing it to fail to accurately represent the smoothness of the target animation display process. To prevent this, the above implementation can first perform frame interpolation processing on the display features of the lost frames, so that the resulting change curve fully includes the display features of each frame.

[0013] In one of the motion effect detection methods described above, in certain implementations of the first aspect, obtaining the display features of the target layer for each frame during the display of the target motion effect without frame loss includes: detecting a trigger signal for a display operation on the target layer of the Nth frame; if the target layer of the Nth frame has been rendered, obtaining the display features of the target layer of the Nth frame; and if the target layer of the Nth frame has not been rendered, determining not to obtain the display features of the target layer; wherein the Nth frame is any frame during the display of the target motion effect.

[0014] In some implementations of the motion effect detection method described above, the display characteristics of the target layer include: the size of the target layer, or the position of the center point of the target layer.

[0015] In one of the motion effect detection methods described above, in certain implementations of the first aspect, the smoothness of the target motion effect display process is determined based on the changing trend of the first change curve, including: calculating the similarity between the first change curve and the second change curve in terms of changing trends, wherein the second change curve is used to describe the target display progress of the target motion effect in each frame, and the similarity is positively correlated with the smoothness of the target motion effect display process.

[0016] In the above implementation, the target display progress refers to the proportion of the current animation process to the total animation process under the condition of no frame drops or other display anomalies. The second change curve is the change curve of the display progress of each frame during the display of the target animation effect under the condition of no frame drops or other display anomalies. The closer the change trend of the first change curve corresponding to the actual display process is to the change trend of the second change curve under the ideal condition, the smoother the actual display process of the target animation effect is; conversely, the closer the change trend is to the second change curve under the ideal condition, the worse the smoothness of the actual display process of the target animation effect is.

[0017] Secondly, this technical solution provides an electronic device, including: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the method of the first aspect or any possible implementation thereof.

[0018] Thirdly, the present invention also provides a chip, the chip including a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface and executes the method in the first aspect or any possible implementation thereof.

[0019] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is configured to execute the instructions stored in the memory. When the instructions are executed, the processor is configured to perform the method in the first aspect or any possible implementation of the first aspect.

[0020] Fourthly, the present invention also provides a computer-readable storage medium storing program code for execution by a device, the program code including instructions for performing the method in the first aspect or any possible implementation thereof. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of a scenario for the motion effect detection method provided in an embodiment of this application;

[0022] Figure 2This is another scenario illustration of the motion effect detection method provided in the embodiments of this application;

[0023] Figure 3 This is a timing control diagram of the animation display process provided in an embodiment of this application;

[0024] Figure 4 This is another timing control diagram of the animation display process provided in the embodiments of this application;

[0025] Figure 5 This is a progress curve diagram of the animation display process provided in the embodiments of this application;

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0027] Figure 7 This is a schematic flowchart of the motion effect detection method provided in the embodiments of this application;

[0028] Figure 8 This is another scenario illustration of the motion effect detection method provided in the embodiments of this application;

[0029] Figure 9 This is another scenario illustration of the motion effect detection method provided in the embodiments of this application;

[0030] Figure 10 This is another scenario illustration of the motion effect detection method provided in the embodiments of this application;

[0031] Figure 11 This is another scenario illustration of the motion effect detection method provided in the embodiments of this application;

[0032] Figure 12 This is another scenario illustration of the motion effect detection method provided in the embodiments of this application;

[0033] Figure 13 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application;

[0034] Figure 14 This is another schematic flowchart of the motion effect detection method provided in the embodiments of this application. Detailed Implementation

[0035] Terminal devices contain various "animation effects," or motion effects. For example, when a user launches an application by clicking the application icon on the main interface, an application launch animation effect is displayed to complete the transition from the application icon to the application interface in an animated form. Similarly, when a user exits an application by swiping up, an application exit animation effect is displayed to complete the transition from the application interface to the application icon in an animated form.

[0036] For application startup scenarios, let's take the settings application as an example, such as... Figure 1 As shown, in the application launch scenario, the application launch animation displays as follows: the application icon gradually enlarges, then switches from icon to application interface, and then the application interface gradually enlarges until it is displayed in full screen. For scenarios where the application is exited via a swipe up, such as... Figure 2 As shown, the process of displaying the application exit animation is as follows: the application interface gradually shrinks and switches from the application interface form to the icon form, and then the application icon gradually shrinks until it becomes the size of a regular application icon.

[0037] During the display of the above animation effects, frame drops may occur. For ease of understanding, the control process for the animation display will be briefly explained first. Specifically, the terminal device can control the rendering and display of each frame based on the vertical synchronization (Vsync) signal. During the animation display process, such as... Figure 3 As shown, upon detecting the (N-1)th Vsync signal, the relevant functional unit in the terminal device begins drawing and rendering the (N-1)th frame of the display screen, while simultaneously displaying the already rendered (N-2)th frame. Upon detecting the Nth Vsync signal, the drawing and rendering of the Nth frame is triggered, and the already rendered (N-1)th frame is displayed. This cycle repeats to achieve normal animation display. However, in some scenarios, such as... Figure 4 As shown, after receiving the Nth Vsync signal, due to factors such as excessive load on the terminal device, the rendering time of the Nth frame may be too long, even exceeding the interval of the Vsync signal. In this case, if the (N+1)th Vsync signal is detected, the terminal device will not be able to display the Nth frame because it has not yet been fully rendered; instead, it will still display the (N-1)th frame. As a result, the Nth frame is dropped, and the user will observe the display remaining on the (N-1)th frame for an extended period, causing a sense of screen stuttering.

[0038] In light of the above situation, it is necessary to test the animation display process to determine its smoothness, which will help to make targeted improvements to the terminal equipment.

[0039] In one technical solution, the cumulative frame drop duration during the animation display process can be detected, and the smoothness of the animation display process can be described by the cumulative frame drop duration. The longer the cumulative frame drop duration, the more obvious the stuttering in the animation display process and the worse the smoothness; conversely, the less obvious the stuttering in the animation display process and the better the smoothness.

[0040] However, the smoothness of the animation display process described based on the cumulative frame drop duration may not be consistent with the smoothness perceived by the user. This is because, during the animation design phase, to make the animation's changes more in line with human perception, many animations are not linear but non-linear, such as changing rapidly first and then slowly, or slowly first and then rapidly. Therefore, for the same cumulative frame drop duration, the degree of stuttering perceived by the user will differ depending on when the frame drops occur in different stages of the animation. Specifically, compared to the slow-changing stages of the animation, the stuttering is more noticeable when frame drops occur in the fast-changing stages of the animation.

[0041] For ease of understanding, Figure 5 A graph illustrating the application startup animation display process is provided, which can be used to describe the display progress of the animation at different moments in the animation display process. Figure 5 As can be seen, during time period T1, the rate of change of the animation display progress is relatively fast. Therefore, the difference between two adjacent frames will be large. Thus, if frame drops occur during this period, users will perceive noticeable stuttering when the displayed image remains unchanged for an extended period. During time period T2, the rate of change of the animation display progress slows down, and the difference between two adjacent frames will decrease. Therefore, if frame drops occur during this period, even if the displayed image remains unchanged for a considerable time, users will not experience noticeable stuttering.

[0042] To address the aforementioned issues, embodiments of this application provide a motion effect detection method that can be used to detect the motion effect display process, determine the smoothness of the motion effect display process, and improve the consistency between the detection results and user perception.

[0043] Figure 6 A schematic diagram of the structure of the electronic device 100 provided in an embodiment of this application is shown.

[0044] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a fingerprint sensor 180H, a pressure sensor 180A, a touch sensor 180K, an ambient light sensor 180L, a proximity light sensor 180G, a temperature sensor 180J, a distance sensor 180F, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer 180E, a bone conduction sensor 180M, etc.

[0045] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0046] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0047] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0048] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0049] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0050] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0051] The MIPI interface can be used to connect the processor 110 and the display screen 194. The processor 110 and the display screen 194 communicate through the DSI interface to realize the display function of the electronic device 100.

[0052] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0053] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0054] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0055] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0056] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as an unlocking function), etc. The data storage area may store data created during the use of electronic device 100 (such as fingerprint data, lock screen interface data, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory disposed in the processor.

[0057] The following embodiments of this application will be used to illustrate having Figure 6 Taking the electronic device with the structure shown as an example, the motion effect detection method provided in the embodiments of this application will be specifically described.

[0058] Figure 7 This is a schematic flowchart of a motion effect detection method provided in an embodiment of this application, such as... Figure 7 As shown, the motion effect detection method provided in this application includes:

[0059] 101, The terminal device determines that the target animation begins.

[0060] 102. The terminal device determines the theoretical change curve corresponding to the target motion effect.

[0061] In this embodiment, the target animation effect can be any kind of animation effect provided by the terminal device system, such as application startup animation effect, application exit animation effect, screen lock animation effect, unlock animation effect, switching animation effect between different pages of the main interface, notification bar display animation effect, etc. Alternatively, the target animation effect can also be any kind of animation effect provided by the target application in the terminal device, such as switching animation effect between different application interfaces of the target application, etc. The target application can be, for example, a self-developed application of the terminal device manufacturer, or, in another implementation, a third-party application.

[0062] The terminal device can determine the theoretical change curve corresponding to the target animation effect after determining that the target animation effect has started. In this embodiment, the theoretical change curve can be used to describe the change curve of the display progress of each frame during the display of the target animation effect under the condition of no frame drops or other display anomalies. The display progress can be characterized, for example, by the proportion of the current animation process to the total animation process. As can be seen from the foregoing description, the display process of the target animation effect is a non-linear change process, such as a rapid change followed by a slow change. That is, the display progress of the target animation effect changes non-linearly with time. For example, assuming the display duration of the target animation effect is T, then... At any given moment, the display progress of the target animation may not be 50%.

[0063] The following explains how to determine the theoretical change curve of the target's dynamic effect.

[0064] First, after determining the start of the target animation, the terminal device can obtain the corresponding animation display parameters. These parameters may include, but are not limited to, the preset display duration of the animation and an interpolator. The preset display duration defines the duration of the target animation, and for the same animation, its display duration remains constant each time it is displayed. The interpolator can be used to control the rate of change of the animation, causing it to change at a preset rate, such as accelerating then decelerating, or decelerating then accelerating.

[0065] Then, the terminal device can determine the timestamp of the first frame of the target animation after the first frame of the animation begins to be drawn. Furthermore, the terminal device can calculate the theoretical change curve of the target animation based on parameters such as the display duration of the target animation, the animation interpolator, the timestamp of the first frame, and the vsync period.

[0066] 103. The terminal device acquires the display characteristics of each frame during the display of the target animation.

[0067] 104. The terminal device generates the actual change curve corresponding to the target animation based on the display characteristics of each frame during the display process of the target animation.

[0068] The display characteristics of each frame during the display of the target animation can be the size of the animation frame, the position of the center point of the animation frame, or the brightness of the animation frame.

[0069] Specifically, the display of a terminal device often involves multiple layers. In a single display process, the terminal device needs to first draw and render each layer to be displayed, then composite the layers according to the set layer order, and finally display them. In the scenario of displaying target animation effects, the display characteristics of each frame during the display of the target animation effect can be characterized by the display characteristics of the layer where the target animation effect is located (hereinafter referred to as the "target layer").

[0070] Taking application startup as an example, Figure 8 A schematic diagram illustrating the changes in the target layer during the display of the target animation is provided, such as... Figure 8 As shown, during application startup, the size of the target layer gradually increases, and the position of the target layer's center point gradually changes. The position of the target layer's center point can be within the display screen. Taking the scenario of swiping up to exit the application as another example, as... Figure 9 As shown, during the process of exiting the application, the size of the target layer gradually decreases, and the position of the center point of the target layer gradually changes.

[0071] It is understood that, corresponding to the changes in the display progress of the target animation, the changes in the size of the target layer and the position of the target layer's center point on the display screen are also non-linear processes. Furthermore, assuming no frame drops or other display anomalies, the trends of the curves corresponding to the change in the target layer size and center point position should be consistent with the aforementioned theoretical trend. Therefore, in this embodiment, during the display of the target animation, the terminal device can acquire the display characteristics of the target layer in each frame, and after the target animation ends, generate the actual change curve corresponding to the display process of the target animation based on the display characteristics of the target layer in each frame. The display characteristics of the target layer in each frame can be, for example, the layer size, the position of the layer's center point, or the layer brightness.

[0072] 105. The terminal device determines the smoothness of the target animation display process based on the actual change curve and the theoretical change curve.

[0073] As explained above, in the absence of frame drops or other display anomalies, the actual change curve of the target layer during the target animation display process should closely approximate the theoretical change curve's trend. Therefore, in this embodiment, the terminal device can detect the degree of similarity between the actual and theoretical change curves in their trends. This similarity can characterize the smoothness of the target animation display process; a higher similarity indicates a smoother display process, and vice versa.

[0074] Figure 10 The diagram illustrates the fitting effect between the actual change curve of the target layer and the theoretical change curve of the target animation in an application startup scenario, assuming no frame drops. Figure 10 As shown, in the absence of frame loss, the two fit well and their trends are highly similar.

[0075] Figure 11This paper presents a schematic diagram showing the fitting effect between the actual change curve of the target layer and the theoretical change curve of the target animation when the target animation changes rapidly and two frames are lost during application startup. For the two lost frames, the terminal device will again control the display of the target layer from the previous frame. Therefore, the display characteristics of the target layer corresponding to the two lost frames have not changed compared to the previous frame. In this case, if... Figure 11 As shown, because the target animation changes at a fast rate, frame dropping will result in a significant discontinuity between the target and adjacent frames, which will significantly reduce the similarity of the change trends between the two change curves.

[0076] Figure 12 This paper presents a schematic diagram showing the fitting effect between the actual change curve of the target layer and the theoretical change curve of the target animation when the target animation changes slowly and two frames are lost during application startup. Consistent with the previous point, for the two lost frames, the terminal device will again control the display of the target layer from the previous frame. Therefore, the display characteristics of the target layer corresponding to the two lost frames do not change compared to the previous frame. However, as... Figure 12 As shown, since the target animation changes slowly at this time, frame dropping will not cause obvious discontinuity between adjacent frames. Therefore, it will not significantly reduce the similarity of the change trends between the two change curves.

[0077] As can be understood from the above, in scenarios where frame drops occur when the target animation changes rapidly, the resulting stuttering sensation from the user's perspective will be quite noticeable. Consistent with user perception, based on the technical solution provided in this application, the similarity in trend between the actual change curve of the target layer and the theoretical change curve corresponding to the target animation is significantly reduced. Conversely, in scenarios where frame drops occur when the target animation changes slowly, the resulting stuttering sensation from the user's perspective will be relatively minor. Consistent with user perception, based on the technical solution provided in this application, the similarity in trend between the actual change curve of the target layer and the theoretical change curve corresponding to the target animation is not significantly reduced.

[0078] As can be seen, the technical solution provided in this application embodiment can effectively detect the smoothness of the target animation display process, and can improve the consistency between the detection results and the user's actual perception.

[0079] In another embodiment of this application, the motion effect detection method provided in this application embodiment is described in conjunction with the software structure of the electronic device 100.

[0080] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0081] Figure 13 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0082] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer (APP), the application framework layer (Framework), the native layer, and the kernel layer (not shown in the diagram).

[0083] The application layer can contain a target application, which can be any application installed on the terminal device. In the target application's usage scenario, the target application can display the target animation. The application layer can also contain a choreographer. The choreographer can reside in the target application's user interface (UI) thread. The choreographer can send the timestamp of the first frame's rendering to the fluency detector after detecting the first frame of the target animation.

[0084] The application framework layer may contain a FluencyDetector. The FluencyDetector can obtain the display duration of the target animation and information about its interpolator from the application layer. Furthermore, after the first frame of the target animation begins to be drawn, the FluencyDetector can also obtain the timestamp of the first frame drawing from Choreographer. Additionally, the FluencyDetector can calculate the theoretical change curve of the target animation's display process based on information such as the display duration, interpolator, and first frame drawing timestamp.

[0085] The local layer may include the display system service SurfaceFlinger. SurfaceFlinger is a service in the Android system that can be used to acquire layer data from different application layers, composite it, and display it. In this embodiment, during the display of the target animation, SurfaceFlinger can be used to acquire the display characteristics of the target layer before each frame of the target layer is displayed, such as the layer size attribute matrix and the layer center point position information. Furthermore, SurfaceFlinger can also be used to generate the actual change curve corresponding to the target animation display process based on the acquired display characteristics of the target layer in each frame, and determine the degree of similarity between the actual change curve and the theoretical change curve in terms of change trend.

[0086] Furthermore, the local layer may also include a monitoring unit, which could be, for example, a hiview service set. The monitoring unit can be used to diagnose faults in the display process of the target animation based on the similarity in trend between the actual and theoretical change curves. If an anomaly is determined in the display process of the target animation, the monitoring unit can also collect relevant logs and upload the fault diagnosis results and related logs to the cloud server.

[0087] The method provided in this application can be applied, for example, to the internal testing phase before a terminal device leaves the factory. In this implementation, the fault diagnosis results and log data uploaded by the maintenance and testing unit to the cloud server can be used by developers to pinpoint the specific cause of motion effect stuttering and optimize the terminal device accordingly to improve user experience. Alternatively, the method provided in this application can be applied, for example, to the actual use of the terminal device by the user. In this implementation, to protect user privacy data, the data uploaded by the maintenance and testing unit to the cloud server may not include log data, but instead include fault diagnosis results. Specifically, the fault diagnosis results may include the similarity in trend between the actual change curve and the theoretical change curve, information about the target application, etc. In this implementation, the fault diagnosis results can be used by developers to perform online upgrades of the terminal device's system or application versions.

[0088] In another embodiment of this application, Figure 13 Taking the electronic device with the architecture shown as an example, the motion effect detection method provided in the embodiments of this application will be further explained.

[0089] Figure 14 This is another schematic flowchart of the motion effect detection method provided in the embodiments of this application, such as... Figure 14 As shown, the motion effect detection method provided in this application includes:

[0090] 201, The target application determines that the target animation begins.

[0091] 202. The target application sends the display duration of the target animation and the interpolator to FluencyDetector.

[0092] In this embodiment, the target application can maintain information on the display duration of different animations within the target application, as well as animation interpolators. The animation display duration information refers to the theoretical display duration of the animation, which is a preset fixed value. The target application can control the start of the target animation. Once the target animation starts, the target application can send the display duration corresponding to the target animation and the interpolator to the FluencyDetector. Specifically, after determining that the target animation has started, the target application can call the interface function beginDetect() provided in this embodiment, through which it passes the display duration corresponding to the target animation and the interpolator to the FluencyDetector.

[0093] 203. FluencyDetector sends a notification message to SurfaceFlinger to start the detection process.

[0094] 204. FluencyDetector registers a notification message with Choreographer for the timestamp of the first frame of the target animation.

[0095] Upon receiving the display duration and interpolator corresponding to the target animation, FluencyDetector can, on the one hand, call the beginDetect() function to send a notification message about the detection process to SurfaceFlinger. On the other hand, FluencyDetector can obtain the timestamp of the first frame after the first frame of the target animation begins to be drawn. In one exemplary implementation, FluencyDetector can set a callback function for the first frame drawing on the Choreographer of the target application's UI thread. Then, when the first frame drawing begins, this callback function can be triggered, at which point the Choreographer can send the timestamp of the first frame drawing to FluencyDetector. In this embodiment, the callback function can be automatically destroyed after the Choreographer sends the timestamp of the first frame drawing to FluencyDetector.

[0096] The embodiments of this application do not limit the execution order of steps 203 and 204. In other implementations, step 204 can be executed first, followed by step 203, or steps 203 and 204 can be executed simultaneously.

[0097] 205, Choreographer determines the start of the first frame drawing.

[0098] 206. Choreographer sends the timestamp of the first frame to FluencyDetector.

[0099] 207. FluencyDetector determines the theoretical change curve of the target animation display process based on the display duration of the target animation, the interpolator, the vsync cycle, and the first frame rendering timestamp.

[0100] In this embodiment of the application, for example, the theoretical change curve of the target animation display process can be described in the form of an array, where each element in the array represents the display progress of each frame in the animation display process. Therefore, at each moment when the target animation changes rapidly, the values ​​of the corresponding elements in the array fluctuate significantly, while at each moment when the target animation changes slowly, the values ​​of the corresponding elements in the array fluctuate less.

[0101] 208. SurfaceFlinger obtains the display characteristics of the target layer in each frame.

[0102] Based on step 203 above, after receiving the notification message to start the detection process, SurfaceFlinger can obtain the display features of the target layer in each frame of the target animation display process. It should be noted that in this embodiment, step 208 can be executed at any time after step 203, without waiting for steps 204 to 207 to complete.

[0103] During the display of the target animation, for any frame of the target layer, after the vsync cycle is reached, SurfaceFlinger can call the doTrasactions() function to perform the display operation. In this embodiment, the onLayerPresent() function can be added inside the doTrasactions() function. Then, after SurfaceFlinger calls the doTrasactions() function, SurfaceFlinger can obtain the matrix attribute of the target layer by calling the onLayerPresent() function.

[0104] In one possible implementation, the `onLayerPresent()` function is called after the `doTrasactions()` function. Based on this implementation, after each `vsync` signal cycle, if the target layer of the frame to be displayed has been rendered, `SurfaceFlinger` can obtain the size information of the target layer. If the target layer has not been rendered, `SurfaceFlinger` can obtain the size information of the target layer of the previous frame. Specifically, when the `vsync` signal used to trigger the display of the Nth frame arrives, if the Nth frame has been rendered, `SurfaceFlinger` can control the display of the Nth frame by calling the `doTrasactions()` function, and `SurfaceFlinger` can obtain the `matrix` attribute of the target layer of the Nth frame by calling the `onLayerPresent()` function. If frame N is not rendered completely, SurfaceFlinger can control the display of frame N-1 by calling the `doTrasactions()` function, and can obtain the matrix attribute of the target layer in frame N-1 by calling the `onLayerPresent()` function. In other words, based on this implementation, if a frame is dropped in frame N, SurfaceFlinger will again control the display of the target layer in frame N-1, and will again obtain the size information of the target layer in frame N-1.

[0105] Understandably, in this implementation, if there are dropped frames during the target animation display process after the target animation ends, for the lost frame N, the size information of the target layer in each frame that SurfaceFlinger has already obtained still contains the data for that frame, and the data for that frame is consistent with the data of the (N-1)th frame. Based on this implementation, in the subsequent step 212, SurfaceFlinger can directly generate the actual change curve of the target animation display process based on the size information of the target layer in each frame that has been obtained.

[0106] In another possible implementation, the `onLayerPresent()` function is called only after `doTrasactions()` has been called and the frame to be displayed has finished rendering. Based on this implementation, after each `vsync` signal cycle, if the target layer of the frame to be displayed has finished rendering, then `SurfaceFlinger` can obtain the size information of the target layer for the current frame. If the target layer of the frame to be displayed has not finished rendering, it indicates that a frame drop has occurred, and `SurfaceFlinger` can determine not to call the `onLayerPresent()` function, thus not needing to obtain the target layer's size information. For example, when the `vsync` signal used to trigger the display of frame N arrives, if frame N has finished rendering, then `SurfaceFlinger` can control the display of the target layer of frame N by calling `doTrasactions()`, and `SurfaceFlinger` can obtain the `matrix` attribute of the target layer of frame N by calling `onLayerPresent()`. When the vsync signal used to trigger the display of the Nth frame arrives, if the Nth frame has not been rendered, it means that the Nth frame has been dropped. In this case, SurfaceFlinger only needs to control the display of the target layer of the N-1th frame by calling the doTrasactions() function, without having to call the onLayerPresent() function to obtain the size information of the target layer.

[0107] Understandably, in this implementation, if there are dropped frames during the target animation display process after the animation ends, the size information of each target layer already obtained by SurfaceFlinger will not include the data corresponding to the lost frames. Based on this implementation, in subsequent step 212, SurfaceFlinger can first perform frame interpolation, adding the size information of the target layers with lost frames to the size information of each target layer already obtained. Then, based on the size information of each target layer obtained after frame interpolation, it generates the actual change curve of the target animation display process.

[0108] 209, The target application determines that the target animation ends.

[0109] 210. The target application sends a notification message to FluencyDetector indicating that the target animation has ended.

[0110] After the target animation ends, the target application can send a notification message to FluencyDetector by calling the endDetect() function. In turn, FluencyDetector can send the calculated theoretical change curve of the target animation display process to SurfaceFlinger.

[0111] 211. FluencyDetector sends the theoretical change curve of the target animation display process to SurfaceFlinger.

[0112] 212. SurfaceFlinger determines the actual change curve of the target animation display process based on the display characteristics of the target layer in each frame.

[0113] After receiving the theoretical change curve sent by FluencyDetector, SurfaceFlinger can determine that the target animation has ended. At this point, SurfaceFlinger can process the size information of the target layer in each frame during the display of the target animation by calling the ProcessLayerPresent() function, and generate the actual change curve corresponding to the display process of the target animation. For example, the actual change curve of the target animation display process can also be described in the form of an array, where each element of the array represents the size of the target layer in each frame during the animation display process.

[0114] 213. SurfaceFlinger determines the similarity in the changing trends of the target animation display process based on the theoretical and actual change curves.

[0115] In this embodiment, although the physical meanings represented by the values ​​at various moments on the theoretical and actual change curves are different, their trends are similar when no frames are dropped. Therefore, the similarity in trend between the theoretical and actual change curves of the target motion effect display process can be determined, and the smoothness of the actual display process can be characterized by the similarity in their trends. The higher the similarity in trend, the smoother the actual display process; conversely, the lower the similarity, the smoother the actual display process.

[0116] Furthermore, when frame drops occur during periods of slow change in the target animation, the impact of frame drops on the similarity between the theoretical and actual change curves is relatively small, and the user's perceived stuttering is also relatively mild. Conversely, when frame drops occur during periods of rapid change in the target animation, the impact of frame drops on the similarity between the theoretical and actual change curves is relatively large, and the user's perceived stuttering is also less noticeable. It is evident that the method provided in this application embodiment can make the obtained detection results more consistent with the user's perception.

[0117] It should be understood that the electronic equipment described here is embodied in the form of functional units. The term "unit" here can be implemented in software and / or hardware, without specific limitation. For example, a "unit" can be a software program, hardware circuit, or a combination of both that implements the above-described functions. The hardware circuit may include application-specific integrated circuits (ASICs), electronic circuits, a processor (e.g., a shared processor, a proprietary processor, or a group processor, etc.) and memory for executing one or more software or firmware programs, integrated logic circuits, and / or other suitable components supporting the described functions. Whether a function is implemented in hardware or in a manner driven by computer software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments.

[0118] The module division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods are possible. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into a single processing module. The integrated modules described above can be implemented in hardware.

[0119] This application also provides an electronic device, which includes a storage medium and a central processing unit. The storage medium may be a non-volatile storage medium, and a computer-executable program is stored in the storage medium. The central processing unit is connected to the non-volatile storage medium and executes the computer-executable program to implement the above-described motion effect detection method.

[0120] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the various steps of the motion detection method of this application.

[0121] This application also provides a computer program product containing instructions that, when run on a computer or any at least one processor, causes the computer to execute the various steps of the motion effect detection method of this application.

[0122] This application also provides a chip, including a processor and a data interface. The processor reads instructions stored in a memory through the data interface to execute the corresponding operations and / or processes performed by the motion detection method provided in this application.

[0123] Optionally, the chip further includes a memory connected to the processor via a circuit or wire, the processor being used to read and execute computer programs stored in the memory. Further optionally, the chip includes a communication interface to which the processor is connected. The communication interface is used to receive data and / or information that needs to be processed, the processor obtaining the data and / or information from the communication interface and processing the data and / or information. The communication interface can be an input / output interface.

[0124] The memory can be read-only memory (ROM), other types of static storage devices that can store static information and instructions, random access memory (RAM), or other types of dynamic storage devices that can store information and instructions. It can also be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices. Alternatively, it can be any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0125] In this embodiment, "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0126] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0127] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0128] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0129] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A motion effect detection method, characterized in that, Applied to a terminal device for displaying target animation effects, the method includes: A first change curve is generated based on the display characteristics of the target layer in each frame during the display of the target animation; the target layer is the layer where the target animation is located; the first change curve is a curve in which the display characteristics change non-linearly over time. Based on the changing trend of the first change curve, the smoothness of the target animation display process is determined; Based on the changing trend of the first change curve, the smoothness of the target animation display process is determined, including: Calculate the similarity between the first change curve and the second change curve in terms of change trend. The second change curve is used to describe the target display progress of the target animation effect in each frame. The target display progress is used to describe the proportion of the current animation process to the total animation process. The second change curve is a curve in which the target display progress changes non-linearly with time. The similarity is positively correlated with the smoothness of the target animation effect display process.

2. The method according to claim 1, characterized in that, Obtain the display characteristics of the target layer in each frame during the display of the target animation, including: Obtain the display characteristics of the target layer for each frame without dropped frames during the display of the target animation; The display features of each frame of the target layer that has not lost frames are subjected to frame interpolation processing to obtain the display features of each frame of the target layer during the display of the target animation.

3. The method according to claim 2, characterized in that, Obtain the display characteristics of the target layer for each frame without dropped frames during the display of the target animation, including: If a trigger signal for a display operation on the target layer of frame N is detected, and the display features of the target layer of frame N are acquired when the target layer of frame N is rendered; and if the target layer of frame N is not rendered, the display features of the target layer are determined not to be acquired. The Nth frame is any frame in the process of displaying the target animation.

4. The method according to claim 1, characterized in that, The display features of the target layer include: the size of the target layer, or the position of the center point of the target layer.

5. An electronic device, characterized in that, include: One or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs including instructions that, when executed by the device, cause the device to perform the method as described in any one of claims 1-4.

6. A chip, characterized in that, The chip includes a processor and a data interface. The processor reads instructions stored in the memory through the data interface and executes the method as described in any one of claims 1-4.

7. A storage medium, characterized in that, The storage medium stores program instructions that, when run on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-4.

Images