Fluency Detection Method, Device, Equipment and Storage Medium

Abstract

The present application relates to a smoothness detection method, device, equipment and storage medium. The method includes: obtaining at least one video to be detected corresponding to each sensitivity, where the sensitivity is the sensitivity of the gyroscope in a target application; for each video to be detected, performing three-dimensional reconstruction processing on each frame image of the video to be detected to determine an angular motion data set of the video to be detected, where the angular motion data set includes angular motion data corresponding to each frame image except the first frame image, and the angular motion data characterizes the angular change of each frame image relative to the previous frame image; performing smoothness detection on the target application according to the angular motion data sets of the videos to be detected corresponding to each sensitivity. By performing three-dimensional reconstruction on each frame image, the present application can more accurately determine whether there is a stuttering phenomenon in the video to be detected, improving the accuracy of smoothness detection.

Description

Technical Field

[0001] The present application relates to the field of testing technologies, and particularly to a method, device, equipment and storage medium for detecting fluency. Background Art

[0002] The running fluency of a game application is directly related to the product experience of users during use. Therefore, before the formal release of a game application, fluency detection is performed on the game application.

[0003] In existing fluency detection solutions, it is usually determined whether there is a lag in the video image directly from consecutive static video frames according to the number of lag frames, the time interval between two adjacent frames or the lag frequency, etc. However, it is difficult to accurately determine the three-dimensional angle change from a two-dimensional image, resulting in a low detection accuracy, which also becomes a problem in the fluency detection of game applications, especially for games with extremely high requirements for the fluency of field-of-view movement, such as 3D mobile games represented by 3D games. Summary of the Invention

[0004] The present application provides a method, device, equipment and storage medium for detecting fluency, which can extract three-dimensional angle motion information from consecutive static video frames and improve the detection accuracy of fluency.

[0005] On the one hand, the present application provides a method for detecting fluency, and the method includes:

[0006] Obtain at least one video to be detected corresponding to each sensitivity that has been pre-recorded, where the sensitivity is the sensitivity of the gyroscope in the target application;

[0007] For each video to be detected, perform three-dimensional reconstruction processing on each frame image of the video to be detected to determine the angle motion data set of the video to be detected. The angle motion data set includes the angle motion data corresponding to each frame image except the first frame image, and the angle motion data represents the angle change of each frame image relative to the previous frame image;

[0008] Perform fluency detection on the target application according to the angle motion data sets of the videos to be detected corresponding to each sensitivity.

[0009] On the other hand, a device for detecting fluency is provided, and the device includes:

[0010] A video acquisition module, configured to obtain at least one video to be detected corresponding to each sensitivity that has been pre-recorded, where the sensitivity is the sensitivity of the gyroscope in the target application;

[0011] A data collection module, which is used to perform three-dimensional reconstruction processing on each frame image of the video to be detected for each of the videos to be detected, so as to determine an angular motion data set of the video to be detected, where the angular motion data set includes angular motion data corresponding to each frame image except the first frame image, and the angular motion data characterizes the angular change of each frame image relative to the previous frame image;

[0012] An analysis module, which is used to perform smoothness detection on the target application according to the angular motion data sets of the videos to be detected corresponding to each sensitivity.

[0013] On the other hand, a computer storage medium is provided, in which at least one instruction or at least one program segment is stored, and the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the smoothness detection method as described above.

[0014] On the other hand, a smoothness detection device is provided, the device includes a processor and a memory, and at least one instruction or at least one program segment is stored in the memory, and the at least one instruction or at least one program segment is loaded and executed by the processor to perform the above smoothness detection method.

[0015] The smoothness detection method, device, equipment and storage medium provided by this application have the following beneficial effects:

[0016] This application performs three-dimensional reconstruction processing on each frame image, extracts three-dimensional angular motion data from continuous two-dimensional still frames, and can more accurately determine whether there is a stuttering phenomenon in the video to be detected; combining the detection results of each video to be detected under each sensitivity setting of the gyroscope, the smoothness of the target application screen under different sensitivities of the gyroscope can be determined, improving the detection accuracy of the target application. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is a schematic diagram of a smoothness detection system provided by an embodiment of the present application.

[0019] Figure 2 is a schematic diagram of the implementation principle of the recording device provided by an embodiment of the present application.

[0020] Figure 3It is a schematic diagram of the implementation principle of the inter-frame analysis model provided by the embodiments of the present application.

[0021] Figure 4 It is a schematic diagram of the implementation principle of the data analysis model provided by the embodiments of the present application.

[0022] Figure 5 It is a schematic flowchart of a smoothness detection method provided by the embodiments of the present application.

[0023] Figure 6 It is a schematic flowchart of determining an angular motion data set provided by the embodiments of the present application.

[0024] Figure 7 It is an example diagram of an angular change curve provided by the embodiments of the present application.

[0025] Figure 8 It is a schematic structural diagram of a smoothness detection device provided by the embodiments of the present application.

[0026] Figure 9 It is a schematic structural diagram of another smoothness detection device provided by the embodiments of the present application.

[0027] Figure 10 It is a schematic structural diagram of a video recording module provided by the embodiments of the present application.

[0028] Figure 11 It is a schematic structural diagram of a data collection module provided by the embodiments of the present application.

[0029] Figure 12 It is a schematic structural diagram of an analysis module provided by the embodiments of the present application.

[0030] Figure 13 It is a schematic structural diagram of a first analysis unit provided by the embodiments of the present application.

[0031] Figure 14 It is a schematic structural diagram of a first analysis unit provided by the embodiments of the present application.

[0032] Figure 15 It is a schematic structural diagram of a second analysis unit provided by the embodiments of the present application.

[0033] Figure 16 It is a schematic structural diagram of another smoothness detection device provided by the embodiments of the present application.

[0034] Figure 17 It is a schematic hardware structure diagram of a device for implementing the method provided by the embodiments of the present application. Detailed implementation manners

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the following will further describe this application in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts belong to the scope protected by this application.

[0036] It should be noted that the terms "first", "second", etc. in the specification, claims, and the above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server including a series of steps or units does not necessarily have to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0037] The objective real space existing in the 3D world is the three-dimensional space, which has three metrics: length, width, and height. Three-dimensional games (also known as 3D games or stereoscopic games) are relative to two-dimensional games (also known as 2D games or flat games). Because they adopt the concept of three-dimensional space coordinates, they appear more real. Exactly for this reason, when detecting the smoothness of three-dimensional games, since consecutive static two-dimensional frame images cannot reflect the changes in the three-dimensional space, traditional detection methods such as using consecutive static frames or based on the number of stuttering frames cannot accurately locate the spatial changes in three-dimensional games, affecting the accuracy of the smoothness detection of three-dimensional games. In the prior art, there is no analysis solution for the influence of the sensitivity of the gyroscope in three-dimensional games on the smoothness of three-dimensional games. In view of this, the embodiments of this application provide a smoothness detection method, device, equipment, and storage medium to solve the above technical problems.

[0038] Please refer to Figure 1 , which shows a schematic diagram of a smoothness detection system provided by an embodiment of this application. As Figure 1 shown, the smoothness detection system may include a recording device 11, a data center 12, an inter-frame analysis model 13, a data analysis model 14, and a mobile terminal 15.

[0039] Specifically, the mobile terminal 15 includes a target application to be detected. The mobile terminal 15 can be used to present the graphical interface of the target application, such as mobile devices that can present graphical interfaces like mobile phones, PADs, etc.

[0040] Specifically, the recording device 11 is used to record at least one video to be detected corresponding to each sensitivity, and save the recorded video to be detected to the data center 12. For the implementation principle, please refer to Figure 2 as shown. The recording device 11 may include a controller 111, a gripper 113, and a recording device 112. The controller 111 includes a control program and a test program.

[0041] Among them, the control program can be used to quantitatively control the gripper; the test program can be used to send a start instruction to the mobile terminal 15 to display a graphical interface on the display of the mobile terminal 15, and a target test scenario is presented in the graphical interface; the gripper 113 may include a device that can control the rotation angle, such as a robotic arm. The robotic arm is a programmable device for controlling the rotation angle, and is used to control the mobile device to rotate in a set direction and angle. The gripper 113 can be used to rotate according to the control of the control program to drive the mobile terminal 15 to rotate; the recording device 112 can be used to record the screen of the graphical interface of the mobile terminal 15 during the rotation of the mobile terminal 15, generate a video to be detected, and store the video to be detected in the data center 12.

[0042] In practical applications, at least one video to be detected at each sensitivity can be obtained by adjusting the sensitivity of the gyroscope in the target application.

[0043] Specifically, the data center 12 may include an independently operating server, or a distributed server, or a server cluster composed of multiple servers. The data center 12 can be used to store the videos to be detected and the results of analyzing the videos to be detected.

[0044] Specifically, the inter-frame analysis model 13 is a model that characterizes the change of 3D world features between sampled video frames and reconstructs the 3D angle change. Refer to Figure 3 as shown. The inter-frame analysis model 13 can be used to obtain the pre-recorded videos to be detected at each sensitivity from the data center 12, and perform three-dimensional reconstruction processing on each video to be detected, so as to determine the angular motion data of the video to be detected.

[0045] Specifically, the data analysis model 14 establishes a credible data model of sensitivity and angle change by analyzing the test case data at different sensitivities. Refer to Figure 4 as shown. The data analysis model 14 can be used to obtain the angular motion data, group according to different test conditions, calculate the angle change at different sensitivities, and perform smoothness detection on the target application.

[0046] The following introduces a smoothness detection method of this application. Figure 5It is a flow chart of a fluency detection method provided in an embodiment of the present application. This specification provides method operation steps as described in the embodiment or flow chart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment) according to the method shown in the embodiment or the accompanying drawings. Specifically, Figure 5 As shown in , the method may include:

[0047] S501, obtaining at least one pre-recorded video to be detected corresponding to each sensitivity, where the sensitivity is the sensitivity of the gyroscope in the target application.

[0048] A gyroscope generally refers to hardware for gravity sensing and direction sensing. It is generally integrated inside a mobile phone to sense gravity and the tilt angle and direction of the phone. It is mainly used on mobile phones as a compass. In an embodiment of the present application, the target application represents a game application that supports a gyroscope, such as Peace Elite, CrossFire, Call of Duty and other mobile games. In game applications, the hardware gyroscope that comes with the mobile terminal is mostly used to rotate the direction of the scene field of view in the game, and an angle correlation relationship is established between the gyroscope in the game application and the hardware gyroscope. The gyroscope is used to control the movement of the game character, such as the movement of the field of view, the movement of the crosshairs, the movement of the muzzle when firing, etc., so the smoothness of the picture in the game application is closely related to the sensitivity setting of the gyroscope in the game application.

[0049] In an embodiment of the present application, before obtaining the pre-recorded at least one video to be detected corresponding to each sensitivity, a step of using a recording device to record at least one video to be detected corresponding to each sensitivity is also included.

[0050] Specifically, the step of using a recording device to record at least one to-be-detected video corresponding to each sensitivity may include:

[0051] (1) The mobile terminal is fixedly placed on the holder of the recording device;

[0052] (2) sending a start instruction to the target application in the mobile terminal through the test program in the controller of the recording device, so that the target test scene is presented in the graphical interface of the mobile terminal;

[0053] (3) Setting the sensitivity of the gyroscope in the target application;

[0054] (4) using a control program in a controller of the recording device to control the holder to rotate regularly, thereby driving the mobile terminal to rotate regularly;

[0055] (5) During the rotation of the mobile terminal, use the recording device of the recording apparatus to perform at least one screen recording operation on the graphical interface to generate at least one video to be detected corresponding to the sensitivity.

[0056] In the embodiment of the present application, at least one virtual character is included in the target test scenario, and a virtual camera is preset on the virtual character. During the rotation of the mobile terminal, the field of view of the virtual camera will be driven to rotate. Taking the robotic arm as the gripper, the game application is opened and logged in through an automated test program, enter the target test scenario, manually / automatically open the game settings, and set the gyroscope sensitivity option; the robotic arm control program controls the robotic arm to rotate in a preset direction. For example, when the robotic arm moves horizontally, it drives the mobile terminal fixed on the robotic arm to move horizontally. When the mobile terminal moves, it triggers the horizontal angular velocity induction of the hardware gyroscope in the mobile terminal, thereby triggering the field of view movement when the gyroscope moves in the game.

[0057] The control program can quantitatively control the robotic arm, precisely control the rotation angle, rotation direction, etc. of the robotic arm through programming, and then quantitatively control the rotation of the gyroscope, and then quantitatively analyze the quantitative relationship between the sensitivity of the gyroscope and the rotation of the field of view in the target application. And combined with automated operation, it can reduce the video frame rate drop caused by improper operations during video recording, such as manually shaking the mobile terminal. Not only can the shaking angle not be kept constant all the time, but also the consistency of the shaking speed cannot be controlled. The robotic arm and the control program solve the problem that humans cannot precisely control the gyroscope to perform quantitative movement. Through the videos recorded by using the recording apparatus, the smoothness of the field of view movement can be detected for any 3D open-world free-view game from the perspective of angle change.

[0058] The recording device can be a device with video recording function such as a camera. In some embodiments, the built-in camera of the mobile terminal can also be used for video recording. However, using the built-in camera will affect the smoothness of the target application's screen. For example, if the screen in the video is stuck, it is difficult to determine whether the cause of the stuck is due to camera recording or the problem of the target application itself.

[0059] S502. For each of the videos to be detected, perform three-dimensional reconstruction processing on each frame image of the video to be detected to determine the angular motion data set of the video to be detected. The angular motion data set includes the angular motion data corresponding to each frame image except the first frame image, and the angular motion data represents the angular change situation of each frame image relative to the previous frame image.

[0060] In the embodiment of the present application, after obtaining the video to be detected, the video to be detected is first unlocked, and each frame of the video to be detected is decoded and saved separately in the form of a picture.

[0061] The inter-frame analysis model uses a preset three-dimensional reconstruction algorithm to perform three-dimensional reconstruction processing on each frame image of the video to be detected, solves the problem that it is difficult to analyze the three-dimensional angle change of the screenshot or video in the game world, and establishes virtual 3D world scene information through continuous single-frame two-dimensional images of the collected video to be detected. Among them, the preset three-dimensional reconstruction algorithm can adopt the SFM (Structure from Motion) algorithm. The SFM algorithm is an algorithm for three-dimensional reconstruction based on various collected unordered pictures, that is, it calculates three-dimensional information from a time series of two-dimensional images. Through SFM, 3D world reconstruction is performed on the 2D pictures of the test samples to collect angle motion data such as the reconstructed angle, position, and direction.

[0062] Specifically, referring to Figure 6 as shown in

[0063] S5021, extract at least one feature point from the first frame image of the video to be detected according to a preset method.

[0064] In the embodiment of the present application, the preset method may include selecting feature points in the middle position area of the first frame image and far from the image edge, or feature points that will appear in the field of view for a period of time and move with the movement of the game field of view. Feature points represent points with obvious features in the target application, such as objects with obvious features in the game such as trees, houses, and stones and are in the 3D scene.

[0065] In the embodiment of the present application, when using the SFM algorithm for three-dimensional reconstruction, at least 5 feature points need to be extracted. After extracting each feature point, the position of each feature point in the first frame image is used as the initial position of each feature point.

[0066] S5022, starting from the second frame image of the video to be detected, taking each frame image as the current frame image, and performing three-dimensional reconstruction processing on the current frame image according to the positions of the feature points in the previous frame image and in the current frame image, to obtain the angle motion data corresponding to the current frame image.

[0067] In the embodiments of the present application, the position changes of each feature point in the previous frame are found in the current frame image. A position pair is formed by the position of each feature point in the previous frame image and the position in the current frame image, and the model is reconstructed with the virtual camera as the center of the field of view. Then, the next frame image is analyzed, and all video frames are analyzed in this way repeatedly. Among them, the virtual camera refers to a device pre-installed on a virtual character in the target test scenario. In the third-person view of game applications, a virtual camera is generally placed on the virtual character.

[0068] In some embodiments, the method further includes: if the feature point does not exist in the current frame image, new feature points are reselected from the current frame image according to the preset method.

[0069] For example, for a stone object, when the field of view of the virtual character moves, at a certain angle during the field of view movement, the stone will appear in the field of view, and beyond this angle, the stone will disappear from the field of view. Then, new feature points need to be reselected. Just like the human eye, the visible field of view is limited. Beyond the field of view, the seen picture will change, and the selected feature points will no longer exist. One of the principles for reselecting feature points is also that the feature points can maintain obvious features within the field of view during the movement of the test field of view.

[0070] Through the SFM algorithm for 3D reconstruction processing, the angular change amount of each frame image relative to the previous frame image can be directly obtained. Then, according to the frame rate interval between two frames, for example, within 1000 milliseconds, if the video recording frame rate is 60, then the frame rate interval between two frames can be approximated as 1000 / 60 ms, and the angular velocity of each frame image relative to the previous frame image can be calculated. For example, if the angular change amount is A1 and the frame rate interval is T1, then the angular velocity is A1 / T1. The angular velocity is the main parameter affecting the field of view movement speed. For example, when the robotic arm moves at a constant speed, the speed of the corresponding gyroscope moving the game field of view is also constant. Therefore, while storing the angular change amount, its corresponding angular velocity value is stored to facilitate the detection of the sensitivity of the gyroscope.

[0071] S5023, the angular motion data sets corresponding to each frame image are used to form the angular motion data set of the video to be detected.

[0072] S503, according to the angular motion data sets of each video to be detected corresponding to each sensitivity, the smoothness of the target application is detected.

[0073] In the embodiments of the present application, the detecting the smoothness of the target application according to the angular motion data sets of each video to be detected corresponding to each sensitivity may include at least one of the following:

[0074] (1) Analyze the angular motion data set of each video to be detected to obtain a first detection result of the video to be detected, where the first detection result includes first indication information and second indication information, the first indication information represents whether there is a stuttering phenomenon in the video to be detected, and the second indication information represents the stuttering position of the video to be detected;

[0075] (2) According to the angular motion data sets of each video to be detected corresponding to each sensitivity, obtain a second detection result of the sensitivity, where the second detection result includes third indication information and fourth indication information, the third indication information represents the stability degree of the target application under the sensitivity, and the fourth indication information represents the stability parameter corresponding to the sensitivity.

[0076] Specifically, the angular motion data includes an angular change amount. Analyzing the angular motion data set of each video to be detected to obtain the first detection result of the video to be detected may include:

[0077] (1) Calculate the mean and variance of the angular change amount to obtain a first mean and a first variance;

[0078] (2) Compare the first variance with a first preset variance to obtain the first indication information;

[0079] (3) If the first variance is greater than the first preset variance, then compare the first mean with each angular change amount to obtain the second indication information.

[0080] Referring to the table below, it shows the data of the angular change amounts corresponding to 3 videos to be detected under sensitivity A obtained in an embodiment of the present application. Among them, YAW ij represents the angular change amount of the i-th frame image relative to the j-th (i≠j, i>j) frame image.

[0081] Table 1: Angular change amounts of each video to be detected under sensitivity A

[0082] Video <![CDATA[YAW 21 > <![CDATA[YAW 32 > <![CDATA[YAW 43 > …… Video 1 0.570312 -0.00253 0.529213 …… Video 2 0.186683 0.322622 0.519906 …… Video 3 0.17121 0.453041 0.360289 ……

[0083] For Video 1, Video 2, and Video 3, calculate the mean and variance of their angular change amounts respectively. For any video, if its corresponding variance is greater than the first preset variance, it can be determined that it is caused by stuttering or unsmoothness, and there is a stuttering phenomenon in this video; combined with the mean, the image with a larger deviation from the mean can be found, and the accurate stuttering position can be obtained. For example Figure 7As shown in [Figure], it is the curve of the angular change amount of each frame image of Video 4 at sensitivity A. From the angular change amount 1 of each frame image and its mean value 2, it is not difficult to see that between the 21st frame image and the 28th frame image, the angular change amount is much lower than the mean value. It can be determined that there is a stuttering phenomenon in the video between the 21st frame image and the 28th frame image.

[0084] In practical applications, to reduce errors, the inter-frame analysis model can analyze in units of a fixed number of frames as the step size. For example, it can analyze in units of 10 frames, or it can analyze in units of 5 frames. The practice of multi-frame interval analysis can reduce the frequent fluctuations between single-frame analyses, which affect the analysis accuracy, and can draw a conclusion closer to the actual situation. For example, the average angular change amount between the first frame image and the tenth frame image is 2, that is, the average angle moves 2 degrees, and the angular velocity is 2 / the time difference between frames. While between the tenth frame image and the eleventh frame image, the angular change amount is 1, that is, the angle moves 1 degree. Then it can be determined that there is a stutter between the first frame image and the tenth frame image. Through the inter-frame analysis model in the embodiments of the present application, the problem of detecting whether the rotation of the character's field of view is smooth in the 3D game world can be solved.

[0085] In some embodiments, the angular motion data includes angular velocity, and the method further includes:

[0086] (1) Calculate the variance of the angular velocity to obtain a second variance;

[0087] (2) Determine whether the second variance matches the motion parameters when recording the video to be detected to obtain the first indication information.

[0088] Angular velocity is the main parameter affecting the speed of the field of view movement. Under the uniform motion control of the robotic arm, the speed of the corresponding gyroscope moving the game field of view is also uniform. At this time, the second variance should be zero. If the second variance is not zero, it indicates that there is a stuttering phenomenon in the video to be detected.

[0089] In the embodiments of the present application, the angular motion data includes the angular change amount. Obtaining the second detection result of the sensitivity according to the angular motion data sets of each video to be detected corresponding to each sensitivity includes:

[0090] (1) For each video to be detected, sum up the angular change amounts to obtain the total angular change amount;

[0091] (2) Calculate the mean and variance of the total angular change amounts to obtain a second mean and a third variance;

[0092] (3) Compare the third variance with a second preset variance to obtain the third indication information;

[0093] (4) Determine the second mean value as the fourth indication information.

[0094] For example, for each angular velocity change amount under sensitivity A in Table 1, the total angular change amounts of Video 1, Video 2, and Video 3 can be calculated separately first, as shown in Table 2. The mean value and variance of the total angular change amounts of these three can be calculated as 12.031 and 0.096 respectively. Since the variance is less than the second preset variance of 0.1, it can be determined that the operation and screen transformation are stable under this sensitivity. Correspondingly, the mean value 12.031 can be determined as the stable parameter under sensitivity A. Of course, the first preset variance and the second preset variance can be set according to different test conditions and are not limited herein.

[0095] Table 2: Total angular change amount of sensitivity A

[0096]

[0097] After analyzing each sensitivity according to the above steps, it is possible to obtain which sensitivity has more stable operation, and give the best experience sensitivity and angular velocity suggestions for using the gyroscope for view movement in the 3D game world. As shown in Table 3, after outputting the first detection result and the second detection result, the rotation parameters and detection results of the robotic arm can be provided to the game planning party to provide reference data for the game planning party, so that the game planning party can combine the game effects and scenarios to obtain the sensitivity value of the gyroscope rotation angle during game design.

[0098] Table 3: Angular change amounts under each sensitivity in different target test scenarios

[0099]

[0100] As can be seen from the technical solutions provided in the embodiments of the present application above, the embodiments of the present application perform three-dimensional reconstruction processing on each frame of image, extract three-dimensional angular motion data from two-dimensional static frames, and can more accurately determine whether there is stuttering in the video to be detected; by combining the detection results of each video to be detected under each sensitivity setting of the gyroscope, the smoothness of the target application screen under different sensitivities of the gyroscope can be determined, improving the detection accuracy of the target application.

[0101] The embodiments of the present application also provide a smoothness detection device, as Figure 8 shown, the device may include:

[0102] A video acquisition module 810, configured to acquire at least one video to be detected corresponding to each sensitivity recorded in advance, where the sensitivity is the sensitivity of the gyroscope in the target application;

[0103] A data collection module 820, configured to perform three-dimensional reconstruction processing on each frame image of the video to be detected for each of the videos to be detected, so as to determine an angular motion data set of the video to be detected, where the angular motion data set includes angular motion data corresponding to each frame image except the first frame image, and the angular motion data characterizes the angular change of each frame image relative to the previous frame image;

[0104] An analysis module 830, configured to perform smoothness detection on the target application according to the angular motion data sets of the videos to be detected corresponding to each sensitivity.

[0105] In some embodiments, as shown in Figure 9 the device may further include:

[0106] A video recording module 840, configured to use a recording device to record at least one video to be detected corresponding to each sensitivity.

[0107] In some embodiments, as shown in Figure 10 the video recording module 840 may include:

[0108] A first preparation unit 8401, configured to fixedly place the mobile terminal on a holder of the recording device;

[0109] A second preparation unit 8402, configured to send a start instruction to the target application in the mobile terminal through a test program in a controller of the recording device, so that a target test scenario is presented in a graphical interface of the mobile terminal;

[0110] A third preparation unit 8403, configured to set a sensitivity of a gyroscope in the target application;

[0111] A control unit 8404, configured to use a control program in a controller of the recording device to control the holder to rotate regularly, so as to drive the mobile terminal to rotate regularly;

[0112] A recording unit 8405, configured to perform at least one screen recording operation on the graphical interface by using a recording device of the recording device during the rotation of the mobile terminal, so as to generate at least one video to be detected corresponding to the sensitivity.

[0113] In some embodiments, as shown in Figure 11 the data collection module 820 may include:

[0114] A feature point selection unit 8201, configured to extract at least one feature point from a first frame image of the video to be detected according to a preset method;

[0115] A data collection unit 8202, configured to start from the second frame image of the video to be detected, take each frame image as the current frame image, and perform three-dimensional reconstruction processing on the current frame image according to the positions of each feature point in the previous frame image and in the current frame image, so as to obtain the angular motion data corresponding to the current frame image;

[0116] A data determination unit 8203, configured to form an angular motion data set of the video to be detected from the angular motion data corresponding to each frame image.

[0117] In some embodiments, as shown in Figure 12 the analysis module 830 may further include:

[0118] A first analysis unit 8301, configured to analyze the angular motion data set of each video to be detected, so as to obtain a first detection result of the video to be detected, where the first detection result includes first indication information and second indication information, the first indication information represents whether there is a freezing phenomenon in the video to be detected, and the second indication information represents the freezing position of the video to be detected;

[0119] A second analysis unit 8302, configured to obtain a second detection result of the sensitivity according to the angular motion data sets of each video to be detected corresponding to each sensitivity, where the second detection result includes third indication information and fourth indication information, the third indication information represents the stability degree of the target application under the sensitivity, and the fourth indication information represents a stability parameter corresponding to the sensitivity.

[0120] In some embodiments, the angular motion data includes an angular change amount. As shown in Figure 13 the first analysis unit 8301 may include:

[0121] A first calculation unit 83011, configured to calculate the mean and variance of the angular change amount to obtain a first mean and a first variance;

[0122] A first comparison unit 83012, configured to compare the first variance with a first preset variance value to obtain the first indication information;

[0123] A second comparison unit 83013, configured to compare the first mean with each angular change amount to obtain the second indication information.

[0124] In some embodiments, the angular motion data includes an angular velocity. As shown in Figure 14 the first analysis unit 8301 may further include:

[0125] A second calculation unit 83014, configured to calculate the variance of the angular velocity to obtain a second variance;

[0126] A third comparison unit 83015, configured to determine whether the second variance matches the motion parameters when the video to be detected is recorded, so as to obtain the first indication information.

[0127] In some embodiments, as shown in Figure 15 , the second analysis unit 8302 may include:

[0128] A third calculation unit 83021, configured to sum up the respective angle change amounts for each video to be detected to obtain a total angle change amount;

[0129] A fourth calculation unit 83022, configured to calculate the mean and variance of the total angle change amounts to obtain a second mean and a third variance;

[0130] A fourth comparison unit 83023, configured to compare the third variance with a second preset variance to obtain the third indication information;

[0131] A determination unit 83024, configured to determine the second mean as the fourth indication information.

[0132] In some embodiments, as shown in Figure 16 , the apparatus may further include:

[0133] A re-acquisition module 850, configured to re-select new feature points from the current frame image according to the preset method if the feature points do not exist in the current frame image.

[0134] The apparatus in the apparatus embodiment and the method embodiment are based on the same inventive concept.

[0135] An embodiment of the present application further provides a computer storage medium, in which at least one instruction or at least one program segment is stored, and the at least one instruction or at least one program segment is loaded and executed by a processor to implement the smoothness detection method provided in the above method embodiment.

[0136] Optionally, in this embodiment, the above computer storage medium may be located in at least one network server among multiple network servers of a computer network. Optionally, in this embodiment, the above storage medium may include, but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disc and other various media that can store program codes.

[0137] An embodiment of the present application further provides a fluency detection device, which includes a processor and a memory. At least one instruction or at least one program segment is stored in the memory, and the at least one instruction or at least one program segment is loaded and executed by the processor to perform the fluency detection method provided in the foregoing method embodiment.

[0138] Furthermore, Figure 17 The figure shows a schematic hardware structure diagram of a device for implementing the method provided in the embodiment of the present application. The device may participate in constituting or include the device or system provided in the embodiment of the present application. As Figure 17 shown, the device 10 may include one or more processors 102 (illustrated as 102a, 102b,..., 102n in the figure) (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 17 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the device 10 may further include more or fewer components than Figure 17 shown in the figure, or have a different configuration from Figure 17 shown in the figure.

[0139] It should be noted that the above one or more processors 102 and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the device 10 (or mobile device). As involved in the embodiment of the present application, the data processing circuit is a processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0140] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage devices corresponding to the methods described in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned smoothness detection method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the device 10 through a network. Examples of the above-mentioned network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.

[0141] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by the communication provider of the device 10. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0142] The display can be, for example, a touch-screen liquid crystal display (LCD), and the liquid crystal display enables a user to interact with the user interface of the device 10 (or mobile device).

[0143] It can be seen from the embodiments of the smoothness detection method, device, equipment, and storage medium provided by the present application that the present application can pre-record the videos to be detected at each sensitivity of the gyroscope by a recording device, and can quantitatively analyze the quantitative relationship between the rotation direction of the field of view between the gyroscope and the target application according to the detection results of each video to be detected; through automated control operations, the inconsistency of manual operations is reduced; by performing three-dimensional reconstruction processing on each frame of image and extracting three-dimensional angular motion data from two-dimensional static frames, it can be more accurately determined whether there is a freeze in the video to be detected; by combining the detection results of each video to be detected at each sensitivity setting of the gyroscope, the smoothness of the target application screen at different sensitivities of the gyroscope can be determined, and the detection accuracy of the target application is improved.

[0144] It should be noted that: the above sequence of the embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of this specification have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0145] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device and the electronic device, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiments.

[0146] The above description has fully disclosed the specific embodiments of the present application. It should be noted that any modifications made by those skilled in the art to the specific embodiments of the present application do not depart from the scope of the claims of the present application. Accordingly, the scope of the claims of the present application is not limited solely to the foregoing specific embodiments.

Claims

1. A fluency detection method, characterized in that The method includes: Obtaining at least one video to be detected corresponding to each sensitivity, where the sensitivity is the sensitivity of the gyroscope in the target application; For each video to be detected, performing three-dimensional reconstruction processing on each frame image of the video to be detected to determine the angular motion data set of the video to be detected. The angular motion data set includes the angular motion data corresponding to each frame image except the first frame image, and the angular motion data characterizes the angular change of each frame image relative to the previous frame image; Performing smoothness detection on the target application according to the angular motion data sets of the videos to be detected corresponding to each sensitivity; The performing smoothness detection on the target application according to the angular motion data sets of the videos to be detected corresponding to each sensitivity includes at least one of the following: Analyzing the angular motion data set of each video to be detected to obtain a first detection result of the video to be detected. The first detection result includes first indication information and second indication information. The first indication information characterizes whether there is a stuttering phenomenon in the video to be detected, and the second indication information characterizes the stuttering position of the video to be detected; Obtaining a second detection result of the sensitivity according to the angular motion data sets of the videos to be detected corresponding to each sensitivity. The second detection result includes third indication information and fourth indication information. The third indication information characterizes the stability degree of the target application under the sensitivity, and the fourth indication information characterizes the stability parameter corresponding to the sensitivity; 2. The method according to claim 1, wherein Before obtaining at least one video to be detected corresponding to each sensitivity, it further includes the step of using a recording device to record at least one video to be detected corresponding to each sensitivity; The using a recording device to record at least one video to be detected corresponding to each sensitivity includes: Fixing the mobile terminal on the holder of the recording device; Sending a start instruction to the target application in the mobile terminal through a test program in the controller of the recording device, so that a target test scenario is presented in the graphical interface of the mobile terminal; Setting the sensitivity of the gyroscope in the target application; Using a control program in the controller of the recording device to control the holder to rotate regularly, so as to drive the mobile terminal to rotate regularly; During the rotation of the mobile terminal, using the recording device of the recording device to perform at least one screen recording operation on the graphical interface to generate at least one video to be detected corresponding to the sensitivity; 3. The method according to claim 1, wherein The performing three-dimensional reconstruction processing on each frame image of the video to be detected to determine the angular motion data set of the video to be detected includes: Extracting at least one feature point from the first frame image of the video to be detected according to a preset method; Starting from the second frame image of the video to be detected, taking each frame image as the current frame image, and performing three-dimensional reconstruction processing on the current frame image according to the positions of each feature point in the previous frame image and in the current frame image to obtain the angular motion data corresponding to the current frame image; The angular motion data corresponding to each frame of the image constitute the angular motion data set of the video to be detected.

4. The method according to claim 1, wherein The angular motion data includes an angular change amount. Analyzing the angular motion data set of each video to be detected to obtain a first detection result of the video to be detected includes: Calculating the mean and variance of the angular change amount to obtain a first mean and a first variance; Comparing the first variance with a first preset variance value to obtain the first indication information; If the first variance is greater than the first preset variance value, comparing the first mean with each angular change amount to obtain the second indication information.

5. The method according to claim 4, characterized in that, The angular motion data includes an angular velocity. The method further includes: Calculating the variance of the angular velocity to obtain a second variance; Determining whether the second variance matches the motion parameter when recording the video to be detected to obtain the first indication information.

6. The method according to claim 1, wherein The angular motion data includes an angular change amount. Obtaining a second detection result of the sensitivity according to the angular motion data set of each video to be detected corresponding to each sensitivity includes: For each video to be detected, summing up each angular change amount to obtain a total angular change amount; Calculating the mean and variance of the total angular change amount to obtain a second mean and a third variance; Comparing the third variance with a second preset variance value to obtain the third indication information; Determining the second mean as the fourth indication information.

7. A fluency detection device, characterized in that, The device includes: A video acquisition module, configured to acquire at least one video to be detected corresponding to each sensitivity pre-recorded, where the sensitivity is the sensitivity of a gyroscope in a target application; A data collection module, configured to perform three-dimensional reconstruction processing on each frame of the video to be detected for each video to be detected to determine the angular motion data set of the video to be detected. The angular motion data set includes the angular motion data corresponding to each frame of the image except the first frame of the image, and the angular motion data characterizes the angular change of each frame of the image relative to the previous frame of the image; An analysis module, configured to perform a smoothness detection on the target application according to the angular motion data sets of each video to be detected corresponding to each sensitivity; Performing a smoothness detection on the target application according to the angular motion data sets of each video to be detected corresponding to each sensitivity includes at least one of the following: Analyzing the angular motion data set of each video to be detected to obtain a first detection result of the video to be detected, where the first detection result includes a first indication information and a second indication information, and the first indication information characterizes whether there is a stuttering phenomenon in the video to be detected, and the second indication information characterizes the stuttering position of the video to be detected; Obtaining a second detection result of the sensitivity according to the angular motion data sets of each video to be detected corresponding to each sensitivity, where the second detection result includes a third indication information and a fourth indication information, and the third indication information characterizes the stability degree of the target application under the sensitivity, and the fourth indication information characterizes the stability parameter corresponding to the sensitivity.

8. A computer storage medium, characterized in that, At least one instruction or at least one program segment is stored in the computer storage medium, and the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the fluency detection method according to any one of claims 1-6.

9. A fluency detection device, characterized in that, The device includes a processor and a memory, and at least one instruction or at least one program segment is stored in the memory. The at least one instruction or at least one program segment is loaded and executed by the processor to implement the fluency detection method according to any one of claims 1-6.

Images