Application evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides an evaluation method and device of an application program, electronic equipment and a storage medium. The method comprises the following steps: determining an association event between a video observation system and a log observation system for evaluating the application program; wherein the video observation system is used for recording an interface change when an application program responds to an associated event; the log observation system is used for recording time information of the application program responding to the associated event; aligning the video observation system and the log observation system in a time dimension based on the association event; and evaluating the application program based on the aligned video observation system and log observation system. By means of the method, the specific consumed time duration of the application program responding to the touch operation can be more accurately determined, and therefore the application program can be more accurately evaluated.

Description

Technical Field

[0001] The present application relates to the field of computers, and more specifically, to an evaluation method, device, electronic device, and storage medium for application programs in the field of computers. Background Art

[0002] In the process of evaluating an application, it is usually necessary to evaluate the performance of the application in responding to touch operation events, and usually a video observation system is used to record the process of the application responding to touch operations, so as to determine the specific time it takes for the application to respond to touch operation events, thereby evaluating the responsiveness of the application.

[0003] Since the signal of a touch operation event needs to go through multiple links in the process of transmission, and each link will be delayed, that is, there is a delay in the process from the application responding to the touch operation event to the application generating interface changes. This will cause the video observation system to determine the specific time it takes for the application to respond to the touch operation event. Inaccurate, resulting in inaccurate evaluation of the application. Summary of the invention

[0004] The present application provides an application evaluation method, device, electronic device and storage medium. The method can more accurately determine the specific time it takes for an application to respond to a touch operation event, thereby more accurately evaluating the application.

[0005] In a first aspect, a method for evaluating an application is provided, the method comprising: determining an associated event between a video observation system and a log observation system used to evaluate the application; wherein the video observation system is used to record interface changes when the application responds to the associated event; the log observation system is used to record time information when the application responds to the associated event; based on the associated event, aligning the video observation system and the log observation system in the time dimension; and evaluating the application based on the aligned video observation system and the log observation system.

[0006] The above technical solution, by determining the associated events between the video observation system and the log observation system used to evaluate the application, can break the isolation between the two different observation systems, can link the related data scattered in different systems, and avoid the problem of inaccurate evaluation of the application caused by the limitations of a single data source. And based on the associated events between the two systems, the video observation system and the log observation system are aligned in the time dimension, which helps to build a unified time frame for evaluating the application. Since the video observation system is used to record the interface changes when the application responds to the associated events, and the log observation system is used to record the time information of the application responding to the associated events, by aligning the video observation system and the log observation system in the time dimension, it can be ensured that the interface changes corresponding to the user operation seen in the video and the response time of the application recorded in the log observation system are consistent. Based on the aligned video observation system and log observation system, the application is evaluated, and the specific time taken for the application to respond to the touch operation event can be determined more accurately, thereby evaluating the application more accurately.

[0007] In combination with the first aspect, in certain implementations of the first aspect, the video observation system and the log observation system are aligned in the time dimension based on the associated event, including: determining a first time point when the application receives the associated event in the video observation system, and determining a second time point when the application receives the associated event in the log observation system; aligning the first time point with the second time point to align the video observation system and the log observation system in the time dimension.

[0008] The above technical solution can ensure that the data about the same event recorded by the two systems accurately corresponds in time by determining the respective time points (i.e., the first time point and the second time point) when the application receives the associated event in the video observation system and the log observation system, and aligning them. This helps to eliminate data errors caused by time inconsistencies, making subsequent evaluations of the application more accurate. After aligning the video observation system and the log observation system in the time dimension, the response time of the application can be more accurately evaluated, thereby more accurately evaluating the application.

[0009] In combination with the first aspect and the above-mentioned implementation methods, in certain implementation methods of the first aspect, determining the first time point when the application receives the associated event in the video observation system includes: drawing a preset mark corresponding to the associated event on the interface of the application when the associated event is received; and determining the first time point based on the time point when the preset mark appears recorded in the video observation system.

[0010] The above technical solution, when receiving the associated event, immediately draws a preset mark corresponding to the associated event on the application interface, and then determines the first time point based on the time point when the video observation system records the appearance of the mark, which can very intuitively and accurately determine the moment when the associated event is received by the application level. In addition, the appearance of the preset mark on the application interface is a visual signal, which is more convenient and intuitive for operators or analysts to determine the time point by observing the appearance of the mark in the video observation system.

[0011] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the first time point is determined based on the time point at which the preset mark appears recorded in the video observation system, including: obtaining a video frame rate corresponding to the video observation system; determining a video refresh duration corresponding to the video frame rate; determining the first time point based on the video refresh duration and the time point at which the preset mark appears recorded in the video observation system; wherein the interval duration between the time point at which the preset mark appears and the first time point is the video refresh duration; and the first time point is earlier than the time point at which the preset mark appears.

[0012] The above technical solution obtains the video frame rate corresponding to the video observation system and determines the video refresh duration based on the video frame rate. This process fully considers the essential characteristics of video recording. By clarifying the video refresh duration, the time point when the application actually receives the associated event can be more accurately located, which can reduce the time error caused by the video frame rate factor, thereby making the first time point determined more accurate.

[0013] In combination with the first aspect and the above-mentioned implementation methods, in certain implementation methods of the first aspect, the application is evaluated based on the aligned video observation system and the log observation system, including: when an operation instruction for the application is received, based on the aligned video observation system and the log observation system, determining in the video observation system a first response duration between the time point when the application receives the operation instruction and the time point when the interface of the application changes, and determining in the log observation system a second response duration between the time point when the application receives the operation instruction and the time point when the application responds to the operation instruction; determining the time difference between the first response duration and the second response duration; and evaluating the application based on the time difference.

[0014] The above technical solution, based on the aligned video observation system and log observation system, respectively determines the first response time and the second response time. These two indicators reflect the response of the application to the operation instruction from different angles. The first response time focuses on the time interval from receiving the operation instruction to the change of the interface, reflecting the visual feedback speed that the user can intuitively feel; the second response time focuses on the time from receiving the instruction to the actual response of the application to the instruction, reflecting the efficiency of the internal processing logic of the application. By clarifying these two time indicators, the response performance of the application can be measured more comprehensively and accurately. The time difference between the first response time and the second response time is calculated. The time difference reflects the time spent by the application in the interface update stage after processing the operation instruction. Evaluating the application based on the time difference can quickly discover possible problems in the response process of the application and can evaluate the application more reasonably.

[0015] In combination with the first aspect and the above-mentioned implementation methods, in certain implementation methods of the first aspect, the application is evaluated based on the time difference, including: if the time difference is greater than the preset difference, determining that the response capability of the application does not meet the preset standard; if the time difference is less than or equal to the preset difference, determining that the response capability of the application meets the preset standard.

[0016] In combination with the first aspect and the above-mentioned implementations, in some implementations of the first aspect, the associated event is a touch operation event on the application; the touch operation event includes a finger press event, a finger move event, and a finger lift event.

[0017] In a second aspect, an evaluation device for an application is provided, the device comprising: a determination module for determining an associated event between a video observation system and a log observation system for evaluating the application; wherein the video observation system is used to record interface changes when the application responds to the associated event; the log observation system is used to record time information when the application responds to the associated event; an alignment module for aligning the video observation system and the log observation system in a time dimension based on the associated event; and an evaluation module for evaluating the application based on the aligned video observation system and the log observation system.

[0018] In combination with the second aspect, in certain implementations of the second aspect, the alignment module is specifically used to: determine a first time point when the application receives the associated event in the video observation system, and determine a second time point when the application receives the associated event in the log observation system; align the first time point and the second time point to align the video observation system and the log observation system in the time dimension.

[0019] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the alignment module includes a determination unit, which is specifically used to: draw a preset mark corresponding to the associated event on the interface of the application when the associated event is received; and determine the first time point based on the time point when the preset mark appears recorded in the video observation system.

[0020] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the determination unit is also specifically used to: obtain the video frame rate corresponding to the video observation system; determine the video refresh duration corresponding to the video frame rate; determine the first time point based on the video refresh duration and the time point when the preset mark appears recorded in the video observation system; wherein the interval between the time point when the preset mark appears and the first time point is the video refresh duration; the first time point is earlier than the time point when the preset mark appears.

[0021] In combination with the second aspect and the above-mentioned implementation methods, in certain implementation methods of the second aspect, the evaluation module is specifically used to: when receiving an operation instruction for the application, based on the aligned video observation system and the log observation system, determine in the video observation system a first response duration between the time point when the application receives the operation instruction and the time point when the interface of the application changes, and determine in the log observation system a second response duration between the time point when the application receives the operation instruction and the time point when the application responds to the operation instruction; determine the time difference between the first response duration and the second response duration; and evaluate the application based on the time difference.

[0022] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the evaluation module includes an evaluation unit, which is specifically used to: if the time difference is greater than the preset difference, determine that the response capability of the application does not meet the preset standard; if the time difference is less than or equal to the preset difference, determine that the response capability of the application meets the preset standard.

[0023] In combination with the second aspect and the above-mentioned implementations, in some implementations of the second aspect, the associated event is a touch operation event on the application; the touch operation event includes a finger press event, a finger move event, and a finger lift event.

[0024] In a third aspect, an electronic device is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the electronic device executes the evaluation method of the application in the first aspect and any possible implementation of the first aspect.

[0025] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the evaluation method of the application in the above-mentioned first aspect and any possible implementation of the first aspect.

[0026] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the evaluation method of the application in the above-mentioned first aspect and any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic flow chart of an application evaluation method provided in an embodiment of the present application;

[0028] Figure 2 is a schematic diagram of an aligned video observation system and a log observation system provided in an embodiment of the present application;

[0029] Figure 3 It is a structural schematic diagram of an evaluation device for an application program provided in an embodiment of the present application;

[0030] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0032] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0033] In the software evaluation system, whether the application responds to the user's touch operation events in a timely manner is an important indicator for measuring the quality of the application. For example, when a user clicks a button or performs a touch operation such as sliding, if the application responds too slowly, it will greatly affect the user experience. Therefore, the quality team needs a way to accurately measure this response time. Usually, an evaluation indicator can be set for the time it takes for the application to respond to touch operations to determine whether the time it takes for the application to respond to touch operations is within an acceptable range. These indicators are usually set based on the type of application, its purpose, and user expectations.

[0034] Since touch operation is an instantaneous behavior and the application's response may be a dynamic process, it is difficult to accurately determine the time it takes for the application to respond to touch operation simply by manual observation. Usually, the entire process from the user performing the touch operation to the application completing the response is recorded by video shooting (i.e., video observation system).

[0035] During the entire touch operation process recorded in the video, it is difficult to directly determine in which frame the application receives the touch operation event in the video frame sequence. This is because a touch operation event signal generally goes through the links of "hardware circuit → driver → operating system → application", and each link will produce delays, which will cause a phenomenon that although the finger touches the screen and slides in the video, the application may not update the interface according to this touch action until a little later.

[0036] Since there is a delay between the time when an application responds to a touch operation event and the time when the application generates an interface change, the specific time taken by the application to respond to the touch operation determined by the video observation system is not accurate enough, resulting in an inaccurate evaluation of the application.

[0037] In order to solve the above technical problems, the embodiment of the present application provides an application evaluation method, the execution subject of the method is an electronic device, for example, the electronic device can be a mobile phone, a computer, a smart TV, a vehicle computer, etc. The following takes the application evaluation method applied to a vehicle computer in a vehicle as an example for explanation:

[0038] Figure 1 It is a schematic flowchart of an application evaluation method provided in an embodiment of the present application.

[0039] For example, Figure 1 As shown, the method 100 includes:

[0040] S101, determining a correlation event between a video observation system and a log observation system for evaluating an application.

[0041] The video observation system is used to record the interface changes when the application responds to the associated events; the log observation system is used to record the time information when the application responds to the associated events.

[0042] S102: Based on the associated events, align the video observation system and the log observation system in the time dimension.

[0043] S103, evaluating the application based on the aligned video observation system and the log observation system.

[0044] In an embodiment of the present application, by determining the associated events between the video observation system and the log observation system used to evaluate the application, the isolation between the two different observation systems can be broken, and the relevant data scattered in different systems can be linked to avoid the problem of inaccurate evaluation of the application caused by the limitations of a single data source. And based on the associated events between the two systems, the video observation system and the log observation system are aligned in the time dimension, which helps to build a unified time frame for evaluating the application. Since the video observation system is used to record the interface changes when the application responds to the associated events, and the log observation system is used to record the time information of the application responding to the associated events, by aligning the video observation system and the log observation system in the time dimension, it can be ensured that the interface changes corresponding to the user operation seen in the video and the response time of the application recorded in the log observation system are consistent. Based on the aligned video observation system and log observation system, the application is evaluated, and the specific time-consuming duration of the application responding to the touch operation can be determined more accurately, thereby evaluating the application more accurately.

[0045] Below Figure 1 The implementation method of each step in the embodiment shown is specifically described:

[0046] As for the above S101, it can be understood that the above associated events refer to events that can correspond to each other in the video observation system and the log observation system. The purpose of determining the above associated events is to establish a common time reference point between the video observation system and the log observation system.

[0047] Exemplarily, the above-mentioned associated events may be some key operations of the user on the application, such as a touch operation, an input operation, etc. of the user.

[0048] In a possible implementation, the associated event is a touch operation event on the application; the touch operation event includes a finger press event, a finger move event, and a finger lift event.

[0049] It can be understood that when a finger is detected touching the screen, the car system will generate an ACTION_DOWN type event (i.e., a finger press event); when the finger is detected continuing to move on the screen, the car system will periodically generate ACTION_MOVE type events (i.e., finger movement events); when the finger is detected lifted and leaves the screen, the car system will generate an ACTION_UP type event (i.e., a finger lift event).

[0050] The finger press event is a start event of a touch operation, which is triggered when the user's finger touches the application interface.

[0051] The finger movement event refers to the time generated when the finger keeps in contact with the screen and moves.

[0052] The finger lift event refers to the end time of the touch operation, which is triggered when the user's finger leaves the application interface.

[0053] Furthermore, a complete touch operation process may consist of a finger press event, several finger move events, and a finger lift event.

[0054] In some embodiments, any one of the finger pressing event, the finger moving event, and the finger lifting event, or multiple events of the finger pressing event, the finger moving event, and the finger lifting event may be determined as associated events.

[0055] It is understandable that the above-mentioned finger pressing event is the starting point of the user's interaction with the application. If the finger pressing event is determined as an associated event, the starting time of the user's operation can be clearly defined, and the application's responsiveness to the initial operation can be evaluated. For example, in an application, the user initiates a command by pressing a button in the application interface. With this event as an associated event, the time from the application receiving the command to responding to the command can be accurately analyzed, and the application's responsiveness can be evaluated based on this.

[0056] The above finger movement events reflect the dynamic interaction process of users on the application interface. If the finger movement events are determined as associated events, we can focus on the real-time feedback capabilities of the application to the user's continuous operations. For example, in a map application, using the finger movement events as associated events can evaluate the smoothness of the map following the finger movement, as well as the timeliness of background data loading and rendering, thereby optimizing the operating experience of the map application.

[0057] The finger lift event marks the end of a touch operation. If the finger lift event is determined as a related event, the responsiveness of the application after the user completes an operation can be determined as a whole. For example, in an online payment application, when confirming a payment operation, after the user completes the payment operation, the finger lift event can be used to determine the time it takes for the application to process the payment request and return the result, thereby determining the responsiveness of the application.

[0058] If the finger press event and the finger lift event are combined and determined as a related event, the overall response process of the application to an independent operation can be fully analyzed, including the total time from receiving the instruction to processing and returning the result. Based on this time, the responsiveness of the application can be fully evaluated.

[0059] Furthermore, the video observation system can record the entire process of the user's operation of the vehicle computer by shooting video, and can also intuitively observe the changes in the interface displayed on the screen of the vehicle computer when the user operates the vehicle computer through the video recorded by the video observation system.

[0060] The above-mentioned log observation system can record various information related to the operation of the application program in the vehicle computer. For example, the log observation system can record the timestamp of each event.

[0061] It can be understood that based on the different characteristics of the video observation system and the log observation system, the video observation system and the log observation system can be associated, so as to closely combine the interface changes of the application observed by the video observation system and the internal information of the application observed by the log observation system, so as to more accurately evaluate the responsiveness of the application.

[0062] Regarding the above S102, it can be understood that the above video observation system and log observation system are independent of each other. Based on the determined associated events, the video observation system and the log observation system can be aligned in the time dimension, that is, the video observation system and the log observation system can be operated based on the same time reference.

[0063] By aligning the video observation system and the log observation system in the time dimension, the time deviation between the two systems can be eliminated as much as possible, and the specific time it takes for the application to respond to the associated event can be determined more accurately.

[0064] Exemplarily, the frame node at which the application receives the associated event recorded in the video observation system can be associated with the timestamp at which the application receives the associated event recorded in the log observation system, thereby achieving alignment of the two observation systems in the time dimension.

[0065] In one possible implementation, based on associated events, the video observation system and the log observation system are aligned in the time dimension, including: determining a first time point when the application receives the associated event in the video observation system, and determining a second time point when the application receives the associated event in the log observation system; aligning the first time point with the second time point to align the video observation system and the log observation system in the time dimension.

[0066] It can be understood that the above-mentioned first time point refers to the time point when the application program determined in the video observation system receives the associated event.

[0067] As mentioned above, since there is a delay between the time when an application responds to an associated event and the time when the application generates an interface change, and this delay is unavoidable, the video observation system is usually unable to directly determine in which frame the application received the associated event, that is, it is unable to accurately determine the first time point when the application received the associated event.

[0068] In some embodiments, the specific delay duration from the time when an application responds to an associated event to the time when the application generates an interface change can be determined first, that is, the delay duration in the process of hardware circuit → driver → operating system → application. Then, based on the determined delay duration and the time point when the user initiates the above-mentioned associated event in the video recorded by the video observation system, the first time point can be determined.

[0069] It is understandable that, under normal circumstances, each link in the chain of associated event transmission can be queried in turn, and the specific delay duration in the signal transmission process of the associated event can be determined based on the information reflected by each link. Then, the time point at which the user initiates the associated event in the video recorded by the video observation system is obtained, and based on this time point, the delay duration in the signal transmission process is considered to obtain the first time point at which the application in the video observation system receives the associated event.

[0070] For example, taking the above-mentioned associated event as a touch operation event, if the specific delay duration in the touch operation event signal transmission process is determined to be 1 second, and the time point when the user initiates the touch operation in the video recorded by the video observation system is 14:00, considering the delay duration of 1 second in the signal transmission process, that is, the time point when the application in the video observation system receives the touch operation event will be delayed by 1 second based on the time point when the user initiates the touch operation, it can be obtained that the first time point when the application in the video observation system receives the associated event is 14:01.

[0071] Since the video recorded by the video observation system can only observe the time point when the user initiates an operation instruction to the application (i.e., generates an associated event), and the time point when the application responds to the operation instruction and generates an interface change (i.e., responds to the associated event), the specific time point when the application actually receives the operation instruction (i.e., receives the associated event) can only be obtained by speculation. In order to more accurately determine the specific time point when the application actually receives the operation instruction (i.e., receives the associated event), when it is determined that the application receives the associated event, a mark can be drawn on the interface of the application, so that the specific time point when the mark appears on the application interface can be directly observed in the video recorded by the video observation system.

[0072] In one possible implementation, the first time point when the application receives the associated event is determined in the video observation system, including: when the associated event is received, drawing a preset mark corresponding to the associated event on the interface of the application; and determining the first time point based on the time point when the preset mark appears recorded in the video observation system.

[0073] It is understandable that when the application receives the associated event, a preset mark is drawn on the interface, and the preset mark can be recorded by the video observation system as a reference for determining the first time point.

[0074] Exemplarily, the preset mark may be a small dot that flashes briefly or a special icon, which may appear at a specific location on the application program interface.

[0075] Furthermore, in order to more accurately distinguish different associated events in the video recorded by the video observation system, a corresponding preset mark may be set for each associated event.

[0076] In some embodiments, the bypass observation mechanism can be designed in advance and integrated into the application program. When the application program receives a correlation event, a preset mark that can be recorded by the video observation system can be drawn on the display interface of the application program.

[0077] It can be understood that the above-mentioned bypass observation mechanism refers to a mechanism for obtaining information in an additional manner outside the main observation system (ie, the video observation system and the log observation system mentioned above).

[0078] Exemplarily, the process of designing the bypass observation mechanism and integrating the designed bypass observation mechanism into the application may be as follows: first, a suitable tag is selected, which may be a tag corresponding to the associated event. Then, the triggering condition of the bypass observation mechanism may be defined, for example, the condition that the application receives the associated event may be set as the triggering condition of the bypass observation mechanism. Finally, necessary logic is added to the front-end code of the application so that the defined bypass observation mechanism is integrated into the application.

[0079] Furthermore, in the architecture of the application, when an associated event occurs, the application framework usually passes the associated event to the corresponding execution module for processing through a set of event distribution mechanisms. In order to determine the timing when the application receives the associated event, when the application framework distributes the associated event to the observed entity (i.e., the execution module in the application for responding to the associated event), the associated event can be intercepted through a callback function to obtain the specific time point when the application receives the associated event, and a mark is drawn on the display interface of the application based on the specific time point when the application receives the associated event.

[0080] It is understandable that the above application framework is responsible for receiving events from the operating system or other input sources, and then judging which execution module in the application program should be assigned to handle these events according to predefined rules and algorithms. For example, in the Android system, Android Framework is a typical application framework.

[0081] The above-mentioned observed entity refers to a specific execution module in the application framework. For example, in a music playing application, when a user clicks a button corresponding to "switch to the next song", the button is the "observed entity".

[0082] Furthermore, the callback function is called before the associated event is about to be processed by the observed entity. By intercepting the associated event through the callback function, the specific time point when the application receives the associated event can be determined.

[0083] Since the mark is drawn immediately after the application receives the associated event, the time point at which the mark appears is ideally very close to the time when the application actually receives the associated event. The time point at which the drawn mark appears can be obtained in the video recorded by the video observation system, and based on this time point, the first time point can be determined more accurately. That is, based on the cooperation of the video observation system and the bypass observation mechanism, the first time point can be determined more accurately.

[0084] The above method, when receiving the associated event, immediately draws a preset mark corresponding to the associated event on the application interface, and then determines the first time point based on the time point when the video observation system records the appearance of the mark, which can very intuitively and accurately determine the moment when the associated event is received by the application level. In addition, the appearance of the preset mark on the application interface is a visual signal, and it is more convenient and intuitive for operators or analysts to determine the time point by observing the appearance of the mark in the video observation system.

[0085] In order to determine the first time point more accurately, in addition to considering the delay in the transmission of the associated event signal, the delay in the mark display process caused by the screen hardware properties can also be considered.

[0086] In a possible implementation, determining a first time point based on a time point at which a preset mark appears recorded in a video observation system includes: acquiring a video frame rate corresponding to the video observation system; determining a video refresh duration corresponding to the video frame rate; determining the first time point based on the video refresh duration and the time point at which the preset mark appears recorded in the video observation system; wherein the interval duration between the time point at which the preset mark appears and the first time point is the video refresh duration; and the first time point is earlier than the time point at which the preset mark appears.

[0087] It is understandable that a video observation system usually captures video at a fixed video frame rate, where the video frame rate refers to the number of frames recorded by the video observation system in one second.

[0088] The above video refresh duration refers to the time interval between every two frames, and the video refresh duration = 1 / video frame rate.

[0089] For example, if the video observation system shoots video at a video frame rate of 60 frames per second, the video refresh duration=1 / video frame rate, and it can be determined that the video refresh duration is approximately 17 milliseconds (ms).

[0090] Since the video recorded in the video observation system will update the video frame at a fixed video refresh time, for example, the video refresh time determined based on the video frame rate is 17ms, which means that the video frame will be updated every 17ms. When the video recorded by the video observation system observes that the preset mark appears in a certain frame in the video, this time point is not the time point when the application actually receives the associated event. In fact, because of the video refresh process, the time point when the application receives the associated event will be earlier.

[0091] For example, if the time point at which the preset mark appears in the video recorded by the video observation system is t 标记, determine the video refresh duration as T, since the interval between the time point when the preset mark appears and the first time point is the video refresh duration, and the first time point is earlier than the time point when the preset mark appears, it can be determined that at t 标记 The time point before and with an interval of T is the first time point.

[0092] Furthermore, the second time point mentioned above refers to a time point determined in the log observation system when the application receives the associated event.

[0093] Since the log observation system itself can record various information related to the operation of the application program, including the timestamp information of each event, based on this, the second time point can be directly determined in the log observation system.

[0094] Furthermore, by aligning the first time point and the second time point, an aligned video observation system and a log observation system can be obtained.

[0095] The above method can ensure that the data about the same event recorded by the two systems accurately corresponds in time by determining the respective time points (i.e., the first time point and the second time point) when the application receives the associated event in the video observation system and the log observation system and aligning them. This helps to eliminate data errors caused by time inconsistency, making subsequent evaluation of the application more accurate. After aligning the video observation system and the log observation system in the time dimension, the response time of the application can be more accurately evaluated, thereby more accurately evaluating the application.

[0096] In some embodiments, the resulting aligned video observation system and log observation system can be as follows: Figure 2 shown.

[0097] For example, Figure 2 As shown, based on the video timeline, it can be determined that the video observation system first observes the touch operation event of the finger touching the screen, but since the touch operation event signal will be delayed during the transmission process, the time point when the application receives the touch operation event is also delayed.

[0098] Since the video observation system cannot directly determine the time point when the application receives the touch operation event, a bypass observation mechanism can be introduced. As mentioned above, based on the bypass observation mechanism, when the application receives the associated event, a preset mark that can be recorded by the video observation system can be drawn on the display interface of the application, and the preset mark is also related to the type of event. For example, with a finger press event (i.e. Figure 2 The mark related to the DOWN event in the DOWN event is the DOWN event mark; the mark related to the finger movement event (i.e. Figure 2The tags related to the MOVE event in are called MOVE event tags.

[0099] Since the video observation system records the process from the generation of a touch operation event to the time when the application responds to the touch operation at a fixed video frame rate, the fixed video frame rate corresponds to a fixed video refresh duration, that is, the video recorded in the video observation system will update the video frame at a fixed video refresh duration. For example, the video refresh duration determined based on the video frame rate is 17ms, which means that the video frame will be updated every 17ms. When a preset mark appears in a certain frame of the video based on the video recorded by the video observation system, this time point is not the time point when the application actually receives the touch operation event. In fact, because of the video refresh process, the time point when the application receives the touch operation event will be earlier, specifically, the actual time point before the preset mark is observed to appear, and the interval between the actual time point and the time point when the preset mark appears is 17ms.

[0100] like Figure 2 As shown, if the DOWN event mark is observed to appear at the marked time point based on the video recorded by the video observation system, it can be determined that the actual time point when the application receives the DOWN event is before the marked time point, and the interval between the actual time point and the marked time point is 17ms. The actual time point of the determined DOWN event is marked on the video timeline.

[0101] Figure 2 The log observation system in can record the timestamp information of each event received by the application. That is, based on the content recorded by the log observation system, the time point when the recorded application receives the finger press event (ACTION_DOWN), the time point when the recorded application receives the finger move event (ACTION_MOVE), and the time point when the recorded application responds to the action can be marked on the log timeline.

[0102] Furthermore, if the finger press event (i.e., DOWN event) is determined as an associated event between the video observation system and the log observation system, the actual time point of the DOWN event marked on the video timeline can be aligned with the time point when the application receives the finger press event (ACTION_DOWN) marked on the log timeline, thereby achieving alignment of the video observation system and the log observation system in the time dimension.

[0103] The video timeline may include the time points of various events recorded by the video observation system through the video, such as the time point of the finger press event, the time point of the finger movement event, the time point of the finger lift event, and the time point when the interface of the application program changes. The log timeline may include the time points of various events recorded by the log observation system through the log, such as the time point when the application program receives the finger press event, the time point when the application program receives the finger movement event, and the time point when the application program starts to respond to the action.

[0104] Since the video observation system will first observe a series of actions such as the user's finger touching the screen, but in fact these actions have not yet been received by the application, Figure 2 The response time of the application observed by the video observation system through the video will be longer than the response time of the application observed by the log observation system through the log.

[0105] For the above S103, it can be understood that based on Figure 2 The aligned video observation system and log observation system shown can accurately evaluate the responsiveness of the application to the operation instructions initiated by the user.

[0106] For example, based on Figure 2 The aligned video observation system and log observation system shown in the figure can observe, through the video recorded by the video observation system, the time point when the application receives an operation instruction initiated by the user and the time point when the interface of the application changes. Combined with the time point when the application receives the operation instruction and the time point when the application responds to the operation instruction recorded in the log observation system, based on the above-determined time points, the application's responsiveness to user-initiated operation instructions can be evaluated.

[0107] In one possible implementation, an application is evaluated based on an aligned video observation system and a log observation system, including: upon receiving an operation instruction for the application, determining in the video observation system a first response duration between a time point when the application receives the operation instruction and a time point when an interface of the application changes, and determining in the log observation system a second response duration between a time point when the application receives the operation instruction and a time point when the application responds to the operation instruction; determining a time difference between the first response duration and the second response duration; and evaluating the application based on the time difference.

[0108] It is understandable that the time point when the application receives the operation instruction in the aligned video observation system and the log observation system is consistent. The above-mentioned first response duration refers to the time from the application actually receiving the operation instruction to the user being able to see obvious changes in the interface, which is determined based on the video recorded by the video observation system, that is, the time it takes for the application to respond to the operation instruction as perceived by the user.

[0109] The log observation system records various events and operation logs within the application. By looking for log records related to the operation instruction, the specific time point when the application responds to the operation instruction can be determined. The second response duration mentioned above refers to the time point when the application receives the operation instruction recorded by the log observation system to the time when the application responds to the operation instruction, that is, the time it takes for the application to actually process the operation instruction.

[0110] The time difference between the first response time and the second response time is a key indicator for evaluating application performance. This time difference reflects the time spent by the application in the interface update phase after processing the operation instruction.

[0111] For example, Figure 2 As shown, the actual time point when the application receives the ACTION_DOWN event and the interface change time point when the application interface changes can be determined in the video timeline, and then the first interval duration between the actual time point and the interface change time point can be calculated.

[0112] You can also determine the time point when the application receives the ACTION_DOWN event in the log timeline (this time point is aligned with the actual time point), and determine the time point of the application action response in the log timeline, and then calculate the second interval duration between the two time points.

[0113] Furthermore, a time difference between the first interval duration and the second interval duration is calculated, and the application is evaluated based on the time difference.

[0114] It is understandable that the above time difference not only includes the process of interface rendering and display update, but may also be affected by network requests, background tasks and other hardware resource limitations. Based on this, a preset difference can be set, and by comparing the above time difference with the preset difference, the application can be evaluated more accurately.

[0115] In one possible implementation, the application is evaluated based on a time difference, including: if the time difference is greater than a preset difference, determining that the responsiveness of the application does not meet a preset standard; if the time difference is less than or equal to the preset difference, determining that the responsiveness of the application meets the preset standard.

[0116] It is understandable that the above preset difference can be set according to actual needs. For example, for a chat application that pursues instant response, the preset difference may be set to a smaller value; while for some professional applications that handle complex tasks, the preset difference may be relatively larger.

[0117] The above time difference refers to the time required from the application completing the processing and preparing the new interface to the new interface actually being displayed on the application interface.

[0118] Under normal circumstances, after receiving an operation instruction, the application will quickly complete the internal processing and immediately start updating the interface.

[0119] If the above time difference is less than or equal to the preset difference, it means that the application can quickly respond to user operations and quickly provide feedback to the user in a short time, that is, the response capability of the application meets the preset standard.

[0120] If the above time difference is greater than the preset difference, it means that the application may have a performance bottleneck, such as slow interface rendering, which results in too long time spent on displaying the interface after processing is completed. For users, they will perceive that the application responds too slowly, that is, the application's responsiveness does not meet the preset standards.

[0121] Exemplarily, assuming that the preset difference is set to 100ms, if the time difference between the first response time and the second response time is greater than 100ms, it means that the application responds too slowly, that is, the response capability of the application does not meet the preset standard; if the time difference between the first response time and the second response time is less than or equal to the preset difference, it means that the application responds faster, that is, the response capability of the application meets the preset standard.

[0122] The above method determines the first response time and the second response time respectively based on the aligned video observation system and the log observation system. These two indicators reflect the response of the application to the operation instruction from different angles. The first response time focuses on the time interval from receiving the operation instruction to the change of the interface, reflecting the visual feedback speed that the user can intuitively feel; the second response time focuses on the time from receiving the instruction to the actual response of the application to the instruction, reflecting the efficiency of the internal processing logic of the application. By clarifying these two time indicators, the response performance of the application can be measured more comprehensively and accurately. The time difference between the first response time and the second response time is calculated. The time difference reflects the time spent by the application in the interface update stage after processing the operation instruction. Evaluating the application based on the time difference can quickly discover possible problems in the response process of the application and can evaluate the application more reasonably.

[0123] Figure 3 It is a structural diagram of an evaluation device for an application provided in an embodiment of the present application.

[0124] For example, Figure 3 As shown, the device 300 includes:

[0125] The determination module 301 is used to determine the correlation event between the video observation system and the log observation system used to evaluate the application.

[0126] The video observation system is used to record the interface changes when the application responds to the associated events; the log observation system is used to record the time information when the application responds to the associated events.

[0127] The alignment module 302 is used to align the video observation system and the log observation system in a time dimension based on the associated event.

[0128] The evaluation module 303 is used to evaluate the application based on the aligned video observation system and the log observation system.

[0129] In one possible implementation, the alignment module is specifically used to: determine a first time point when the application receives the associated event in the video observation system, and determine a second time point when the application receives the associated event in the log observation system; align the first time point and the second time point to align the video observation system and the log observation system in the time dimension.

[0130] In one possible implementation, the alignment module includes a determination unit, which is specifically used to: draw a preset mark corresponding to the associated event on the interface of the application when the associated event is received; and determine the first time point based on the time point at which the preset mark appears recorded in the video observation system.

[0131] In a possible implementation, the determination unit is also specifically used to: obtain the video frame rate corresponding to the video observation system; determine the video refresh duration corresponding to the video frame rate; determine the first time point based on the video refresh duration and the time point at which the preset mark appears recorded in the video observation system; wherein the interval between the time point at which the preset mark appears and the first time point is the video refresh duration; and the first time point is earlier than the time point at which the preset mark appears.

[0132] In one possible implementation, the evaluation module is specifically used to: upon receiving an operation instruction for the application, determine in the video observation system a first response duration between a time point when the application receives the operation instruction and a time point when the interface of the application changes, and determine in the log observation system a second response duration between a time point when the application receives the operation instruction and a time point when the application responds to the operation instruction; determine a time difference between the first response duration and the second response duration; and evaluate the application based on the time difference.

[0133] In one possible implementation, the evaluation module includes an evaluation unit, which is specifically used to: if the time difference is greater than a preset difference, determine that the responsiveness of the application does not meet the preset standard; if the time difference is less than or equal to the preset difference, determine that the responsiveness of the application meets the preset standard.

[0134] In a possible implementation, the associated event is a touch operation event on the application; the touch operation event includes a finger press event, a finger move event, and a finger lift event.

[0135] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0136] For example, Figure 4 As shown, the electronic device 400 includes: a memory 401 and a processor 402, wherein the memory 401 stores an executable program code 4011, and the processor 402 is used to call and execute the executable program code 4011 to perform an evaluation method of an application.

[0137] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to execute an evaluation method for an application provided in an embodiment of the present application.

[0138] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0139] In the case of dividing each functional module according to each function, the device may also include a determination module, an alignment module, an evaluation module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0140] It should be understood that the device provided in this embodiment is used to execute the above-mentioned evaluation method of an application program, and thus can achieve the same effect as the above-mentioned implementation method.

[0141] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to an electronic device, the processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device in executing relevant program codes and data.

[0142] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits shown in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0143] In addition, the device provided in the embodiments of the present application may specifically be a chip, component or module, and the chip may include a connected processor and memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute an evaluation method of an application provided in the above embodiment.

[0144] This embodiment also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement an evaluation method for an application provided in the above embodiment.

[0145] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement an application evaluation method provided by the above embodiment.

[0146] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0147] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0148] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0149] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for evaluating an application, characterized in that: The method comprises: Determine an associated event between a video observation system and a log observation system for evaluating an application; wherein the video observation system is used to record interface changes when the application responds to the associated event; and the log observation system is used to record time information when the application responds to the associated event; Based on the associated event, aligning the video observation system and the log observation system in a time dimension; The application is evaluated based on the aligned video observation system and the log observation system.

2. The method according to claim 1, characterized in that: The step of aligning the video observation system and the log observation system in a time dimension based on the associated event includes: Determine in the video observation system a first time point at which the application receives the associated event, and determine in the log observation system a second time point at which the application receives the associated event; The first time point and the second time point are aligned to align the video observation system and the log observation system in a time dimension.

3. The method according to claim 2, characterized in that Determining in the video observation system the first time point at which the application receives the associated event includes: When the associated event is received, a preset mark corresponding to the associated event is drawn on the interface of the application program; The first time point is determined based on a time point at which the preset mark appears, recorded in the video observation system.

4. The method according to claim 3, characterized in that: The determining the first time point based on the time point at which the preset mark appears recorded in the video observation system includes: Obtaining a video frame rate corresponding to the video observation system; Determine a video refresh duration corresponding to the video frame rate; The first time point is determined based on the video refresh duration and the time point at which the preset mark appears recorded in the video observation system; wherein the interval between the time point at which the preset mark appears and the first time point is the video refresh duration; and the first time point is earlier than the time point at which the preset mark appears.

5. The method according to claim 1, characterized in that The evaluating the application based on the aligned video observation system and the log observation system includes: In the case of receiving an operation instruction for the application, based on the aligned video observation system and log observation system, determining in the video observation system a first response duration between a time point when the application receives the operation instruction and a time point when the interface of the application changes, and determining in the log observation system a second response duration between a time point when the application receives the operation instruction and a time point when the application responds to the operation instruction; Determine a time difference between the first response time and the second response time; Based on the time difference, the application is evaluated.

6. The method according to claim 5, characterized in that The step of evaluating the application based on the time difference includes: If the time difference is greater than a preset difference, determining that the responsiveness of the application does not meet a preset standard; If the time difference is less than or equal to the preset difference, it is determined that the responsiveness of the application meets the preset standard.

7. The method according to any one of claims 1 to 3, characterized in that The associated event is a touch operation event on the application; the touch operation event includes a finger press event, a finger move event and a finger lift event.

8. An application program evaluation device, characterized in that: The device comprises: A determination module, used to determine the associated event between the video observation system and the log observation system used to evaluate the application; wherein the video observation system is used to record the interface changes when the application responds to the associated event; and the log observation system is used to record the time information of the application responding to the associated event; an alignment module, configured to align the video observation system and the log observation system in a time dimension based on the associated event; An evaluation module is used to evaluate the application based on the aligned video observation system and the log observation system.

9. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images